Method for generating content related to movement of vehicle, and electronic device supporting same
The electronic device in vehicles integrates interior and exterior sensors to generate high-quality content using AI models, addressing the limitations of existing technologies by capturing user interactions and external environments for enhanced content creation.
Patent Information
- Application Number
- PCT/KR2025/095172
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-07-03
- Filing Date
- 2025-04-08
- Publication Date
- 2025-12-26
AI Technical Summary
Existing technologies fail to effectively generate content related to the movement of vehicles using data from both interior and exterior cameras and microphones, limiting the creation of high-quality content during autonomous driving.
An electronic device mounted on a vehicle uses interior and exterior cameras, microphones, and processors to identify events, select relevant images, and generate content using artificial intelligence models, integrating data from sensors and external sources to enhance content creation.
Enables the generation of high-quality content by capturing user interactions and external environments, enhancing user experience and content creation capabilities in autonomous vehicles.
Smart Images

Figure KR2025095172_26122025_PF_FP_ABST
Abstract
Description
Method for generating content related to the movement of a vehicle and electronic device supporting the same
[0001] The present disclosure relates to a method for generating content related to the movement of a vehicle and an electronic device supporting the same. Specifically, the present disclosure relates to a method for generating content related to movement in a movable electronic device mounted on a vehicle and an electronic device supporting the same.
[0002] Self-driving cars powered by information and communication technology (ICT) and artificial intelligence (AI) and vehicle-related services based on electronic devices are advancing rapidly. The automotive industry's paradigm, fused with ICT and AI, is rapidly shifting from internal combustion engines (personal driving) to electric power (autonomous driving). Vehicles offering autonomous driving capabilities can be equipped with cameras facing outside the vehicle for autonomous driving, and based on images captured by the cameras, they can provide object detection capabilities utilizing deep learning technology.
[0003] Meanwhile, technology that provides content generated by grouping content captured on electronic devices registered to a user account by subject and location is now available through devices such as smartphones. Recently, with the active development of deep learning-based AI systems, these systems are being utilized to search video captured by cameras, recognize objects within the video, and enhance the captured footage.
[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.
[0005] According to one embodiment, a method for an electronic device in a vehicle to generate content related to the movement of the vehicle may include an operation of obtaining at least one user image by photographing at least one user inside the vehicle using at least one first camera facing inside the vehicle while the vehicle is moving. The method may include an operation of obtaining voice data of the at least one user inside the vehicle using a microphone disposed inside the vehicle while the vehicle is moving. The method may include an operation of identifying a purpose of generating the content related to the movement of the vehicle based on the at least one user image or the voice data when a preset event occurs, and an operation of monitoring a situation related to the movement of the vehicle. The method may include an operation of selecting at least one image from among the at least one user image acquired using the at least one first camera or a plurality of external images acquired using at least one second camera facing outside the vehicle based on the purpose of generating the content and the monitored situation, and generating the content including the at least one selected image. The action of generating the above content may be to generate the content by inputting at least one selected image into an artificial intelligence model trained to generate the content related to the movement of the vehicle.
[0006] According to one embodiment, an electronic device within a vehicle may include at least one first camera facing the interior of the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one second camera facing the exterior of the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one microphone disposed within the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one processor. According to one embodiment, an electronic device within a vehicle may include a memory storing instructions and an artificial intelligence model. The artificial intelligence model may be configured through training for generating content related to the movement of the vehicle. The instructions may be executed by the at least one processor to cause the electronic device to acquire at least one user image by photographing at least one user within the vehicle using the at least one first camera while the vehicle is moving. The artificial intelligence model may be configured through training for generating content related to the movement of the vehicle. The commands, executed by the at least one processor, may cause the electronic device to obtain voice data of the at least one user inside the vehicle using a microphone disposed inside the vehicle while the vehicle is moving. The commands, executed by the at least one processor, may cause the electronic device to identify a purpose of generating the content related to the movement of the vehicle based on the at least one user image or the voice data when a preset event occurs. The commands, executed by the at least one processor, may cause the electronic device to monitor a situation related to the movement of the vehicle.The above commands may be executed by the at least one processor to cause the electronic device to select at least one image from among at least one user image acquired using the at least one first camera or a plurality of external images acquired using the at least one second camera based on the purpose of generating the content and the monitored situation, and to generate the content including the at least one selected image. The operation of generating the content may be to generate the content by inputting the at least one selected image into an artificial intelligence model trained for generating the content related to the movement of the vehicle.
[0007] FIG. 1 is a diagram for explaining an overview of a method for generating content using a plurality of acquired images based on data acquired through a camera or microphone by an electronic device according to the present disclosure.
[0008] FIG. 2 is a diagram illustrating the structure of an electronic device mounted on a vehicle according to one embodiment.
[0009] FIG. 3 is a flowchart illustrating a process of generating content using a plurality of acquired images based on data acquired through a camera or microphone in an electronic device according to one embodiment.
[0010] FIG. 4 is a flowchart illustrating a specific process of identifying the purpose of generating content using acquired user images and voices inside a vehicle in an electronic device according to one embodiment.
[0011] FIG. 5 is an exemplary diagram illustrating an electronic device according to one embodiment of the present invention identifying the purpose of generating content related to the movement of a vehicle based on a result of recognizing an object in an image acquired through a camera inside the vehicle.
[0012] FIG. 6 is an exemplary diagram illustrating an electronic device according to one embodiment of the present invention identifying the purpose of generating content related to the movement of a vehicle based on the result of interpreting voice data acquired through a microphone inside the vehicle.
[0013] FIG. 7 is a drawing for explaining a process in which an electronic device mounted on a vehicle according to one embodiment activates a content shooting mode.
[0014] FIG. 8 is a flowchart illustrating a process in which an electronic device mounted on a vehicle according to one embodiment acquires a plurality of images by controlling driving of the vehicle in response to the mode of the vehicle being determined to be a content creation mode.
[0015] FIG. 9 is a diagram illustrating a method for an electronic device mounted on a vehicle according to one embodiment to select at least one external image from among a plurality of external images acquired using a camera facing the outside of the vehicle.
[0016] FIG. 10 is a flowchart illustrating a process in which an electronic device according to one embodiment generates high-quality content using low-quality external images acquired using a camera facing the outside of a vehicle.
[0017] FIG. 11 is a diagram illustrating a method for an electronic device according to one embodiment to select at least one image from among images acquired using a camera, and a method for generating content based on the selected at least one image using an artificial intelligence model.
[0018] FIG. 12 is a flowchart illustrating a process of generating an input prompt to be input to an artificial intelligence model based on an image or voice obtained from inside or outside a vehicle in an electronic device according to one embodiment.
[0019] FIG. 13 is a diagram illustrating an example of generating an input prompt to be input to an artificial intelligence model based on images or voices obtained from inside and outside a vehicle in an electronic device according to one embodiment.
[0020] FIG. 14 is a diagram illustrating an example of an environment in which an electronic device mounted on a vehicle according to one embodiment is connected to and operates with an external electronic device located around the vehicle.
[0021] FIG. 15 is a flowchart illustrating a process in which an electronic device mounted on a vehicle according to one embodiment is connected to an external electronic device located around the vehicle to generate content and transmit the generated content to the external electronic device.
[0022] FIG. 16 is a flowchart illustrating a process of generating content while connected to an external electronic device when an electronic device mounted on a vehicle according to one embodiment receives an external image capture request from an external electronic device located around the vehicle.
[0023] FIG. 17 is a diagram illustrating an environment in which an electronic device mounted on a vehicle according to one embodiment shares a network with external electronic devices located around the vehicle and operates by being connected to a server through the shared network.
[0024] FIG. 18 is a flowchart illustrating a process in which an electronic device mounted on a vehicle according to one embodiment is connected to an external electronic device and a server located around the vehicle through a network, and a server generates content based on data obtained from the electronic device and the external electronic device, and transmits the generated content to the external electronic device.
[0025] FIG. 19 is an exemplary drawing of a screen for controlling an electronic device mounted on a vehicle from an external electronic device when the electronic device mounted on a vehicle is connected to an external electronic device according to one embodiment.
[0026] FIG. 20 is a block diagram of an electronic device within a network environment according to various embodiments.
[0027] FIG. 21 is a diagram illustrating a system including a generative artificial intelligence model according to one embodiment.
[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the disclosed embodiments may be implemented in various different forms and are not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present disclosure, and similar parts have been designated with similar reference numerals throughout the specification.
[0029] The terms used in this disclosure are described as currently common terms, taking into account the functions mentioned herein. However, these terms may mean various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.
[0030] Additionally, while terms such as first, second, etc. may be used to describe various components, the components are not limited by these terms. These terms are used to distinguish one component from another.
[0031] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where the parts are "directly connected," but also the case where the parts are "electrically connected" or "operatively connected" with other elements intervening therebetween. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise stated.
[0032] The phrases “in one embodiment” and the like appearing in various places throughout this disclosure do not necessarily all refer to the same embodiment.
[0033] An embodiment of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.
[0034] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.
[0035] The electronic device of the present disclosure may be mounted on a vehicle. The electronic device of the present disclosure may be mounted inside or outside the vehicle. The electronic device of the present disclosure may be the vehicle itself or may be a device mounted on the vehicle as at least a part of the vehicle. However, for convenience, the electronic device is described as being mounted on the vehicle in the present disclosure.
[0036] The vehicle in the present disclosure may include, for example, a passenger car, a truck, a motorcycle, and a bus. The vehicle described in the present disclosure may include an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, an electric vehicle having an electric motor as a power source, and the like. The vehicle may be an autonomous vehicle or an unmanned vehicle (driverless car) that can drive itself by recognizing the driving environment and controlling the vehicle without driver operation. Alternatively, the vehicle may be a manually operated vehicle driven by driver operation, or a vehicle that combines manual operation and autonomous driving methods. The vehicle according to the embodiment of the present document is not limited to the types of vehicles described above, and may include new types of vehicles according to technological advancements.
[0037] The artificial intelligence-related functions according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in the memory. If one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model (or deep learning model) is learned using a plurality of learning data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.
[0038] In the present disclosure, an artificial intelligence model (or deep learning model) may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and performs neural network operations through operations between the operation results of the previous layer and the plurality of weights. The plurality of weights of the plurality of neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the plurality of weights may be updated so that a loss value or a cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q networks.
[0039] An electronic device mounted on a vehicle according to the present disclosure can provide content corresponding to a specific situation by generating content using an artificial intelligence model based on data acquired while the electronic device is moving.
[0040] In the present disclosure, content related to the movement of a vehicle may include all possible types of content including images, videos, and voices.
[0041] In the present disclosure, a user may mean at least one person riding inside a vehicle equipped with an electronic device.
[0042] In the present disclosure, an exterior image may refer to an image captured of the exterior of a vehicle. For example, it may include an image captured of a range including the exterior of the vehicle.
[0043] The present disclosure will be described in detail with reference to the attached drawings below.
[0044] FIG. 1 is a drawing for explaining an overview of a method for generating content using a plurality of acquired images based on data acquired through a camera or microphone by an electronic device (100, 2001) according to the present disclosure.
[0045] According to one embodiment, the electronic device (100, 2001) may be mounted inside the vehicle (105). According to one embodiment, the electronic device (100, 2001) may be the vehicle (105) itself or at least a part of the vehicle (105). According to one embodiment, the electronic device (100, 2001) may be placed inside or outside the vehicle (105). In one embodiment, the electronic device (100, 2001) may be a set of parts that are separately placed inside and outside the vehicle (105) and can communicate with each other. In one embodiment, the electronic device (100, 2001) may be a set of parts that are placed inside and outside the vehicle (105) and are connected to each other.
[0046] In one embodiment, the electronic device (100, 2001) may include a first camera (110) facing the interior of the vehicle (105). In one embodiment, the first camera (110) may detect at least one of a user's gaze, expression, posture, or motion, and a change therein, while the user is seated in a seat within the vehicle (105). For example, the first camera (110) may detect that the user's gaze changes from being directed forward to being directed to the right while the user is seated in a seat within the vehicle (105).
[0047] In one embodiment, the electronic device (100, 2001) may include a microphone (150) positioned inside the vehicle (105). In one embodiment, the microphone (150) may detect the voice of a user while the user is seated inside the vehicle (105). For example, at least one microphone (150) may be positioned to correspond to at least some seats inside the vehicle (105), and the voice of a user seated in a seat corresponding to each microphone may be acquired.
[0048] In one embodiment, the electronic device (100, 2001) may include a second camera (120) facing the exterior of the vehicle (105). In one embodiment, the second camera (120) may be a camera mounted on the vehicle (105) for autonomous driving. In one embodiment, the second camera (120) may acquire images of the exterior of the vehicle (105) while the vehicle (105) is driving. In one embodiment, the second camera (120) may acquire images of the exterior of the vehicle (105) while the vehicle (105) is powered on. For example, the second camera (120) may continuously acquire images of the exterior of the vehicle (105) while the vehicle (105) is powered on.
[0049] In one embodiment, the second camera (120) may be positioned to capture an all-round view of the exterior of the vehicle relative to the vehicle (105). In one embodiment, the number and placement of the second cameras (120) positioned on one vehicle (105) may vary depending on the angle of view of each second camera (120).
[0050] According to one embodiment, the electronic device (100, 2001) can determine whether an event causing content generation has occurred based on at least one piece of data acquired through the first camera (110), the second camera (120), the microphone (150), or the sensor module (e.g., 2076 of FIG. 20 ). In one embodiment, the event causing content generation may be preset. According to one embodiment, the electronic device (100, 2001) can determine whether an event causing content generation has occurred using an artificial intelligence module.
[0051] According to one embodiment, the electronic device (100, 2001) may identify the purpose of generating the content and monitor a situation related to the movement of the vehicle when it determines that an event that causes the generation of content has occurred. In one embodiment, the electronic device (100, 2001) may select at least one external image from among a plurality of external images (181, 189) acquired through the second camera (120) based on the identified purpose of generating the content and the situation related to the movement of the monitored vehicle. According to one embodiment, the electronic device (100, 2001) may generate content (190) including the at least one selected external image.
[0052] FIG. 2 is a diagram illustrating the structure of an electronic device mounted on a vehicle according to one embodiment.
[0053] In one embodiment, the electronic device (100, 2001) may include a first camera (110) facing the interior of the vehicle (105). In one embodiment, the first camera (110) may be positioned inside the vehicle (105) facing the interior of the vehicle (105), or may be positioned from the outside of the vehicle (105) facing the interior of the vehicle (105). In one embodiment, there may be one first camera (110) or there may be multiple first cameras. For example, the first cameras (110) may be positioned to correspond to each seat in the vehicle (105).
[0054] In one embodiment, the first camera (110) may detect at least one of the user's gaze, facial expression, or motion, and changes therein, while the user is seated in the vehicle (105). For example, the first camera (110) may detect that the user's gaze changes from looking forward to looking to the right while the user is seated in the vehicle (105).
[0055] In one embodiment, the first camera (110) may be positioned toward a seat within the vehicle (105). In one embodiment, the first camera (110) may be positioned to view the front of the user when the user is seated within the vehicle (105). For example, the first camera (110) may be positioned to view the face of the user when the user is seated within the vehicle (105). For example, the first camera (110) may be positioned to capture the upper body of the user when the user is seated within the vehicle (105).
[0056] In one embodiment, at least one first camera (110) may be positioned inside the vehicle (105) to face at least one user. For example, at least one first camera (110) may be positioned at the front of the vehicle (105) to face the rear of the vehicle (105). For example, one first camera (110) may be positioned inside the vehicle (105) to face a user seated in the driver's seat. For example, multiple first cameras (110) may be positioned in front of each seat inside the vehicle (105) to view the front of each seat. For example, the number of first cameras (110) may be equal to the maximum number of passengers that can ride in the vehicle (105). In this case, each first camera (110) may be positioned to face the front of each user when each user is seated in the vehicle (105).
[0057] In one embodiment, the electronic device (100, 2001) may include a microphone (150) facing the interior of the vehicle (105). In one embodiment, the microphone (150) may be placed inside the vehicle (105). In one embodiment, there may be one or more microphones (150). In one embodiment, the microphone (150) may be placed so as to face the user when the user is seated in a seat inside the vehicle (105). For example, the microphone (150) may be placed at a location adjacent to a seat inside the vehicle (105). For example, the microphone (150) may be placed to correspond to each seat inside the vehicle (105). For example, the microphone (150) may be placed at a location adjacent to the driver's seat inside the vehicle (105) and facing the driver's seat. For example, the microphone (150) may be placed for each row of seats inside the vehicle (105).
[0058] In one embodiment, the microphone (150) can detect the voice of a user while the user is seated in the vehicle (105). For example, at least one microphone (150) can be positioned to correspond to each seat in the vehicle (105) to acquire the voice of a user seated in each seat.
[0059] For example, if there are a total of five seats in a vehicle (105), five microphones (150) can be placed inside the vehicle (105) to correspond to each seat. In this case, the five microphones (150) corresponding to each seat can acquire the voices of users corresponding to each microphone (150) while the users are seated in each seat.
[0060] In one embodiment, the electronic device (100, 2001) may include a second camera (120) facing the exterior of the vehicle (105). In one embodiment, the second camera (120) may be positioned inside the vehicle (105) facing the exterior of the vehicle (105), or may be positioned outside the vehicle (105) facing the exterior of the vehicle (105). In one embodiment, there may be one second camera (120) or there may be multiple second cameras.
[0061] In one embodiment, the second camera (120) may be a camera mounted on the vehicle (105) for autonomous driving. In one embodiment, the second camera (120) may acquire images of the exterior of the vehicle (105) while the vehicle (105) is driving. In one embodiment, the second camera (120) may acquire images of the exterior of the vehicle (105) while the vehicle (105) is powered on. In one embodiment, the second camera (120) may acquire images of the exterior of the vehicle (105) while the vehicle (105) is powered on. For example, the second camera (120) may continuously acquire images of the exterior of the vehicle (105) while the vehicle (105) is powered on.
[0062] In one embodiment, the second camera (120) may be positioned to capture an all-round view of the exterior of the vehicle relative to the vehicle (105). In one embodiment, the number and placement of the second cameras (120) positioned on one vehicle (105) may vary depending on the angle of view of each second camera (120).
[0063] An electronic device (100, 2001) according to one embodiment may include a processor (140, 2020). Functions related to artificial intelligence according to the present disclosure are operated through the processor (140, 2020). The processor (140, 2020) may be composed of one or more processors (140, 2020). In this case, the one or more processors (140, 2020) may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. The one or more processors (140, 2020) control input data to be processed according to predefined operation rules or artificial intelligence models stored in a memory. When the one or more processors (140, 2020) are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.
[0064] Although not shown, an electronic device (100, 2001) according to one embodiment may include a sensor module (e.g., 2076 of FIG. 20). The sensor module (2076) may detect an operating state (e.g., power or temperature) of the electronic device (100, 2001), an operating state (e.g., speed, acceleration, proximity of an external obstacle, etc.) of the vehicle (105), or an external environmental state (e.g., user state), and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (2076) may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0065] For example, the sensor module (2076) may include a motion sensor (e.g., an acceleration sensor or a gyro sensor) that detects movement of the electronic device (100, 2001) or the vehicle (105). The electronic device (100, 2001) may determine location information or movement information of the electronic device (100, 2001) or the vehicle (105) based on information detected through the acceleration sensor or the gyro sensor. The acceleration sensor may obtain information about the speed at which the electronic device (100, 2001) or the vehicle (105) moves. In one embodiment, the gyro sensor may obtain information about a change in the direction in which the electronic device (100, 2001) moves. For example, the electronic device (100, 2001) can determine a situation related to the movement of the vehicle (105) based on information about changes in the speed or direction in which the electronic device (100, 2001) moves, acquired through an acceleration sensor or a gyro sensor. For example, the electronic device (100, 2001) can select at least one image from among a plurality of images acquired through the second camera (120) based on information about changes in the speed or direction in which the electronic device (100, 2001) moves, acquired through an acceleration sensor or a gyro sensor. For example, the electronic device (100, 2001) can generate content by correcting the selected at least one image based on information about changes in the speed or direction in which the electronic device (100, 2001) moves, acquired through an acceleration sensor or a gyro sensor.
[0066] For example, the sensor module (2076) may include a light sensor. The light sensor may obtain information about the brightness of light or the color of light. For example, the electronic device (100, 2001) may determine current weather information based on the information about the brightness of light or the color of light obtained through the light sensor. For example, the electronic device (100, 2001) may determine a situation related to the movement of the vehicle (105) based on the information about the brightness of light or the color of light obtained through the light sensor. For example, the electronic device (100, 2001) may select at least one image from among a plurality of images obtained through the second camera (120) based on the information about the brightness of light or the color of light obtained through the light sensor. For example, the electronic device (100, 2001) may generate content by correcting the selected at least one image based on the information about the brightness of light or the color of light obtained through the light sensor.
[0067] For example, the sensor module (2076) may include a global positioning system (GPS) receiver. The GPS sensor may receive a signal for obtaining location information of the electronic device (100, 2001). The electronic device (100, 2001) may identify the geographical location of the electronic device (100, 2001) based on the signal received by the GPS sensor.
[0068] For example, the sensor module (2076) may include an altimeter sensor. The altimeter sensor may measure altitude based on the location of each electronic device (100, 2001). The altimeter sensor may acquire information while linked with a GPS sensor. The electronic device (100, 2001) may acquire information regarding altitude changes obtained by the altimeter sensor through the altimeter sensor.
[0069] For example, the sensor module (2076) may include a temperature sensor that detects temperature. The temperature sensor may detect the temperature inside or outside the vehicle (105) in which the electronic device (100, 2001) is mounted. For example, the temperature sensor may be placed on the exterior surface of the vehicle (105).
[0070] In one embodiment, the sensor module (2076) may include a light sensor. The light sensor may measure the illuminance of light incident on the light sensor from around the electronic device (100, 2001). For example, the light sensor may be positioned on the upper surface of the exterior of the vehicle (105).
[0071] FIG. 3 is a flowchart illustrating a process of generating content using a plurality of acquired images based on data acquired through a camera (110, 120) or a microphone (150) in an electronic device (100, 2001) according to one embodiment.
[0072] In the identification number 313, the electronic device (100, 2001) can obtain an image of a user inside the vehicle (105) using the first camera (110). In one embodiment, the electronic device (100, 2001) can obtain an image of a user boarding the vehicle (105) using the first camera (110) when at least one user is boarding the vehicle (105). In one embodiment, while the vehicle (105) is driving, the electronic device (100, 2001) can obtain an image of at least one user using at least one first camera (110) facing the interior of the vehicle (105). For example, the electronic device (100, 2001) can obtain an image of at least one user using at least one first camera (110) arranged at the front inside the vehicle (105) and arranged to face the rear of the vehicle (105). For example, the electronic device (100, 2001) can obtain an image of a user positioned in a seat corresponding to each first camera (110) by using at least one first camera (110) arranged to correspond to at least one seat inside the vehicle (105).
[0073] In one embodiment, the electronic device (100, 2001) can obtain user-related information from a user image acquired through the first camera (110). This will be described in detail in FIG. 4.
[0074] In the identification number 317, the electronic device (100, 2001) can acquire a voice inside the vehicle (105) using the microphone (150). According to one embodiment, while the vehicle (105) is driving, the electronic device (100, 2001) can acquire a voice of a user inside the vehicle using at least one microphone (150) disposed inside the vehicle (105). For example, the electronic device (100, 2001) can acquire a voice of at least one user inside the vehicle (105) through at least one microphone (150), analyze the acquired voice, and identify the user who uttered the acquired voice. For example, the electronic device (100, 2001) can acquire multiple voices of multiple users inside the vehicle (105) through at least one microphone (150), synthesize and analyze the acquired voices, and identify the user corresponding to each acquired voice. For example, an electronic device (100, 2001) can acquire the voice of at least one user within a vehicle (105) through at least one microphone (150) and interpret the meaning of the acquired voice. Details regarding the process of interpreting the acquired user's voice will be described in detail in the description of FIG. 4 below.
[0075] At identification number 320, the electronic device (100, 2001) can determine whether a preset event has occurred.
[0076] According to one embodiment, the preset event may be an event identified based on a user image inside a vehicle (105) acquired through the first camera (110). In one embodiment, the preset event may include a case where a change in a user image is detected by the first camera (110). For example, if a change in the user's gaze is detected in a user image acquired by at least one first camera (110), the electronic device (100, 2001) may identify that a preset event has occurred. For example, if a change in the user's facial expression is detected in a user image acquired by at least one first camera (110), the electronic device (100, 2001) may identify that a preset event has occurred.
[0077] According to one embodiment, the preset event may be an event identified based on a voice inside the vehicle (105) acquired through the microphone (150). In one embodiment, the preset event may include a case where a voice with a predetermined attribute and / or a predetermined meaning is detected by the microphone (150). For example, if the volume of the voice acquired by at least one microphone (150) is identified as being greater than a predetermined volume, the electronic device (100, 2001) may identify that a preset event has occurred. For example, if the electronic device (100, 2001) identifies that the meaning of the voice derived by interpreting the voice acquired by at least one microphone (150) is related to a predetermined external situation, the electronic device (100, 2001) may identify that a preset event has occurred. An electronic device (100, 2001) according to one embodiment can identify whether the meaning of a voice obtained by interpreting a voice through at least one microphone (150) is related to a predetermined external situation by using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the predetermined external situation may include a situation related to at least one of the following: the moving speed of the current vehicle (105), the weather outside the vehicle (105), the scenery, the location of the current vehicle (105), whether a landmark exists within a predetermined distance from the location of the current vehicle (105), whether a tourist attraction exists within a predetermined distance from the location of the current vehicle (105), or whether a person or an animal exists within a predetermined distance from the location of the current vehicle (105).
[0078] According to one embodiment, the preset event may be an event identified based on an image of the outside of the vehicle acquired through the second camera (120). In one embodiment, the preset event may include a case where a predetermined object is detected outside the vehicle (105) by the second camera (120). According to one embodiment, the electronic device (100, 2001) may identify that the preset event has occurred when it is determined that an image of a predetermined object is included in an image acquired by at least one second camera (120). The electronic device (100, 2001) according to one embodiment may identify whether an image of a predetermined object is included in an image acquired by at least one second camera (120) using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the image of the predetermined object may be an image of an object including at least one of weather, scenery, a landmark within a predetermined distance, a person, and an animal outside the vehicle (105). For example, the electronic device (100, 2001) may identify that an image of a given object identified as being included in an image acquired by at least one second camera (120) corresponds to a given landmark. In this case, the electronic device (100, 2001) may identify that a preset event has occurred.
[0079] According to one embodiment, the preset event may be an event identified based on data acquired through a sensor module (e.g., 2076 of FIG. 20) included in the electronic device (100, 2001) or another electronic device connected to the electronic device (e.g., the external electronic device (1400) of FIG. 14, the external electronic device (1400) of FIG. 17). In one embodiment, the preset event may include a case where predetermined data is acquired through a sensor of the sensor module (e.g., 2076 of FIG. 20). In one embodiment, the electronic device (100, 2001) may identify that the data acquired through the sensor of the sensor module (e.g., 2076 of FIG. 20) is related to a predetermined external situation. In one embodiment, the electronic device (100, 2001) can identify whether data acquired through a sensor of a sensor module (e.g., 2076 of FIG. 20) is related to a predetermined external situation using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the predetermined external situation may include a situation related to at least one of the current moving speed of the vehicle, the weather outside the vehicle, the scenery, the current location, whether a landmark exists within a predetermined distance from the current location, whether a predetermined tourist attraction exists within a predetermined distance from the current location, or whether a person or an animal exists within a predetermined distance from the current location.
[0080] According to one embodiment, the preset event may be an event identified based on data stored in a memory (e.g., 2030 of FIG. 20) of the electronic device (100, 2001). In one embodiment, the preset event may include a case where predetermined data is acquired from the memory (e.g., 2030 of FIG. 20) of the electronic device (100, 2001). In one embodiment, the electronic device (100, 2001) may identify that the data acquired from the memory (e.g., 2030 of FIG. 20) is related to a predetermined external situation. In one embodiment, the electronic device (100, 2001) may identify whether the data acquired from the memory (e.g., 2030 of FIG. 20) is related to a predetermined external situation using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, a given external situation may include a situation related to at least one of the current vehicle (105) moving speed, weather outside the vehicle, scenery, current location, whether a landmark exists within a given distance from the current location, whether a given tourist attraction exists within a given distance from the current location, or congestion within a given distance from the current location. For example, the electronic device (100, 2001) may use a map stored in a memory (e.g., 2030 of FIG. 20) to identify that the current location of the vehicle (105) is within a given distance from a location corresponding to a landmark stored in the map. In this case, the electronic device (100, 2001) may identify that a given landmark exists within a given distance from the current location of the vehicle (105) and may identify that a preset event has occurred.
[0081] According to one embodiment, the preset event may be an event identified based on data acquired through a server (e.g., server 1700 of FIG. 17) connected to the electronic device (100, 2001). In one embodiment, the preset event may include a case where predetermined data is received from the server (e.g., server 1700 of FIG. 17). In one embodiment, the electronic device (100, 2001) may identify that the data acquired through the server (e.g., server 1700 of FIG. 17) is related to a predetermined external situation. In one embodiment, the electronic device (100, 2001) may identify whether the data acquired from the server (e.g., server 1700 of FIG. 17) is related to a predetermined external situation using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, a given external situation may include a situation related to at least one of the moving speed of the current vehicle (105), the weather outside the vehicle, the scenery, the current location, whether a landmark exists within a given distance from the current location, whether a given tourist attraction exists within a given distance from the current location, or the level of congestion within a given distance from the current location. For example, the electronic device (100, 2001) may receive information about at least one of the number of other terminal devices located within a given distance from the current vehicle (105), a communication status, or a map through a server (e.g., the server (1700) of FIG. 17), and may identify the level of congestion within a given distance from the location of the current vehicle (105) by analyzing the received information. For example, the electronic device (100, 2001) may receive information from a server (e.g., the server (1700) of FIG. 17) that there are terminal devices exceeding a given number within a given distance from the location of the current vehicle (105). In this case, the electronic device (100, 2001) can identify that a predetermined tourist attraction exists within a predetermined distance from the current location of the vehicle (105), and can identify that a preset event has occurred.
[0082] According to one embodiment, the preset event can be identified by analyzing at least one piece of data acquired through at least one of the first camera (110), the microphone (150), the second camera (120), the sensor module (e.g., 2076 of FIG. 20), and the memory (e.g., 2030 of FIG. 20). For example, the preset event can be identified by comprehensively analyzing a plurality of pieces of data acquired through at least one of the first camera (110), the microphone (150), the second camera (120), and the sensor module (e.g., 2076 of FIG. 20).
[0083] If the electronic device (100, 2001) determines that a preset event has not occurred at identification number 320, the electronic device (100, 2001) can acquire an image of a user inside the vehicle (105) using the first camera (110) and acquire a voice inside the vehicle using the microphone (150) again according to identification numbers 313 and 317.
[0084] If the electronic device (100, 2001) determines that a preset event has occurred at identification number 320, the electronic device (100, 2001) can identify the purpose of creating the content at identification number 330. For example, the electronic device (100, 2001) can identify the purpose of creating the content based on the properties of an object identified from a user image captured by the first camera (110) or the meaning of a voice acquired by the microphone (150). For example, the electronic device (100, 2001) can determine the user's intention to create a video for uploading to Instagram by interpreting the voice acquired through the microphone (150). In this case, the electronic device (100, 2001) can create an image with a width-to-height ratio of 9:16 so that it can be uploaded to Instagram. Other embodiments will be described in detail with reference to FIGS. 4 to 6 .
[0085] If the electronic device (100, 2001) determines that a preset event has occurred at identification number 320, the electronic device (100, 2001) can monitor a situation related to the movement of the vehicle (105) at identification number 340. For example, information about a situation related to the movement of the vehicle (105) may include a situation related to at least one of the purpose of movement of the user within the vehicle (105), the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location, whether a landmark exists within a predetermined distance from the current location, whether a predetermined tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or an animal exists within a predetermined distance from the current location.
[0086] If the electronic device (100, 2001) determines that a preset event has occurred in the identification number 320, the electronic device (100, 2001) may set the properties of the operation of capturing an image by the first camera (110) or the second camera (120). For example, if the electronic device (100, 2001) determines that a preset event has occurred, the electronic device (100, 2001) may set the properties of the operation of capturing an image by the first camera (110) or the second camera (120) based on the properties of the generated event. Based on the purpose of generating the content identified in the identification number 330, the electronic device (100, 2001) may set the properties of the operation of capturing an image by the first camera (110) or the second camera (120). Based on the result of monitoring the situation related to the movement of the vehicle (105) in the identification number 340, the electronic device (100, 2001) can set the properties of the operation of the first camera (110) or the second camera (120) to capture an image. The properties of the operation of the first camera (110) or the second camera (120) to capture an image may include, for example, properties related to at least one of the shutter speed, the field of view (FOV), the exposure value, or the position of the focus of the camera (110, 120), but are not limited thereto, and may include all properties related to the shooting of the camera (110, 120).
[0087] In the identification number 350, the electronic device (100, 2001) can select at least one image from among at least one user image acquired using at least one first camera (110) or a plurality of external images acquired using the second camera (120). The operation of the electronic device (100, 2001) selecting at least one image will be described in detail in FIGS. 9 and 11, and will be omitted here for convenience.
[0088] In the identification number 360, the electronic device (100, 2001) can generate content related to the movement of a vehicle (105) including at least one selected image. The operation of the electronic device (100, 2001) to generate content related to the movement of a vehicle (105) including at least one selected image will be described in detail with reference to FIGS. 11 to 13, and will be omitted here for convenience.
[0089] FIG. 4 is a flowchart illustrating a specific process of identifying the purpose of creating content using user images and voices inside an acquired vehicle (105) in an electronic device (100, 2001) according to one embodiment.
[0090] Referring to identification number 413, as the electronic device (100, 2001) acquires an image of a user inside a vehicle (105) using the first camera (110) at identification number 313, the electronic device (100, 2001) can recognize an object within the user image. In one embodiment, the electronic device (100, 2001) can recognize at least one user included in the user image.
[0091] In one embodiment, the electronic device (100, 2001) can obtain information related to an object included in the obtained user image by analyzing the user image obtained through the first camera (110). In one embodiment, the electronic device (100, 2001) can obtain information related to a user in the user image obtained through the first camera (110). For example, the electronic device (100, 2001) can obtain and update information related to a user in real time in the user image obtained through the first camera (110). For example, the electronic device (100, 2001) can obtain information related to at least one of a direction of each user's gaze, a change in gaze, an expression, a change in expression, a movement, a gesture, or a change in posture, based on the user image obtained from the first camera (110) corresponding to each seat.
[0092] According to one embodiment, the electronic device (1000) can recognize a person in a captured image by inputting an image (e.g., a still image and a moving image) captured through the first camera (110) into an artificial intelligence model (e.g., a vision model) trained for object recognition. The artificial intelligence model trained for object recognition may include, but is not limited to, an artificial intelligence model based on convolutional neural networks (CNN) and an artificial intelligence model based on region-based convolution neural networks (R-CNN).
[0093] Referring to identification number 417, as the electronic device (100, 2001) obtains voice data inside the vehicle (105) using the microphone (150) at identification number 317, the electronic device (100, 2001) can interpret the meaning of the obtained voice data. According to one embodiment, the electronic device (100, 2001) can obtain the voice of at least one user inside the vehicle (105) through at least one microphone (150) and interpret the meaning of the obtained voice.
[0094] According to one embodiment, the electronic device (100, 2001) can identify that the meaning of the voice obtained by interpreting the voice by at least one microphone (150) is related to a predetermined external situation. For example, the predetermined external situation may include a situation related to at least one of the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location of the vehicle (105), whether a landmark exists within a predetermined distance from the current location of the vehicle (105), whether a tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or an animal exists within a predetermined distance from the current location of the vehicle (105).
[0095] According to one embodiment, the electronic device (100, 2001) can obtain text from speech acquired by at least one microphone (150) and input the obtained text into an artificial intelligence model trained for natural language interpretation (e.g., a natural language understanding (NLU) model, a large language model (LLM)), thereby obtaining an output value representing the meaning of the speech. The text from the speech acquired by at least one microphone (150) can be generated from the speech using, for example, automatic speech recognition (ASR) technology and / or speech to text (STT) technology. According to one embodiment, the speech can be analyzed using audio analysis technology (e.g., ASR technology), thereby being used to distinguish speakers, identify the emotions of the speakers, and / or understand the content of the conversation.
[0096] According to one embodiment, the electronic device (100, 2001) can obtain information related to a given external situation by analyzing and synthesizing multiple meanings derived by interpreting multiple voices acquired by at least one microphone (150).
[0097] An electronic device (100, 2001) according to one embodiment can acquire information by synthesizing data included in the acquired image and voice by inputting at least one image and at least one voice into an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the artificial intelligence model (e.g., 2150 of FIG. 21) may include a multi-modal artificial intelligence model. For example, the artificial intelligence model (e.g., 2150 of FIG. 21) can generate information related to at least one of a user or an external situation by synthesizing multiple types of acquired data. For example, the artificial intelligence model (e.g., 2150 of FIG. 21) can identify relationships between multiple acquired data and generate information related to at least one of a user or an external situation based on the relationships between the identified data.
[0098] Referring to identification number 420, the electronic device (100, 2001) can determine whether a preset event has occurred based on the object recognition result and the meaning of voice data.
[0099] The preset event may include a case where a change in a user image is detected by the first camera (110). For example, the electronic device (100, 2001) may, as a result of recognizing an object in a user image acquired by at least one first camera (110), acquire information related to at least one of a direction of gaze, a change in gaze, an expression, a change in expression, a movement, a gesture, or a change in posture of at least one user in the user image.
[0100] In one embodiment, a memory (e.g., 2030 of FIG. 20) of an electronic device (100, 2001) may store predetermined characteristics regarding an object that can be recognized in a user image acquired from a first camera (110). In one embodiment, a server (e.g., server (1700) of FIG. 17) that can be connected to an electronic device (100, 2001) may store predetermined characteristics regarding an object that can be recognized in a user image acquired from a first camera (110). In this case, the electronic device (100, 2001) may receive predetermined characteristics regarding an object from a server (e.g., server (1700) of FIG. 17). For example, certain characteristics about an object stored in memory (e.g., 2030 of FIG. 20) or a server (e.g., server (1700) of FIG. 17) may include characteristics about at least one of a direction of the user's gaze, a change in the gaze, an expression, a change in the expression, a movement, a gesture, or a change in the posture.
[0101] According to one embodiment, the electronic device (100, 2001) can identify whether a preset event has occurred by comparing the result of recognizing an object in a user image with predetermined characteristics stored in a memory (e.g., 2030 of FIG. 20) or a server (e.g., server 1700 of FIG. 17). For example, if the electronic device (100, 2001) determines that the result of recognizing an object in a user image corresponds to predetermined characteristics stored in a memory (e.g., 2030 of FIG. 20) or a server (e.g., 1700 of FIG. 17), the electronic device can identify that a preset event has occurred. For example, the electronic device (100, 2001) can detect that the user's expression inside a vehicle (105) changes to a surprised expression as a result of recognizing an object in a user image, and then detect a motion of the user pointing outside the vehicle (105). In this case, the electronic device (100, 2001) can match the characteristic of the user's facial expression changing to a surprised expression and the characteristic of the user's movement pointing outside the vehicle (105) among the predetermined characteristics stored in the memory (e.g., 2030 of FIG. 20) with the result of recognizing the object. In this case, the electronic device (100, 2001) can identify that a preset event has occurred when the user's facial expression and movement inside the vehicle correspond to the predetermined characteristics stored in the memory (e.g., 2030 of FIG. 20).
[0102] For example, if a change in the user's gaze is recognized in a user image acquired by at least one first camera (110), the electronic device (100, 2001) can identify that a preset event has occurred. For example, if a change in the user's facial expression is recognized in a user image acquired by at least one first camera (110), the electronic device (100, 2001) can identify that a preset event has occurred.
[0103] In one embodiment, the preset event may be an event identified based on voices inside the vehicle acquired through the microphone (150). In one embodiment, the preset event may include a case where a user's voice is detected by the microphone (150). For example, if the volume of the voice acquired by at least one microphone (150) is identified as being greater than a predetermined volume, the electronic device (100, 2001) may identify that the preset event has occurred.
[0104] For example, the electronic device (100, 2001) may identify that a preset event has occurred when it identifies that the meaning of the voice obtained by interpreting the voice by at least one microphone (150) is related to a predetermined external situation. For example, the predetermined external situation may include a situation related to at least one of the following: the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location of the vehicle (105), whether a landmark exists within a predetermined distance from the current location of the vehicle (105), whether a predetermined tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or an animal exists within a predetermined distance from the current location of the vehicle (105).
[0105] Referring to the identification number 430, if the electronic device (100, 2001) determines that a preset event in the identification number 420 has occurred, the electronic device (100, 2001) can identify the purpose of generating content based on the object recognition result and the meaning of the interpreted voice data. The purpose of generating content may include at least one of the following: the type of content to be generated, the type of object to be expressed by the content, the object preferred by the user, the type of platform to which the content to be generated will be uploaded, the type of travel destination preferred by the user, the standard of the content, or the type of voice data to be included in the content. For example, the purpose of generating content may be the purpose of generating a photo album of landmarks taken along the route to the travel destination, the purpose of generating content of objects of interest to users on the way to the destination, the purpose of generating content summarizing events that occurred during the travel period, or the purpose of capturing the scenery around the vehicle during a predetermined time period from the occurrence of a predetermined event.
[0106] According to one embodiment, the electronic device (100, 2001) can identify objects and attributes of objects within an image of a user inside a vehicle acquired through the first camera (110). For example, the electronic device (100, 2001) can, as a result of recognizing an object within the user image acquired through the first camera (110), acquire information related to at least one of a direction of gaze, a change in gaze, an expression, a change in expression, a movement, a gesture, or a change in posture of at least one user within the user image. In one embodiment, the electronic device (100, 2001) can identify the purpose of creating content based on the result of recognizing an object within the acquired user image.
[0107] For example, the electronic device (100, 2001) can detect the direction of each user's gaze from the user image acquired through the first camera (110). For example, the electronic device (100, 2001) can detect a gaze looking in a predetermined direction from the user image acquired through the first camera (110). In this case, the electronic device (100, 2001) can synthesize and analyze at least one piece of gaze information detected from an image acquired through at least one first camera (110). In this case, the electronic device (100, 2001) can identify the purpose of creating content to acquire an image taken in a predetermined direction with respect to the vehicle (105) based on the result of analyzing the gaze information.
[0108] In one embodiment, the electronic device (100, 2001) can interpret voices inside the vehicle acquired through the microphone (150) and determine their meaning. For example, the voices acquired through the microphone (150) may include voices uttered in relation to at least one of the user's intention to create content, the purpose of movement of the vehicle (105), or the characteristics of data to be included in the content.
[0109] According to one embodiment, the electronic device (100, 2001) can interpret the meaning of the extracted voice. For example, the electronic device (100, 2001) can interpret the meaning of the extracted voice using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, if the extracted voice includes a voice in which a user utters, "Wow, guys, look at the left rear!", the electronic device (100, 2001) can, by interpreting the voice, identify that the meaning is to look at an object located at the left rear of the vehicle (105). In this case, the electronic device (100, 2001) can identify the purpose of generating content to acquire an image of the left rear of the vehicle (105) based on the meaning derived by interpreting the extracted voice. For example, as a result of identifying an object included in an external image acquired through a second camera (120) facing the left rear of a vehicle (105), the external image may include an image of a location visited by multiple users. In this case, the purpose of creating content including an image of a location visited by multiple users can be identified.
[0110] For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the left rear of a vehicle (105) may perform an operation to generate an image of the left rear of the vehicle (105). For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the left rear of the vehicle (105) may select external images (622, 623) captured by the second cameras (124, 125) facing the left rear of the vehicle (105) from among a plurality of second cameras (121, 122, 123, 124, 125, 126). The electronic device (100, 2001) may generate an image or video (630) of the left rear of the vehicle (105) based on the selected external images (622, 623).
[0111] Specific examples of identifying the purpose of content creation based on the meaning of object recognition results or voice data are described in detail in FIGS. 5 and 6.
[0112] FIG. 5 is an exemplary diagram illustrating an electronic device (100, 2001) according to one embodiment of the present invention identifying the purpose of generating content related to the movement of a vehicle (105) based on the result of recognizing an object in an image acquired through a camera (110) inside the vehicle (105).
[0113] An electronic device (100, 2001) according to one embodiment may include at least one first camera (110) facing the interior of a vehicle (105). There is no limitation on the number of first cameras (110) included in the electronic device (100, 2001), and there is no limitation on the number of objects that one first camera (110) can acquire. However, for the convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes four first cameras (111, 112, 113, 114) corresponding to four seats respectively inside the vehicle (105), and the following will be described.
[0114] An electronic device (100, 2001) according to one embodiment may include at least one second camera (120) facing the outside of a vehicle (105). There is no limitation on the number of second cameras (120) included in the electronic device (100, 2001), but for the sake of convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes a second camera (121) facing the front of the vehicle (105), two second cameras (122, 123) facing the right side of the vehicle, a second camera (124) facing the rear of the vehicle, and two second cameras (125, 126) facing the left side of the vehicle, and the following will be described.
[0115] The number of users who can board a vehicle (105) according to one embodiment and the number of users who board the vehicle may vary without limitation depending on the type and situation of the vehicle (105), but in the description of this drawing, for the convenience of explanation, it is assumed that the number of seats inside the vehicle (105) is four and that the number of people boarding inside the vehicle is four, with one person per seat, and the following will be described.
[0116] According to one embodiment, the electronic device (100, 2001) can acquire an image of a user inside a vehicle through the first camera (111, 112, 113, 114). For example, the electronic device (100, 2001) can detect the direction of each user's gaze from the user image acquired through the first camera (111, 112, 113, 114). For example, the electronic device (100, 2001) can detect a gaze (511) looking upward to the right from a user image acquired through the first camera (111), a gaze (512) looking toward the right from a user image acquired through the first camera (112), a gaze (513) looking toward the right from a user image acquired through the first camera (113), and a gaze (514) looking downward to the right from a user image acquired through the first camera (114). In this case, the electronic device (100, 2001) can synthesize and analyze each gaze information (511, 512, 513, 514) detected from the images acquired through each of the first cameras (111, 112, 113, 114). In this case, the electronic device (100, 2001) can identify the purpose of generating content to obtain an image of the right side of the vehicle (105) based on the result of analyzing the gaze information (511, 512, 513, 514).
[0117] For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the right side of a vehicle (105) may perform an operation to generate an image of the right side of the vehicle (105). For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the right side of a vehicle (105) may select external images (522, 523) captured by second cameras (122, 123) facing the right side of the vehicle (105) from among a plurality of second cameras (121, 122, 123, 124, 125, 126). The electronic device (100, 2001) may generate an image or video (530) of the right side of the vehicle (105) based on the selected external images (522, 523).
[0118] FIG. 6 is an exemplary diagram illustrating an electronic device (100, 2001) according to one embodiment of the present invention identifying the purpose of generating content related to the movement of a vehicle (105) based on the result of interpreting voice data acquired through a microphone inside the vehicle (105).
[0119] An electronic device (100, 2001) according to one embodiment may include at least one microphone (150) facing the interior of a vehicle (105). There is no limitation on the number of microphones (150) included in the electronic device (100, 2001), but for convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes two microphones (151, 152) corresponding to two rows of seats respectively inside the vehicle (105).
[0120] An electronic device (100, 2001) according to one embodiment may include at least one second camera (120) facing the outside of a vehicle (105). There is no limitation on the number of second cameras (120) included in the electronic device (100, 2001), but for the sake of convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes a second camera (121) facing the front of the vehicle (105), two second cameras (122, 123) facing the right side of the vehicle, a second camera (124) facing the rear of the vehicle, and two second cameras (125, 126) facing the left side of the vehicle, and the following will be described.
[0121] The number of users who can board a vehicle (105) according to one embodiment and the number of users who board the vehicle may vary without limitation depending on the type and situation of the vehicle (105), but in the description of this drawing, for the convenience of explanation, it is assumed that the number of seats inside the vehicle (105) is four and that the number of people boarding inside the vehicle is four, with one person per seat, and the following will be described.
[0122] According to one embodiment, the electronic device (100, 2001) can acquire voices inside the vehicle through the microphones (151, 152). For example, the electronic device (100, 2001) can detect the user's voice within the voices acquired through the microphones (151, 152). For example, the electronic device (100, 2001) can acquire the user's voice through the microphone (152) located adjacent to the second row of the vehicle (105). For example, the voice acquired through the microphone (152) located adjacent to the second row of the vehicle (105) may include the user's voice saying, "Wow, guys, look to the left, behind!" In this case, the electronic device (100, 2001) can extract the user's voice saying, "Wow, guys, look to the left, behind!" within the acquired voice.
[0123] According to one embodiment, the electronic device (100, 2001) can interpret the meaning of the extracted voice. For example, the electronic device (100, 2001) can interpret the meaning of the extracted voice using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, if the extracted voice includes a voice in which a user utters, "Wow, guys, look at the left rear!", the electronic device (100, 2001) can, by interpreting the voice, identify that the meaning is to look at an object located at the left rear of the vehicle (105). In this case, the electronic device (100, 2001) can identify the purpose of generating content to acquire an image of the left rear of the vehicle (105) based on the meaning derived by interpreting the extracted voice.
[0124] For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the left rear of a vehicle (105) may perform an operation to generate an image of the left rear of the vehicle (105). For example, an electronic device (100, 2001) that has identified the purpose of generating content to obtain an image of the left rear of the vehicle (105) may select external images (622, 623) captured by the second cameras (124, 125) facing the left rear of the vehicle (105) from among a plurality of second cameras (121, 122, 123, 124, 125, 126). The electronic device (100, 2001) may generate an image or video (630) of the left rear of the vehicle (105) based on the selected external images (622, 623).
[0125] FIG. 7 is a drawing for explaining a process in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment activates a content shooting mode.
[0126] A vehicle (105) equipped with an electronic device (100, 2001) according to one embodiment may be a vehicle providing an autonomous driving function. According to one embodiment, when the vehicle (105) equipped with the electronic device (100, 2001) is a vehicle capable of providing an autonomous driving function, the vehicle (105) may include a second camera (120) facing the exterior of the vehicle. For example, the second camera (120) may acquire an image of the exterior of the vehicle (105) to monitor the exterior of the vehicle (105) while the vehicle (105) is performing an autonomous driving function. For example, the electronic device (100, 2001) may provide a continuous recording function for the exterior of the vehicle (105) through the second camera (120). An electronic device (100, 2001) according to one embodiment can generate content related to the movement of a vehicle (105) based on an external image of the vehicle (105) acquired by a second camera (120) while the vehicle (105) is performing an autonomous driving function. An electronic device (100, 2001) according to one embodiment can generate content related to the movement of a vehicle (105) based on an external image of the vehicle (105) acquired by the second camera (120) for autonomous driving.
[0127] According to the identification number 701, the mode of the electronic device (100, 2001) may be set to a mode for generating content based on a user's selection. In one embodiment, as the mode of the electronic device (100, 2001) is set to a mode for generating content based on a user's selection, the mode of the vehicle (105) may be changed to a mode for generating content. For example, the user's selection of the mode of the electronic device (100, 2001) may be identified by receiving a user's input through a display (e.g., a display module (2060) of FIG. 20 ), as in the example of the identification number 701. In one embodiment, the electronic device (100, 2001) may be set to a mode for generating content based on an input received through at least one of a display (e.g., a display module (2060) of FIG. 20 ), a touchpad, or a physical button inside the vehicle (105). For example, an input may be received via a display within the vehicle (105) (e.g., a display module (2060) of FIG. 20) that causes the electronic device (100, 2001) to be set to a mode for generating content. For example, via a display within the vehicle (105) (e.g., a display module (2060) of FIG. 20), the electronic device (100, 2001) may provide a GUI that prompts the user to input information regarding whether to set the electronic device (100, 2001) to a mode for generating content.
[0128] Identification number 702 is a flowchart illustrating a process of generating content when the mode of an electronic device (100, 2001) or a vehicle (105) equipped with an electronic device (100, 2001) is determined to be an autonomous driving mode.
[0129] According to identification number 710, in one embodiment, the mode of the electronic device (100, 2001) or the vehicle (105) equipped with the electronic device (100, 2001) may be set to an autonomous driving mode. In one embodiment, as the mode of the electronic device (100, 2001) or the vehicle (105) equipped with the electronic device (100, 2001) is set to an autonomous driving mode, the second camera (120) may acquire an external image of the vehicle (105) to monitor the exterior of the vehicle (105).
[0130] Referring to the identification number 720, in one embodiment, the mode of the electronic device (100, 2001) may be set to a mode for generating content. In one embodiment, the electronic device (100, 2001) may be set to a mode for generating content when the mode of the vehicle (105) is set to an autonomous driving mode. In one embodiment, the electronic device (100, 2001) may set the mode of the electronic device (100, 2001) to a mode for generating content in a given situation even when the mode of the vehicle (105) is not set to an autonomous driving mode. For example, when the electronic device (100, 2001) determines that a preset event of the identification number 320 has occurred, the electronic device (100, 2001) may set the mode of the electronic device (100, 2001) to a mode for generating content. For example, if the electronic device (100, 2001) determines that a user who has previously used the electronic device (100, 2001) has boarded the vehicle (105), and if the electronic device (100, 2001) determines that an object preferred by the user is located within a predetermined distance from the vehicle (105) based on the user's history of using the electronic device (100, 2001), the electronic device (100, 2001) may set the mode of the electronic device (100, 2001) to a mode for generating content.
[0131] For example, if the electronic device (100, 2001) determines that the vehicle (105) is located on a predetermined path stored in the memory (e.g., 2030 of FIG. 20) of the electronic device (100, 2001), the electronic device (100, 2001) may set the mode of the electronic device (100, 2001) to a mode for generating content. For example, if the electronic device (100, 2001) receives an input from the user to set the electronic device (100, 2001) to a mode for generating content through at least one of a display (e.g., a display module (2060) of FIG. 20), a touch pad, or a physical button, the electronic device (100, 2001) may set the mode of the electronic device (100, 2001) to a mode for generating content.
[0132] For example, the electronic device (100, 2001) may receive an input from a user to select at least one camera (110, 120) to acquire an image. For example, the electronic device (100, 2001) may receive an input from a user to select at least one camera (110, 120) to acquire an image through at least one of a display (e.g., a display module (2060) of FIG. 20), a touchpad, or a physical button. When the electronic device (100, 2001) receives an input from a user to select at least one camera (110, 120) to acquire an image, the electronic device (100, 2001) may acquire an image through the selected at least one camera (110, 120). When the electronic device (100, 2001) receives an input from a user to select at least one camera (110, 120) to acquire an image, the electronic device (100, 2001) may activate the selected at least one camera (110, 120). When the electronic device (100, 2001) receives an input from a user for selecting at least one camera (110, 120) to acquire an image, the electronic device (100, 2001) can select at least one external image acquired by the selected at least one camera (110, 120) and generate content based on the selected at least one image. When the electronic device (100, 2001) receives an input from a user for selecting at least one camera (110, 120) to acquire an image, the electronic device (100, 2001) can select at least one image acquired by the selected at least one camera (110, 120) and input the selected at least one image into an artificial intelligence model, thereby generating content using the artificial intelligence model.
[0133] In one embodiment, when the mode of the electronic device (100, 2001) is set to a mode for generating content, the electronic device (100, 2001) can obtain a user image inside the vehicle using the first camera (110) (see 313 of FIG. 3 and FIG. 7). In one embodiment, when the mode of the electronic device (100, 2001) is set to a mode for generating content, the electronic device (100, 2001) can obtain a voice inside the vehicle using the microphone (150) (see 317 of FIG. 3 and FIG. 7). In one embodiment, when the mode of the electronic device (100, 2001) is set to a mode for generating content, the electronic device (100, 2001) can obtain an external image of the vehicle using the second camera (120).
[0134] FIG. 8 is a flowchart illustrating a process in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment acquires a plurality of images by controlling the driving of the vehicle (105) in response to the mode of the vehicle (105) being determined to be a content creation mode.
[0135] Referring to identification number 810, the electronic device (100, 2001) can determine whether the vehicle (105) is located at a predetermined location. According to one embodiment, the electronic device (100, 2001) can determine whether the vehicle (105) is located at a predetermined location using a sensor included in the sensor module (2076). For example, the electronic device (100, 2001) can determine whether the vehicle (105) is located at a predetermined location using a GPS sensor or an altimeter sensor of the sensor module (2076). According to one embodiment, the electronic device (100, 2001) can determine whether the vehicle (105) is located at a predetermined location using an artificial intelligence model.
[0136] According to one embodiment, the electronic device (100, 2001) may determine that the vehicle (105) is located at a predetermined location when it determines that a preset event of identification number 320 of FIG. 3 has occurred. A detailed description related to an example of determining that a preset event has occurred corresponds to the description of identification number 320 of FIG. 3, and therefore will be omitted herein.
[0137] Referring to identification number 820, the electronic device (100, 2001) can control the driving of the vehicle (105). According to one embodiment, the electronic device (100, 2001) can change the driving properties of the vehicle (105) when it is determined that the vehicle (105) is located at a predetermined location as indicated by identification number 810. For example, the electronic device (100, 2001) can control at least one of the driving speed, driving direction, turning direction, driving path, whether to change lanes, whether to set a waypoint, or whether to stop the vehicle (105) when it is determined that the vehicle (105) is located at a predetermined location.
[0138] For example, the electronic device (100, 2001) can identify, using a map stored in a memory (e.g., 2030 of FIG. 20), that the current location of the vehicle (105) is within a predetermined distance from a location corresponding to a landmark stored in the map. In this case, the electronic device (100, 2001) can determine that the vehicle (105) is located at a predetermined location. In this case, for example, the electronic device (100, 2001) can reduce the speed at which the vehicle (105) is traveling. In this case, for example, the electronic device (100, 2001) can change the driving route of the vehicle (105) to a shortest route toward the landmark stored in the map. In this case, for example, the electronic device (100, 2001) can change the driving direction of the vehicle (105) so that the landmark stored on the map can be within the range that can be captured by the second camera (120) of the vehicle (105).
[0139] Referring to identification number 830, the electronic device (100, 2001) can capture multiple images using the second camera (120). For example, when the electronic device (100, 2001) determines that the vehicle (105) is located at a predetermined location, the electronic device (100, 2001) can set the multiple second cameras (120) to acquire relatively high-quality images. For example, when the electronic device (100, 2001) determines that the vehicle (105) is located at a predetermined location, the electronic device (100, 2001) can acquire an external image using a camera capable of acquiring relatively high-quality images among the multiple second cameras (120).
[0140] When the electronic device (100, 2001) acquires a plurality of images using the second camera (120) according to the identification number 830, the electronic device (100, 2001) may select at least one external image from among the acquired plurality of images (see identification number 350 of FIGS. 3 and 8). For example, the electronic device (100, 2001) may select at least one external image from among the plurality of external images captured in advance before or while determining that the vehicle (105) is located at a predetermined location. According to one embodiment, the electronic device (100, 2001) may select at least one external image from among the acquired plurality of images using an artificial intelligence model.
[0141] FIG. 9 is a drawing for explaining a method for an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment to select at least one external image from among a plurality of external images acquired using a camera facing the outside of the vehicle (105).
[0142] An electronic device (100, 2001) according to one embodiment may include at least one first camera (110) facing the interior of a vehicle (105). There is no limitation on the number of first cameras (110) included in the electronic device (100, 2001), and there is no limitation on the number of objects that one first camera (110) can acquire. However, for the convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes four first cameras (111, 112, 113, 114) corresponding to four seats respectively inside the vehicle (105), and the following will be described.
[0143] An electronic device (100, 2001) according to one embodiment may include at least one microphone (150) facing the interior of a vehicle (105). There is no limitation on the number of microphones (150) included in the electronic device (100, 2001), but for convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes two microphones (151, 152) corresponding to two rows of seats respectively inside the vehicle (105).
[0144] An electronic device (100, 2001) according to one embodiment may include at least one second camera (120) facing the outside of a vehicle (105). There is no limitation on the number of second cameras (120) included in the electronic device (100, 2001), but for the sake of convenience of explanation, in the description of this drawing, it is assumed that the electronic device (100, 2001) includes a second camera (121) facing the front of the vehicle (105), two second cameras (122, 123) facing the right side of the vehicle, a second camera (124) facing the rear of the vehicle, and two second cameras (125, 126) facing the left side of the vehicle, and the following will be described.
[0145] The number of users who can board a vehicle (105) according to one embodiment and the number of users who board the vehicle may vary without limitation depending on the type and situation of the vehicle (105), but in the description of this drawing, for the convenience of explanation, it is assumed that the number of seats inside the vehicle (105) is four and that the number of people boarding inside the vehicle is four, with one person per seat, and the following will be described.
[0146] According to one embodiment, the electronic device (100, 2001) can monitor a situation related to the movement of the vehicle (see identification number 340 in FIG. 3) and select at least one external image from among a plurality of images acquired using the second camera (see identification number 350 in FIG. 3) by identifying the purpose of generating content (see identification number 330 in FIG. 3).
[0147] In one embodiment, the electronic device (100, 2001) may acquire voices inside the vehicle through the microphones (151, 152). For example, the electronic device (100, 2001) may detect the user's voice within the voices acquired through the microphones (151, 152). For example, the electronic device (100, 2001) may acquire the user's voice through the microphone (152) located adjacent to the second row of the vehicle (105). With reference to identification number 901, for example, the voice acquired through the microphone (152) located adjacent to the second row of the vehicle (105) may include the voice of the user uttering "Look over there! The view is really beautiful!" at 9:37:14 AM. In this case, the electronic device (100, 2001) can extract the user's voice, "Look over there! The view is really beautiful!", from the acquired voice.
[0148] According to one embodiment, the electronic device (100, 2001) can interpret the meaning of the extracted voice. For example, the electronic device (100, 2001) can interpret the meaning of the extracted voice using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, if the extracted voice includes a voice in which a user utters, "Look over there! The view is really beautiful!", the electronic device (100, 2001) can, by interpreting the voice, identify that the meaning is to look at an object located in a predetermined direction from the vehicle (105). In this case, the electronic device (100, 2001) can select an image taken in a predetermined direction from the vehicle (105) based on the meaning derived by interpreting the extracted voice. In this case, the electronic device (100, 2001) can determine the predetermined direction. In this case, the electronic device (100, 2001) may select at least one image acquired through the second camera (121, 122, 123, 124, 125, 126) facing a determined direction from among a plurality of images acquired through the second camera (121, 122, 123, 124, 125, 126). Referring to FIG. 3, for example, the electronic device (100, 2001) may determine a predetermined direction based on at least one of a preset event that has occurred (see identification number 320 of FIG. 3), a purpose of generating identified content (see identification number 330 of FIG. 3), or a situation monitored in relation to the movement of the vehicle (see identification number 340 of FIG. 3).For example, if the electronic device (100, 2001) decides to select an image taken in a predetermined direction from a vehicle (105) as a result of interpreting the user's voice extracted from the voice acquired at 9:37:14 AM, the electronic device (100, 2001) may determine the predetermined direction based on at least one of a preset event (see identification number 320 in FIG. 3) that occurred within a time interval adjacent to 9:37:14 AM, a purpose of generating the identified content (see identification number 330 in FIG. 3), or a situation monitored in relation to the movement of the vehicle (see identification number 340 in FIG. 3). For example, the electronic device (100, 2001) may acquire gaze information of users at 9:37:17 AM on the same day through the first camera (110) and determine the predetermined direction based on the acquired gaze information.
[0149] According to one embodiment, the electronic device (100, 2001) can acquire an image of a user inside a vehicle through the first camera (111, 112, 113, 114). For example, the electronic device (100, 2001) can detect the direction of each user's gaze from the user image acquired through the first camera (111, 112, 113, 114). For example, the electronic device (100, 2001) can determine a predetermined direction based on the direction of each user's gaze detected from the user image acquired through the first camera (111, 112, 113, 114). For example, the electronic device (100, 2001) can detect a gaze (911) looking upward to the right from a user image acquired through the first camera (111), detect a gaze (912) looking toward the right from a user image acquired through the first camera (112), detect a gaze (913) looking toward the right from a user image acquired through the first camera (113), and detect a gaze (914) looking downward to the right from a user image acquired through the first camera (114). In one embodiment, the electronic device (100, 2001) can synthesize and analyze each gaze information (511, 512, 513, 514) detected from images acquired through each of the first cameras (111, 112, 113, 114). In this case, the electronic device (100, 2001) can identify that a predetermined direction is a direction toward the right side of the vehicle (105) based on the result of analyzing the line of sight information (511, 512, 513, 514). In this case, the electronic device (100, 2001) can select an external image (922, 923) captured by the second camera (122, 123) facing the right side of the vehicle (105) from among a plurality of external images acquired through the second camera (121, 122, 123, 124, 125, 126).
[0150] In one embodiment, when the electronic device (100, 2001) decides to select at least one external image from among a plurality of images acquired using the second camera (120), the electronic device may select an external image (922, 923) captured using the second camera (120) prior to the time of the decision. For example, when the electronic device (100, 2001) decides to select at least one external image from among a plurality of images acquired using the second camera (120), the electronic device may select an external image (922, 923) captured using the second camera (120) at the time when a preset event (see identification number 320 of FIG. 3) that is the basis of the decision occurs.
[0151] For example, if the electronic device (100, 2001) decides to select an image taken in a predetermined direction from the vehicle (105) as a result of interpreting the user's voice extracted from the voice acquired at 9:37:14 AM, and identifies that the predetermined direction is toward the right side of the vehicle (105) based on the gaze information acquired at 9:37:17 AM on the same day, the electronic device may select an external image (922, 923) taken at 9:37:13 AM on the same day through the second camera (122, 123) facing the right side of the vehicle (105).
[0152] In one embodiment, the electronic device (100, 2001) can generate content (930) based on at least one selected external image (922, 923). According to one embodiment, the electronic device (100, 2001) can generate content (930) using an artificial intelligence model based on at least one selected external image (922, 923).
[0153] FIG. 10 is a flowchart illustrating a process in which an electronic device (100, 2001) according to one embodiment generates high-quality content using a low-quality image acquired using a camera (110, 120).
[0154] Referring to identification number 1010, the electronic device (100, 2001) can acquire a low-quality image through the first camera (110) or the second camera (120). According to one embodiment, the vehicle (105) equipped with the electronic device (100, 2001) may be a vehicle that provides an autonomous driving function. For example, the second camera (120) can acquire an image of the exterior of the vehicle (105) to monitor the exterior of the vehicle (105) while the vehicle (105) is performing the autonomous driving function. For example, the electronic device (100, 2001) can provide a continuous recording function for the exterior of the vehicle (105) through the second camera (120). For example, the electronic device (100, 2001) can capture a plurality of external images having a relatively low image quality compared to the highest image quality of an external image that can be acquired through the second camera (120). As a result, the electronic device (100, 2001) can further save the capacity of the memory (e.g., 2030 of FIG. 20) for storing the plurality of external images. The first camera (110) according to one embodiment can acquire an image having the lowest image quality that can detect the user's gaze in order to acquire at least one user image. As a result, the electronic device (100, 2001) can further save the capacity of the memory (e.g., 2030 of FIG. 20) for storing the user image.
[0155] Referring to identification number 1020, the electronic device (100, 2001) can select at least one image from among a plurality of images acquired through the first camera (110) or the second camera (120) using an artificial intelligence model. A method for selecting at least one image from among a plurality of images acquired through the first camera (110) or the second camera (120) and a detailed description thereof correspond to the descriptions of FIGS. 9 and 11, and therefore will be omitted herein.
[0156] Referring to identification number 1030, the electronic device (100, 2001) can generate a high-quality image based on a selected external image using an artificial intelligence model. According to one embodiment, the electronic device (100, 2001) can correct the selected external image using the artificial intelligence model. The corrected external image can have a relatively higher image quality than the external image before correction. A detailed description of the artificial intelligence model and a method of using the same corresponds to the description of FIG. 21 and is therefore omitted herein.
[0157] FIG. 11 is a drawing for explaining a method for an electronic device (100, 2001) according to one embodiment to select at least one image from among images acquired through a camera (110, 120) and a method for generating content based on the selected at least one image using an artificial intelligence model.
[0158] According to one embodiment, the electronic device (100, 2001) may select at least one image (1122, 1123) from among images acquired using at least one of a first camera (110) inside the vehicle (105) or a second camera (120, 121, 122, 123, 124, 125, 126) facing the outside of the vehicle (105).
[0159] According to one embodiment, the electronic device (100, 2001) may select at least one image from among the images acquired using at least one of the first camera (110) inside the vehicle (105) or the second camera (120) facing the outside of the vehicle (105), based on the user image inside the vehicle acquired through the first camera (110). In one embodiment, the electronic device (100, 2001) may select at least one user image from among the user images acquired using the first camera (110) inside the vehicle (105), based on a change in the user image detected by the first camera (110). For example, based on a property of a change in the user's expression in the user image acquired by at least one first camera (110), the electronic device (100, 2001) may select at least one user image acquired immediately after the user's expression changes from among the user images acquired using the first camera (110). In one embodiment, the electronic device (100, 2001) may select at least one external image from among external images acquired using a second camera (120) facing the outside of the vehicle (105) based on a change in the user image detected by the first camera (110). For example, the electronic device (100, 2001) may select at least one external image from among external images acquired using a second camera (120) facing the outside of the vehicle (105) based on a property of a change in the user's gaze within the user image acquired by the at least one first camera (110).For example, based on an attribute related to at least one of a direction of the user's gaze, a change in the gaze, an expression, a change in the expression, a movement, a gesture, or a change in the posture within the user image acquired by at least one first camera (110), the electronic device (100, 2001) may select at least one external image from among external images acquired using a second camera (120) facing the outside of the vehicle (105).
[0160] In one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) inside the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on a voice inside the vehicle acquired through a microphone (150). In one embodiment, the electronic device (100, 2001) may select at least one external image from among external images acquired using a second camera (120) facing the outside of the vehicle (105) based on the user's voice when the user's voice is detected by the microphone (150). For example, if the electronic device (100, 2001) identifies that the meaning of the voice derived by interpreting the voice acquired by at least one microphone (150) is related to a predetermined external situation, the electronic device (100, 2001) may select at least one external image from among external images acquired by using the second camera (120) facing the outside of the vehicle (105) based on the meaning of the derived voice. According to one embodiment, the electronic device (100, 2001) may identify whether the meaning of the voice derived by interpreting the voice acquired by at least one microphone (150) is related to a predetermined external situation by using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the predetermined external situation may include a situation related to at least one of the following: the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location of the vehicle (105), whether a landmark exists within a predetermined distance from the current location of the vehicle (105), whether a tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or an animal exists within a predetermined distance from the current location of the vehicle (105).
[0161] According to one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using the first camera (110) inside the vehicle (105) or the second camera (120) facing the outside of the vehicle (105), based on an image of the outside of the vehicle acquired through the second camera (120). In one embodiment, the electronic device (100, 2001) may include a case where an object outside of the vehicle (105) is detected by the second camera (120). According to one embodiment, when the electronic device (100, 2001) identifies that an image of a predetermined object is included in an image acquired by at least one second camera (120), the electronic device (100, 2001) may select at least one external image from among external images acquired using the second camera (120) facing the outside of the vehicle (105) based on an attribute of the identified image. An electronic device (100, 2001) according to one embodiment may identify whether an image of a predetermined object is included in an image acquired by at least one second camera (120) using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the image of the predetermined object may be an image of an object including at least one of weather outside the vehicle (105), scenery, a landmark within a predetermined distance, a person, and an animal. For example, the electronic device (100, 2001) may identify an image of a predetermined object identified as being included in an image acquired by at least one second camera (120) as an image corresponding to a predetermined landmark. In this case, the electronic device (100, 2001) may select at least one image captured in a direction in which the predetermined landmark is located from among images acquired using a first camera (110) inside the vehicle (105) or a second camera (120) facing the outside of the vehicle (105).
[0162] According to one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105), based on data acquired through a sensor module (e.g., 2076 of FIG. 20) included in the electronic device (100, 2001) or another electronic device connected to the electronic device (100, 2001) (e.g., an external electronic device (1400) of FIG. 14, an external electronic device (1400) of FIG. 17). In one embodiment, when predetermined data is acquired through a sensor of a sensor module (e.g., 2076 of FIG. 20), the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on an attribute of the acquired data. In one embodiment, the electronic device (100, 2001) may identify that the data acquired through a sensor of the sensor module (e.g., 2076 of FIG. 20) is related to a predetermined external situation. In one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on an attribute of the identified predetermined external situation. In one embodiment, the electronic device (100, 2001) can use an artificial intelligence model (e.g., 2150 of FIG. 21) to identify whether data acquired through a sensor of a sensor module (e.g., 2076 of FIG. 20) is related to a certain external situation.For example, a given external situation may include a situation related to at least one of the current vehicle's moving speed, the weather outside the vehicle, the scenery, the current location, whether a landmark exists within a given distance from the current location, whether a given tourist attraction exists within a given distance from the current location, or whether a person or animal exists within a given distance from the current location.
[0163] According to one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using the first camera (110) within the vehicle (105) or the second camera (120) facing the outside of the vehicle (105) based on data stored in the memory (e.g., 2030 of FIG. 20) of the electronic device (100, 2001). In one embodiment, when predetermined data is acquired from the memory (e.g., 2030 of FIG. 20) of the electronic device (100, 2001), the electronic device (100, 2001) may select at least one image from among images acquired using the first camera (110) within the vehicle (105) or the second camera (120) facing the outside of the vehicle (105) based on attributes of the acquired predetermined data. In one embodiment, the electronic device (100, 2001) can identify that data acquired from a memory (e.g., 2030 of FIG. 20) is related to a predetermined external situation. In one embodiment, the electronic device (100, 2001) can identify whether data acquired from a memory (e.g., 2030 of FIG. 20) is related to a predetermined external situation using an artificial intelligence model (e.g., 2150 of FIG. 21). In one embodiment, the electronic device (100, 2001) can select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on an attribute of the predetermined external situation. For example, a given external situation may include a situation related to at least one of the current moving speed of the vehicle (105), the weather outside the vehicle, the scenery, the current location, whether a landmark exists within a given distance from the current location, whether a given tourist attraction exists within a given distance from the current location, or the level of congestion within a given distance from the current location.For example, the electronic device (100, 2001) can identify, by using a map stored in a memory (e.g., 2030 of FIG. 20), that the current location of the vehicle (105) is within a predetermined distance from a location corresponding to a landmark stored in the map, and that the landmark is located in front of the vehicle (105). In this case, the electronic device (100, 2001) can identify that a predetermined landmark exists in front of the current vehicle (105) and within a predetermined distance from the vehicle (105), and can select at least one external image acquired through the second camera (120) facing the front of the vehicle (105) from among images acquired using the first camera (110) or the second camera (120) in the vehicle (105).
[0164] According to one embodiment, the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on data acquired through a server (e.g., server (1700) of FIG. 17) connected to the electronic device (100, 2001). In one embodiment, when predetermined data is received from a server (e.g., server (1700) of FIG. 17), the electronic device (100, 2001) may select at least one image from among images acquired using a first camera (110) within the vehicle (105) or a second camera (120) facing the outside of the vehicle (105) based on attributes of the received predetermined data. In one embodiment, the electronic device (100, 2001) can identify that data acquired through a server (e.g., server (1700) of FIG. 17) is related to a predetermined external situation. In this case, the electronic device (100, 2001) can select at least one image based on an attribute of the identified predetermined external situation. In one embodiment, the electronic device (100, 2001) can identify whether data acquired from a server (e.g., server (1700) of FIG. 17) is related to a predetermined external situation using an artificial intelligence model (e.g., 2150 of FIG. 21). For example, the predetermined external situation may include a situation related to at least one of the current moving speed of the vehicle (105), the weather outside the vehicle, the scenery, the current location, whether a landmark exists within a predetermined distance from the current location, whether a predetermined tourist attraction exists within a predetermined distance from the current location, or the level of congestion within a predetermined distance from the current location.For example, the electronic device (100, 2001) can receive information about at least one of the number of other terminal devices, communication status, or maps located within a predetermined distance from the current vehicle (105) through a server (e.g., the server (1700) of FIG. 17), and analyze the received information to identify the level of congestion within a predetermined distance from the location of the current vehicle (105). For example, the electronic device (100, 2001) can receive information from a server (e.g., the server (1700) of FIG. 17) that there are terminal devices exceeding a predetermined number within a predetermined distance in front of the current vehicle (105). In this case, the electronic device (100, 2001) can identify that a predetermined tourist attraction exists within a predetermined distance from the front of the current vehicle (105), and can select at least one external image taken toward the front of the vehicle (105) from among images acquired using the first camera (110) or the second camera (120) within the vehicle (105).
[0165] According to one embodiment, the electronic device (100, 2001) may select at least one image from among external images acquired using the first camera (110) or the second camera (120) in the vehicle (105) based on a result of analyzing at least one data acquired through at least one of the first camera (110), the microphone (150), the second camera (120), the sensor module (e.g., 2076 of FIG. 20), and the memory (e.g., 2030 of FIG. 20). For example, the electronic device (100, 2001) may synthesize and analyze a plurality of data acquired through at least one of the first camera (110), the microphone (150), the second camera (120), and the sensor module (e.g., 2076 of FIG. 20). In this case, the electronic device (100, 2001) can select at least one image from among the images acquired using the first camera (110) or the second camera (120) in the vehicle (105) based on the analysis results.
[0166] In one embodiment, the electronic device (100, 2001) may use an artificial intelligence model to generate content (1130) based on at least one selected image (1122, 1123). A specific method by which the electronic device (100, 2001) generates content (1130) will be described in detail in FIG. 12.
[0167] FIG. 12 is a flowchart illustrating a process of generating an input prompt to be input to an artificial intelligence model based on an image or voice obtained from inside or outside a vehicle (105) in an electronic device (100, 2001) according to one embodiment.
[0168] In the identification number 1210, the electronic device (100, 2001) can identify the purpose of generating the content. According to one embodiment, the electronic device (100, 2001) can identify the purpose of generating the content based on an image acquired through the first camera (110). According to one embodiment, the electronic device (100, 2001) can identify the purpose of generating the content based on the result of identifying an object in the image acquired through the first camera (110). According to one embodiment, the electronic device (100, 2001) can determine the purpose of generating the content based on a voice acquired through the microphone (150). According to one embodiment, the electronic device (100, 2001) can determine the purpose of generating the content based on an input received from a user inside the vehicle (105). For example, through a display (e.g., display module (2060) of FIG. 20) within the vehicle (105), the electronic device (100, 2001) may provide a GUI that prompts the user to input a purpose for generating content. For example, the electronic device (100, 2001) may receive a touch input related to the purpose for generating content from the user. For example, the electronic device (100, 2001) may receive a touch input related to the purpose for generating content from the user through a physical button included in the vehicle (105).
[0169] A specific description of the operation for determining the purpose of content creation corresponds to the description of FIGS. 4 to 6, and therefore is omitted here.
[0170] In the identification number 1220, the electronic device (100, 2001) can obtain information about a situation related to the movement of the vehicle (105). According to one embodiment, the electronic device (100, 2001) can obtain information about a situation related to the movement of the vehicle (105) based on an image obtained through the second camera (120). According to one embodiment, the electronic device (100, 2001) can obtain information about a situation related to the movement of the vehicle (105) based on an attribute of data obtained by a sensor of a sensor module (e.g., 2076 of FIG. 20). For example, information about the situation regarding the movement of the vehicle (105) may include information related to at least one of the purpose of movement of the user within the vehicle (105), the current movement speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location, whether a landmark exists within a predetermined distance from the current location, whether a predetermined tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or animal exists within a predetermined distance from the current location.
[0171] In the identification number 1230, the electronic device (100, 2001) can acquire at least one image. According to one embodiment, the electronic device (100, 2001) can acquire at least one image through the first camera (110) or the second camera (120).
[0172] In the identification number 1240, the electronic device (100, 2001) may generate an input prompt to be input into the artificial intelligence model. In one embodiment, the electronic device (100, 2001) may generate an input prompt to be input into the artificial intelligence model based on the purpose of generating the identified content. In one embodiment, the electronic device (100, 2001) may generate an input prompt to be input into the artificial intelligence model based on information regarding the situation related to the movement of the acquired vehicle (105).
[0173] In one embodiment, the input prompt to be input to the artificial intelligence model may include information related to the purpose of creating the content. According to one embodiment, the input prompt to be input to the artificial intelligence model may include information related to the purpose of creating the content identified based on the result of recognizing an object in an image acquired through the first camera (110). For example, the input prompt to be input to the artificial intelligence model may include information related to the purpose of creating the content, such as to acquire an image taken in a predetermined direction relative to the vehicle (105), identified based on the result of analyzing gaze information. According to one embodiment, the input prompt to be input to the artificial intelligence model may include information related to the purpose of creating the content, identified based on the result of interpreting a voice acquired through the microphone (150). For example, the input prompt to be input to the artificial intelligence model may include information spoken by the user related to at least one of the intention to create the content, the purpose of moving the vehicle (105), or the characteristics of data to be included in the content.
[0174] In one embodiment, the input prompt to be input to the artificial intelligence model may include information related to the situation regarding the movement of the vehicle (105). For example, the input prompt to be input to the artificial intelligence model may include a command that causes the artificial intelligence model to generate content related to at least one of the user's purpose of movement within the vehicle (105), the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location, whether a landmark exists within a predetermined distance from the current location, whether a predetermined tourist attraction exists within a predetermined distance from the current location of the vehicle (105), or whether a person or an animal exists within a predetermined distance from the current location.
[0175] In one embodiment, the input prompt to be input into the artificial intelligence model may include information input by the user. According to one embodiment, the electronic device (100, 2001) may receive input from the user through at least one of a display (e.g., the display module (2060) of FIG. 20 ), a touchpad, or a physical button inside the vehicle (105) regarding the content to be included in the input prompt. For example, the electronic device (100, 2001) may receive input from the user inside the vehicle (105). For example, the electronic device (100, 2001) may provide a GUI that allows the user to input information related to the purpose of generating the content to be input into the artificial intelligence model and the situation regarding the movement of the vehicle (105) through the display (e.g., the display module (2060) of FIG. 20 ) inside the vehicle (105). For example, the electronic device (100, 2001) may receive touch input from a user related to information regarding the purpose of generating content and the situation regarding the movement of the vehicle (105) through at least one of a display (e.g., a display module (2060) of FIG. 20), a touch pad, or a physical button within the vehicle (105).
[0176] At identification number 1250, the electronic device (100, 2001) can input the generated prompt and multiple external images into the artificial intelligence model.
[0177] As indicated by reference numeral 1260, the electronic device (100, 2001) may generate content using an artificial intelligence model. According to one embodiment, the artificial intelligence model may include a deep learning model. According to one embodiment, the artificial intelligence model may include a generative artificial intelligence model. According to one embodiment, the artificial intelligence model may be created through learning. According to one embodiment, the artificial intelligence model may include predefined operation rules or an artificial intelligence model set to perform a desired characteristic or purpose by being trained using a plurality of learning data by a learning algorithm. In one embodiment, the training of the artificial intelligence model may be performed within the electronic device (100, 2001) or may be performed through a separate server (e.g., server 1700 of FIG. 17). According to one embodiment, the artificial intelligence model may include an artificial intelligence model trained to generate content related to the movement of the vehicle (105). In one embodiment, the artificial intelligence model may include an artificial intelligence model trained through learning to generate content related to the movement of the vehicle (105) based on at least one of voice, image, or video data.
[0178] FIG. 13 is a diagram illustrating an example of generating an input prompt to be input to an artificial intelligence model based on images or voices obtained from inside and outside a vehicle (105) in an electronic device (100, 2001) according to one embodiment.
[0179] According to one embodiment, the electronic device (100, 2001) may include a memory (e.g., memory (2130) of FIGS. 13 and 21). Referring to identification number 1311, the memory (2130) or the server (e.g., 1700 of FIG. 17) according to one embodiment may store information related to a user. For example, the memory (2130) or the server (e.g., 1700 of FIG. 17) may store information related to a user who has previously used the vehicle (105) and the electronic device (100, 2001) mounted on the vehicle (100, 2001). For example, the memory (2130) or the server (e.g., 1700 of FIG. 17) may store information related to a user who has previously ridden in the vehicle (105). For example, the memory (2130) may store information related to a given user, such as information that the user likes the sea and frequently takes pictures of seascapes.
[0180] Referring to identification numbers 1313 and 1315, the electronic device (100, 2001) can acquire data through at least one of the first camera (110), the microphone (150), the second camera (120), or the sensor of the sensor module (e.g., 2076 of FIG. 20).
[0181] For example, referring to identification number 1313, the electronic device (100, 2001) can acquire a user's voice uttering "Upload today's travel video to Instagram" via the microphone (151). In this case, the electronic device (100, 2001) can interpret the acquired voice and determine that the user's voice means to upload a video of a day's travel while the vehicle is in motion to Instagram.
[0182] For example, referring to identification number 1315, the electronic device (100, 2001) can acquire a voice of a user saying "Wow, it's finally summer vacation!" through a microphone (152). In this case, the electronic device (100, 2001) can determine, by interpreting the acquired voice, that the user is currently on summer vacation.
[0183] Referring to identification number 1317, the electronic device (100, 2001) can identify the purpose of creating content based on data acquired through at least one of the first camera (110), the microphone (150), the second camera (120), or the sensor of the sensor module (e.g., 2076 of FIG. 20).
[0184] For example, the electronic device (100, 2001) can identify that the user's intention to create content is to upload a video to Instagram based on the result of interpreting the voice acquired at identification number 1313. For example, the electronic device (100, 2001) can identify that the purpose of movement of the vehicle (105) is related to travel based on the result of interpreting the meaning of the voice acquired at identification number 1315.
[0185] For example, the electronic device (100, 2001) may identify that the purpose of movement of the vehicle (105) is related to a trip for a summer vacation, based on the result of interpreting the voice acquired from the identification number 1315.
[0186] For example, referring to identification number 1317, the electronic device (100, 2001) can identify that the purpose of creating the content is to upload a video related to a summer vacation at the beach to Instagram by analyzing the results of interpreting the voices obtained from identification numbers 1313 and 1315.
[0187] Referring to identification number 1319, the electronic device (100, 2001) may generate an input prompt to be input into the artificial intelligence model based on the purpose of generating the identified content. For example, the input prompt to be input into the artificial intelligence model may include information spoken by the user related to at least one of the following: the intention to generate the content, the purpose of moving the vehicle (105), or the characteristics of the data to be included in the content.
[0188] For example, referring to identifier 1319, the electronic device (100, 2001) may generate an input prompt to "generate landscape photos that people who like the ocean like while on their summer vacation" based on the purpose of generating the content being identified as uploading images related to a summer vacation at the ocean to Instagram.
[0189] FIG. 14 is a drawing showing an example of an environment in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment operates while connected to an external electronic device (1400).
[0190] According to one embodiment, the electronic device (100, 2001) may be connected to and operate with an external electronic device (1400). For example, the external electronic device (1400) may include a smart phone.
[0191] In one embodiment, the electronic device (100, 2001) may be connected to an external electronic device (1400) via an antenna (e.g., an antenna module (2097) of FIG. 20). In one embodiment, the electronic device (100, 2001) may communicate with the external electronic device (1400) via a communication module (e.g., a communication module (2090) of FIG. 20). For example, the electronic device (100, 2001) may be connected to and operate with the external electronic device (100, 2001) via Bluetooth communication.
[0192] For example, the electronic device (100, 2001) and the external electronic device (1400) may store artificial intelligence models in their respective memories. In this case, at least some of the operations included in the process of generating content may be performed by the artificial intelligence model of the electronic device (100, 2001), and at least some of the operations may be performed by the artificial intelligence model of the external electronic device (1400).
[0193] For example, the electronic device (100, 2001) may be connected to and operate with an external electronic device (1400) and a server (e.g., server (1700) of FIG. 17). This will be described in detail in FIGS. 17 and 18.
[0194] FIG. 15 is a flowchart illustrating a process in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment is connected to an external electronic device (1400) located around the vehicle (105) to generate content and transmit the generated content to the external electronic device (1400).
[0195] In operation 1510, if the external electronic device (1400) detects that the electronic device (100, 2001) is located within a predetermined distance from the external electronic device (1400), the external electronic device (1400) may request a connection to the electronic device (100, 2001). For example, if the external electronic device (1400) detects that a user has boarded a vehicle (105), the external electronic device (1400) may request a connection to the electronic device (100, 2001). According to one embodiment, if the electronic device (100, 2001) receives a connection request from the external electronic device (1400), the electronic device may attempt to connect to the external electronic device (1400) via an antenna or a communication module.
[0196] In operation 1520, when the electronic device (100, 2001) is connected to an external electronic device (1400), it can transmit information about this to the external electronic device (1400).
[0197] In operation 1530, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to generate content. For example, when the external electronic device (1400) receives an input from a user to generate content, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to generate content based on the user's input. For example, the external electronic device (1400) may provide the user with a GUI asking whether to generate content through a display, and may receive an input requesting the generation of content by receiving a touch input from the user to the GUI. In this case, the external electronic device (1400) may transmit a signal to the electronic device (100, 2001) to turn on the second camera (120) to generate content.
[0198] In operation 1540, when the electronic device (100, 2001) receives a signal requesting content generation from an external electronic device (1400), the electronic device (100, 2001) may capture an external image through the second camera (120) of the vehicle (105) and generate content based on the captured external image. According to one embodiment, the process of generating content may be performed using an artificial intelligence model. For example, the process of generating content may be performed using a generative artificial intelligence model. The process of capturing an external image and generating content has been described in detail in FIGS. 3 to 13, and thus will be omitted herein.
[0199] In operation 1550, the electronic device (100, 2001) may transmit the generated content to an external electronic device (1400). In operation 1560, the external electronic device (1400) may receive the generated content and store the received content in the memory of the external electronic device (1400).
[0200] FIG. 16 is a flowchart illustrating a process of generating content while connected to an external electronic device (1400) when an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment receives an external image capture request from the external electronic device (1400).
[0201] In operation 1610, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to capture an external image for content generation while connected to the electronic device (100, 2001).
[0202] In operation 1620, for example, the external electronic device (1400) may transmit information that serves as the basis for content creation to the electronic device (100, 2001). For example, the information that serves as the basis for content creation may include information stored in the memory or server of the external electronic device (100, 2001). For example, the information that serves as the basis for content creation may include information stored in the memory or server of the external electronic device (100, 2001) in relation to the user of the electronic device (100, 2001).
[0203] In operation 1630, the electronic device (100, 2001) that receives a signal requesting to capture an external image from an external electronic device (1400) can capture an external image through the second camera (120) of the vehicle (105) and generate content based on the captured external image. The process of capturing an external image and generating content has been described in detail in FIGS. 3 to 13, and thus will be omitted here.
[0204] In operation 1640, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to complete capturing of an external image. For example, if the external electronic device (1400) receives an input from a user to stop capturing, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to complete capturing of an external image. In one embodiment, the electronic device (100, 2001) may terminate capturing of an external image for generating content upon receiving a signal requesting the completion of capturing of an external image from the external electronic device (1400).
[0205] In operation 1650, the electronic device (100, 2001) may generate content based on external images captured during a period prior to the time at which the signal was received, upon receiving a signal requesting completion of capturing an external image from the external electronic device (1400), and transmit the generated content to the external electronic device (1400).
[0206] Referring to operation 1660, in one embodiment, upon receiving the generated content, the external electronic device (1400) may store the content in the memory of the external electronic device (1400). For example, upon receiving the generated content, the external electronic device (1400) may generate new content based on the received content and data stored in the memory of the external electronic device (1400).
[0207] FIG. 17 is a drawing illustrating an environment in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment shares a network with an external electronic device (1400) located around the vehicle (105) and operates by being connected to a server through the shared network.
[0208] According to one embodiment, the electronic device (100, 2001) may be connected to and operate with an external electronic device (1400) and a server (1700). According to one embodiment, the electronic device (100, 2001) may share a network (1750) with an external electronic device (1400) located around a vehicle (105), and may be connected to and operate with a server (1700) through the shared network (1750).
[0209] For example, the electronic device (100, 2001), the external electronic device (1400), and the server (1700) may store artificial intelligence models (1701, 1702, 1703) in their respective memories. In this case, at least some of the operations included in the process of generating content may be performed by the first artificial intelligence model (1701) of the electronic device (100, 2001), at least some of them may be performed by the second artificial intelligence model (1702) of the external electronic device (1400), and at least some of them may be performed by the third artificial intelligence model (1703) of the server (1700). For example, the determination of whether a preset event has occurred may be performed by a second artificial intelligence model (1702) of an external electronic device (1400), and the operation of identifying information related to a predetermined external situation by interpreting a voice acquired through a microphone (150) may be performed by a third artificial intelligence model (1703) of a server (1700).
[0210] FIG. 18 is a flowchart illustrating a process in which an electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment is connected to an external electronic device (1400) and a server located around the vehicle (105) through a network, and a server generates content based on data obtained from the electronic device (100, 2001) and the external electronic device (1400), and transmits the generated content to the external electronic device (100, 2001).
[0211] In operation 1810, an external electronic device (1400) may detect that a user has boarded a vehicle (105). In one embodiment, the electronic device (100, 2001) may be connected to the external electronic device (1400) via an antenna (e.g., an antenna module (2097) of FIG. 20). In one embodiment, the electronic device (100, 2001) may communicate with the external electronic device (1400) via a communication module (e.g., a communication module (2090) of FIG. 20). For example, the electronic device (100, 2001) may be connected to and operate with the external electronic device (100, 2001) via Bluetooth communication. Therefore, for example, an external electronic device (1400) can be connected to an electronic device (100, 2001) via Bluetooth communication, and when the external electronic device (1400) and the electronic device (100, 2001) are connected via Bluetooth communication, the external electronic device (1400) can detect that a user has boarded the vehicle (105).
[0212] In operation 1815, the external electronic device (1400) may request a connection to the electronic device (100, 2001) from the external electronic device (1400) if the electronic device (100, 2001) is located within a predetermined distance from the external electronic device (1400). For example, the external electronic device (1400) may request a connection to the electronic device (100, 2001) if it is detected that a user has boarded a vehicle (105).
[0213] In operation 1820, when an electronic device (100, 2001) according to one embodiment receives a connection request from an external electronic device (1400), it attempts to connect to the external electronic device (1400) through an antenna or a communication module, and when the connection is completed, it can transmit information about the connection to the external electronic device (1400).
[0214] In operation 1825, an external electronic device (1400) according to one embodiment may transmit a signal requesting the electronic device (100, 2001) to generate content. For example, when the external electronic device (1400) receives an input from a user to generate content, the external electronic device (1400) may transmit a signal requesting the electronic device (100, 2001) to generate content based on the user's input.
[0215] In operation 1830, when the electronic device (100, 2001) receives a signal requesting content generation from an external electronic device (1400), the electronic device (100, 2001) may acquire data through the first camera (110), the second camera (120), the microphone (150), or the sensor module (e.g., 2076 of FIG. 20) of the vehicle (105). For example, when the electronic device (100, 2001) receives a signal requesting content generation from the external electronic device (1400), the electronic device (100, 2001) may activate the second camera (120) of the vehicle (105) to capture an external image.
[0216] An external electronic device (1400) can capture images of the interior or exterior of a vehicle through a camera, record audio through a microphone, or detect certain data through a sensor module. In one embodiment, the external electronic device (1400) can generate content based on captured images of the interior or exterior of the vehicle, recorded audio, or certain detected data.
[0217] In operation 1835, the electronic device (100, 2001) may transmit acquired data or captured external images to the server (1700). Furthermore, in operation 1840, the external electronic device (1400) may transmit captured images of the interior or exterior of the vehicle, recorded voices, or predetermined sensed data to the server (1700). According to one embodiment, the external electronic device (1400) may transmit generated content to the server (1700). According to one embodiment, the server (1700) may generate new content based on data, external images, or content received from at least one of the electronic device (100, 2001) or the external electronic device (1400).
[0218] In operation 1850, the server (1700) may transmit content generated by the server (1700) to an external electronic device (1400). In operation 1855, the external electronic device (1400) may store the content received from the server (1700) in memory.
[0219] FIG. 19 is an exemplary drawing of a screen for controlling an electronic device (100, 2001) mounted on a vehicle (105) from an external electronic device (1400) when the electronic device (100, 2001) mounted on a vehicle (105) according to one embodiment is connected to an external electronic device (1400).
[0220] Referring to identification number 1910, an external electronic device (1400) can control an electronic device (100, 2001) mounted on a vehicle (105). For example, the external electronic device (1400) can display a GUI (1901) representing a menu for controlling the electronic device (100, 2001) mounted on the vehicle (105) through a display.
[0221] Referring to the identification number 1920, the external electronic device (1400) can control the second camera (120) included in the electronic device (100, 2001) of the vehicle (105). For example, the external electronic device (1400) can display a GUI (1921) through a display indicating whether it is connected to the electronic device (100, 2001) of the vehicle (105) or the type of the connected vehicle (105). For example, the external electronic device (1400) can control at least one of whether the second camera (120) included in the electronic device (100, 2001) of the vehicle (105) is activated or turned on. For example, the external electronic device (1400) may display a GUI (1923) for controlling at least one of the activation or power of the second camera (120) included in the electronic device (100, 2001) of the vehicle (105) through a display. Referring to identification number 1930, for example, the external electronic device (1400) may display a GUI (1935) for controlling the activation or power of each of the second cameras (120) included in the electronic device through a display. For example, when the electronic device (100, 2001) is set to a mode for generating content (see identification number 701 of FIG. 7), the external electronic device (1400) may receive a user's input for at least a part of the GUI (1935) for controlling the activation or power of each of the second cameras (120). In this case, when the external electronic device (1400) transmits a signal regarding the user's input to the electronic device (100, 2001), the electronic device (100, 2001) that receives the signal can select an image captured by the second camera (120) corresponding to the GUI (1935) where the user's input was received, and generate content based on the selected image.
[0222] Referring to identification number 1930, the external electronic device (1400) can control the first camera (110), microphone (150), or sensor module included in the electronic device (100, 2001) of the vehicle (105). For example, the external electronic device (1400) can control whether the electronic device (100, 2001) of the vehicle (105) detects a gaze through the first camera (110). For example, the external electronic device (1400) can display a GUI (1931) through a display for controlling whether to detect a gaze using the first camera (110). For example, the external electronic device (1400) can control at least one of whether the first camera (110) is activated or powered on. For example, the external electronic device (1400) may display a GUI (1933) through a display for controlling at least one of the activation or power of the first camera (110). For example, the external electronic device (1400) may display a GUI (1934) through a display for controlling whether to capture an image through the first camera (110).
[0223] According to one embodiment, an external electronic device (1400) can control an electronic device (100, 2001) by transmitting a signal corresponding to a GUI in which a user's input is received to the electronic device (100, 2001) when a user's input is received for a GUI (1901, 1921, 1923, 1931, 1932, 1933, 1934, 1935) displayed through a display.
[0224] According to one embodiment, a method for an electronic device in a vehicle to generate content related to the movement of the vehicle may include an operation of obtaining at least one user image by photographing at least one user inside the vehicle using at least one first camera facing inside the vehicle while the vehicle is moving. The method may include an operation of obtaining voice data of the at least one user inside the vehicle using a microphone disposed inside the vehicle while the vehicle is moving. The method may include an operation of identifying a purpose of generating the content related to the movement of the vehicle based on the at least one user image or the voice data when a preset event occurs, and an operation of monitoring a situation related to the movement of the vehicle. The method may include an operation of selecting at least one image from among the at least one user image acquired using the at least one first camera or a plurality of external images acquired using at least one second camera facing outside the vehicle based on the purpose of generating the content and the monitored situation, and generating the content including the at least one selected image. The action of generating the above content may be to generate the content by inputting at least one selected image into an artificial intelligence model trained to generate the content related to the movement of the vehicle.
[0225] A method for an electronic device in a vehicle according to one embodiment to generate content related to movement of the vehicle may include an operation of controlling driving of the vehicle to generate the content according to the generation purpose based on the purpose of generating the content and the monitored situation. A method for an electronic device in a vehicle according to one embodiment to generate content related to movement of the vehicle may include an operation of acquiring the plurality of external images using the at least one second camera of the vehicle that is being driven by the control.
[0226] In one embodiment, the operation of controlling the driving of the vehicle may include an operation of changing the driving properties of the vehicle as the vehicle moves to a predetermined location. The driving properties of the vehicle may include at least one of the vehicle's driving speed, driving direction, turning direction, driving path, whether a lane is changed, whether a waypoint is set, or whether the vehicle is stopped.
[0227] According to one embodiment, the operation of selecting the at least one user image or the at least one image from among the plurality of external images may be selecting the at least one user image or the at least one image from among the plurality of external images based on a direction in which the gaze of the at least one user in the vehicle is directed while acquiring the at least one user image or the at least one image from among the plurality of external images.
[0228] According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of recognizing an object within the user image. According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of interpreting the meaning of the voice data. According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of identifying the purpose of generating the content based on the result of recognizing the object and the meaning of the interpreted voice data.
[0229] In one embodiment, the voice data may include a voice requesting the creation of the content.
[0230] According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of identifying the purpose of generating the content based on preference information of at least one user determined in advance for generating the content.
[0231] According to one embodiment, the information about the situation regarding the movement of the vehicle may include at least one of the purpose of movement of the user within the vehicle (105), the current moving speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location, and whether a landmark exists within a predetermined distance from the current location.
[0232] In one embodiment, the artificial intelligence model may include a generative artificial intelligence model. In one embodiment, the operation of generating the content may include an operation of generating an input prompt to be input into the generative artificial intelligence model based on the purpose of generating the content and the monitored situation. In one embodiment, the operation of generating the content may include an operation of inputting the input prompt and the at least one image into the generative artificial intelligence model.
[0233] A method for an electronic device in a vehicle, according to one embodiment, to generate content related to the movement of the vehicle may include an operation of connecting with an external electronic device. A method for an electronic device in a vehicle, according to one embodiment, to generate content related to the movement of the vehicle may include an operation of transmitting the generated content to the external electronic device.
[0234] According to one embodiment, an electronic device within a vehicle may include at least one first camera facing the interior of the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one second camera facing the exterior of the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one microphone disposed within the vehicle. According to one embodiment, an electronic device within a vehicle may include at least one processor. According to one embodiment, an electronic device within a vehicle may include a memory storing instructions and an artificial intelligence model. The artificial intelligence model may be configured through training for generating content related to the movement of the vehicle. The instructions may be executed by the at least one processor to cause the electronic device to acquire at least one user image by photographing at least one user within the vehicle using the at least one first camera while the vehicle is moving. The artificial intelligence model may be configured through training for generating content related to the movement of the vehicle. The commands, executed by the at least one processor, may cause the electronic device to obtain voice data of the at least one user inside the vehicle using a microphone disposed inside the vehicle while the vehicle is moving. The commands, executed by the at least one processor, may cause the electronic device to identify a purpose of generating the content related to the movement of the vehicle based on the at least one user image or the voice data when a preset event occurs. The commands, executed by the at least one processor, may cause the electronic device to monitor a situation related to the movement of the vehicle.The above commands may be executed by the at least one processor to cause the electronic device to select at least one image from among at least one user image acquired using the at least one first camera or a plurality of external images acquired using the at least one second camera based on the purpose of generating the content and the monitored situation, and to generate the content including the at least one selected image. The operation of generating the content may be to generate the content by inputting the at least one selected image into an artificial intelligence model trained for generating the content related to the movement of the vehicle.
[0235] In an electronic device within a vehicle according to one embodiment, the commands may be executed by the at least one processor to cause the electronic device to control driving of the vehicle to generate the content according to the generation purpose based on the generation purpose of the content and the monitoring situation. In an electronic device within a vehicle according to one embodiment, the commands may cause the electronic device to acquire the plurality of external images using the at least one second camera of the vehicle being driven by the control.
[0236] According to one embodiment, the operation of controlling the driving of the vehicle may include an operation of changing a driving property of the vehicle as the vehicle moves to a predetermined location.
[0237] According to one embodiment, the driving properties of the vehicle may include at least one of the driving speed of the vehicle, the driving direction, the turning direction, the driving path, whether a lane is changed, whether a waypoint is set, or whether the vehicle is stopped.
[0238] According to one embodiment, the operation of selecting the at least one user image or the at least one image from among the plurality of external images may be selecting the at least one user image or the at least one image from among the plurality of external images based on a direction in which the gaze of the at least one user in the vehicle is directed while acquiring the at least one user image or the at least one image from among the plurality of external images.
[0239] According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of recognizing an object within the user image or an operation of interpreting the meaning of the voice data. According to one embodiment, the operation of identifying the purpose of generating the content may include an operation of identifying the purpose of generating the content based on the object recognition result and the meaning of the interpreted voice data.
[0240] In one embodiment, the voice data may include a voice requesting the creation of the content.
[0241] According to one embodiment, the operation of identifying the purpose of generating the content may be identifying the purpose of generating the content based on preference information of at least one user determined in advance for generating the content.
[0242] According to one embodiment, the information about the situation regarding the movement of the vehicle may include information about at least one of the purpose of movement of the user within the vehicle (105), the current movement speed of the vehicle (105), the weather outside the vehicle (105), the scenery, the current location, and whether a landmark exists within a predetermined distance from the current location.
[0243] In one embodiment, the artificial intelligence model may include a generative artificial intelligence model. In one embodiment, the operation of generating the content may include an operation of generating an input prompt to be input into the generative artificial intelligence model based on the purpose of generating the content and the monitored situation. In one embodiment, the operation of generating the content may include an operation of inputting the input prompt and at least one selected image into the generative artificial intelligence model.
[0244] In one embodiment, the commands, executed by the at least one processor, may cause the electronic device to connect with an external electronic device. In one embodiment, the commands, executed by the at least one processor, may cause the electronic device to transmit the generated content to the external electronic device.
[0245] Below, with reference to FIGS. 20 and 21, we specify and expand upon devices to which various embodiments disclosed in this document can be applied or expanded.
[0246] FIG. 20 is a block diagram of an electronic device (100, 2001) within a network environment (2000) according to various embodiments. Referring to FIG. 20, in the network environment (2000), the electronic device (2001) may communicate with the electronic device (2002) via a first network (2098) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (2004) or the server (2008) via a second network (2099) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (2001) may communicate with the electronic device (2004) via the server (2008). According to one embodiment, the electronic device (2001) may include a processor (2020), a memory (2030), an input module (2050), an audio output module (2055), a display module (2060), an audio module (2070), a sensor module (2076), an interface (2077), a connection terminal (2078), a haptic module (2079), a camera module (2080), a power management module (2088), a battery (2089), a communication module (2090), a subscriber identification module (2096), or an antenna module (2097). In some embodiments, the electronic device (2001) may omit at least one of these components (e.g., the connection terminal (2078)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (2076), camera module (2080), or antenna module (2097)) may be integrated into a single component (e.g., display module (2060)).
[0247] The processor (2020) may, for example, execute software (e.g., a program (2040)) to control at least one other component (e.g., a hardware or software component) of the electronic device (2001) connected to the processor (2020) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (2020) may store commands or data received from other components (e.g., a sensor module (2076) or a communication module (2090)) in the volatile memory (2032), process the commands or data stored in the volatile memory (2032), and store the resulting data in the non-volatile memory (2034). According to one embodiment, the processor (2020) may include a main processor (2021) (e.g., a central processing unit or an application processor) or a secondary processor (2023) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (2021). For example, when the electronic device (2001) includes the main processor (2021) and the secondary processor (2023), the secondary processor (2023) may be configured to use less power than the main processor (2021) or to be specialized for a given function. The secondary processor (2023) may be implemented separately from the main processor (2021) or as a part thereof.
[0248] The auxiliary processor (2023) may control at least a portion of functions or states associated with at least one component of the electronic device (2001) (e.g., the display module (2060), the sensor module (2076), or the communication module (2090)), for example, on behalf of the main processor (2021) while the main processor (2021) is in an inactive (e.g., sleep) state, or together with the main processor (2021) while the main processor (2021) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (2023) (e.g., an image signal processor or a communication processor) may be implemented as part of another functionally related component (e.g., a camera module (2080) or a communication module (2090)). According to one embodiment, the auxiliary processor (2023) (e.g., a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model (e.g., 2150 of FIG. 21). The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, in the electronic device (2001) itself on which the artificial intelligence model is executed, or may be performed through a separate server (e.g., server (2008)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.
[0249] The memory (2030) can store various data used by at least one component (e.g., the processor (2020) or the sensor module (2076)) of the electronic device (2001). The data can include, for example, software (e.g., the program (2040)) and input data or output data for commands related thereto. The memory (2030) can include volatile memory (2032) or non-volatile memory (2034).
[0250] The program (2040) may be stored as software in memory (2030) and may include, for example, an operating system (2042), middleware (2044), or an application (2046).
[0251] The input module (2050) can receive commands or data to be used in a component of the electronic device (2001) (e.g., a processor (2020)) from an external source (e.g., a user) of the electronic device (2001). The input module (2050) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0252] The audio output module (2055) can output audio signals to the outside of the electronic device (2001). The audio output module (2055) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.
[0253] The display module (2060) can visually provide information to an external party (e.g., a user) of the electronic device (2001). The display module (2060) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (2060) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.
[0254] The audio module (2070) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (2070) can acquire sound through the input module (2050), output sound through the sound output module (2055), or an external electronic device (e.g., electronic device (2002)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (2001).
[0255] The sensor module (2076) can detect the operating status (e.g., power or temperature) of the electronic device (2001) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (2076) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0256] The interface (2077) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (2001) to an external electronic device (e.g., the electronic device (2002)). In one embodiment, the interface (2077) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0257] The connection terminal (2078) may include a connector through which the electronic device (2001) may be physically connected to an external electronic device (e.g., the electronic device (2002)). In one embodiment, the connection terminal (2078) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0258] The haptic module (2079) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (2079) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
[0259] The camera module (2080) can capture still images and videos. In one embodiment, the camera module (2080) may include one or more lenses, image sensors, image signal processors, or flashes.
[0260] The power management module (2088) can manage power supplied to the electronic device (2001). According to one embodiment, the power management module (2088) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).
[0261] A battery (2089) may power at least one component of the electronic device (2001). In one embodiment, the battery (2089) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0262] The communication module (2090) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (2001) and an external electronic device (e.g., electronic device (2002), electronic device (2004), or server (2008)), and the performance of communication through the established communication channel. The communication module (2090) may operate independently from the processor (2020) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (2090) may include a wireless communication module (2092) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (2094) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (2004) via a first network (2098) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (2099) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (2092) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (2096) to identify or authenticate the electronic device (2001) within a communication network such as the first network (2098) or the second network (2099).
[0263] The wireless communication module (2092) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (2092) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (2092) can support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (2092) can support various requirements specified in the electronic device (2001), an external electronic device (e.g., the electronic device (2004)), or a network system (e.g., the second network (2099)). According to one embodiment, the wireless communication module (2092) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.
[0264] The antenna module (2097) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (2097) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (2097) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (2098) or the second network (2099), may be selected from the plurality of antennas, for example, by the communication module (2090). A signal or power may be transmitted or received between the communication module (2090) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (2097).
[0265] According to various embodiments, the antenna module (2097) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.
[0266] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).
[0267] According to one embodiment, commands or data may be transmitted or received between the electronic device (2001) and an external electronic device (2004) via a server (2008) connected to a second network (2099). Each of the external electronic devices (2002 or 2004) may be the same or a different type of device as the electronic device (2001). According to one embodiment, all or part of the operations executed in the electronic device (2001) may be executed in one or more of the external electronic devices (2002, 2004, or 2008). For example, when the electronic device (2001) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (2001) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (2001). The electronic device (2001) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (2001) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (2004) may include an Internet of Things (IoT) device. The server (2008) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (2004) or server (2008) may be included within the second network (2099). The electronic device (2001) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.
[0268] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.
[0269] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.
[0270] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0271] Various embodiments of the present document may be implemented as software (e.g., a program (2040)) including one or more instructions stored in a storage medium (e.g., an internal memory (2036) or an external memory (2038)) readable by a machine (e.g., an electronic device (2001)). For example, a processor (e.g., a processor (2020)) of the machine (e.g., an electronic device (2001)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.
[0272] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0273] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
[0274] FIG. 21 is a diagram illustrating a system including a generative artificial intelligence model according to one embodiment.
[0275] Referring to FIG. 21, the User Query / Response Interface (2110) can receive a user's input. The user's input may be in the form of natural language, images, and / or videos. Furthermore, context information may also be transmitted when the user's input is transmitted. Context information may include various additional information at the time of user input. For example, information on the application currently being used by the user or information on the user's location. Furthermore, the user's input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user's input may also be in a non-natural language form, such as selecting a menu. The User Query / Response Interface (2110) can output the results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The User Query Interface can output the results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user.
[0276] The AI framework (2120) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.
[0277] User input received from the User Query / Response Interface (2110) can be transmitted to the Prompt design component (2121). The Prompt design component (2121) can be used to generate a prompt suitable for inputting the user input into a Large Language Model (LLM) or a Large Multimodal Model (LMM). The Prompt design component (2121) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The Prompt design component (2121) can access a knowledge component (e.g., knowledge repositories (2140)) containing user preference data, a prompt library, and prompt examples based on the user input to generate a prompt, and transmit the generated prompt to the LLM or LMM.
[0278] The API / Plug-in management component (2123) can communicate with external information when there is a request for additional information when passing user input as input to a generative model. The API / Plug-in management component (2123) can establish a channel for communicating with the outside of the AI Interface through the API, and can enable access to various data sources (e.g., knowledge repositories (2140)) through the established channel. In addition, if the API / Plug-in management component (2123) needs to perform an action that performs the user input as a final result rather than an intermediate result in an application or service, it can request the action to the application / service component (2130) through the API. Information obtained from an external source can be used to generate a prompt in the prompt design component (2121) together with the user input, or can be passed as input to the generative model.
[0279] The Refiner component (e.g., the output modification component (2125)) can fine-tune the output from a generative model. For example, the Refiner component can verify that the content generated by the LLM and / or LMM is not irrelevant, biased, or harmful. Furthermore, the Refiner component can determine the degree to which the output matches the user's desired result and, if necessary, perform additional processing. The Refiner component can also configure and provide users with hints to avoid undesirable output.
[0280] Generative AI Model (2150) can generally refer to an artificial intelligence neural network that creates new types of data based on user input information. Generative AI Model (2150) can include an image-generating model and / or a language-generating model. Representative models for generating images include a generative adversarial network (GAN) and a variational autoencoder (VAE), and examples include a diffusion-based generative model that uses a VAE and a transformer structure. A language-generating model is a model trained to statistically output the most appropriate output value based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there is also an LMM that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.
[0281] According to one embodiment, the electronic device (xxx) of FIGS. 1 to 20 may be configured to include at least a portion of the User Query / Response Interface (2310), the AI framework (2320), the application / service component (2330), the knowledge repositories (2340), or the Generative AI Model (2350) of FIG. 21. According to one embodiment, at least a portion of the User Query / Response Interface (2310), the AI framework (2320), the application / service component (2330), the knowledge repositories (2340), or the Generative AI Model (2350) of FIG. 21 may be included in another electronic device (e.g., an external electronic device and / or a server).
[0282] The methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.
[0283] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. The one or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the claims or specification of the present disclosure.
[0284] In the present disclosure, a function or operation performed by an electronic device may be performed by one or more processors executing one or more instructions stored in a memory. The function or operation of the electronic device mentioned in the present disclosure may be performed by one processor executing one or more instructions, or may be performed by a combination of multiple processors executing one or more instructions. The processor mentioned in the present disclosure may be understood to include a circuit for performing an operation or controlling other components of the electronic device. For example, the one or more processors may include at least one of a central processing unit (CPU), a microprocessor unit (MPU), an application processor (AP), a communication processor (CP), a neural processing unit (NPU), a system on chip (SoC), an application-specific integrated circuit (ASIC), or an integrated circuit (IC) configured to execute one or more instructions. The one or more processors may be configured to perform the operations of the electronic device described above.
[0285] In the present disclosure, a program (software module, software) may be stored in a non-volatile memory including a random access memory (RAM), a flash memory, a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a magnetic disc storage device, a compact disc ROM (CD-ROM), digital versatile discs (DVDs) or other forms of optical storage devices, a magnetic cassette. Or, it may be stored in a memory formed by a combination of some or all of these. The memory may be formed by a single storage medium, or may be formed by a combination of a plurality of storage media. The one or more commands may be stored in a single storage medium, or may be distributed and stored in a plurality of storage media.
[0286] Additionally, the program may be stored on an attachable storage device that is accessible via a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure via an external port. Additionally, a separate storage device on the communication network may be connected to a device performing an embodiment of the present disclosure.
[0287] In the specific embodiments of the present disclosure described above, components included in the disclosure are expressed in the singular or plural form, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in the plural form may be composed of singular elements, or components expressed in the singular form may be composed of plural elements.
[0288] Additionally, in the present disclosure, terms such as “part”, “module”, etc. may refer to a hardware component such as a processor or circuit, and / or a software component executed by a hardware component such as a processor.
[0289] A "component" or "module" may be implemented by a program stored in an addressable storage medium and executed by a processor. For example, a "component" or "module" may be implemented by components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.
[0290] The specific implementations described in this disclosure are merely exemplary and do not limit the scope of the present disclosure in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted.
[0291] Additionally, in the present disclosure, “comprising at least one of a, b, or c” may mean “comprising only a, including only b, including only c, or including a combination of two or more (including a and b, including b and c, including a and c, or including all of a, b, and c).
[0292] While the detailed description of this disclosure has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.
Claims
1. A method for generating content related to the movement of a vehicle by an electronic device in a vehicle, An action of obtaining at least one user image by photographing at least one user inside the vehicle using at least one first camera facing inside the vehicle while the vehicle is moving; An action of acquiring voice data of at least one user inside the vehicle using a microphone placed inside the vehicle while the vehicle is moving; An action of identifying the purpose of generating the content related to the movement of the vehicle based on at least one user image or the voice data when a preset event occurs; An action to monitor the situation related to the movement of the above vehicle; and Based on the purpose of generating the content and the monitored situation, an operation of selecting at least one image from among the at least one user image acquired using the at least one first camera or a plurality of external images acquired using the at least one second camera facing the outside of the vehicle, and generating the content including the at least one selected image, A method for generating the content, wherein the operation of generating the content comprises generating the content by inputting at least one selected image into an artificial intelligence model trained to generate the content related to the movement of the vehicle.
2. In claim 1, An operation of controlling the driving of the vehicle to generate the content according to the generation purpose based on the purpose of generating the content and the monitored situation; An operation of acquiring the plurality of external images by using the at least one second camera of the vehicle while it is running, by the above control; How to include more.
3. In claim 1, The action of identifying the purpose of creation of the above content is: An action to recognize an object within the user image, An action to interpret the meaning of the above voice data, An operation for identifying the purpose of generating the content based on the result of recognizing the object and the meaning of the interpreted voice data, method.
4. In claim 1, The above artificial intelligence model includes a generative artificial intelligence model, The action of generating the above content is: An operation of generating an input prompt to be input into the generative artificial intelligence model based on the purpose of generating the content and the monitored situation; and comprising an action of inputting the input prompt and at least one selected image into the generative artificial intelligence model; method.
5. In claim 1, The act of connecting to an external electronic device; An action of transmitting the generated content to the external electronic device; including, method.
6. For electronic devices in vehicles, At least one first camera facing the interior of the vehicle; At least one second camera facing the exterior of the vehicle; At least one microphone placed inside the vehicle; at least one processor; and Contains memory for storing commands and artificial intelligence models, The above artificial intelligence model is configured through training for generating content related to the movement of the vehicle. The above instructions are individually or collectively executed by the at least one processor, so that the electronic device: While the vehicle is moving, at least one user image is acquired by photographing at least one user inside the vehicle using at least one first camera, While the vehicle is moving, voice data of at least one user inside the vehicle is acquired using a microphone placed inside the vehicle, When a preset event occurs, identifying the purpose of generating the content related to the movement of the vehicle based on at least one user image or the voice data, Monitor the situation related to the movement of the above vehicle, and Based on the purpose of generating the content and the monitored situation, at least one image is selected from among at least one user image acquired using the at least one first camera or a plurality of external images acquired using the at least one second camera, and the content including the selected at least one image is generated. The action of generating the above content is to generate the content by inputting at least one selected image into an artificial intelligence model trained to generate the content related to the movement of the vehicle. Electronic devices.
7. In claim 6, The above instructions are individually or collectively executed by the at least one processor, such that the electronic device: Based on the purpose of generating the above content and the monitoring situation, the driving of the vehicle is controlled to generate the above content according to the purpose of generating the above content, and By using the at least one second camera of the vehicle being driven by the above control, the plurality of external images are acquired. Electronic devices.
8. In claim 7, The above commands are individually or collectively executed by the at least one processor to cause the electronic device to change the driving properties of the vehicle as the vehicle moves to a predetermined location, The driving properties of the vehicle include at least one of the driving speed, driving direction, turning direction, driving path, whether a lane is changed, whether a waypoint is set, or whether the vehicle is stopped. Electronic devices.
9. In claim 7, The instructions are individually or collectively executed by the at least one processor to cause the electronic device to select the at least one user image or at least one image from among the plurality of external images based on a direction of the gaze of the at least one user within the vehicle while acquiring the at least one user image or at least one of the plurality of external images. Electronic devices.
10. In claim 6, The above commands are individually or collectively executed by the at least one processor to cause the electronic device to recognize an object in the user image or interpret the meaning of the voice data, and Based on the result of recognizing the object and the meaning of the interpreted voice data, the purpose of generating the content is identified. Electronic devices.
11. In claim 10, The above voice data includes a voice requesting the creation of the content, Electronic devices.
12. In claim 11, The instructions are individually or collectively executed by the at least one processor to cause the electronic device to identify a purpose of generating the content based on the at least one user's preference information determined in advance for generating the content. Electronic devices.
13. In claim 6, Information about the situation regarding the movement of the vehicle includes at least one of the user's movement purpose within the vehicle, the current movement speed of the vehicle, the weather outside the vehicle, the scenery, the current location, and whether a landmark exists within a certain distance from the current location. Electronic devices.
14. In claim 6, The above artificial intelligence model includes a generative artificial intelligence model, The above instructions are individually or collectively executed by the at least one processor, such that the electronic device: Based on the purpose of generating the content and the monitored situation, generating an input prompt to be input into the generative artificial intelligence model; and Inputting the above input prompt and at least one selected image into the generative artificial intelligence model, Electronic devices.
15. In claim 6, The above instructions are individually or collectively executed by the at least one processor, such that the electronic device: Connected to external electronic devices, and To transmit the generated content to the external electronic device, Electronic devices.
Citation Information
Patent Citations
Self-driving traveling vehicle short video automatic generation method and system and vehicle thereof
CN116132611A
On-vehicle photographing system
JP2021012072A
Driver assistance apparatus and control method for the same
KR1020160134075A
Road surface severity estimation device
KR1020240176185A
KR20190067380A