Server for providing VAR video and method of operation thereof

Abstract

A method for providing VAR images and an electronic device for performing said method are disclosed. An electronic device according to various embodiments may receive a plurality of images from a plurality of cameras, determine the type of objects included in the plurality of images based on said images, determine the position of said objects within a pre-generated 3D coordinate system, generate a VAR image based on said determined type of objects and said determined position of said objects, and transmit said generated VAR image to an external electronic device.

Description

Technology Field

[0001] Various embodiments of the present disclosure relate to a server for providing VAR images and a method of operating the same. Background Technology

[0002] Existing methods for providing VAR (video assistant referees) footage faced difficulties in rapidly delivering information to consumers because production was time-consuming and the footage could not be provided to viewers in real time. Additionally, there were issues regarding the labor required of the video provider, such as editing recorded footage or users directly creating animations to produce VAR videos.

[0003] Therefore, in order to solve the aforementioned problem, there is a need to provide a method that uses artificial intelligence to determine the location and type of an object and generates a new VAR image using only the determined location and type of the object. The problem to be solved

[0004] The object of the present application is to provide a server for providing VAR images and a method of operation thereof in order to form a platform for providing VAR images to automatically assist in judgment.

[0005] The problems that this application aims to solve are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the attached drawings. means of solving the problem

[0006] According to various embodiments, a method of operation of an electronic device may be provided, comprising: receiving a plurality of images from a plurality of cameras; determining the type of objects included in the plurality of images based on the plurality of images; determining the position of the objects within a pre-generated 3D coordinate system; generating a VAR image based on the determined type of objects and the determined position of the objects; and transmitting the generated VAR image to an external electronic device.

[0007] According to various embodiments, a server may be provided comprising: a communication circuit; a memory; and at least one processor; wherein the at least one processor receives a plurality of images from a plurality of cameras through the communication circuit, determines the type of objects included in the plurality of images based on the plurality of images, determines the position of the objects within a pre-generated 3D coordinate system, generates a VAR image based on the determined type of objects and the determined position of the objects, and transmits the generated VAR image to an external electronic device.

[0008] According to various embodiments, a non-transient recording medium for storing at least one program executable by a computer may be provided, wherein when the at least one program is executed, at least one processor is implemented to: receive a plurality of images from a plurality of cameras through the communication circuit, determine the type of objects included in the plurality of images based on the plurality of images, determine the position of the objects within a pre-generated 3D coordinate system, generate a VAR image based on the determined type of objects and the determined position of the objects, and transmit the generated VAR image to an external electronic device.

[0009] The means of solving the problem are not limited to the problems mentioned above, and other unmentioned problems will be clearly understood by those skilled in the art to which this disclosure belongs from the description below. Effects of the invention

[0010] The present invention can acquire multiple images from multiple cameras and determine the type of object and the location of the object included in the image based on the acquired multiple images. Accordingly, a VAR image can be generated based on the determined type of object and the location of the object.

[0011] In addition, by producing VAR footage in various formats as needed, it is possible to provide users with VAR footage in a format appropriate for the type of sport, and there is the advantage of saving human resources as VAR footage is generated independently of user editing. Brief explanation of the drawing

[0012] FIG. 1 is a block diagram of a system according to various embodiments. FIG. 2 is a drawing for illustrating examples of systems according to various embodiments. FIG. 3 is a block diagram showing examples of servers according to various embodiments. FIG. 4 is a block diagram showing examples of programs according to various embodiments. FIG. 5 is a block diagram showing examples of user terminals according to various embodiments. FIG. 6 is a block diagram showing examples of camera modules according to various embodiments. FIG. 7a is a drawing for illustrating a first camera according to various embodiments. FIG. 7b is a drawing for illustrating a second camera according to various embodiments. FIG. 7c is a drawing for explaining a shooting method of a plurality of camera modules according to various embodiments. FIG. 8 is a flowchart illustrating an example of a service for generating VAR images of a server according to various embodiments. FIG. 9 is a flowchart illustrating an example of a method for receiving multiple images from multiple camera modules of a server according to various embodiments. FIG. 10 is a drawing for explaining an example of a method for receiving multiple images from multiple camera modules of a server according to various embodiments. FIG. 11 is a flowchart illustrating an example of a method for determining the type of object of a server according to various embodiments. FIG. 12 is a diagram illustrating a method for determining the type of object of a server according to various embodiments. FIG. 13 is a flowchart illustrating a method for determining the location of an object on a server according to various embodiments. FIG. 14 is a diagram illustrating a method for determining the location of an object of a server according to various embodiments. FIG. 15 is a flowchart illustrating a method for generating a graphic VAR image of a server according to various embodiments. FIG. 16 is a diagram illustrating a method for generating a graphic VAR image of a server according to various embodiments. FIG. 17 is a flowchart illustrating a method for generating an overlapping VAR image of a server according to various embodiments. FIG. 18 is a diagram illustrating a method for generating an overlapping VAR image of a server according to various embodiments. FIG. 19 is a flowchart illustrating a method for transmitting a VAR image to a user terminal that has received a request from a user of a server, according to various embodiments. FIG. 20 is a diagram illustrating a method for transmitting a VAR image to a user terminal that has received a request from a user of a server, according to various embodiments. FIG. 21 is a flowchart illustrating a method for transmitting VAR images based on information associated with a user terminal of a server to a user terminal according to various embodiments. FIG. 22 is a diagram illustrating a method for transmitting VAR images to a user terminal based on information associated with a user terminal of a server, according to various embodiments. Specific details for implementing the invention

[0013] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0014] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0015] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory) or external memory that is readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of a machine (e.g., an electronic device) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0016] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0017] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0018] According to various embodiments, a method of operation of an electronic device may be provided, comprising: receiving a plurality of images from a plurality of cameras; determining the type of objects included in the plurality of images based on the plurality of images; determining the position of the objects within a pre-generated 3D coordinate system; generating a VAR image based on the determined type of objects and the determined position of the objects; and transmitting the generated VAR image to an external electronic device.

[0019] According to various embodiments, an operation method may be provided in which the operation of receiving the plurality of images includes the operation of receiving first images acquired using a plurality of fixed first cameras and the operation of receiving second images acquired using a plurality of second cameras, wherein the plurality of second cameras operate by means of a control signal transmitted from the electronic device.

[0020] According to various embodiments, a method of operation may be provided in which the operation of generating the VAR image includes the operation of generating a graphic image based on the type of object and the location of the object.

[0021] According to various embodiments, a method of operation may be provided in which the operation of generating the VAR image includes the operation of generating an image in which an auxiliary object is overlapped with at least one of the received plurality of images.

[0022] According to various embodiments, a method of operation may be provided in which the VAR image includes analysis data associated with the object.

[0023] According to various embodiments, a method of operation may be provided in which the plurality of second cameras capture a set area when the position of the object is included in a set area.

[0024] According to various embodiments, a server may be provided comprising a communication circuit, a memory, and at least one processor, wherein the at least one processor receives a plurality of images from a plurality of cameras through the communication circuit, determines the type of objects included in the plurality of images based on the plurality of images, determines the position of the objects within a pre-generated 3D coordinate system, generates a VAR image based on the determined type of objects and the determined position of the objects, and transmits the generated VAR image to an external electronic device.

[0025] According to various embodiments, the at least one processor is implemented to receive first images acquired using a plurality of fixed first cameras and second images acquired using a plurality of second cameras as at least part of the operation of receiving the plurality of images, and the plurality of second cameras may be provided with a server that operates by a control signal transmitted from the electronic device.

[0026] According to various embodiments, the at least one processor may be provided with a server implemented to generate a graphic image based on the type of object and the location of the object as at least part of the operation of generating the VAR image.

[0027] According to various embodiments, the at least one processor may be provided with a server implemented to generate an image in which an auxiliary object is overlapped on at least one of the received plurality of images as at least part of the operation of generating the VAR image.

[0028] According to various embodiments, the VAR image may be provided with a server including analysis data associated with the object.

[0029] According to various embodiments, a server may be provided that captures the plurality of second cameras including the set area when the location of the object is included in the set area.

[0030] According to various embodiments, a non-transient recording medium for storing at least one program executable by a computer may be provided, wherein when the at least one program is executed, at least one processor is implemented to: receive a plurality of images from a plurality of cameras through the communication circuit, determine the type of objects included in the plurality of images based on the plurality of images, determine the position of the objects within a pre-generated 3D coordinate system, generate a VAR image based on the determined type of objects and the determined position of the objects, and transmit the generated VAR image to an external electronic device.

[0031] According to various embodiments, the system (10) receives a plurality of images (e.g., a first image, a second image) from a plurality of camera modules (300) using a server (100), determines the type of object (e.g., a first object type (1200), a second object type (1210)) and the location of the object (location of the first object (1420), location of the second object (1430)) from the plurality of images (e.g., a first image (1000a), a second image (1000b)), and based on the type of object (a first object type (1200), a second object type (1210)) and the location of the object (location of the first object (1410), location of the second object (1420)), a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR It may be a system (10) implemented to generate an image (2200a) and a second VAR image (2200b).

[0032] The above object (700) may include people (e.g., players, referees, spectators), items (e.g., balls, rackets, goalposts), and / or fields (e.g., sports fields, indoor gymnasiums) included in the plurality of images (e.g., first image (1000a), second image (1000b)), but is not limited to the examples described and may include objects necessary for producing VAR images of various types of sports. Below, examples of the system (10) according to various embodiments will be further described.

[0033] FIG. 1 is a block diagram illustrating examples of a system (10) according to various embodiments. FIG. 1 will be described below with reference to FIG. 2.

[0034] FIG. 2 is a drawing for explaining examples of a system (10) according to various embodiments.

[0035] Referring to FIG. 1, the system (10) may include a server (100), a user terminal (200), and a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)). A server (100), a user terminal (200), and / or a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) included in the system (10) may transmit / receive a plurality of images (e.g., a first image (1000a), a second image (1000b)) and / or VAR images (e.g., a graphic VAR image (1600a), an overlapping VAR image (1600b), or a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)) or provide a predetermined service based on the activation of a communication function with each other.

[0036] According to various embodiments, a server (100), with reference to FIG. 2, receives a plurality of images (e.g., a first image (1000a), a second image (1000b)) from a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)), stores them in the server (100), and uses the plurality of images (e.g., a first image (1000a), a second image (1000b)) to determine the type of object (e.g., a first object type (1200), a second object type (1210)) and the location of the object (location of the first object (1410), location of the second object (1420)). In addition, using the types of objects (e.g., types of first objects (1200), types of second objects (1210)) and the locations of objects (locations of first objects (1410), locations of second objects (1420)), VAR images (e.g., graphic VAR images (1600a), overlap VAR images (1600b), and modified VAR images (e.g., first VAR images (2200a), second VAR images (2200b)) can be generated. The detailed operation of the server (100) will be described later with reference to FIGS. 8 to 22.

[0037] According to various embodiments, the user terminal (200) may be a user terminal (200) of a producer (or consumer) of a VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). The user terminal (200) receives the VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) from the server (100) and can play the VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). For example, a user A terminal (200) receives a plurality of VAR images (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), and modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b)), and can transmit a request to a server (100) to play at least one of the plurality of VAR images (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), and modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b)). For example, using an application (240) installed on a user terminal (200), at least one VAR An interface for requesting playback of a video (e.g., graphic VAR video (1600a), overlapping VAR video (1600b), or modified VAR video (e.g., first VAR video (2200a), second VAR video (2200b)) may be provided on the display (250) of the user terminal (200).The detailed operation of the user terminal (200) will be described later with reference to FIGS. 19 and FIGS. 20.

[0038] Additionally, the user terminal (200) may transmit to the server (100) a request to change at least one of the plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). At this time, using an application (240) installed on the user terminal (200), an interface for requesting to change at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) is provided by the user It may be provided on the display (250) of the terminal (200). The detailed operation of the user terminal (200) will be described later with reference to FIGS. 19 and FIGS. 20.

[0039] According to various embodiments, a plurality of camera modules (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) can capture a plurality of images (e.g., a first image (1000a), a second image (1000b)) and transmit them to a server (100), with reference to FIG. 2. Details regarding the operation of the plurality of camera modules (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) will be described later with reference to FIG. 7c. Details regarding the plurality of images (e.g., a first image (1000a), a second image (1000b)) will be described later with reference to FIG. 10.

[0040] FIG. 3 is a block diagram showing examples of a server (100) according to various embodiments.

[0041] According to various embodiments, the server (100) may include a first processor (110), a first memory (120) containing a program (140), and a first communication circuit (130). However, the server (100) may be implemented to include more devices and / or fewer devices, not limited to the described and / or illustrated examples.

[0042] According to various embodiments, the first processor (110) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the first processor (110) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the first processor (110) can store commands or data received from other components (e.g., VAR image generation module (146), VAR image determination module (147)) in a volatile first memory (120), process the commands or data stored in the first memory (120), and store result data in the first memory (120). According to one embodiment, the first processor (110) may include a main processor (not shown) (e.g., a central processing unit or an application processor) or an auxiliary processor (not shown) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the first processor (110) includes a main processor (not shown) and an auxiliary processor (not shown), the auxiliary processor (not shown) may be configured to use less power than the main processor (not shown) or to be specialized for a designated function. The auxiliary processor (not shown) may be implemented separately from the main processor (not shown) or as part thereof.

[0043] An auxiliary processor (not shown) may control at least some of the functions or states associated with at least one component of the server (100) (e.g., first memory (120), first communication circuit (130)) on behalf of the main processor (not shown) while the main processor (not shown) is in an inactive (e.g., sleep) state, or together with the main processor (not shown) while the main processor (not shown) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (not shown) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (not shown)). According to one embodiment, the auxiliary processor (not shown) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the server (100) itself where the artificial intelligence is performed, or through a separate server. Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above. An artificial intelligence model may include multiple artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0044] According to various embodiments, the first memory (120) may store various data used by at least one component of the server (100) (e.g., processor (110)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The first memory (120) may include volatile memory (not shown) or non-volatile memory (not shown).

[0045] According to various embodiments, the program (140) may be stored as software in the first memory (120) and, when executed, may cause the first processor (110) to perform at least one operation. Examples of functions provided by the program (140) will be described later, and the operation of the modules of the program (140) may be understood as the operation of the first processor (110) based on the execution of the program (140).

[0046] According to various embodiments, the first communication circuit (130) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the server (100) and an external electronic device (e.g., user terminal (200), camera module (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2))) and the performance of communication through the established communication channel. The first communication circuit (130) may include one or more communication processors that operate independently of the first processor (110) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the first communication circuit (130) may include a wireless communication module (not shown) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (not shown) (e.g., LAN (local area network) communication module, or power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (e.g., user terminal (200)) through a first network (not shown) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (not shown) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). A wireless communication module (not shown) can identify or authenticate a server (100) within a communication network such as the first network (not shown) or the second network (not shown) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in a subscriber identification module (not shown).

[0047] FIG. 4 is a block diagram showing examples of a program (140) according to various embodiments.

[0048] According to various embodiments, the program (140) may include a 3D coordinate system generation module (141), an object type generation module (142), an object location determination module (143), a shooting signal generation module (144), a data analysis module (145), a VAR image generation module (146) and / or a VAR image determination module (147). The plurality of modules may be implemented as executable computer code, instructions, and / or APIs.

[0049] According to various embodiments, a 3D coordinate system generation module (141) can acquire a plurality of first images (1000a-1, 1000a-2) from a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) using a communication circuit (130). At this time, the 3D coordinate system generation module (141) can generate a virtual 3D coordinate system using the plurality of first images (1000a-1, 1000a-2). For example, the 2D coordinates of the plurality of first images (1000a-1, 1000a-2) can be acquired using the shooting positions of the plurality of first images (1000a-1, 1000a-2). At this time, a reference point (1410) is determined, and a virtual 3D coordinate system for the plurality of first images (1000a-1, 1000a-2) can be generated using the distance between the 2D coordinates. However, the generation method is not limited to the described example and may include a wider variety of generation methods.

[0050] According to various embodiments, the object type determination module (142) can determine the type of an object (e.g., the type of a first object (1200), the type of a second object (1210)) based on a plurality of first images (1000a-1, 1000a-2). For example, the object type determination module (142) can determine the type of a first object (1200) using a first artificial intelligence model on the plurality of first images (1000a-1, 1000a-2), and determine the type of a second object (1210) using a second artificial intelligence model on the plurality of first images (1000a-1, 1000a-2). For example, the object type determination module (142) can determine the type of a first object (1200) and the type of a second object (1210) using a third artificial intelligence model. However, the determination method for determining the type of the described object (e.g., type of the first object (1200), type of the second object (1210)) is not limited to a variety of other determination methods. Details of the operation for determining the type of the object (e.g., type of the first object (1200), type of the second object (1210)) will be described later with reference to FIGS. 11 and 12.

[0051] According to various embodiments, the object position determination module (143) can determine the position of an object (e.g., the position of a first object (1420), the position of a second object (1430)) in a pre-generated 3D coordinate system (1400). For example, the object position determination module (143) can identify an object (700) within the 3D coordinate system (1400) and determine the position of the object (e.g., the position of a first object (1420), the position of a second object (1430)) by matching coordinates corresponding to the 3D coordinate system. However, the method for determining the position of the object (e.g., the position of a first object (1420), the position of a second object (1430)) is not limited to a variety of other determination methods. An operation to determine the location of the above object (e.g., the location of the first object (1420), the location of the second object (1430)) will be described later with reference to FIGS. 13 and 14.

[0052] According to various embodiments, the shooting signal generation module (144) receives a type of object (e.g., a type of first object (1200), a type of second object (1210)) and a location of an object (e.g., a location of first object (1410), a location of second object (1420)), and based on the received type of object (e.g., a type of first object (1200), a type of second object (1210)) and location of an object (e.g., a location of first object (1410), a location of second object (1420)), it can transmit a shooting control signal to a second camera (300b-1, 300b-2). For example, the shooting signal generation module (144) can predict the movement path of an object (700) using the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)), and can transmit a shooting control signal to the second camera (300b-1, 300b-2) to produce a natural image according to the movement path. However, the operation of transmitting a shooting signal using the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) is not limited to the operation of transmitting a shooting signal using the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)), and may include a shooting signal transmission operation for a wider variety of purposes.

[0053] According to various embodiments, the data analysis module (145) can analyze data using the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)). For example, the data analysis module (145) can obtain the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)), determine the positional relationship between a plurality of object locations (e.g., location of first object (1410), location of second object (1420)), and analyze the judgment result according to the set rule or the behavior of the object (700) through the determined positional relationship. However, the operation of analyzing data using the above-mentioned types of objects (e.g., types of first objects (1200), types of second objects (1210)) and locations of objects (e.g., locations of first objects (1410), locations of second objects (1420)) is not limited to the operation of analyzing data, and various operations for analyzing data may be included. The detailed operation of the data analysis module (145) will be described later with reference to FIGS. 15 to 18.

[0054] According to various embodiments, the VAR image generation module (146) can generate a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), and a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)) using the type of object (e.g., a first object type (1200), a second object type (1210)) and the location of the object (e.g., a first object location (1410), a second object location (1420)). For example, the VAR image generation module (146) can generate a graphic VAR image (1600a) using the type of object (e.g., a first object type (1200), a second object type (1210)) and the location of the object (e.g., a first object location (1410), a second object location (1420)). The graphic VAR The image (1600a) may be a computer graphics image generated regardless of the first image (1000a-1, 1000a-2) or the second image (1000b-1, 1000b-2) acquired from the camera module (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)). Additionally, the VAR image generation module (146) may generate an overlapping VAR image using the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)). The overlapping VAR image (1600b) may be generated by the camera module (300) (e.g., first camera (300a-1, 300a-2), second The image may be generated by overlapping an auxiliary object (1720) on a first image (1000a-1, 1000a-2) or a second image (1000b-1, 1000b-2) obtained from a camera (300b-1, 300b-2) based on a set rule or action.Additionally, the VAR image generation module (146) can generate a new VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) by modifying the VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). Details of the operation of the VAR image generation module (146) will be described later with reference to FIGS. 15 to 22.

[0055] According to various embodiments, the VAR image determination module (147) can determine the VAR image to be transmitted to the user terminal (200). For example, the VAR image determination module (147) may determine at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) among a plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) based on at least one of the network available resources of the user terminal (200), hardware information of the user terminal (200), or whether the user terminal (200) subscribes to a program. However, the above VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR images (e.g., first VAR image (2200a), second VAR image) The elements for determining the image (2200b) are not limited to various elements and may include various elements. The operation of the VAR image determination module (147) will be described later with reference to FIGS. 21 and 22.

[0056] FIG. 5 is a block diagram showing examples of user terminals (200) according to various embodiments.

[0057] According to various embodiments, the user terminal (200) may include a second processor (210), a second communication circuit (220), a second memory (230), an application (240) stored in the second memory, a display (250), and an input module (260). However, the user terminal (200) may be implemented to include more devices and / or fewer devices, provided that it is not limited to the described and / or illustrated examples.

[0058] According to various embodiments, the second processor (210), the second communication circuit (220), and the second memory (230) can each be implemented as described above as the first processor (110), the first communication circuit (130), and the first memory (120), so redundant descriptions are omitted.

[0059] According to various embodiments, the application (240) may be implemented in various forms. In one embodiment, when the application (240) is implemented as an on-device type, the application (240) may be implemented to perform an operation similar and / or identical to the operation of the aforementioned program (140). In another embodiment, when the application (240) is implemented as a server type, the application (240) may be implemented to include a function for simply receiving VAR images and / or a function for transmitting a request to the server (100) to change VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), or modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b)). Accordingly, based on the execution of the aforementioned program (140) on the server, as VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), or modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) received from the server (100) are provided through the application (240), a plurality of VAR images (e.g., graphic At least one of the following transmissions may be performed: a VAR image (1600a), an overlapping VAR image (1600b), and a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)).

[0060] According to various embodiments, the display (250) may visually provide information to the outside of the user terminal (200) (e.g., user). The display (250) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display (250) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0061] The input module (260) can receive commands or data to be used for a component of the user terminal (200) (e.g., the second processor (210)) from outside the user terminal (200) (e.g., the user). The input module (260) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0062] FIG. 6 is a block diagram showing examples of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) according to various embodiments.

[0063] According to various embodiments, a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) may include a shooting device (301), a fixing device (350), and an actuator (360).

[0064] According to various embodiments, the imaging device (301) may include a third processor (310), an image sensor (320), a shutter (340), and a third communication circuit (330).

[0065] According to various embodiments, the imaging device (301) can capture a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) and transmit the plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) to a server (100) using a third communication circuit (330).

[0066] According to various embodiments, the third processor (310) and the second communication circuit (330) can each be implemented as described above as the first processor (110) and the first communication circuit (130), so redundant descriptions are omitted.

[0067] According to various embodiments, the image sensor (320) can generate a plurality of images (e.g., first image (1000a-1, 1000a-2), second image (1000b-1, 1000b-2)) by converting the intensity of light into an electrical signal.

[0068] According to various embodiments, the shutter (340) can adjust the clarity of a plurality of images (e.g., first image (1000a-1, 1000a-2), second image (1000b-1, 1000b-2)) by adjusting the speed. For example, the shutter (340) can increase the speed of the shutter (340) in the set area (710) to take a picture when the object (700) is in the set area (710) in order to obtain the exact location of the object (e.g., location of the first object (1410), location of the second object (1420)).

[0069] According to various embodiments, the fixing device (350) can fix the shooting device (301) to specify the shooting direction and shooting angle.

[0070] According to various embodiments, the actuator (360) can move the shooting device (301) to change the shooting direction and shooting angle.

[0071] FIG. 7a is a drawing for explaining a first camera (300a-1, 300a-2) according to various embodiments.

[0072] According to various embodiments, the first camera (300a-1, 300a-2) may include a shooting device (301) and a fixing device (350).

[0073] Redundant descriptions of the shooting device (301) and the fixing device (350) are omitted.

[0074] According to various embodiments, the first camera (300a-1, 300a-2) may be fixed to a fixed device (350) to capture a certain area and generate a first image (1000a-1, 1000a-2). For example, the first camera (300a-1, 300a-2) may be fixed to capture a certain area to serve as a reference for the position of a camera module (300) that generates a 3D coordinate system (1400). Accordingly, a first image (1000a-1, 1000a-2) containing a first area for determining the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)).

[0075] FIG. 7b is a drawing for illustrating a second camera (300b-1, 300b-2) according to various embodiments.

[0076] According to various embodiments, the second camera (300b-1, 300b-2) may include a shooting device (301) and an actuator (360).

[0077] Redundant descriptions of the imaging device (301) and actuator (360) are omitted.

[0078] According to various embodiments, the second camera (300b-1, 300b-2) can generate a second image (1000b-1, 1000b-2) by using an actuator (360) to take a picture with a shooting direction and shooting angle corresponding to a control signal received from the server (100). Additionally, the second camera (300b-1, 300b-2) may be a camera having higher clarity or resolution than the first camera (300a-1, 300a-2).

[0079] FIG. 7c is a drawing for explaining a shooting method of a plurality of camera modules (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) according to various embodiments.

[0080] According to various embodiments, a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) can transmit a plurality of images (e.g., a first image (1000a-1, 1000a-2), a second image (1000b-1, 1000b-2)) to a server (100). Accordingly, the shooting method of the camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) can be changed according to a control signal generated according to the transmitted plurality of images (e.g., a first image (1000a-1, 1000a-2), a second image (1000b-1, 1000b-2)). For example, the first camera (300a-1, 300a-2) can capture a fixed and set area (710) and transmit the first image (1000a-1, 1000a-2) containing the set area to the server (100). At this time, if the server (100) identifies that an object (700) exists in the set area (710), it can transmit a control signal to the second camera (300b-1, 300b-2) to capture the set area (710) with high resolution or high clarity. Accordingly, when the second camera (300b-1, 300b-2) receives the control signal, the second camera (300b-1, 300b-2) can change the shooting direction or shooting angle of the shooting device (301) using the actuator (360) and adjust at least one of the shutter (340) or image sensor (320) to capture the set area (710) with high resolution or high clarity. However, the operation of the plurality of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) is not limited to the illustrated operation, and more operations may be performed and / or fewer operations may be performed.

[0081] FIG. 8 is a flowchart illustrating an example of a service for generating VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations shown and / or described, and more operations may be performed and / or fewer operations may be performed.

[0082] According to various embodiments, the server (100) can receive a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) from a plurality of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) in operation 801. The operation of receiving the plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) is described in detail later in operations 901 to 903.

[0083] According to various embodiments, the server (100) can determine the type of object (e.g., type of first object (1200), type of second object (1210)) based on a plurality of images (e.g., first image (1000a-1, 1000a-2), second image (1000b-1, 1000b-2)) in operation 803. The operation of determining the type of object (e.g., type of first object (1200), type of second object (1210)) is described in detail later in operations 1101 to 1103.

[0084] According to various embodiments, the server (100) can determine the location of an object (e.g., the location of a first object (1410), the location of a second object (1420)) within a pre-generated 3D coordinate system (1400) in operation 805. The operation of determining the location of the object (e.g., the location of a first object (1410), the location of a second object (1420)) is described in detail later in operations 1301 to 1303.

[0085] According to various embodiments, the server (100) can generate VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) based on the type of object (e.g., first object type (1200), second object type (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) in operation 807. The operation of generating the above VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) is described in detail below in operations 1501 to 1503 and operations 2101 to 2103.

[0086] According to various embodiments, the server (100) may transmit the generated VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to an external electronic device in operation 809. The operation of transmitting the VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to an external electronic device is described in detail later in operations 2105 to 2107.

[0087] FIG. 9 is a flowchart illustrating an example of a method for receiving a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) from a plurality of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations shown and / or described, and more operations may be performed and / or fewer operations may be performed. FIG. 9 will be described below with reference to FIG. 10.

[0088] FIG. 10 is a drawing for illustrating an example of a method for receiving a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) from a plurality of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) of a server (100) according to various embodiments.

[0089] According to various embodiments, the server (100) may receive a plurality of first images (1000a-1, 1000a-2) from a plurality of first cameras (300a-1, 300a-2) in operation 901. For example, with reference to FIG. 10, the server (100) may receive a first image (1000a-1) containing a first area from a first camera (300a-1) capturing a first area, and receive a second image (1000a-2) containing a second area from a first camera (300a-2) capturing a second area.

[0090] According to various embodiments, the server (100) may receive a plurality of second images (1000b-1, 1000b-2) from a plurality of second cameras (300b-1, 300b-2) in operation 903. For example, with reference to FIG. 10, the server (100) may receive a second image (1000b-1) including a third area from a second camera (300b-1) that captures a third area that is dynamically changed according to a control signal of the server (100), and may receive a second image (1000b-2) including a fourth area from a second camera (300a-2) that captures a fourth area that is dynamically changed according to a control signal of the server (100).

[0091] FIG. 11 is a flowchart illustrating an example of a method for determining the type of object of a server (100) (e.g., a first type of object (1200), a second type of object (1210)) according to various embodiments. The actions may be performed regardless of the order of the actions shown and / or described, and more or fewer actions may be performed. FIG. 11 will be described below with reference to FIG. 12.

[0092] FIG. 12 is a drawing for explaining a method of determining the type of object of a server (100) (e.g., the type of a first object (1200), the type of a second object (1210)) according to various embodiments.

[0093] According to various embodiments, the server (100) can determine the type of the first object (1200) based on the first image (1000a-1, 1000a-2) in operation 1101. For example, the server (100) can determine the object (700) that is of the type of person among the objects (700) included in the first image (1000a-1, 1000a-2) by using an artificial intelligence model for determining a person among the types of objects (e.g., type of the first object (1200), type of the second object (1210)) in the first image (1000a-1, 1000a-2) with reference to FIG. 12. The operation of determining the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) using the above artificial intelligence model can utilize well-known artificial intelligence models such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, and Mask R-CNN, so a detailed description is omitted.

[0094] According to various embodiments, the server (100) can determine the type of the second object (1210) based on the first image (1000a-1, 1000a-2) in operation 1103. For example, the server (100) can determine the type of the second object (700) among the objects (700) included in the first image (1000a-1, 1000a-2) by using an artificial intelligence model for determining the type of object (e.g., type of the first object (1200), type of the second object (1210)) in the first image (1000a-1, 1000a-2) with reference to FIG. 12. The operation of determining the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) using the above model can utilize well-known artificial intelligence models such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, and Mask R-CNN, so a detailed description is omitted.

[0095] According to various embodiments, the artificial intelligence model for determining the type of the first object (1200) and the type of the second object (1210) may be the same artificial intelligence model. In this case, operations 1101 and 1103 may be performed at the same time.

[0096] FIG. 13 is a flowchart illustrating a method for determining the location of an object of a server (100) (e.g., the location of a first object (1410), the location of a second object (1420)) according to various embodiments. The actions may be performed regardless of the order of the actions shown and / or described, and more or fewer actions may be performed. FIG. 13 will be described below with reference to FIG. 14.

[0097] FIG. 14 is a drawing for explaining a method of determining the location of an object of a server (100) (e.g., the location of a first object (1410), the location of a second object (1420)) according to various embodiments.

[0098] According to various embodiments, the server (100) can identify an object (700) from a pre-generated 3D coordinate system (1400) in operation 1301. For example, with reference to FIG. 14, the server (100) can identify a person and a ball as objects (700) to locate in the pre-generated 3D coordinate system (1400). Since the operation of identifying the object (700) can utilize well-known technologies such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, Mask R-CNN, etc., a detailed description is omitted.

[0099] According to various embodiments, the server (100) can determine the position of an object (700) within a pre-generated 3D coordinate system (1400) in operation 1303. For example, with reference to FIG. 14, the server (100) may include a reference point (1410) in the pre-generated 3D coordinate system (1400). At this time, the x and y coordinates of the plurality of first images (1000a-1, 1000a-2) can be obtained by using the positions of the plurality of first cameras (300a-1, 300a-2) stored in the plurality of first images (1000a-1, 1000a-2). Accordingly, the z coordinate can be obtained by using the reference point (1410), x coordinate, and y coordinate, and the total coordinates of the plurality of first images (1000a-1, 1000a-2) at a specific point in time can be determined. The position of the object (700) can be determined by matching it with the overall coordinates (e.g., the position of the first object (1410), the position of the second object (1420)). Since the operation of determining the 3D coordinates can utilize well-known technologies such as programming languages, triangulation, 3D scanning, LiDAR, ToF (Time of Flight), and stereo vision, a detailed description is omitted.

[0100] FIG. 15 is a flowchart illustrating a method for generating a graphic VAR image (1600a) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the illustrated and / or described operations, and more operations may be performed and / or fewer operations may be performed. FIG. 15 will be described below with reference to FIG. 16.

[0101] FIG. 16 is a drawing for explaining a method of generating a graphic VAR image (1600a) of a server (100) according to various embodiments.

[0102] According to various embodiments, the server (100) can analyze data based on the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) in operation 1501. For example, with reference to FIG. 16, the server (100) can analyze that a goal event has occurred when the location of a ball and the location of a goalpost among a plurality of objects (700) are determined, and the determined location of the ball has completely crossed the line of the goalpost. Also, for example, the server (100) can analyze that a foul event has occurred when the locations of a plurality of people among a plurality of objects (700) overlap, by determining the location between people and the feature point information of the people according to the time. However, the operation of data analysis described above is not limited to the above, and may include a wider variety of data analysis operations that can be performed based on the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) after pre-setting rules and actions. Since the operation of recognizing the gesture may utilize well-known gesture recognition technologies, such as vision-based gesture recognition, a detailed description is omitted. Furthermore, since the operation of determining the feature point information may utilize well-known technologies such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), Gabor filter, Harris corner detector, LBP (Local Binary Pattern), and HOG (Histogram of Oriented Gradients), a detailed description is omitted.

[0103] According to various embodiments, the server (100) can generate a graphic VAR image (1600a) based on the type of object (e.g., type of first object (1200), type of second object (1210)), the location of the object (e.g., location of first object (1410), location of second object (1420)) and analysis data (1620) in operation 1503. For example, the server (100), referring to FIG. 16, can generate a graphic VAR image (1600a) including a graphic (1610) corresponding to the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) using a first artificial intelligence model. At this time, the first artificial intelligence model can be trained to generate a graphic VAR video (1600a) that includes a graphic (1610) corresponding to the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)). Also, for example, if the type of first object (1200) is a goalpost and the type of second object (1210) is a ball, the server (100) can use the first artificial intelligence model to generate a graphic VAR video (1600a) that includes a graphic (1610) corresponding to the ball and a graphic (1610) corresponding to the goalpost. At this time, the graphic VAR video (1600a) can be modified by including analysis data (1620) that analyzed the goal event analyzed in operation 1501. However, if the analysis data (1620) is not included in the graphic VAR image (1600a), operation 1501 may be omitted.The above artificial intelligence model may utilize well-known artificial intelligence models such as real-time graphics, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Image-to-Image Translation, Pix2Pix, CycleGAN, Neural Style Transfer, and 3D ShapeNets, so a detailed description is omitted.

[0104] FIG. 17 is a flowchart illustrating a method for generating an overlap VAR image (1600b) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the illustrated and / or described operations, and more operations may be performed and / or fewer operations may be performed. FIG. 17 will be described below with reference to FIG. 18.

[0105] FIG. 18 is a drawing for explaining a method of generating an overlapping VAR image (1600b) of a server (100) according to various embodiments.

[0106] According to various embodiments, the server (100) can analyze data based on the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) in operation 1701. Since operation 1701 can be implemented as in operation 1501, a redundant description is omitted.

[0107] According to various embodiments, the server (100) can generate an overlapping VAR image (1600b) based on the type of object (e.g., type of first object (1200), type of second object (1210)), the location of the object (e.g., location of first object (1410), location of second object (1420)) and analysis data (1620) in operation 1703. For example, the server (100) can generate an overlapping VAR image (1600b) by using a second artificial intelligence model, with reference to FIG. 18, to overlap an auxiliary object (1720) with an object (700) included in the received second image (1000b-1, 1000b-2). At this time, the server (100) can train the second artificial intelligence model to overlap the type and location of the auxiliary object (1720) based on a set rule or the behavior of the object (700). For example, the server (100) can generate an overlapping VAR image (1600b) by using the second artificial intelligence model, where the type of the first object (1200) is a ball and the type of the second object (1210) is a person, in which an auxiliary object (1720) with a shape that surrounds the ball is overlapped and an auxiliary object (1720) connecting feature points is overlapped on the person. However, the example described is not limited to the example, and a wider variety of shapes of auxiliary objects (1720) to aid in judgment may be included. At this time, the overlapping VAR image (1600b) may be modified by including analysis data (1620) that analyzes the goal event analyzed in operation 1701. However, if the analysis data (1620) is not included in the overlapping VAR image (1600b), operation 1701 may be omitted.The above artificial intelligence model may utilize well-known artificial intelligence models such as real-time graphics, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Image-to-Image Translation, Pix2Pix, CycleGAN, Neural Style Transfer, and 3D ShapeNets, so a detailed description is omitted.

[0108] FIG. 19 is a flowchart illustrating a method for transmitting VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) to a user terminal (200) for which a user of the server (100) has requested reception, according to various embodiments. Operations may be performed regardless of the order of the illustrated and / or described operations, and more operations may be performed and / or fewer operations may be performed. FIG. 19 will be described below with reference to FIG. 20.

[0109] FIG. 20 is a diagram illustrating a method for transmitting VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) to a user terminal (200) according to various embodiments, for which a user of the server (100) has requested reception.

[0110] According to various embodiments, when the server (100) receives a request for reception of a specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) of a user terminal (200) in operation 1901, the server may determine the specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))). For example, the server (100), referring to FIG. 20, may receive a request for a specific VAR image associated with a specific rule (e.g., graphic VAR image (1600a), overlap VAR image (1600b), change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) based on the input of the selection interface of the user terminal (200). For example, with reference to FIG. 20, when the server (100) receives a request for a VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) for assisting reading of the goal line based on the input of the goal line interface among the selection interfaces, it can determine the VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) capable of assisting reading of the goal line.For example, when the server (100) receives a request to receive a VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) with the type of VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))), whether an auxiliary object (1720) is included, and whether analysis data (1620) is included, based on the input of a setting interface among selection interfaces, the VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) The type of image (2200b), whether an auxiliary object (1720) or analysis data (1620) is included, can be determined as a specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) with the inclusion of an auxiliary object (1720) or analysis data (1620).

[0111] According to various embodiments, the server (100) can transmit the VAR image determined in operation 1901 (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to the user terminal (200) in operation 1903. For example, with reference to FIG. 20, the server (100) can transmit the VAR image capable of assisting in reading a goal line (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to the user terminal (200) when a specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) is determined as a specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) is determined as a VAR image capable of assisting in reading a goal line (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)))Also, for example, if the server (100) determines a specific VAR image (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) in which the analysis data (1620) has been removed from the graphic VAR image (1600a) capable of assisting in reading the goal line (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))), the server determines a VAR image in which the analysis data (1620) has been removed from the graphic VAR image (1600a) capable of assisting in reading the goal line (e.g., graphic VAR image (1600a), overlapping VAR image (1600b), modified VAR image (e.g., first VAR The image (2200a) and the second VAR image (2200b) can be transmitted to the user terminal (200).

[0112] FIG. 21 is a flowchart illustrating a method for transmitting VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) based on information associated with a user terminal (200) of a server (100) according to various embodiments to a user terminal (200). Operations may be performed regardless of the order of the illustrated and / or described operations, and more operations may be performed and / or fewer operations may be performed. FIG. 21 will be described below with reference to FIG. 22.

[0113] FIG. 22 is a diagram illustrating a method for transmitting VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) based on information associated with a user terminal (200) of a server (100) according to various embodiments to a user terminal (200).

[0114] According to various embodiments, information associated with a first user terminal (200a) and a second user terminal (200b) can be identified. For example, the server (100) can identify at least one element of the operating system, application (240) settings, CPU, RAM, memory, battery, network type, network speed, network available resources, subscription status, or security of the first user terminal (200a), and at least one element of the operating system, application (240) settings, CPU, RAM, memory, battery, network type, network speed, network available resources, subscription status, or security of the second user terminal (200b). However, the examples described are not limited to, and more diverse information associated with the user terminal (200) can be identified.

[0115] According to various embodiments, the server (100) can generate a first VAR image (2200a) and a second VAR image (2200b) based on identified information in operation 2103. For example, if the server (100) identifies the network speed of the first user terminal (200a) with reference to FIG. 22, it can generate a first VAR image (2200a) by compressing the capacity of the graphic VAR image (1600a) to a capacity corresponding to the network speed. Also, for example, if the server (100) identifies whether the second user terminal (200b) is subscribed with reference to FIG. 22 and identifies that the second user terminal is not subscribed, it can generate a second VAR image by inserting an advertisement into the graphic VAR image (1600a). However, the operation of generating the first VAR image (2200a) and the second VAR image (2200b) described is not limited to examples, and the first VAR image (2200a) and the second VAR image (2200b) corresponding to information associated with the user terminal (200) can be generated.

[0116] According to various embodiments, the server (100) can transmit the generated first VAR image (2200a) to the first user terminal (200a) in operation 2105. For example, with reference to FIG. 22, the server (100) can transmit the first VAR image (2200a) to the first user terminal (200a) if the first VAR image (2200a) is a graphic VAR image (1600a) compressed in correspondence with the network speed of the first user terminal (200a).

[0117] According to various embodiments, the server (100) may transmit the generated second VAR image (2200b) to the second user terminal (200b) in operation 2107. For example, with reference to FIG. 22, the server (100) may transmit the second VAR image (2200b) to the first user terminal (200a) if the second VAR image (2200b) is a graphic VAR image (1600a) with an advertisement inserted corresponding to the non-subscription status of the second user terminal (200b). By transmitting a first VAR image (2200a) to the first user terminal (200a) and transmitting a second VAR image (2200b) to the second user terminal (200b), different VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b))) are provided to each user, thereby enabling a variety of customized images to be provided.

Claims

Claim 1 A method of operation of an electronic device comprises: receiving a plurality of first images acquired using a plurality of fixed first cameras; receiving a plurality of second images acquired using a plurality of second cameras whose operation is controlled by the electronic device; determining the type of objects included in the plurality of first images based on the plurality of first images; determining the position of the objects within a pre-generated 3D coordinate system; generating a VAR image from the plurality of second images based on the determined type of objects and the determined position of the objects; and transmitting the generated VAR image to an external electronic device; wherein the operation of determining the position of the objects includes: determining the coordinates of the plurality of first images within the 3D coordinate system based on the positions of the plurality of first cameras; and matching the coordinates of the object included in at least one of the plurality of first images within the 3D coordinate system using the coordinates of the plurality of first images. Claim 2 delete Claim 3 In claim 1, the operation of generating the VAR image includes the operation of generating a graphic image based on the type of object and the location of the object. Claim 4 A method of operation according to claim 1, wherein the operation of generating the VAR image includes the operation of generating an image in which an auxiliary object is overlapped on at least one of the received plurality of images. Claim 5 A method of operation according to either claim 3 or 4, wherein the VAR image includes analysis data associated with the object. Claim 6 A method of operation in which, in claim 1, the plurality of second cameras photograph including the set area when the position of the object is included in the set area. Claim 7 A server comprising: a communication circuit; a memory; and at least one processor; wherein the at least one processor receives a plurality of first images acquired using a plurality of first cameras fixed through the communication circuit; receives a plurality of second images acquired using a plurality of second cameras whose operation is controlled by the server; determines the type of objects included in the plurality of first images based on the plurality of first images and determines the position of the objects within a pre-generated 3D coordinate system; generates a VAR image from the plurality of second images based on the determined type of objects and the determined position of the objects; and transmits the generated VAR image to an external electronic device; wherein the at least one processor determines the coordinates of the plurality of first images within the 3D coordinate system based on the positions of the plurality of first cameras, and uses the coordinates of the plurality of first images to match the coordinates of the object included in at least one of the plurality of first images within the 3D coordinate system. Claim 8 delete Claim 9 In claim 7, the at least one processor is implemented to generate a graphic image based on the type of object and the location of the object as at least part of the operation of generating the VAR image, a server. Claim 10 In claim 7, the at least one processor is implemented to generate an image in which an auxiliary object is overlapped on at least one of the received plurality of images as at least part of the operation of generating the VAR image, the server. Claim 11 In either of claims 9 and 10, the VAR image comprises analysis data associated with the object, on a server. Claim 12 In claim 7, the plurality of second cameras capture including the set area when the location of the object is included in the set area, the server. Claim 13 A non-transient recording medium for storing at least one program executable by a computer, wherein when the at least one program is executed, at least one processor is implemented to: receive a plurality of first images acquired using a plurality of first cameras fixed through a communication circuit, and receive a plurality of second images acquired using a plurality of second cameras whose operation is controlled by the at least one processor; determine the type of objects included in the plurality of first images based on the plurality of first images, determine the position of the objects within a pre-generated 3D coordinate system, generate a VAR image from the plurality of second images based on the determined type of objects and the determined position of the objects, and transmit the generated VAR image to an external electronic device; and the at least one processor is implemented to determine the coordinates of the plurality of first images within the 3D coordinate system based on the positions of the plurality of first cameras, and match the coordinates of the object included in at least one of the plurality of first images within the 3D coordinate system using the coordinates of the plurality of first images.

Images