Server for providing VAR image and operating method thereof

Abstract

A method for providing a VAR image and an electronic device for performing the method are disclosed. An electronic device according to various embodiments may: receive a plurality of images from a plurality of cameras; determine the types of objects included in the plurality of images on the basis of the received plurality of images; determine the positions of the objects in a pre-generated 3D coordinate system; generate a VAR image on the basis of the determined types of the objects and the determined positions of the objects; and transmit the generated VAR image to an external electronic device.

Description

Server for providing VAR video and its operating method

[0001] Various embodiments of the present disclosure relate to a server for providing VAR images and a method of operating the same.

[0002]

[0003] The existing method of providing VAR (video assistant referees) videos required a long time to produce and was unable to be delivered to consumers in real time, making it difficult to quickly provide information to consumers. Furthermore, VAR videos required video providers to edit the footage or create animations themselves, requiring significant labor.

[0004] Therefore, in order to solve the aforementioned problem, there is a need to develop a method for determining the location and type of an object using artificial intelligence and generating a new VAR image using only the determined location and type of the object.

[0005]

[0006] The purpose of this application is to provide a server for providing VAR images and an operating method thereof, in order to form a platform for automatically providing VAR images to assist in judgment.

[0007] The problems to be solved by this application are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which this application belongs from this specification and the attached drawings.

[0008]

[0009] According to various embodiments, an operating method of an electronic device may be provided, including: receiving a plurality of images from a plurality of cameras; determining, based on the plurality of images, types of objects included in the plurality of images; determining positions of the objects within a pre-generated 3D coordinate system; generating a VAR image based on the types of the objects determined and the positions of the objects determined; and transmitting the generated VAR image to an external electronic device.

[0010] According to various embodiments, a server may be provided, comprising: a communication circuit; a memory; and at least one processor; wherein the at least one processor is configured to receive a plurality of images from a plurality of cameras through the communication circuit; determine, based on the plurality of images, types of objects included in the plurality of images; determine positions of the objects within a pre-generated 3D coordinate system; generate a VAR image based on the types of the determined objects and the positions of the determined objects; and transmit the generated VAR image to an external electronic device.

[0011] According to various embodiments, a non-transitory recording medium storing at least one program executable by a computer may be provided, wherein when the at least one program is executed, at least one processor is configured to: receive a plurality of images from a plurality of cameras through the communication circuit, determine types of objects included in the plurality of images based on the plurality of images, determine positions of the objects within a pre-generated 3D coordinate system, generate a VAR image based on the types of the determined objects and the positions of the determined objects, and transmit the generated VAR image to an external electronic device.

[0012] The means of solving the problem are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0013]

[0014] The present invention acquires multiple images from multiple cameras and, based on the acquired images, determines the type and location of objects included in the images. Accordingly, a VAR image can be generated based on the determined type and location of the objects.

[0015] In addition, by producing VAR videos in various formats as needed, VAR videos in an appropriate format corresponding to the type of sport can be provided to users using VAR, and since VAR videos are created regardless of user editing, there is an advantage of saving human resources.

[0016]

[0017] FIG. 1 is a block diagram of a system according to various embodiments.

[0018] FIG. 2 is a drawing for explaining an example of a system according to various embodiments.

[0019] FIG. 3 is a block diagram illustrating an example of a server according to various embodiments.

[0020] FIG. 4 is a block diagram illustrating an example of a program according to various embodiments.

[0021] FIG. 5 is a block diagram illustrating an example of a user terminal according to various embodiments.

[0022] FIG. 6 is a block diagram illustrating an example of a camera module according to various embodiments.

[0023] FIG. 7A is a drawing for explaining a first camera according to various embodiments.

[0024] FIG. 7b is a drawing for explaining a second camera according to various embodiments.

[0025] FIG. 7c is a drawing for explaining a shooting method of multiple camera modules according to various embodiments.

[0026] FIG. 8 is a flowchart illustrating an example of a service for generating a VAR image of a server according to various embodiments.

[0027] FIG. 9 is a flowchart illustrating an example of a method for receiving multiple images from multiple camera modules of a server according to various embodiments.

[0028] FIG. 10 is a diagram illustrating an example of a method for receiving multiple images from multiple camera modules of a server according to various embodiments.

[0029] FIG. 11 is a flowchart illustrating an example of a method for determining the type of an object of a server according to various embodiments.

[0030] FIG. 12 is a diagram illustrating a method for determining the type of an object of a server according to various embodiments.

[0031] FIG. 13 is a flowchart illustrating a method for determining the location of an object of a server according to various embodiments.

[0032] FIG. 14 is a diagram illustrating a method for determining the location of an object of a server according to various embodiments.

[0033] FIG. 15 is a flowchart illustrating a method of generating a graphic VAR image of a server according to various embodiments.

[0034] FIG. 16 is a diagram illustrating a method for generating a graphic VAR image of a server according to various embodiments.

[0035] FIG. 17 is a flowchart illustrating a method of generating an overlap VAR image of a server according to various embodiments.

[0036] FIG. 18 is a diagram for explaining a method of generating an overlap VAR image of a server according to various embodiments.

[0037] FIG. 19 is a flowchart illustrating a method of transmitting a VAR image to a user terminal at the user's request of a server according to various embodiments.

[0038] FIG. 20 is a diagram for explaining a method of transmitting a VAR image to a user terminal at the user's request of a server according to various embodiments.

[0039] FIG. 21 is a flowchart illustrating a method of transmitting a VAR image based on information associated with a user terminal of a server to a user terminal according to various embodiments.

[0040] FIG. 22 is a diagram for explaining a method of transmitting a VAR image based on information associated with a user terminal of a server to a user terminal according to various embodiments.

[0041]

[0042] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0043] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0044] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0045] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0046] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0047] According to various embodiments, an operating method of an electronic device may be provided, including: an operation of receiving a plurality of images from a plurality of cameras; an operation of determining, based on the plurality of images, types of objects included in the plurality of images; an operation of determining positions of the objects within a pre-generated 3D coordinate system; an operation of generating a VAR image based on the types of the objects determined and the positions of the objects determined; and an operation of transmitting the generated VAR image to an external electronic device.

[0048] According to various embodiments, an operation method may be provided in which the operation of receiving the plurality of images includes an operation of receiving first images acquired using a plurality of fixed first cameras and an operation of receiving second images acquired using a plurality of second cameras, wherein the plurality of second cameras are operated by a control signal transmitted from the electronic device.

[0049] According to various embodiments, an operation method may be provided in which the operation of generating the VAR image includes an operation of generating a graphic image based on the type of the object and the location of the object.

[0050] According to various embodiments, a method of operation may be provided, wherein the operation of generating the VAR image includes an operation of generating an image in which an auxiliary object overlaps at least one image among the plurality of received images.

[0051] According to various embodiments, the VAR image may be provided with an operating method that includes analysis data associated with the object.

[0052] According to various embodiments, the plurality of second cameras may be provided with an operating method for taking pictures including the set area when the location of the object is included in the set area.

[0053] According to various embodiments, a server may be provided, comprising a communication circuit, a memory, and at least one processor, wherein the at least one processor is configured to receive a plurality of images from a plurality of cameras through the communication circuit, determine types of objects included in the plurality of images based on the plurality of images, determine positions of the objects within a pre-generated 3D coordinate system, generate a VAR image based on the types of the determined objects and the positions of the determined objects, and transmit the generated VAR image to an external electronic device.

[0054] According to various embodiments, a server may be provided in which the at least one processor is configured to receive, as at least part of the operation of receiving the plurality of images: first images acquired using a plurality of fixed first cameras, and second images acquired using a plurality of second cameras, wherein the plurality of second cameras are operated by a control signal transmitted from the electronic device.

[0055] According to various embodiments, a server may be provided wherein the at least one processor is configured to, at least as part of the operation of generating the VAR image: generate a graphical image based on the type of the object and the location of the object.

[0056] According to various embodiments, a server may be provided in which the at least one processor is configured to, as at least part of the operation of generating the VAR image: generate an image in which an auxiliary object overlaps at least one of the received plurality of images.

[0057] According to various embodiments, the VAR image may be provided to a server that includes analysis data associated with the object.

[0058] According to various embodiments, a server may be provided in which the plurality of second cameras capture images including the set area when the location of the object is included in the set area.

[0059] According to various embodiments, a non-transitory recording medium storing at least one program executable by a computer may be provided, wherein when the at least one program is executed, at least one processor is configured to: receive a plurality of images from a plurality of cameras through the communication circuit, determine types of objects included in the plurality of images based on the plurality of images, determine positions of the objects within a pre-generated 3D coordinate system, generate a VAR image based on the types of the determined objects and the positions of the determined objects, and transmit the generated VAR image to an external electronic device.

[0060] According to various embodiments, the system (10) receives a plurality of images (e.g., a first image, a second image) from a plurality of camera modules (300) using a server (100), and determines, from the plurality of images (e.g., a first image (1000a), a second image (1000b)), a type of an object (e.g., a type of a first object (1200), a type of a second object (1210)) and a location of an object (a location of a first object (1420), a location of a second object (1430)), and, based on the type of the object (a type of the first object (1200), a type of the second object (1210)) and the location of the object (a location of a first object (1410), a location of a second object (1420)), a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR It may be a system (10) implemented to generate a first image (2200a), a second VAR image (2200b).

[0061] The above object (700) may include a person (e.g., a player, a referee, a spectator), an object (e.g., a ball, a racket, a goal post), and / or a field (e.g., a sports field, an indoor gymnasium) included in the plurality of images (e.g., a first image (1000a), a second image (1000b), but is not limited to the described examples and may include objects necessary for producing VAR images of various types of sports. Hereinafter, examples of the system (10) according to various embodiments will be described in more detail.

[0062] FIG. 1 is a block diagram illustrating an example of a system (10) according to various embodiments. Hereinafter, FIG. 1 will be described with reference to FIG. 2.

[0063] FIG. 2 is a drawing for explaining an example of a system (10) according to various embodiments.

[0064] Referring to FIG. 1, the system (10) may include a server (100), a user terminal (200), and a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)). The server (100), user terminal (200) and / or camera module (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) included in the system (10) can transmit / receive a plurality of images (e.g., first image (1000a), second image (1000b)) and / or VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b)) or provide a predetermined service based on the activation of a communication function.

[0065] According to various embodiments, the server (100), referring to FIG. 2, receives a plurality of images (e.g., a first image (1000a), a second image (1000b)) from a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)), stores the images in the server (100), and determines the type of object (e.g., a type of first object (1200), a type of second object (1210)) and the location of the object (a location of the first object (1410), a location of the second object (1420)) using the plurality of images (e.g., a first image (1000a), a second image (1000b)). In addition, by using the types of the objects (e.g., type of first object (1200), type of second object (1210)) and the positions of the objects (position of first object (1410), position of second object (1420)), a VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) can be generated. The detailed operation of the server (100) will be described later with reference to FIGS. 8 to 22.

[0066] According to various embodiments, the user terminal (200) may be a user terminal (200) of a producer (or consumer) of a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)). The user terminal (200) may receive a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)) from the server (100) and play back the VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)). For example, For example, the user terminal (200) may receive a plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR images (e.g., first VAR image (2200a), second VAR image (2200b)), and transmit a request to the server (100) to play at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) among the plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), and modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). For example, by using an application (240) installed in the user terminal (200), at least one VAR An interface for requesting playback of an image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), or modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) may be provided on a display (250) of a user terminal (200).The detailed operation of the user terminal (200) is described later with reference to FIGS. 19 and 20.

[0067] In addition, the user terminal (200) may transmit a request to the server (100) to change at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) among the plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). At this time, the user terminal (200) may use the application (240) installed in the user terminal (200) to request a change in at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), change VAR image (e.g., first VAR image (2200a), second VAR image (2200b)). The interface may be provided on the display (250) of the user terminal (200). The detailed operation of the user terminal (200) will be described later with reference to FIGS. 19 and 20.

[0068] According to various embodiments, the plurality of camera modules (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)) can capture a plurality of images (e.g., the first image (1000a), the second image (1000b)) and transmit them to the server (100) as shown in FIG. 2. The detailed description of the operation of the plurality of camera modules (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)) will be described later with reference to FIG. 7c. The detailed description of the plurality of images (e.g., the first image (1000a), the second image (1000b)) will be described later with reference to FIG. 10.

[0069] FIG. 3 is a block diagram illustrating an example of a server (100) according to various embodiments.

[0070] According to various embodiments, the server (100) may include a first processor (110), a first memory (120) including a program (140), and a first communication circuit (130). However, without being limited to the described and / or illustrated examples, the server (100) may be implemented to include more devices and / or fewer devices.

[0071] According to various embodiments, the first processor (110) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the first processor (110) by executing, for example, software (e.g., a program (140)), and may perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the first processor (110) may store commands or data received from other components (e.g., a VAR image generation module (146), a VAR image determination module (147)) in a volatile first memory (120), process the commands or data stored in the first memory (120), and store result data in the first memory (120). According to one embodiment, the first processor (110) may include a main processor (not shown) (e.g., a central processing unit or an application processor) or an auxiliary processor (not shown) that can operate independently or together with the main processor (not shown) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the first processor (110) includes a main processor (not shown) and an auxiliary processor (not shown), the auxiliary processor (not shown) may be configured to use lower power than the main processor (not shown) or to be specialized for a given function. The auxiliary processor (not shown) may be implemented separately from the main processor (not shown) or as a part thereof.

[0072] The auxiliary processor (not shown) may control at least a part of functions or states related to at least one component (e.g., the first memory (120), the first communication circuit (130)) of the server (100), for example, on behalf of the main processor (not shown) while the main processor (not shown) is in an inactive (e.g., sleep) state, or together with the main processor (not shown) while the main processor (not shown) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (not shown) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (not shown)). According to one embodiment, the auxiliary processor (not shown) (e.g., a neural network processing device) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. Such learning may be performed, for example, in the server (100) itself where artificial intelligence is performed, or may be performed through a separate server. Learning algorithms may include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. An artificial intelligence model may include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0073] According to various embodiments, the first memory (120) may store various data used by at least one component (e.g., processor (110)) of the server (100). The data may include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The first memory (120) may include volatile memory (not shown) or non-volatile memory (not shown).

[0074] According to various embodiments, the program (140) may be stored as software in the first memory (120) and, when executed, may cause the first processor (110) to perform at least one operation. Examples of functions provided by the program (140) will be described below, and the operations of the modules of the program (140) may be understood as operations of the first processor (110) based on the execution of the program (140).

[0075] According to various embodiments, the first communication circuit (130) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between a server (100) and an external electronic device (e.g., a user terminal (200), a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) and the performance of communication through the established communication channel. The first communication circuit (130) may operate independently from the first processor (110) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the first communication circuit (130) may include a wireless communication module (not shown) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (not shown) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module may communicate with an external electronic device (e.g., a user terminal (200)) via a first network (not shown) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (not shown) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as a plurality of separate components (e.g., multiple chips). A wireless communication module (not shown) can identify or authenticate a server (100) within a communication network, such as a first network (not shown) or a second network (not shown), using subscriber information (e.g., international mobile subscriber identity (IMSI)) stored in a subscriber identification module (not shown).

[0076] FIG. 4 is a block diagram illustrating an example of a program (140) according to various embodiments.

[0077] According to various embodiments, the program (140) may include a 3D coordinate system generation module (141), an object type generation module (142), an object position determination module (143), a shooting signal generation module (144), a data analysis module (145), a VAR image generation module (146), and / or a VAR image determination module (147). The plurality of modules may be implemented as executable computer codes, instructions, and / or APIs.

[0078] According to various embodiments, the 3D coordinate system generation module (141) may obtain a plurality of first images (1000a-1, 1000a-2) from a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) using a communication circuit (130). At this time, the 3D coordinate system generation module (141) may generate a virtual 3D coordinate system using the plurality of first images (1000a-1, 1000a-2). For example, the 2D coordinates of the plurality of first images (1000a-1, 1000a-2) may be obtained using the shooting positions of the plurality of first images (1000a-1, 1000a-2). At this time, a reference point (1410) is determined, and a virtual 3D coordinate system for the plurality of first images (1000a-1, 1000a-2) can be generated using the distance between the 2D coordinates. However, the present invention is not limited to the described generation example, and may include more diverse generation methods.

[0079] According to various embodiments, the object type determination module (142) may determine the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) based on the plurality of first images (1000a-1, 1000a-2). For example, the object type determination module (142) may determine the type of the first object (1200) using a first artificial intelligence model for the plurality of first images (1000a-1, 1000a-2), and may determine the type of the second object (1210) using a second artificial intelligence model for the plurality of first images (1000a-1, 1000a-2). Also, for example, the object type determination module (142) may determine the type of the first object (1200) and the type of the second object (1210) using a third artificial intelligence model. However, the method for determining the type of the described object (e.g., the type of the first object (1200), the type of the second object (1210)) is not limited to the method for determining the type of the described object, and may include more diverse methods for determining the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)). Details of the operation for determining the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) will be described later with reference to FIGS. 11 and 12.

[0080] According to various embodiments, the object position determination module (143) may determine the position of an object (e.g., the position of a first object (1420), the position of a second object (1430)) in a pre-generated 3D coordinate system (1400). For example, the object position determination module (143) may identify an object (700) in the 3D coordinate system (1400) and match coordinates corresponding to the 3D coordinate system to determine the position of the object (e.g., the position of a first object (1420), the position of a second object (1430)). However, the method for determining the position of the object (e.g., the position of a first object (1420), the position of a second object (1430)) is not limited to this method, and may include more diverse determination methods. The operation for determining the position of the above object (e.g., the position of the first object (1420), the position of the second object (1430)) is described below with reference to FIGS. 13 and 14.

[0081] According to various embodiments, the shooting signal generation module (144) may receive the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)), and transmit a shooting control signal to the second camera (300b-1, 300b-2) based on the received type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)). For example, the shooting signal generation module (144) can predict the movement of the object (700) by using the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)), and transmit a shooting control signal to the second camera (300b-1, 300b-2) to produce a natural image according to the movement. However, the operation of transmitting the shooting signal by using the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)) is not limited to the operation of transmitting the shooting signal, and a shooting signal transmission operation for a more diverse purpose may be included.

[0082] According to various embodiments, the data analysis module (145) can analyze data by using the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) and the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)). For example, the data analysis module (145) can obtain the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) and the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)), determine a location relationship between the locations of a plurality of objects (e.g., the location of the first object (1410), the location of the second object (1420)), and analyze a judgment result based on a set rule or a behavior of the object (700) through the determined location relationship. However, the operation is not limited to analyzing data using the type of the object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)), and various operations for analyzing data may be included. The detailed operation of the data analysis module (145) will be described later with reference to FIGS. 15 to 18.

[0083] According to various embodiments, the VAR image generation module (146) may generate a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)) using the type of the object (e.g., a type of the first object (1200), a type of the second object (1210)) and the position of the object (e.g., a position of the first object (1410), a position of the second object (1420)). For example, the VAR image generation module (146) may generate a graphic VAR image (1600a) using the type of the object (e.g., a type of the first object (1200), a type of the second object (1210)) and the position of the object (e.g., a position of the first object (1410), a position of the second object (1420)). The graphic VAR The image (1600a) may be a computer graphics image generated regardless of the first image (1000a-1, 1000a-2) or the second image (1000b-1, 1000b-2) obtained from the camera module (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)). In addition, the VAR image generation module (146) may generate an overlap VAR image using the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)). The overlap VAR image (1600b) may be generated by the camera module (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)). It may be an image generated by overlapping an auxiliary object (1720) on a first image (1000a-1, 1000a-2) or a second image (1000b-1, 1000b-2) obtained from a camera (300b-1, 300b-2) based on a set rule or action.In addition, the VAR image generation module (146) can generate a new VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)) by modifying the VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)). Details of the operation of the VAR image generation module (146) will be described later with reference to FIGS. 15 to 22.

[0084] According to various embodiments, the VAR image determination module (147) can determine a VAR image to be transmitted to the user terminal (200). For example, the VAR image determination module (147) may determine at least one VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) among a plurality of VAR images (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)) based on at least one element of the network availability of the user terminal (200), hardware information of the user terminal (200), or program subscription of the user terminal (200). However, the VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR The elements for determining the image (2200b) are not limited to these elements, and various elements may be included. The operation of the VAR image determination module (147) is described below with reference to FIGS. 21 and 22.

[0085] FIG. 5 is a block diagram illustrating an example of a user terminal (200) according to various embodiments.

[0086] According to various embodiments, the user terminal (200) may include a second processor (210), a second communication circuit (220), a second memory (230), an application (240) stored in the second memory, a display (250), and an input module (260). However, without being limited to the described and / or illustrated examples, the user terminal (200) may be implemented to include more devices, and / or to include fewer devices.

[0087] According to various embodiments, each of the second processor (210), the second communication circuit (220), and the second memory (230) can be implemented like the first processor (110), the first communication circuit (130), and the first memory (120) described above, and therefore, redundant descriptions are omitted.

[0088] According to various embodiments, the application (240) may be implemented in various forms. In one embodiment, when the application (240) is implemented in an on-device type, the application (240) may be implemented to perform operations similar to and / or identical to those of the program (140) described above. In another embodiment, when the application (240) is implemented as a server type, the application (240) may be implemented to include a function for simply receiving a VAR image and / or a function for transmitting a request to the server (100) to change a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)). Accordingly, based on the above-described program (140) being executed on the server, a plurality of VAR images (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) received from the server (100) are provided through the application (240), thereby providing the server (100) with a plurality of VAR images. At least one of the following may be transmitted: an image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b)).

[0089] According to various embodiments, the display (250) can visually provide information to an external party (e.g., a user) of the user terminal (200). The display (250) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display (250) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0090] The input module (260) can receive commands or data to be used in a component of the user terminal (200) (e.g., the second processor (210)) from an external source (e.g., a user) of the user terminal (200). The input module (260) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0091] FIG. 6 is a block diagram illustrating an example of a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) according to various embodiments.

[0092] According to various embodiments, a camera module (300) (e.g., a first camera (300a-1, 300a-2), a second camera (300b-1, 300b-2)) may include a photographing device (301), a fixing device (350), and an actuator (360).

[0093] According to various embodiments, the photographing device (301) may include a third processor (310), an image sensor (320), a shutter (340), and a third communication circuit (330).

[0094] According to various embodiments, the photographing device (301) can photograph a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) and transmit the plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) to the server (100) using the third communication circuit (330).

[0095] According to various embodiments, each of the third processor (310) and the second communication circuit (330) can be implemented like the first processor (110) and the first communication circuit (130) described above, and therefore, redundant descriptions are omitted.

[0096] According to various embodiments, the image sensor (320) can generate a plurality of images (e.g., first image (1000a-1, 1000a-2), second image (1000b-1, 1000b-2)) by converting the intensity of light into an electrical signal.

[0097] According to various embodiments, the shutter (340) can adjust the sharpness of multiple images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) by controlling the speed. For example, when the object (700) is in the set area (710), the shutter (340) can increase the speed of the shutter (340) and take pictures in the set area (710) in order to obtain an accurate object location (e.g., the location of the first object (1410), the location of the second object (1420)).

[0098] According to various embodiments, the fixing device (350) can fix the photographing device (301) and designate the photographing direction and angle.

[0099] According to various embodiments, the actuator (360) can change the shooting direction and shooting angle by moving the shooting device (301).

[0100] FIG. 7a is a drawing for explaining a first camera (300a-1, 300a-2) according to various embodiments.

[0101] According to various embodiments, the first camera (300a-1, 300a-2) may include a photographing device (301) and a fixing device (350).

[0102] Duplicate descriptions of the shooting device (301) and the fixing device (350) are omitted.

[0103] According to various embodiments, the first camera (300a-1, 300a-2) may be fixed to a fixed device (350) to capture a certain area and generate a first image (1000a-1, 1000a-2). For example, the first camera (300a-1, 300a-2) may be fixed and capture a certain area to serve as a reference for the position of the camera module (300) that generates a 3D coordinate system (1400). Accordingly, a first image (1000a-1, 1000a-2) including a first area for determining the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) and the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)) may be generated.

[0104] FIG. 7b is a drawing for explaining a second camera (300b-1, 300b-2) according to various embodiments.

[0105] According to various embodiments, the second camera (300b-1, 300b-2) may include a photographing device (301) and an actuator (360).

[0106] Duplicate descriptions of the shooting device (301) and actuator (360) are omitted.

[0107] According to various embodiments, the second camera (300b-1, 300b-2) may generate a second image (1000b-1, 1000b-2) by taking pictures in a shooting direction and shooting angle corresponding to a control signal received from the server (100) using an actuator (360). In addition, the second camera (300b-1, 300b-2) may be a camera having a higher definition or resolution than the first camera (300a-1, 300a-2).

[0108] FIG. 7c is a drawing for explaining a shooting method of a plurality of camera modules (300) (e.g., first camera (300a-1, 300a-2), second camera (300b-1, 300b-2)) according to various embodiments.

[0109] According to various embodiments, the camera module (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)) can transmit a plurality of images (e.g., the first image (1000a-1, 1000a-2), the second image (1000b-1, 1000b-2)) to the server (100). Accordingly, the shooting method of the camera module (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)) can be changed according to a control signal generated according to the transmitted plurality of images (e.g., the first image (1000a-1, 1000a-2), the second image (1000b-1, 1000b-2)). For example, the first camera (300a-1, 300a-2) can capture a fixed and set area (710) and transmit a first image (1000a-1, 1000a-2) including the set area to the server (100). At this time, if the server (100) identifies that an object (700) exists in the set area (710), it can transmit a control signal to the second camera (300b-1, 300b-2) to capture the set area (710) with high resolution or high clarity. Accordingly, when the second camera (300b-1, 300b-2) receives the control signal, the second camera (300b-1, 300b-2) can change the shooting direction or shooting angle of the shooting device (301) using the actuator (360) and adjust at least one of the shutter (340) and the image sensor (320) to capture the set area (710) with high resolution or high clarity. However, the operations of the plurality of camera modules (300) (e.g., the first camera (300a-1, 300a-2), the second camera (300b-1, 300b-2)) are not limited to the illustrated operations, and more operations may be performed and / or fewer operations may be performed.

[0110] FIG. 8 is a flowchart illustrating an example of a service for generating VAR images (e.g., graphic VAR images (1600a), overlap VAR images (1600b), and modified VAR images (e.g., first VAR images (2200a), second VAR images (2200b))) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed.

[0111] According to various embodiments, the server (100) may, in operation 801, receive a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) from a plurality of camera modules (300) (e.g., first cameras (300a-1, 300a-2), second cameras (300b-1, 300b-2)). The operation of receiving the plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) will be described in detail in operations 901 to 903.

[0112] According to various embodiments, the server (100) may determine the type of object (e.g., type of first object (1200), type of second object (1210)) based on a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) in operation 803. The operation of determining the type of object (e.g., type of first object (1200), type of second object (1210)) is specifically described below in operations 1101 to 1103.

[0113] According to various embodiments, the server (100) may determine, in operation 805, the location of an object (e.g., the location of a first object (1410), the location of a second object (1420)) within a pre-generated 3D coordinate system (1400). The operation of determining the location of the object (e.g., the location of a first object (1410), the location of a second object (1420)) is specifically described below in operations 1301 to 1303.

[0114] According to various embodiments, the server (100) may generate a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) based on the type of the object (e.g., a type of the first object (1200), a type of the second object (1210)) and the location of the object (e.g., a location of the first object (1410), a location of the second object (1420)) in operation 807. The operations for generating the above VAR images (e.g., graphic VAR images (1600a), overlap VAR images (1600b), and modified VAR images (e.g., first VAR images (2200a), second VAR images (2200b))) are specifically described in operations 1501 to 1503 and operations 2101 to 2103.

[0115] According to various embodiments, the server (100) may transmit the generated VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to an external electronic device in operation 809. The operation of transmitting the VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to the external electronic device is described in detail in operations 2105 to 2107.

[0116] FIG. 9 is a flowchart illustrating an example of a method for receiving a plurality of images (e.g., first images 1000a-1, 1000a-2, second images 1000b-1, 1000b-2)) from a plurality of camera modules (300) of a server (100) (e.g., first cameras 300a-1, 300a-2, second cameras 300b-1, 300b-2)) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Hereinafter, FIG. 9 will be described with reference to FIG. 10.

[0117] FIG. 10 is a drawing for explaining an example of a method for receiving a plurality of images (e.g., first images (1000a-1, 1000a-2), second images (1000b-1, 1000b-2)) from a plurality of camera modules (300) (e.g., first cameras (300a-1, 300a-2), second cameras (300b-1, 300b-2)) of a server (100) according to various embodiments.

[0118] According to various embodiments, the server (100) may, in operation 901, receive a plurality of first images (1000a-1, 1000a-2) from a plurality of first cameras (300a-1, 300a-2). For example, referring to FIG. 10, the server (100) may receive a first image (1000a-1) including a first area from a first camera (300a-1) that photographs a first area, and may receive a second image (1000a-2) including a second area from a first camera (300a-2) that photographs a second area.

[0119] According to various embodiments, the server (100) may, in operation 903, receive a plurality of second images (1000b-1, 1000b-2) from a plurality of second cameras (300b-1, 300b-2). For example, referring to FIG. 10, the server (100) may receive a second image (1000b-1) including a third area from a second camera (300b-1) that photographs a third area that dynamically changes according to a control signal of the server (100), and may receive a second image (1000b-2) including a fourth area from a second camera (300a-2) that photographs a fourth area that dynamically changes according to a control signal of the server (100).

[0120] FIG. 11 is a flowchart illustrating an example of a method for determining the type of an object (e.g., type of first object (1200), type of second object (1210)) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Hereinafter, FIG. 11 will be described with reference to FIG. 12.

[0121] FIG. 12 is a drawing for explaining a method for determining the type of an object of a server (100) (e.g., type of first object (1200), type of second object (1210)) according to various embodiments.

[0122] According to various embodiments, the server (100) may determine the type (1200) of the first object based on the first image (1000a-1, 1000a-2) in operation 1101. For example, referring to FIG. 12, the server (100) may determine an object (700) of type human among objects (700) included in the first image (1000a-1, 1000a-2) by using an artificial intelligence model for determining a human among the types of objects (e.g., the type of the first object (1200), the type of the second object (1210)) in the first image (1000a-1, 1000a-2). The operation of determining the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) using the above artificial intelligence model can be performed using well-known artificial intelligence models such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, and Mask R-CNN, so a detailed description thereof is omitted.

[0123] According to various embodiments, the server (100) may determine the type (1210) of the second object based on the first image (1000a-1, 1000a-2) in operation 1103. For example, referring to FIG. 12, the server (100) may use an artificial intelligence model to determine a type of object (e.g., type of first object (1200), type of second object (1210)) in the first image (1000a-1, 1000a-2) to determine a type of an object (700) among objects (700) included in the first image (1000a-1, 1000a-2) that is a public object. The operation of determining the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) using the above model can be performed using well-known artificial intelligence models such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, and Mask R-CNN, so a detailed description thereof is omitted.

[0124] According to various embodiments, the artificial intelligence model for determining the type of the first object (1200) and the type of the second object (1210) may be the same artificial intelligence model. In this case, operations 1101 and 1103 may be performed at the same time.

[0125] FIG. 13 is a flowchart illustrating a method for determining the location of an object (e.g., the location of a first object (1410), the location of a second object (1420)) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Hereinafter, FIG. 13 will be described with reference to FIG. 14.

[0126] FIG. 14 is a diagram for explaining a method for determining the location of an object of a server (100) (e.g., location of a first object (1410), location of a second object (1420)) according to various embodiments.

[0127] According to various embodiments, the server (100) can identify an object (700) from a pre-generated 3D coordinate system (1400) in operation 1301. For example, referring to FIG. 14, the server (100) can identify a person and a ball as objects (700) whose locations are to be found in the pre-generated 3D coordinate system (1400). The operation of identifying the object (700) can utilize well-known technologies such as R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO, Convolutional Neural Networks (CNN), U-Net, and Mask R-CNN, and thus a detailed description thereof will be omitted.

[0128] According to various embodiments, the server (100) may determine the position of the object (700) within a pre-generated 3D coordinate system (1400) in operation 1303. For example, referring to FIG. 14, the pre-generated 3D coordinate system (1400) may include a reference point (1410). At this time, the server (100) may obtain the x and y coordinates of the plurality of first images (1000a-1, 1000a-2) by using the positions of the plurality of stored first cameras (300a-1, 300a-2) in the acquired plurality of first images (1000a-1, 1000a-2). Accordingly, by using the reference point (1410), the x-coordinate, and the y-coordinate, the z-coordinate may be obtained, and the entire coordinates of a specific point in time of the plurality of first images (1000a-1, 1000a-2) may be determined. The object (700) can be matched with the global coordinates to determine the position of the object (e.g., the position of the first object (1410), the position of the second object (1420)). The operation of determining the 3D coordinates can utilize a programming language or a well-known technology such as Triangulation, 3D scanning, LiDAR, ToF (Time of Flight), Stereo Vision, etc., so a detailed description thereof will be omitted.

[0129] FIG. 15 is a flowchart illustrating a method for generating a graphic VAR image (1600a) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Below, FIG. 15 will be described with reference to FIG. 16.

[0130] FIG. 16 is a diagram for explaining a method of generating a graphic VAR image (1600a) of a server (100) according to various embodiments.

[0131] According to various embodiments, the server (100) may analyze data based on the type of object (e.g., the type of the first object (1200), the type of the second object (1210)) and the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)) in operation 1501. For example, referring to FIG. 16, when the location of a ball and the location of a goal are determined among a plurality of objects (700), the server (100) may analyze that a goal event has occurred if the determined location of the ball completely crosses the line of the goal. In addition, for example, when the locations of a plurality of people among a plurality of objects (700) overlap, the server (100) may determine the location between the people and the characteristic point information of the people according to the viewpoint, and analyze that a foul event has occurred. However, the data analysis operation described above is not limited to the above, and may include more diverse data analysis operations that can be analyzed based on the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) after presetting rules and actions. The gesture recognition operation may utilize a well-known gesture recognition technology such as vision-based gesture recognition, so a detailed description thereof will be omitted. In addition, the operation of determining the feature point information may utilize a well-known technology such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), Gabor filter, Harris corner detector, LBP (Local Binary Pattern), and HOG (Histogram of Oriented Gradients), so a detailed description thereof will be omitted.

[0132] According to various embodiments, the server (100) may generate a graphic VAR image (1600a) based on the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)), the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)) and the analysis data (1620) in operation 1503. For example, referring to FIG. 16, the server (100) may generate a graphic VAR image (1600a) including graphics (1610) corresponding to the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)) and the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)) using a first artificial intelligence model. At this time, the first artificial intelligence model can be trained to generate a graphic VAR image (1600a) including a graphic (1610) corresponding to the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)). In addition, for example, when the type of the first object (1200) is a goal and the type of the second object (1210) is a ball, the server (100) can generate a graphic VAR image (1600a) including a graphic (1610) corresponding to the ball and a graphic (1610) corresponding to the goal using the first artificial intelligence model. At this time, the graphic VAR image (1600a) can be changed by including analysis data (1620) that analyzes the goal event analyzed in operation 1501. However, if the graphic VAR image (1600a) does not include analysis data (1620), operation 1501 may be omitted.The above artificial intelligence model can be used with well-known artificial intelligence models such as real-time graphics, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Image-to-Image Translation, Pix2Pix, CycleGAN, Neural Style Transfer, and 3D ShapeNets, so a detailed description is omitted.

[0133] FIG. 17 is a flowchart illustrating a method for generating an overlap VAR image (1600b) of a server (100) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations and / or fewer operations may be performed. Below, FIG. 17 will be described with reference to FIG. 18.

[0134] FIG. 18 is a diagram for explaining a method of generating an overlap VAR image (1600b) of a server (100) according to various embodiments.

[0135] According to various embodiments, the server (100) may analyze data based on the type of object (e.g., type of first object (1200), type of second object (1210)) and the location of the object (e.g., location of first object (1410), location of second object (1420)) in operation 1701. Since operation 1701 may be implemented similarly to operation 1501, redundant descriptions thereof will be omitted.

[0136] According to various embodiments, the server (100) may generate an overlap VAR image (1600b) based on the type of the object (e.g., the type of the first object (1200), the type of the second object (1210)), the location of the object (e.g., the location of the first object (1410), the location of the second object (1420)) and the analysis data (1620) in operation 1703. For example, referring to FIG. 18, the server (100) may generate an overlap VAR image (1600b) by overlapping an auxiliary object (1720) with an object (700) included in a received second image (1000b-1, 1000b-2) using a second artificial intelligence model. At this time, the server (100) can train the second artificial intelligence model to overlap the type and location of the auxiliary object (1720) based on the set rule or the behavior of the object (700). For example, the server (100) can use the second artificial intelligence model to generate an overlap VAR image (1600b) in which, when the type of the first object (1200) is a ball and the type of the second object (1210) is a person, the ball is overlapped with an auxiliary object (1720) having a shape that surrounds the border, and the person is overlapped with an auxiliary object (1720) that connects feature points. However, the present invention is not limited to the described examples, and more diverse shapes of auxiliary objects (1720) to assist in judgment may be included. At this time, the overlap VAR image (1600b) can be changed by including analysis data (1620) that analyzes the goal event analyzed in operation 1701. However, if the analysis data (1620) is not included in the overlap VAR image (1600b), operation 1701 may be omitted.The above artificial intelligence model can be used with well-known artificial intelligence models such as real-time graphics, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Image-to-Image Translation, Pix2Pix, CycleGAN, Neural Style Transfer, and 3D ShapeNets, so a detailed description is omitted.

[0137] FIG. 19 is a flowchart illustrating a method for transmitting a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) requested by a user of a server (100) to a user terminal (200) according to various embodiments. The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Hereinafter, FIG. 19 will be described with reference to FIG. 20.

[0138] FIG. 20 is a drawing for explaining a method of transmitting a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) that a user of a server (100) has requested to receive to a user terminal (200) according to various embodiments.

[0139] According to various embodiments, when the server (100) receives a reception request for a specific VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) of the user terminal (200) in operation 1901, the server (100) may determine the specific VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))). For example, referring to FIG. 20, the server (100) may receive a request for a specific VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) associated with a specific rule based on an input of a selection interface of a user terminal (200). For example, referring to FIG. 20, when the server (100) receives a request for receiving a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) for assisting in reading the goal line, based on an input of a goal line interface among selection interfaces, the server (100) may determine a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) capable of assisting in reading the goal line.For example, when the server (100) receives a request to receive a VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), changed VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) with the type of the VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), changed VAR image (e.g., first VAR image (2200a), second VAR image (2200b))), whether the auxiliary object (1720) is included, and whether the analysis data (1620) is included, the server (100) receives a request to receive a VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), changed VAR image (e.g., first VAR image (2200a), second VAR The type of the image (2200b)), whether the auxiliary object (1720) is included, or whether the analysis data (1620) is included, can be changed. A VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) can be determined as a specific VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))).

[0140] According to various embodiments, the server (100) may transmit, in operation 1903, the VAR image determined in operation 1901 (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) to the user terminal (200). For example, referring to FIG. 20, the server (100) may transmit a VAR image capable of assisting in reading a goal line (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) to the user terminal (200) when a VAR image capable of assisting in reading a goal line (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) is determined as a specific VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) to the user terminal (200).For example, the server (100) determines a VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))) from which analysis data (1620) is removed from a graphic VAR image (1600a) capable of assisting goal-line reading (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))), from which analysis data (1620) is removed from a graphic VAR image (1600a) capable of assisting goal-line reading (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b))). The video (2200a), the second VAR video (2200b))) can be transmitted to the user terminal (200).

[0141] FIG. 21 is a flowchart illustrating a method for transmitting a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) based on information associated with a user terminal (200) of a server (100) according to various embodiments to a user terminal (200). The operations may be performed regardless of the order of the operations illustrated and / or described, and more operations may be performed and / or fewer operations may be performed. Hereinafter, FIG. 21 will be described with reference to FIG. 22.

[0142] FIG. 22 is a drawing for explaining a method of transmitting a VAR image (e.g., a graphic VAR image (1600a), an overlap VAR image (1600b), a modified VAR image (e.g., a first VAR image (2200a), a second VAR image (2200b))) based on information associated with a user terminal (200) of a server (100) according to various embodiments to a user terminal (200).

[0143] According to various embodiments, information associated with the first user terminal (200a) and the second user terminal (200b) can be identified. For example, the server (100) can identify at least one element among the operating system, application (240) settings, CPU, RAM, memory, battery, network type, network speed, available network resources, subscription status, or security of the first user terminal (200a), and at least one element among the operating system, application (240) settings, CPU, RAM, memory, battery, network type, network speed, available network resources, subscription status, or security of the second user terminal (200b). However, the present invention is not limited to the described examples, and more diverse information associated with the user terminal (200) can be identified.

[0144] According to various embodiments, the server (100) may generate a first VAR image (2200a) and a second VAR image (2200b) based on the identified information in operation 2103. For example, referring to FIG. 22, if the server (100) identifies the network speed of the first user terminal (200a), the server (100) may generate the first VAR image (2200a) by compressing the capacity of the graphic VAR image (1600a) to a capacity corresponding to the network speed. Also, referring to FIG. 22, if the server (100) identifies whether the second user terminal (200b) is subscribed and identifies the non-subscription status of the second user terminal, the server (100) may insert an advertisement into the graphic VAR image (1600a) to generate the second VAR image. However, the operation is not limited to the example of generating the first VAR image (2200a) and the second VAR image (2200b) described above, and the first VAR image (2200a) and the second VAR image (2200b) corresponding to information associated with the user terminal (200) can be generated.

[0145] According to various embodiments, the server (100) may transmit the generated first VAR image (2200a) to the first user terminal (200a) in operation 2105. For example, referring to FIG. 22, if the first VAR image (2200a) is a compressed graphic VAR image (1600a) corresponding to the network speed of the first user terminal (200a), the server (100) may transmit the first VAR image (2200a) to the first user terminal (200a).

[0146] According to various embodiments, the server (100) may transmit the generated second VAR image (2200b) to the second user terminal (200b) in operation 2107. For example, referring to FIG. 22, if the second VAR image (2200b) is a graphic VAR image (1600a) in which an advertisement is inserted in response to the unsubscribed status of the second user terminal (200b), the server (100) may transmit the second VAR image (2200b) to the first user terminal (200a). By transmitting the first VAR image (2200a) to the first user terminal (200a) and transmitting the second VAR image (2200b) to the second user terminal (200b), each user is provided with a different VAR image (e.g., graphic VAR image (1600a), overlap VAR image (1600b), modified VAR image (e.g., first VAR image (2200a), second VAR image (2200b)), thereby providing diversity in which customized images can be provided.

Claims

1. In the method of operating an electronic device, An action of receiving multiple images from multiple cameras; An operation of determining the types of objects included in the plurality of images based on the plurality of images; An action to determine the positions of the objects within a pre-generated 3D coordinate system; An operation of generating a VAR image based on the types of the determined objects and the locations of the determined objects; and An operation of transmitting the generated VAR image to an external electronic device; How to operate.

2. In paragraph 1, The operation of receiving the above multiple images is: An operation of receiving first images acquired using a plurality of fixed first cameras; and An operation of receiving second images acquired using multiple second cameras Including, The above plurality of second cameras, Operated by a control signal transmitted from the above electronic device, How to operate.

3. In paragraph 1, The operation of generating the above VAR image is as follows: An operation of generating a graphic image based on the type of the object and the location of the object. including, How to operate.

4. In paragraph 1, The operation of generating the above VAR image is as follows: An operation of generating an image in which an auxiliary object overlaps at least one image among the plurality of received images, How it works.

5. In either of paragraphs 3 and 4, The above VAR image includes analysis data associated with the object. How it works.

6. In paragraph 2, The above plurality of second cameras, when the location of the object is included in the set area, take pictures including the set area. How to operate.

7. As a server, communication circuit; memory; comprising at least one processor; At least one processor, Receive multiple images from multiple cameras through the above communication circuit, Based on the above multiple images, determine the types of objects included in the above multiple images, Determine the positions of the above objects within a pre-generated 3D coordinate system, Based on the types of objects determined above and the locations of the objects determined above, a VAR image is generated, Implemented to transmit the above generated VAR image to an external electronic device Server.

8. In paragraph 7, The at least one processor, at least as part of the operation of receiving the plurality of images: Receive first images acquired using a plurality of fixed first cameras, It is implemented to receive second images acquired using multiple second cameras, The above plurality of second cameras, Operated by a control signal transmitted from the above electronic device, Server.

9. In paragraph 7, The at least one processor, at least as part of the operation of generating the VAR image: Implemented to generate a graphic image based on the type of the object and the location of the object, Server.

10. In paragraph 7, The at least one processor, at least as part of the operation of generating the VAR image: It is implemented to generate an image in which an auxiliary object overlaps at least one image among the plurality of received images. Server.

11. In any one of paragraphs 9 and 10, The above VAR image includes analysis data associated with the object. Server.

12. In paragraph 8, The above plurality of second cameras, when the location of the object is included in the set area, take pictures including the set area. Server.

13. In a non-transitory storage medium storing at least one program executable by a computer, when the at least one program is executed, at least one processor: Receive multiple images from multiple cameras through a communication circuit, Based on the above multiple images, determine the types of objects included in the above multiple images, Determine the positions of the above objects within a pre-generated 3D coordinate system, Based on the types of objects determined above and the locations of the objects determined above, a VAR image is generated, Implemented to transmit the above generated VAR image to an external electronic device Non-transitory recording media.

Images