Automatic relay method and electronic device for performing same

Abstract

Disclosed are an automatic relay method and an electronic device for performing the method. The electronic device according to various embodiments may: receive, from a plurality of detection cameras, a plurality of first images captured with respect to a photographing space; identify, for each set frame, the type of at least one object included in the plurality of first images; determine, for each set frame, the position of the at least one object in a three-dimensional coordinate system set with respect to the photographing space; determine a control signal for generating a relay image by inputting the type and the position of the at least one object to a trained model; and determine the relay image by using at least one second image among a plurality of second images obtained from a plurality of photographing cameras, on the basis of the control signal.

Description

METHOD OF AUTOMATED BROADCASTING AND ELECTRONIC DEVICE PERFROMING THE METHOD

[0001] The disclosure below relates to an automatic relay method and an electronic device performing the method.

[0002]

[0003] In order to broadcast a sports game, cameras and equipment for broadcasting are installed, and personnel are hired to operate the cameras and equipment.

[0004] Because each game requires dedicated equipment and personnel, broadcasting sports events is expensive. Furthermore, when broadcasting footage is determined by humans, the quality of the broadcast can be influenced by the individual judgment, experience, and capabilities of the operating staff.

[0005] The background technology described above is technology that the inventor possessed or acquired in the process of deriving the disclosure of the present application, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the present application.

[0006]

[0007] According to various embodiments, when broadcasting an event such as a sporting event, the electronic device may automatically determine and / or generate a broadcast image from among images acquired from a plurality of cameras for shooting, and transmit the broadcast image.

[0008] According to various embodiments, the electronic device can improve the quality of the relay image by providing analyzed data together with the relay image.

[0009] According to various embodiments, the electronic device can identify an event that occurred in a shooting space and determine a relay image based on the identified event.

[0010] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

[0011]

[0012] According to various embodiments, an electronic device includes at least one processor and a memory electrically connected to the at least one processor and storing at least one command executed by the processor, wherein the processor, when the at least one command is executed, causes the electronic device to receive a plurality of first images captured with respect to a shooting space from a plurality of detection cameras, identify a type of at least one object included in the plurality of first images for each set frame, determine a position of the at least one object in a three-dimensional coordinate system set with respect to the shooting space for each set frame, input the type and position of the at least one object into a learned model, and determine a control signal for generating a relay image, and determine the relay image based on the control signal by using at least one second image among a plurality of second images acquired from the plurality of shooting cameras.

[0013] The at least one processor can control the plurality of shooting cameras based on the control signal.

[0014] The at least one processor may determine data about the at least one object based on the type and location of the at least one object.

[0015] The at least one processor can determine the relay image including the data.

[0016] The at least one processor may input the location of the at least one object into the model, determine a control signal for generating a replay image, and determine the replay image based on a plurality of second images of previous frames acquired from the plurality of shooting cameras according to the control signal for generating the replay image.

[0017] The at least one processor may process the relay image based on information of a user terminal that is connected to the electronic device and transmit the processed relay image to the user terminal.

[0018] According to various embodiments, an electronic device includes at least one processor and a memory electrically connected to the at least one processor and storing at least one command executed by the processor, wherein the processor, when the at least one command is executed, causes the electronic device to receive a plurality of first images captured with respect to a shooting space from a plurality of detection cameras, identify at least one object included in the plurality of first images for each set frame, determine a position of the at least one object in a three-dimensional coordinate system set with respect to the shooting space for each set frame, input a type and position of the at least one object into a learned model, thereby identifying an event occurring in the shooting space, and using the model, determine a control signal for generating a relay image according to the event, and, based on the control signal, determine the relay image using at least one second image among a plurality of second images acquired from the plurality of shooting cameras.

[0019] The at least one processor can determine data about the at least one object based on the type and location of the at least one object, and determine the relay image including the data based on the event.

[0020] An automatic relay method according to various embodiments may include an operation of receiving a plurality of first images captured with respect to a shooting space from a plurality of detection cameras, an operation of identifying at least one object included in the plurality of first images for each set frame, an operation of determining a position of the at least one object in a three-dimensional coordinate system set with respect to the shooting space for each set frame, an operation of inputting a type and position of the at least one object into a learned model and determining a control signal for generating a relay image, and an operation of determining the relay image using at least one second image among a plurality of second images acquired from the plurality of shooting cameras based on the control signal.

[0021] The operation of determining the above relay image may include an operation of controlling the plurality of shooting cameras based on the control signal.

[0022] The above automatic relay method may further include an operation of determining data regarding the at least one object based on the type and location of the at least one object.

[0023] The operation of determining the above relay image can determine the above relay image including the above data.

[0024] The operation of determining the control signal may include an operation of determining a control signal for generating a replay image by inputting the type and location of the at least one object into the model, and the operation of determining the relay image may include an operation of determining the replay image based on a plurality of second images of previous frames acquired from the plurality of shooting cameras according to the control signal for generating the replay image.

[0025] The above automatic relay method may further include an operation of processing the relay image based on information of a user terminal that is connected to the electronic device and an operation of transmitting the processed relay image to the user terminal.

[0026]

[0027] An automatic relay method and electronic device according to various embodiments can reduce costs and / or manpower consumed for relaying a sports game (or event) by determining a relay image in real time and transmitting the relay image.

[0028] The automatic relay method and electronic device according to various embodiments can improve the quality of relay images by providing analyzed data and relay images.

[0029]

[0030] FIG. 1A is a schematic block diagram of an electronic device, a user terminal, and a camera module according to various embodiments.

[0031] FIG. 1b is a schematic block diagram of an electronic device according to various embodiments.

[0032] FIG. 2 is a flowchart illustrating an operation of an automatic relay method performed by an electronic device according to various embodiments.

[0033] FIG. 3 is a drawing showing a plurality of detection cameras and a plurality of shooting cameras arranged in a shooting space according to various embodiments.

[0034] FIGS. 4A, 4B, 4C, and 4D are diagrams illustrating an operation of an electronic device recognizing an object according to various embodiments.

[0035] FIGS. 5A, 5B, 5C, and 5D are diagrams illustrating an operation of an electronic device according to various embodiments to determine the position of an object.

[0036] FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, and 6H are diagrams showing relay images determined by an electronic device according to various embodiments.

[0037] FIG. 7 is a flowchart illustrating an operation of an automatic relay method performed by an electronic device according to various embodiments.

[0038]

[0039] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concepts described in the embodiments.

[0040] Although terms such as "first" or "second" may be used to describe various components, these terms should be interpreted solely to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

[0041] When it is said that a component is "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0042] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this document, phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can each include any one of the items listed together in that phrase, or all possible combinations thereof. In this specification, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0043] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0044] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0045] The term "~part" as used in this document refers to a software or hardware component such as an FPGA or ASIC, and the "~part" performs certain roles. However, the "~part" is not limited to software or hardware. The "~part" may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. For example, the "~part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "~parts" may be combined into a smaller number of components and "~parts" or further separated into additional components and "~parts." Furthermore, the components and "~parts" may be implemented to execute one or more CPUs within a device or a secure multimedia card. Additionally, '~bu' may include one or more processors.

[0046] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0047]

[0048] FIG. 1A is a schematic block diagram of an electronic device (100), a user terminal (200), and a camera module (300) according to various embodiments.

[0049] As shown in FIG. 1a, the system (10) may include an electronic device (100), a user terminal (200), and a camera module (300).

[0050] Referring to FIG. 1A, an electronic device (100) according to various embodiments may receive a plurality of first images from a plurality of first cameras (300a) (e.g., a plurality of detection cameras (300a)). The electronic device (100) may identify at least one object existing in a shooting space using the plurality of first images. For example, the electronic device (100) may input the plurality of first images into a trained first model (140) to identify at least one object. For example, when a plurality of images are input, the first model (140) may be trained to output the types of objects included in the plurality of images (e.g., a player, a ball, a referee, a goal, a table tennis table, etc.). The electronic device (100) may identify an object for each set frame.

[0051] For example, the electronic device (100) can determine the location of an identified object using a plurality of first images. For example, the electronic device (100) can determine the location of the identified object in a three-dimensional coordinate system set with respect to the shooting space. The electronic device (100) can determine the location of the object for each set frame.

[0052] For example, the electronic device (100) may determine a control signal based on the type and location of the identified object. For example, the electronic device (100) may input the location of the identified object into a learned second model (150) to determine the control signal. The second model (150) may be trained to output a control signal when the type and location of the identified object are input.

[0053] For example, the electronic device (100) can transmit a control signal to a plurality of second cameras (300b) (e.g., a plurality of cameras for shooting (300b)). The electronic device (100) can control the plurality of second cameras (300b) using the control signal.

[0054] For example, the electronic device (100) can determine a relay image based on a control signal. The electronic device (100) can determine a relay image using at least one second image among a plurality of second images received from a plurality of second cameras (300b).

[0055] For example, the control signal may include information about a second camera (300b) selected from among a plurality of second cameras (300b) to generate a relay image, and information about the operation of the selected second camera (300b) (e.g., PTZ (pan, tilt, zoom) values, direction / angle / magnification values, etc.). The electronic device (100) may determine a second camera (300b) to acquire a second image to be used as a relay image according to the control signal. The electronic device (100) may determine a second image acquired from a second camera (300b) selected from among a plurality of second cameras (300b) as a relay image.

[0056] For example, the plurality of second cameras (300b) may be controlled according to a control signal received from the electronic device (100). For example, the plurality of second cameras (300b) may each operate according to information included in the control signal (e.g., PTZ (pan, tilt, zoom) values, direction / angle / magnification values, etc.). For example, the plurality of second cameras (300b) may each include an operation module (e.g., an actuator, a motor, etc.). Each of the operation modules of the plurality of second cameras (300b) may operate according to the control signal.

[0057] For example, the electronic device (100) can transmit a relay image to a user terminal (200). The electronic device (100) can receive information about the user terminal (200) from the user terminal (200) (e.g., terminal type, terminal specifications, user information, etc.). The electronic device (100) can post-process the relay image based on the information about the user terminal (200). The electronic device (100) can transmit the post-processed relay image to the user terminal (200).

[0058] For example, post-processing may include changing the format of an image, compressing an image, or resizing an image. For example, post-processing may include changing the content of a broadcast image, such as adding additional images (e.g., advertisements) or auxiliary objects to the broadcast image.

[0059]

[0060] FIG. 1b is a schematic block diagram of an electronic device (100) according to various embodiments.

[0061] Referring to FIG. 1B, an electronic device (100) according to various embodiments may include a processor (110), a memory (120), a communication circuit (130), a first model (140), and a second model (150).

[0062] The memory (120) can store various data used by at least one component (e.g., processor (110)) of the electronic device (100). For example, the data can include input data or output data for software (or programs, applications, etc.) and commands related thereto. The memory (120) can include volatile memory or non-volatile memory.

[0063] For example, the processor (110) may execute software (or a program, an application, etc.) to control at least one other component of the electronic device (100) connected to the processor (110). The processor (110) may execute the software and display or output the processed result through a display module or an audio output module (e.g., a speaker). The processor (110) may execute the software to perform data processing or calculation. For example, the processor (110) may store commands or data received from other components of the electronic device (100) (e.g., the communication circuit (130), the first model (140), the second model (150)) in the memory (120). The processor (110) may process commands or data stored in the memory (120) and store result data in the memory (110).

[0064] For example, the communication circuit (130) can establish a direct communication channel or a wireless communication channel between the electronic device (100) and an external electronic device (e.g., user terminal (200), first camera (300a), second camera (300b), camera module (300)), and support wired / wireless communication through the established communication channel.

[0065] For example, the electronic device (100) can establish a communication channel with the user terminal (200) using the communication circuit (130) and transmit a relay image. For example, the electronic device can establish a communication channel with the first camera (300a) using the communication circuit (130) and receive a first image from the first camera (300a). For example, the electronic device can establish a communication channel with the second camera (300b) using the communication circuit (130) and receive a second image from the second camera (300b). The electronic device (100) can transmit a control signal to the second camera (300b).

[0066] For example, the first model (140) can be trained to output the types of objects included in a plurality of images (e.g., a plurality of first images) when input. For example, the description of a neural network model for recognizing objects in images known to the first model (140) can be substantially identically applied. For example, the description of a known object detection model and an object recognition model can be substantially identically applied to the first model (140).

[0067] According to one embodiment, the electronic device (100) can determine the location of at least one identified object. For example, the electronic device (100) can determine the location of the object identified by the first model (140). The electronic device (100) can determine the location (e.g., 3D coordinates) of the identified object within a set 3D coordinate system.

[0068] For example, the electronic device (100) can input multiple first images into the first model (140) to identify additional information. For example, the additional information may include player jersey numbers, referee gestures, and information displayed on the scoreboard. The additional information is not limited to the examples above and may include objects and / or information that can be recognized by analyzing the multiple first images.

[0069] For example, the second model (150) can be trained to output a control signal for generating a relay image when the type and location of an object are input. For example, the type and location of an object can be input to the second model (150) for each set frame. The second model (150) can be trained to output a control signal using the type and location of an object input for each set frame.

[0070] For example, the electronic device (100) can determine the type of at least one object included in a plurality of first images for each set frame. The electronic device (100) can determine the location of at least one object included in a plurality of first images for each set frame. The electronic device (100) can input the type and location of at least one object for each set frame into the second model (150) to determine a control signal.

[0071] For example, when a table tennis match is being played in a shooting space, the electronic device (100) can identify the type of each object (e.g., player, referee, table tennis ball, table tennis table, scoreboard, etc.) using a plurality of first images acquired from a plurality of detection cameras (300a). The electronic device (100) can determine the location of each identified object.

[0072] A table tennis match can include events such as player entrance, warm-up, match start, serve, rally, and scoring. Depending on the nature of the events involved in a table tennis match, the positions of each object may have different characteristics.

[0073] For example, when a player enters the court, they may stand in a waiting position. Furthermore, the positions of players during warm-ups before the start of a match may differ from those during a serve / rally. When serving, a player assumes a ready position, then throws the ball a certain height above the ground and strikes it. During a serve, the ball may remain in a low position for a certain period of time, rise above a certain height, then descend again to be struck by the player.

[0074] During a rally, the ball can travel back and forth between the two sides of the table at high speed. During a rally, players can move on either side of the table.

[0075] A point may be scored if the ball leaves the table without hitting the table, hits an area other than the opponent's table, or if the player fails to hit the ball.

[0076] As described above, the positions (or temporal changes in positions) of objects (players, balls, referees, etc.) may have different characteristics for each event (e.g., player entrance, warm-up, serve, rally, score, etc.). The cameras used to capture the video to be broadcast may be determined differently for each event. The direction, angle, and magnification of the selected cameras may also vary for each event.

[0077] For example, in a table tennis match, serves, rallies, and scoring may occur. The electronic device (100) can determine the type (e.g., player, ball, referee, etc.) and location of each object for each set frame. The electronic device (100) can input the type and location of each object for each set frame into the second model (150).

[0078] For example, when a sub is in progress, the electronic device (100) can input the type and location of the object when the sub is in progress to the second model (150) for each set frame. The second model (150) can output a control signal using the type and location of the object when the sub is in progress. For example, the control signal can include information for determining a close-up image of the player performing the sub as the relay image.

[0079] The electronic device (100) can transmit a control signal to a plurality of filming cameras (300b). Based on the control signal, the electronic device (100) can determine an image acquired from a filming camera that takes a close-up shot of a player serving as a relay image.

[0080] For example, when a rally is in progress, the electronic device (100) can input the type and location of objects during the rally to the second model (150) for each set frame. The second model (150) can output a control signal using the type and location of objects during the rally. For example, the control signal can include information for determining a video taken from behind one player in the direction of the table tennis table as a relay video.

[0081] The electronic device (100) can transmit a control signal to a plurality of filming cameras (300b). Based on the control signal, the electronic device (100) can determine an image acquired from a filming camera that captures the direction of the table tennis table from behind one player as a relay image.

[0082] For example, when a goal is scored, the electronic device (100) can input the type and location of the object at the time of the goal to the second model (150) for each set frame. The second model (150) can output a control signal using the type and location of the object at the time of the goal. For example, the control signal can include information for determining a close-up image of the player who scored as the relay image.

[0083] The electronic device (100) can transmit a control signal to a plurality of shooting cameras (300b). Based on the control signal, the electronic device (100) can determine an image acquired from a shooting camera that takes a close-up shot of a scoring player as a relay image.

[0084] As described above, a sports game can be divided into individual events. When each event that constitutes a sports game occurs, each object may have different location characteristics (or temporal position changes). The electronic device (100) inputs the type and location of each object into the second model (150), selects at least one camera for shooting among a plurality of cameras for shooting (300b) arranged in a shooting space, and controls the operating characteristics (e.g., PTZ value, direction / angle / magnification, etc.) of the selected camera for shooting.

[0085] In the above example, the operation of the electronic device (100) was described when a table tennis match was in progress, but is not limited thereto. For example, even when various games or events, such as soccer, basketball, tennis, or hockey, are in progress, the electronic device (100) can operate substantially in the same manner as described above.

[0086] For example, training data for training the second model (150) may include input data and correct answer data. For example, the training data may include the types and locations of each object acquired in a shooting space where multiple detection cameras (300a) and multiple shooting cameras (300b) are installed, as well as control signals for generating relay images.

[0087] For example, the learning data may include the types and locations of each object, as well as relay images, acquired in a shooting space where multiple detection cameras (300a) and multiple recording cameras (300b) are installed. The relay image may be understood as being substantially identical to a control signal for generating the relay image. For example, the relay image may indicate the recording camera that captured the image used in the relay image, and operation information of the recording camera (e.g., PTZ value, direction / angle / magnification, etc.).

[0088] The second model (150) can be trained to output a control signal for generating a relay signal using the type and location of each input object. The electronic device (100) can generate a relay image according to the location of each object using the control signal.

[0089] For example, the electronic device (100) can input the type and location of at least one object into the second model (150) to identify an event that occurred in the shooting space. The electronic device (100) can use the second model (150) to determine a control signal for generating a relay image according to the event. For example, training data for training the second model (150) can include the type and location of each object, an event, and a control signal. The second model (150) can be trained to output an event when the type and location of each object is input. The second model (150) can be trained to output a control signal when the type and location of each object and an event are input. The event can be intermediate output data of the second model (150), and the control signal can be final output data of the second model (150).

[0090] For example, the electronic device (100) may input the type and location of an object into the second model (150) to generate a control signal for generating a replay image. For example, the control signal for generating a replay image may include information about a second image to be used as a replay image among a plurality of second images of a previous frame. The control signal for generating a replay image may include information about the second image to be used as a replay image, such as the start time, end time, and playback speed (e.g., 0.5x speed, 2x speed, etc.) of the replay image.

[0091] For example, the control signal may include information for generating a replay image from a previously transmitted relay image. The electronic device (100) may generate a replay image using the previously transmitted relay image to the user terminal (200).

[0092] For example, the control signal may include information for generating a replay image from a plurality of second images of previous frames that were not used in the relay image. The electronic device (100) may generate a replay image using at least one second image among the plurality of second images that were not transmitted to the user terminal (200).

[0093]

[0094] FIG. 2 is a flowchart of an automatic relay method performed by an electronic device (100) according to various embodiments.

[0095] For example, the electronic device (100) may receive a plurality of first images captured with respect to a shooting space from a plurality of detection cameras (300a) in operation (210). For example, the plurality of detection cameras (300a) may each have fixed positions and movements. The plurality of detection cameras (300a) may each have fixed movement information (e.g., PTZ values, direction / angle / magnification, etc.).

[0096] For example, the electronic device (100) can identify at least one object included in a plurality of first images for each frame set in operation (220). For example, the electronic device (100) can input the plurality of first images into a first model (140) to identify at least one object included in the plurality of first images. For example, the first model (140) can output the type of at least one object included in the plurality of first images.

[0097] For example, the electronic device (100) can determine the position of at least one object in a three-dimensional coordinate system set with respect to the shooting space for each frame set in the operation (230).

[0098] For example, the electronic device (100) can determine the position (3D coordinates) of the object in a set 3D coordinate system based on the 2D coordinates of the object in the plurality of first images and the positions of the plurality of detection cameras (300a).

[0099] For example, the electronic device (100) may determine one or more locations of an object, depending on the type of object. For example, if the object is a person, the electronic device (100) may determine the head location and the center of the chest location as the person's location. For example, if the object is a ball, the electronic device (100) may determine the center location of the ball as the ball's location.

[0100] The operation of determining the position of an object according to the type of the object described above is exemplary, and the electronic device (100) may determine the position of the object differently from the above example. For example, if the object is a person, the electronic device (100) may determine the skeleton information of the person and determine the position of each joint with respect to the person as the position of the person.

[0101] For example, the electronic device (100) can input the type and location of at least one object in the operation (240) into a learned model (e.g., a second model (150)) to determine a control signal for generating a relay image.

[0102] For example, the second model (150) can be trained to output a control signal using the type and location of the input object. For example, the control signal can include information for generating a relay image. The control signal can include information for controlling the operation of a plurality of shooting cameras (300b). The control signal can include information regarding a shooting camera among the plurality of shooting cameras (300b) to be used as a relay image.

[0103] For example, the electronic device (100) may determine a relay image by using at least one second image among a plurality of second images acquired from a plurality of shooting cameras (300b) based on a control signal in operation (250). The electronic device (100) may determine an appropriate second image as a relay image based on the type and location of an object by generating a relay image according to the control signal output from the second model (150).

[0104] For example, the electronic device (100) can input the type and location of at least one object into the model to determine a control signal for generating a replay image.

[0105] For example, the electronic device (100) can input the type and location of an object into the second model (150) to generate a control signal for generating a replay image.

[0106] For example, the electronic device (100) can determine a replay image based on a plurality of second images of previous frames acquired from a plurality of shooting cameras (300b) according to a control signal for generating a replay image.

[0107] For example, the control signal may include information for generating a replay image from a previously transmitted relay image. For example, the control signal may include information for generating a replay image from a plurality of second images of previous frames that are not used in the relay image.

[0108] The operations (210) to (250) illustrated in FIG. 2 can be performed substantially identically by the processor (110) of the electronic device (100).

[0109] The operations (210) to (250) illustrated in FIG. 2 may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0110]

[0111] FIG. 3 is a drawing showing a plurality of detection cameras (300a) and a plurality of shooting cameras (300b) arranged in a shooting space according to various embodiments.

[0112] Figure 3 is a drawing showing a plurality of detection cameras (300a) and a plurality of shooting cameras (300b) deployed when the shooting location is a soccer stadium.

[0113] For example, the electronic device (100) can receive a plurality of first images from a plurality of detection cameras (300a) positioned at a shooting location. The electronic device (100) can identify at least one object using the plurality of first images. The electronic device (100) can determine the location of at least one identified object using the plurality of first images. The electronic device (100) can determine the location (e.g., 3D coordinates) of at least one object in a set 3D coordinate system.

[0114] For example, the electronic device (100) may input the type and location of at least one object into a learned model (e.g., the second model (150)) to determine a control signal for generating a relay image. The electronic device (100) may receive a plurality of second images from a plurality of capturing cameras (300b). The electronic device (100) may generate a relay image using at least one second image among the plurality of second images according to the control signal.

[0115] The electronic device (100) can transmit the generated relay image to the user terminal (200).

[0116] The arrangement of the plurality of detection cameras (300a) and the plurality of photographing cameras (300b) illustrated in FIG. 3 is exemplary and is not limited to the example illustrated in FIG. 3.

[0117]

[0118] FIG. 4a, FIG. 4b, FIG. 4c, and FIG. 4d are drawings showing an operation of an electronic device (100) recognizing an object according to various embodiments.

[0119] FIG. 4a is a drawing showing a plurality of detection cameras (300a), a plurality of shooting cameras (300b), an object (410), and an environment (420-1, 420-2) placed at a shooting location.

[0120] For example, the electronic device (100) can identify an object (410) using a plurality of first images received from a plurality of detection cameras (300a). The electronic device (100) can determine the location of the object (410) using a plurality of first images received from a plurality of detection cameras (300a).

[0121] For example, the electronic device (100) can identify additional information using a plurality of first images. For example, the electronic device (100) can identify additional information from objects included in the plurality of first images. For example, the additional information may include player jersey numbers, referee gestures, and information displayed on an electronic scoreboard. The electronic device (100) can identify the environment (420-1, 420-2) from the plurality of first images.

[0122] In the above example, the electronic device (100) can input a plurality of first images into a learned model (e.g., the first model (140)) to identify the environment (420-1, 420-2) or additional information.

[0123] FIG. 4b is a drawing showing a plurality of first images (311, 313, 315, 317) received from a plurality of detection cameras (300a).

[0124] In FIG. 4b, each of the plurality of first images (311, 313, 315, 317) may include an object (411). Each of the plurality of first images (311, 313, 315, 317) may be captured by a plurality of detection cameras (300a) positioned at different locations. The locations of the objects (411) within each of the plurality of first images (311, 313, 315, 317) may be different from each other.

[0125] The electronic device (100) can identify an object (411) by inputting a plurality of first images (311, 313, 315, 317) into the first model (140). The electronic device (100) can determine the type of the object (411) as a player.

[0126] Fig. 4c is a diagram showing a plurality of first images (311, 313, 315, 317) received from a plurality of detection cameras (300a). Unlike Fig. 4b, Fig. 4c shows that the first images (311) and (313) include an object (411), but the first images (315) and (317) do not include the object (411). For example, if the detection cameras that captured the first images (315) and (317) are not facing the object (411), the first images (315) and (317) may not include the object (411). For example, even if the detection camera that captured the first image (315) and the first image (317) is directed toward the object (411), if the object (411) is obscured by an obstacle or another object, the first image (315) and the first image (317) may not include the object (411).

[0127] As shown in FIG. 4c, even if the first image (311) and the first image (313) include an object (411), but the first image (315) and the first image (317) do not include an object (411), the electronic device (100) can identify the object (411) using a plurality of first images (311, 313, 315, 317).

[0128] FIG. 4d is a drawing showing objects (411, 413, 415, 417) recognized using a plurality of first images (311, 313, 315, 317).

[0129] The electronic device (100) can identify objects (411, 413, 415, 417), as shown in FIG. 4D. For example, the electronic device (100) can identify objects (411) and (413) as players. For example, the electronic device (100) can identify object (415) as a referee. For example, the electronic device (100) can identify object (417) as a soccer ball.

[0130]

[0131] FIG. 5a, FIG. 5b, FIG. 5c, and FIG. 5d are drawings showing an operation of an electronic device (100) determining the position of an object according to various embodiments.

[0132] FIG. 5a is a diagram showing a three-dimensional coordinate system (500) and object coordinates (520, 530) according to various embodiments.

[0133] For example, a three-dimensional coordinate system (500) may be set with respect to the shooting space. For example, a plurality of detection cameras (300a) may be positioned around the shooting space. Through calibration, distortion of a plurality of first images acquired from a plurality of detection cameras (300a) positioned around the shooting space may be eliminated.

[0134] For example, the origin (510) of the three-dimensional coordinate system (500) can be determined as an arbitrary location in the shooting location. As shown in Fig. 5a, one corner of a soccer field can be determined as the origin (510) of the three-dimensional coordinate system (500).

[0135] For example, a calibration plate placed in a shooting location can be captured by at least one detection camera. In one or more images captured by the calibration plate, pixels indicating the calibration plate can be understood as capturing the same location. By repeatedly capturing calibration plates placed in different locations within the shooting location using one or more detection cameras, the positions of each detection camera can be estimated (or determined), and a three-dimensional coordinate system (with an arbitrary location within the shooting location as its origin) can be established.

[0136] For example, when the origin (510) of the 3D coordinate system (500) is determined, the 3D coordinate system (500) can be set with respect to the shooting location. The positions (e.g., 3D coordinates) of the plurality of detection cameras (300a) can be determined in the 3D coordinate system (500). The positions (e.g., 3D coordinates) of the plurality of shooting cameras (300b) can be determined in the 3D coordinate system (500).

[0137] As shown in FIG. 5A, the electronic device (100) can determine the position of an object (e.g., 3D coordinates). For example, the electronic device (100) can determine the position of a person as coordinates (520) ((x24, y11, z1) or (24, 11, 1)). The electronic device (100) can determine the position of a ball as coordinates (520) ((x64, y47, z3) or (64, 47, 3)).

[0138] FIG. 5b is a drawing showing the coordinates (321, 323, 325, 327) of an object in a plurality of first images (311, 313, 315, 317) received from a plurality of detection cameras (300a).

[0139] A plurality of first images (311, 313, 315, 317) are two-dimensional, and the coordinates of the object (321, 323, 325, 327) within each image can be expressed as two-dimensional coordinates (e.g., (x1, y1), (x2, y2), (x3, y3), (x4, y4)).

[0140] For example, the position of the detection camera that captured the first image (311) may be (x1, y1, z1). Using the coordinates (321) of the object in the first image (311), the direction and / or angle of viewing the object from the position (x1, y1, z1) of the detection camera may be calculated.

[0141] Using the position (x1', y1', z1') of the detection camera and the direction and / or angle at which the detection camera views the object from the position (x1', y1', z1'), a virtual first straight line passing through the position (x1', y1', z1') of the detection camera and the object can be calculated.

[0142] In a manner substantially the same as calculating the position (x1', y1', z1') of the detection camera and the virtual first straight line passing through the object, the position (x2', y2', z2') of the detection camera and the virtual second straight line passing through the object can be calculated using the coordinates (x2, y2) of the object and the position (x2', y2', z2') of the detection camera in the second image (313). The position (x3', y3', z3') of the detection camera and the virtual third straight line passing through the object, and the position (x4', y4', z4') of the detection camera and the virtual fourth straight line passing through the object can also be calculated substantially the same as in the above example.

[0143] For example, the electronic device (100) can calculate the intersection point of the first straight line to the fourth straight line. The electronic device (100) can determine the coordinates of the intersection point of the first straight line to the fourth straight line as the coordinates of the object.

[0144] The description of the operation of the above electronic device (100) to determine the coordinates of an object by using the coordinates (321, 323, 325, 327) of the object in the plurality of first images (311, 313, 315, 317) and the positions of the plurality of detection cameras (300a) is exemplary and is not limited to the above example. The electronic device (100) can determine the coordinates of an object by applying a known method for calculating the positions of the plurality of detection cameras (300a) in a set three-dimensional coordinate system or a method that can be easily derived by a person skilled in the art.

[0145] FIG. 5C is a diagram showing a plurality of first images (311, 313, 315, 317) received from a plurality of detection cameras (300a). Unlike FIG. 5B, FIG. 5C shows that the first images (311) and (313) include an object (411), but the first images (315) and (317) do not include an object (411). The electronic device (100) can determine the position of the object by using the coordinates (321, 323) of the object in the first images (311) and (313), and the positions of the detection cameras that captured the first images (311) and (313).

[0146] FIG. 5D is a diagram illustrating an operation of an electronic device (100) to determine the position of an object in time series. The electronic device (100) can determine the position of the object for each set frame. For example, when each of a plurality of detection cameras (300a) as in FIG. 5D captures an image at 8.3 ms / 120 fps, the electronic device (100) can determine the position of the object for each frame. The electronic device (100) can determine the position (541) of an object (e.g., a ping-pong ball) as (x1, y1, z1) in one frame. The electronic device (100) can determine the position (542) of an object (e.g., a ping-pong ball) as (x2, y2, z2) in two frames. The electronic device (100) can determine the position (543) of an object (e.g., a ping-pong ball) as (x3, y3, z3) in three frames. The electronic device (100) can determine the position (544) of an object (e.g., a ping-pong ball) as (x4, y4, z4) in the 4th frame. The electronic device (100) can determine the position (545) of an object (e.g., a ping-pong ball) as (x5, y5, z5) in the 5th frame. The electronic device (100) can determine the position (546) of an object (e.g., a ping-pong ball) as (x6, y6, z6) in the 6th frame. The electronic device (100) can determine the position (547) of an object (e.g., a ping-pong ball) as (x7, y7, z7) in the 7th frame. The electronic device (100) can determine the position (548) of an object (e.g., a ping-pong ball) as (x8, y8, z8) in the 8th frame.

[0147] The electronic device (100) can input the location of the object into a learned model (e.g., a second model (150)) for each set frame (e.g., 1 frame).

[0148] The method by which the electronic device (100) determines the position of the object is not limited to the examples illustrated in FIGS. 5A, 5B, 5C, and 5D. For example, the electronic device (100) can determine the three-dimensional coordinates of the object by using the distance between the plurality of detection cameras (300a) and the object and the positions of the detection cameras (300a). For example, the electronic device (100) can calculate the direction from the detection camera to the object by using the coordinates (321) of the object in the first image (321). The electronic device (100) can determine the position (e.g., three-dimensional coordinates) of the object within the three-dimensional coordinate system by using the positions of the detection cameras, the directions from the detection cameras to the object, and the distance between the detection cameras and the object.

[0149] The distance between the multiple detection cameras (300a) and the object can be calculated using various known methods. For example, the multiple capture cameras (300a) may be depth cameras. For example, if the multiple capture cameras (300a) are a single camera, various methods and / or algorithms for estimating the distance between the single camera and the object can be applied.

[0150]

[0151] FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, and 6H are diagrams showing relay images determined by an electronic device (100) according to various embodiments.

[0152] Hereinafter, the operation of the electronic device (100) and the data generated will be described using the example of a table tennis match being held at a filming location, but is not limited thereto. For example, even if a ball game such as soccer, basketball, baseball, or volleyball, or a sport such as track and field or high jump, is being held at the filming location, the description of the operation of the electronic device (100) and the data generated can be applied substantially in the same manner.

[0153] Figures 6a to 6d are drawings showing relay images (610, 620, 630, 640) determined according to the type and location of an object when a table tennis match is in progress.

[0154] As shown in FIG. 6a, when a table tennis match begins and a player enters, the electronic device (100) can determine a broadcast image (610) based on the type of object (e.g., player, ball) and location.

[0155] As shown in FIG. 6b, when a player prepares a serve, the electronic device (100) can determine a relay image (620) based on the type of object (e.g., player, ball) and location.

[0156] As shown in FIG. 6c, when the sub starts, the electronic device (100) can determine the relay image (630) based on the type (e.g., player, ball) and location of the object.

[0157] As shown in FIG. 6d, when a score occurs, the electronic device (100) can determine a relay image (640) based on the type (e.g., player, ball) and location of the object.

[0158] In FIGS. 6A to 6D , the electronic device (100) can determine relay images (610 to 640) using a plurality of second images received from a plurality of shooting cameras (300b) according to a control signal. The electronic device (100) can determine the control signal by inputting the type and location of an object into the learned second model (150).

[0159] In FIGS. 6A to 6D , the electronic device (100) can input the type and location of an object to the second model (150) to determine a control signal. The electronic device (100) can control the operation of at least one of the plurality of photographing cameras (300b) according to the control signal. The electronic device (100) can determine a relay image (610 to 640) using at least one second image among the plurality of received second images.

[0160] As shown in FIGS. 6A to 6D, when a table tennis match is in progress, the electronic device (100) can automatically determine a broadcast image (610 to 640). The electronic device (100) can use the second model (150) to determine a broadcast image (610 to 640) for smooth and natural broadcasting of the table tennis match depending on the type and location of the object.

[0161] The progress of the table tennis match in the relay images (610 to 640) illustrated in FIGS. 6A to 6D above is illustrative and is not limited to the illustrated examples. For example, even in cases such as player entrances, warm-ups, team changes, breaks, and the end of the match, the electronic device (100) can input the type and location of an object into the second model (150) to determine a control signal, and determine a relay image based on the control signal.

[0162] FIG. 6e is a drawing showing a relay image (650) containing data.

[0163] For example, the electronic device (100) may determine data regarding at least one object based on the type and location of at least one object. For example, if the object is a person or a ping-pong ball, the electronic device (100) may determine data regarding the locations of the person and the ping-pong ball (e.g., movement distance, movement trajectory, average movement speed, attack pattern, defense pattern, serve success rate, serve failure rate, etc.).

[0164] For example, the electronic device (100) can determine prediction data using the type and location of an object and data. For example, the prediction data may include a player's probability of winning.

[0165] FIG. 6e illustrates a replay image (650) determined by an electronic device (100) using one or more second images. The electronic device (100) can determine a replay image (650) that includes a second image (653). The second image displayed in the background of the replay image (650) may be a different image from the second image (653).

[0166] The electronic device (100) can determine a replay image using at least one second image of a previous frame. The electronic device (100) can input the type of object (e.g., a person, a ping-pong ball) and its location into the second model (150) to determine a control signal for generating a replay image (650). Based on the control signal, the electronic device (100) can determine the start and end points of the replay image (650) and at least one second image for generating the replay image (650). For example, the electronic device (100) can generate the replay image (650) after a score occurs and before the next game begins.

[0167] For example, the electronic device (100) can generate a replay image (650) using at least one second image and data. As shown in FIG. 6E, the replay image (650) can include data (651). The data (651) can be data regarding the position of an object (e.g., a ping-pong ball) (e.g., a trajectory of the ball). The electronic device (100) can determine a replay image (650) including one or more second images and data.

[0168] In FIG. 6e, the case where the image (650) is a replay image is described as an example, but the present invention is not limited thereto. For example, when a rally is in progress, the electronic device (100) can determine a relay image such as the image (650).

[0169] FIG. 6f is a drawing showing a relay image (660) containing data.

[0170] As illustrated in FIG. 6F, the electronic device (100) can determine a relay image (660) representing the trajectory of a table tennis ball. For example, the electronic device (100) can determine the relay image (660) after a score is scored and before the players proceed to the next game. The electronic device (100) can determine the relay image (660) containing data based on the type (e.g., person, table tennis ball) and location of the object.

[0171] FIG. 6g is a drawing showing a relay image (670) displaying data provided by an electronic device (100).

[0172] As shown in Figure 6g, the data may include data about Player 1 in Match 1. The data may include the percentage of the area attacked by Player 1, and the average / maximum speed of the ping pong ball hit by Player 1.

[0173] The data shown in Fig. 6g is exemplary and is not limited to the example shown.

[0174] FIG. 6h is a diagram showing data and relay images (680) provided by an electronic device (100).

[0175] As shown in FIG. 6h, the electronic device (100) can determine a relay image (680) containing data.

[0176] For example, the electronic device (100) can generate a relay image in which visualized data overlaps a second image, as in the image (650) of FIG. 6e. For example, the electronic device (100) can generate a relay image (680) in which visualized data is separated from the second image, as in FIG. 6h.

[0177] For example, the data may include, but is not limited to, scoring rate / number of points, attack point percentage, serve point scoring rate / number of points, average speed, and top speed of Player 1's attack patterns (e.g., drives, backhand drives, cuts).

[0178]

[0179] FIG. 7 is a flowchart of an automatic relay method performed by an electronic device (100) according to various embodiments.

[0180] For example, the electronic device (100) can receive a plurality of first images captured with respect to a shooting space from a plurality of detection cameras (300a) in operation (710).

[0181] For example, the electronic device (100) can identify at least one object included in a plurality of first images for each frame set in operation (720).

[0182] For example, the electronic device (100) can determine the position of at least one object in a three-dimensional coordinate system set with respect to the shooting space for each frame set in the operation (730).

[0183] For example, the electronic device (100) may input the type and location of at least one object into a learned model (e.g., the second model (150)) in operation (740) to identify an event occurring in the shooting space. For example, if a table tennis match is being played in the shooting space, the event may include events such as a player entering, a warm-up, a serve preparation, a serve, a serve fault, a rally, a score occurrence, and a change of sides.

[0184] Events are not limited to the examples described above, and may include all events that may occur from the start to the end of a table tennis match. Furthermore, even if a game or event other than a table tennis match is in progress at the filming location, the electronic device (100) can identify events. For example, the electronic device (100) can identify events that may occur in other games (e.g., in a soccer game, a kickoff, offside, foul, goal, penalty kick, shot, pass, etc.). For example, if a performance is in progress, the electronic device (100) can identify events that may occur during the performance.

[0185] For example, the electronic device (100) may use the model in operation (750) to determine a control signal for generating a relay image based on an event. For example, the identified event may be an intermediate output of the second model (150). The second model (150) may output a control signal based on the type and location of the input object and the event.

[0186] For example, the electronic device (100) can determine a relay image by using at least one second image among a plurality of second images acquired from a plurality of shooting cameras (300b) based on a control signal in operation (760).

[0187] With respect to the above-described operations (710), (720), (730), and (760), the descriptions of operations (210), (220), (230), and (250) of FIG. 2 can be substantially equally applied.

[0188] The operations (710) to (760) illustrated in FIG. 7 can be performed substantially identically by the processor (110) of the electronic device (100).

[0189] The operations (710) to (760) illustrated in FIG. 7 may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0190]

[0191] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the operating system. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0192] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0193] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0194] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0195] Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the described embodiments. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0196] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In electronic devices, at least one processor; and A memory electrically connected to at least one processor and storing at least one instruction executed by the processor. Including, The above processor, When at least one of the above commands is executed, the electronic device receives a plurality of first images captured with respect to the shooting space from a plurality of detection cameras; Identifying the type of at least one object included in the plurality of first images for each set frame; For each of the above-set frames, determine the position of at least one object in a three-dimensional coordinate system set with respect to the shooting space; By inputting the type and location of at least one object into the learned model, a control signal for generating a relay image is determined; Based on the above control signal, the relay image is determined by using at least one second image among a plurality of second images acquired from a plurality of shooting cameras. Electronic devices.

2. In paragraph 1, At least one processor, Based on the above control signal, controlling the plurality of shooting cameras, Electronic devices.

3. In paragraph 1, At least one processor, Determining data about the at least one object based on the type and location of the at least one object, Electronic devices.

4. In paragraph 3, At least one processor, Determining the relay image including the above data, Electronic devices.

5. In paragraph 1, At least one processor, By inputting the type and location of at least one object into the model, a control signal for generating a replay image is determined, In accordance with a control signal for generating the replay image, determining the replay image based on a plurality of second images of previous frames acquired from the plurality of shooting cameras. Electronic devices.

6. In paragraph 1, At least one processor, Processing the relay image based on information of a user terminal (200) connected to the electronic device; Transmitting the processed relay image to the user terminal (200), Electronic devices.

7. In electronic devices, at least one processor; and A memory electrically connected to at least one processor and storing at least one instruction executed by the processor. Including, The above processor, When at least one of the above commands is executed, the electronic device receives a plurality of first images captured with respect to the shooting space from a plurality of detection cameras; Identifying at least one object included in the plurality of first images for each set frame; For each of the above-set frames, determine the position of at least one object in a three-dimensional coordinate system set with respect to the shooting space; By inputting the type and location of at least one object into the learned model, an event occurring in the shooting space is identified; Using the above model, a control signal for generating a relay image according to the above event is determined; Based on the above control signal, the relay image is determined by using at least one second image among a plurality of second images acquired from a plurality of shooting cameras. Electronic devices.

8. In paragraph 7, At least one processor, Based on the type and location of the at least one object, determine data about the at least one object; Based on the above event, determining the relay image including the data, Electronic devices.

9. In the automatic relay method, An operation of receiving a plurality of first images captured with respect to a shooting space from a plurality of detection cameras; An operation of identifying at least one object included in the plurality of first images for each set frame; For each of the above-set frames, an operation of determining the position of at least one object in a three-dimensional coordinate system set with respect to the shooting space; An operation of inputting the type and location of at least one object into a learned model to determine a control signal for generating a relay image; and An operation of determining the relay image by using at least one second image among a plurality of second images acquired from a plurality of shooting cameras based on the above control signal. including, Automatic relay method.

10. In paragraph 9, The action of determining the above relay image is: An operation of controlling the plurality of shooting cameras based on the above control signal. including, Automatic relay method.

11. In paragraph 9, An operation of determining data regarding at least one object based on the type and location of the at least one object. including more, Automatic relay method.

12. In paragraph 11, The action of determining the above relay image is: Determining the relay image including the above data, Automatic relay method.

13. In paragraph 1, The operation of determining the above control signal is: An operation of determining a control signal for generating a replay image by inputting the type and location of at least one object into the model. Including, The action of determining the above relay image is: An operation of determining the replay image based on a plurality of second images of previous frames acquired from the plurality of shooting cameras according to a control signal for generating the replay image. including, Automatic relay method.

14. In paragraph 9, An operation of processing the relay image based on information of a user terminal (200) connected to the electronic device; and An operation of transmitting the processed relay image to the user terminal (200). including more, Automatic relay method.

15. A computer program stored on a computer-readable recording medium for executing the method of any one of claims 9 to 14 in combination with hardware.

Images