Methods, systems and media for analyzing spherical video content

Abstract

Methods, systems, and media for analyzing spherical video content are provided. More specifically, methods, systems, and media for detecting the misuse of two-dimensional videos placed on a sphere in spherical video content are provided. In some embodiments, the method includes: receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views; and selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, wherein neighboring points of the first frame are mapped to corresponding neighboring points of the area according to the mapping, and one or more continuous portions of the frame are respectively mapped to corresponding multiple continuous portions of the area; identifying a partition within the area that meets a criterion, the criterion indicating that the area has a probability greater than a threshold value of including a specific type of content; in response to identifying a partition within the area that meets the criterion, analyzing the identified partition of the area using a video fingerprinting technique; in response to determining using the video fingerprinting technique that content associated with the identified partition of the area matches a reference content item in a plurality of reference content items, generating an indication of a match associated with the identifier of the spherical video content item.

Description

Technical Field

[0001] The disclosed subject matter relates to methods, systems, and media for analyzing spherical video content and, in some embodiments, detecting misuse of two-dimensional video placed on a sphere in the spherical video content by tiling the sphere. More specifically, the disclosed subject matter relates to identifying a match between content in the spherical video content and one of a plurality of reference content items. Background Art

[0002] People often view video content via different video service providers, for example, by streaming content from a service provider. In many cases, the video content is compared to a database of reference video files to determine whether the video content matches one of the reference video files. For example, this helps determine whether the video content includes content that is already copyrighted by another content creator or entity.

[0003] Immersive or 360-degree spherical video content has become increasingly popular, including video content recorded from multiple views that can be manipulated and navigated by a user. However, it can be difficult to determine where copyrighted video content is superimposed on, inserted into, or stitched with spherical video content.

[0004] Therefore, it is desirable to provide methods, systems, and media for analyzing spherical video content. Summary of the invention

[0005] New and useful methods, systems, and media are provided for analyzing spherical video content.

[0006] In particular, methods, systems and media are provided for the technical problem of detecting a match between spherical video content and a plurality of reference content items.

[0007] In an example, the methods, systems, and media can be used to detect misuse of spherical video content by tiling the sphere to place a two-dimensional video on a sphere. In this case, the methods, systems, and media can be used to detect potential copyrighted content. In another example, the methods, systems, and media can be used to index spherical video content, such as by assigning tags to the spherical video content related to matching reference items. This allows for an automated indexing process.

[0008] According to some embodiments of the disclosed subject matter, a computer-implemented method for analyzing spherical video content is provided, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views, wherein the spherical video content item is encoded as multiple two-dimensional video frames; selecting a first frame of the multiple two-dimensional video frames associated with the spherical video content item; dividing the first frame into multiple tiles, the tiles spanning the first frame of the spherical video content item; for each tile of the multiple tiles, calculating a probability (i.e., a probability value) that the tile includes a specific type of content; determining for each tile whether the probability exceeds a predetermined threshold; in response to determining that the probability exceeds the predetermined threshold for a particular tile, causing content associated with the tile to be analyzed using a video fingerprinting technique; in response to determining that the content associated with the tile matches a reference content item in a plurality of reference content items using the video fingerprinting technique, generating a match indication associated with the identifier of the spherical video content item.

[0009] In some embodiments, the particular type of content includes content in motion.

[0010] In some embodiments, the particular type of content includes faces.

[0011] In some embodiments, the particular type of content includes a border that is part of a border of a panel in which the video content item is inserted into the spherical video content item.

[0012] In other words, the specific type of content may include any one or more elements of the group consisting of: content in motion; content including faces; content including an edge that is part of a panel boundary in which the video content item is inserted into the spherical video content item.

[0013] In some embodiments, the method further comprises determining a size of the tile.

[0014] In some embodiments, the method further comprises projecting the content associated with the tile onto a two-dimensional plane prior to causing the content associated with the tile to be analyzed using the video fingerprinting technique.

[0015] In some embodiments, the method further comprises, in response to content associated with the tile matching a reference content item in the plurality of reference content items, prohibiting presentation of the spherical video content item.

[0016] In some embodiments, the method further includes, in response to determining that the probability for the particular tile does not exceed a predetermined threshold, discarding the particular tile by prohibiting content associated with the particular tile from being analyzed using a video fingerprinting technique.

[0017] According to some embodiments of the disclosed subject matter, a system for analyzing spherical video content is provided, the system comprising: a memory storing computer-executable instructions; and a hardware processor, which, when executing the computer-executable instructions stored in the memory, is configured to: receive an identifier of a spherical video content item, wherein the spherical video content item has multiple views, and wherein the spherical video content item is encoded into multiple two-dimensional video frames; select a first frame of the multiple two-dimensional video frames associated with the spherical video content item; divide the first frame into multiple tiles, the tiles spanning the first frame of the spherical video content item; for each tile of the multiple tiles, calculate a probability that the tile includes a specific type of content; determine for each tile whether the probability exceeds a predetermined threshold; in response to determining that the probability exceeds the predetermined threshold for a particular tile, cause content associated with the tile to be analyzed using a video fingerprinting technique; in response to determining that the content associated with the tile matches a reference content item in a plurality of reference content items using the video fingerprinting technique, generate a match indication associated with the identifier of the spherical video content item.

[0018] According to some embodiments of the disclosed subject matter, a computer program product (e.g., a computer-readable medium, which may be a non-transitory computer-readable medium) contains computer-executable instructions that, when executed by a processor, cause the processor to perform a method for analyzing spherical video content, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views, and wherein the spherical video content item is encoded into multiple two-dimensional video frames; selecting a first frame of the multiple two-dimensional video frames associated with the spherical video content item; dividing the first frame into multiple tiles, the tiles spanning the first frame of the spherical video content item; for each tile of the multiple tiles, calculating a probability that the tile includes a specific type of content; determining for each tile whether the probability exceeds a predetermined threshold; in response to determining that the probability exceeds the predetermined threshold for a particular tile, causing content associated with the tile to be analyzed using a video fingerprinting technique; in response to determining that the content associated with the tile matches a reference content item in a plurality of reference content items using the video fingerprinting technique, generating an indication of a match associated with the identifier of the spherical video content item.

[0019] According to some embodiments of the disclosed subject matter, a system for analyzing spherical video content is provided, the system comprising: a device for receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views and wherein the spherical video content item is encoded into multiple two-dimensional video frames; a device for selecting a first frame of the multiple two-dimensional video frames associated with the spherical video content item; a device for dividing the first frame into multiple tiles, the tiles spanning the first frame of the spherical video content item; a device for calculating, for each tile of the multiple tiles, a probability that the tile includes a specific type of content; a device for determining, for each tile, whether the probability exceeds a predetermined threshold; a device for analyzing content associated with the tile using a video fingerprinting technique in response to determining that the probability exceeds the predetermined threshold for a particular tile; and a device for generating a match indication associated with the identifier of the spherical video content item in response to determining, using the video fingerprinting technique, that the content associated with the tile matches a reference content item of multiple reference content items.

[0020] According to some embodiments of the disclosed subject matter, a method for analyzing spherical video content is provided, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views; and selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional region using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the region according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the region; identifying a partition within the region that satisfies a criterion, the criterion indicating that the region has a likelihood greater than a threshold of including a particular type of content; in response to identifying a partition within the region that satisfies the criterion, analyzing the identified partition of the region using a video fingerprinting technique; and in response to determining using the video fingerprinting technique that content associated with the identified partition of the region matches a reference content item from a plurality of reference content items, generating an indication of a match associated with the identifier of the spherical video content item.

[0021] In some embodiments, two or more spaced-apart consecutive portions of the frame are each mapped to two corresponding portions of the region.

[0022] In some embodiments, the two-dimensional region is spanned by first and second transverse axes, and wherein a first continuous portion of the frame is mapped to two portions of the region having different positions relative to the first axis. A second continuous portion of the frame is mapped to two portions of the region having different positions relative to the second axis.

[0023] In some embodiments, the method further comprises defining a coordinate system for the frame based on latitude and longitude, wherein the longitude corresponds to the first axis and the latitude corresponds to the second axis.

[0024] In some embodiments, the first continuous portion of the sphere includes a pole according to the coordinate system.

[0025] In some embodiments, the method further comprises preventing the spherical video content item from being presented on the user device in response to determining, using the video fingerprinting technique, that content associated with the identified partition of the modified equirectangular projection matches a reference content item.

[0026] In some embodiments, based on object recognition techniques applied to the projection, regions are identified that meet criteria indicating that the region has a likelihood greater than a threshold of including a particular type of content.

[0027] In some embodiments, the particular type of content included in the identified partitions within the area is a two-dimensional video content item.

[0028] In some embodiments, the plurality of reference content items are a plurality of copyrighted media content items.

[0029] In some embodiments, the method further comprises analyzing audio content associated with the spherical video content item using an audio fingerprinting technique.

[0030] According to some embodiments of the disclosed subject matter, a system for analyzing spherical video content is provided, the system comprising: a memory storing computer-executable instructions; and a hardware processor, when executing the computer-executable instructions stored in the memory, configured to receive an identifier of a spherical video content item, wherein the spherical video content item has multiple views; and select a first frame of the spherical video content item; project the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the area according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the area; identifying a partition within the area that meets a criterion indicating that the area has a likelihood greater than a threshold of including a particular type of content; in response to identifying the partition within the area that meets the criterion, analyzing the identified partition of the area using a video fingerprinting technique; and in response to determining using the video fingerprinting technique that content associated with the identified partition of the area matches a reference content item from a plurality of reference content items, generating an indication of a match associated with the identifier of the spherical video content item.

[0031] According to some embodiments of the disclosed subject matter, a computer program product (e.g., a computer-readable medium, which may be a non-transitory computer-readable medium) is provided, which contains computer-executable instructions that, when executed by a processor, cause the processor to perform a method for analyzing spherical video content, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views; and selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the area according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the area; identifying a partition within the area that meets a criterion indicating that the area has a likelihood greater than a threshold of including content of a particular type; in response to identifying the partition within the area that meets the criterion, analyzing the identified partition of the area using a video fingerprinting technique; and in response to determining using the video fingerprinting technique that content associated with the identified partition of the area matches a reference content item from a plurality of reference content items, generating an indication of a match associated with the identifier of the spherical video content item.

[0032] According to some embodiments of the disclosed subject matter, a system for analyzing spherical video content is provided, the system comprising: means for receiving an identifier of a spherical video content item, wherein the spherical video content item has multiple views; means for selecting a first frame of the spherical video content item; means for projecting the first frame of the spherical video content item to a two-dimensional region using a projection defined by a mapping, wherein neighboring points of the first frame are mapped to corresponding neighboring points of the region according to the mapping, and one or more consecutive portions of the frame are each mapped to corresponding multiple consecutive portions of the region; means for identifying a partition within the region that meets a criterion indicating that the region has a likelihood greater than a threshold of including a particular type of content; means for analyzing the identified partition of the region using a video fingerprinting technique in response to identifying the partition within the region that meets the criterion; and means for generating an indication of a match associated with the identifier of the spherical video content item in response to determining using the video fingerprinting technique that content associated with the identified partition of the region matches a reference content item from a plurality of reference content items.

[0033] Embodiments of the disclosed subject matter can have the following technical effects: matches between spherical video content and reference content items can be reliably identified with substantially no human involvement (i.e., automatically), or at least candidate matches can be identified for subsequent evaluation by another technique (such as a human operator). BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various objects, features, and advantages of the disclosed subject matter may be more fully understood by reference to the following detailed description of the disclosed subject matter when considered in conjunction with the following drawings, wherein like reference numerals represent like elements.

[0035] Figure 1A , 1B , 1C, and 1D show illustrative examples of schematic diagrams for detecting potential copyrighted content in spherical video content, according to some embodiments of the disclosed subject matter.

[0036] Figure 2 A schematic diagram of an illustrative system suitable for implementing the mechanisms described herein for detecting potential copyrighted content in spherical video content is shown, in accordance with some embodiments of the disclosed subject matter.

[0037] Figure 3 It is shown that according to some embodiments of the disclosed subject matter, Figure 2 Detailed examples of the hardware used in the server and / or user devices.

[0038] Figure 4 An illustrative example of a process for detecting potential copyrighted content in spherical video content is shown in accordance with some embodiments of the disclosed subject matter.

[0039] Figure 5 An illustrative example of a process for detecting regions of a spherical video content item that satisfy criteria indicating that the region has a likelihood greater than a threshold of including an embedded video is shown in accordance with some embodiments of the disclosed subject matter.

[0040] Figure 6 An illustrative example of a schematic diagram for projecting spherical video using equirectangular projection is shown in accordance with some embodiments of the disclosed subject matter. DETAILED DESCRIPTION

[0041] According to various embodiments, mechanisms (which may include methods, systems, and media) are provided for detecting misuse of two-dimensional video placed on a sphere in spherical video content by tiling the sphere.

[0042] In some embodiments, the mechanism described herein can be used to detect specific types of video content superimposed on spherical video content and / or stitched together with spherical video content. For example, in some embodiments, the mechanism can be used to detect potential copyrighted video content or potential copyrighted audio content that has been superimposed on or inserted into spherical video content. It should be noted that various fingerprinting techniques have been used to identify copyrighted content, such as by matching a portion of a content item with a reference content item in a database of reference content items. However, these fingerprinting techniques sometimes fail to identify copyrighted video content that has been inserted into a spherical video content item. In particular, in some cases, copyrighted video content may be inserted into a non-default viewing position of the spherical video content, which may make it difficult for these fingerprinting techniques to identify the position of the spherical video content where the copyrighted video content has been inserted. In addition, in the case of encoding spherical video content as two-dimensional content, the copyrighted video content may be distorted, which may also make it difficult for the video fingerprinting technique to match the content with the undistorted reference content item.

[0043] In some embodiments, the mechanisms described herein may process spherical video content items (as described in greater detail herein) prior to applying one or more fingerprinting techniques to the processed spherical video content items to detect and identify particular video content items, such as potentially copyrighted video content items. For example, the mechanisms described herein may process a spherical video content item prior to applying one or more fingerprinting techniques that compare the content to one or more two-dimensional reference files to determine whether the spherical video content item is likely to contain a full movie in a non-default viewport.

[0044] Note that, as described herein, spherical video content refers to video content recorded from multiple views (e.g., simultaneously) using any suitable number of cameras (e.g., one, two, five, and / or any other suitable number) that is stitched together to form an image across any suitable spatial range (e.g., 180 degrees, 270 degrees, 360 degrees, and / or any other suitable spatial range) around an axis. More specifically, in some embodiments, multiple views of the same scene along different directions may be recorded simultaneously by one or more video capture devices, and the resulting video content may be stitched together to allow a user to change the viewpoint of the spherical video content. For example, in some embodiments, a user may manipulate the viewpoint of the spherical video content by clicking and / or dragging the spherical video content using a user input device (e.g., when using a touch-sensitive display), changing the user's gaze point (e.g., when using a gaze tracking device associated with a user device presenting the spherical video content), and / or manipulating the spherical video content in any other suitable manner, thereby causing the viewpoint of the spherical video content to rotate accordingly. Additionally, as described herein, a viewport of a spherical video content item may be described as a portion of the spherical video content item viewed by a viewer located at the center of a sphere represented by the spherical video content item looking in a particular direction.

[0045] In some embodiments, the mechanisms described herein may divide a frame of spherical video content into a plurality of tiles, and may determine whether each tile may include content to be further analyzed (e.g., analyzed using fingerprinting techniques to determine whether the content corresponds to copyrighted content, and / or further analyzed in any other suitable manner). In some embodiments, the mechanisms described herein may calculate, for each tile, a probability that the tile contains content to be further analyzed, and if the probability does not exceed a predetermined threshold, the tile may be discarded. Alternatively, in the event that the probability of a tile does exceed a predetermined threshold, the content of the tile may then be further analyzed, for example, using one or more fingerprinting techniques (e.g., video fingerprinting, audio fingerprinting, and / or any suitable combination thereof) to identify spherical video content items that include a particular video content item superimposed and / or inserted into the spherical video content.

[0046] In some embodiments, the probability of a tile may indicate a likelihood that the content corresponding to the tile includes motion, a likelihood that the content of the tile includes a particular type of object (e.g., a face, a logo, text, and / or any other suitable type of object), a likelihood that the content of the tile includes a border, an edge, and / or any other suitable visual feature, and / or any other suitable information.

[0047] In some embodiments, a spherical video content item that includes one or more tiles exceeding a predetermined threshold may be marked with an appropriate indication. For example, in some embodiments, an indication may be stored together with an identifier of the spherical video content item, indicating the frame number in which content matching content in a reference database is found, an identifier of a matching reference content item, and / or any other suitable information. In some such embodiments, the spherical video content item may be marked for manual viewing, for example, by placing the identifier of the marked spherical video content item in a queue of content items to be manually viewed. Alternatively, the presentation of the spherical video content item may be prohibited or altered—for example, by replacing the spherical video content item with a message interface, by distorting the video or audio content, by not allowing the user to navigate within the three-dimensional space of the spherical video content item, and the like.

[0048] In some embodiments, the mechanisms described herein may identify a portion of a frame of a spherical video content item that may contain an embedded video item. In some embodiments, the mechanisms may then further analyze the identified portion of the frame of the spherical video content item, for example by performing fingerprint analysis on the portion of the frame to identify the embedded video item.

[0049] In some embodiments, the mechanisms may use any suitable technique to identify a portion of a frame of a spherical video content item. For example, in some embodiments, the mechanisms may project a frame of a spherical video content item using a projection such as a modified equirectangular projection that extends the spherical image and wraps the extended spherical image through the equator and around the poles of the sphere, thereby allowing identification of video items embedded anywhere along the sphere (i.e., including the edges and / or poles of the equirectangular projection) in the modified equirectangular projection.

[0050] In another example, the mechanism may select a frame of a spherical video content item, and may project the frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, adjacent points of the frame being mapped to corresponding adjacent points of the area according to the mapping, and one or more continuous portions of the frame being each mapped to corresponding multiple continuous portions of the area. Continuing with this example, two or more spaced-apart continuous portions of the frame may each be mapped to two corresponding portions of the area. It should be noted that the two-dimensional area may be spanned by first and second horizontal axes, wherein a first continuous portion of the frame may be mapped to two portions of the area having different positions relative to the first axis, and a second continuous portion of the frame may be mapped to two portions of the area having different positions relative to the second axis. In some embodiments, the mechanism may define a coordinate system for the frame based on latitude and longitude, wherein the longitude corresponds to the first axis, the latitude corresponds to the second axis, and wherein according to the coordinate system the first continuous portion of the sphere includes a pole.

[0051] Combine the following Figure 5 and Figure 6 More detailed techniques are shown and described for identifying a portion of a frame of a spherical video content item that may include an embedded video item (e.g., identifying a partition within a two-dimensional projected area of ​​a selected frame that satisfies a criterion that is greater than a threshold likelihood indicating that the area has a likelihood of including an embedded video content item).

[0052] It should be noted that this mechanism can be executed at any suitable time. For example, these mechanisms can be executed in response to a request to upload a spherical video content item to a content database. In a more specific example, the mechanism can be used to perform fingerprint analysis to identify embedded video items within the spherical video content item at the time of upload (e.g., as opposed to offline processing). In another more specific example, the mechanism can determine whether to use (i) a first mechanism that performs fingerprint analysis at the time of upload to identify embedded video items in the spherical video content item, or to use (ii) a second mechanism that divides a frame of the spherical video content into multiple tiles and determines whether each tile may contain content to be further analyzed.

[0053] This article combines Figures 1A to 6 These and other features for analyzing spherical video content are described.

[0054] return Figure 1A , according to some embodiments of the disclosed subject matter, an illustrative example of a schematic diagram of a frame of a spherical video content item is shown. Figure 1A As shown, the frames of the spherical video content item 100 may include video content that has been recorded from multiple viewpoints of any suitable number of cameras (one, two, five, and / or any other suitable number) and has been stitched together to create a browsable content item. For example, in some embodiments, the spherical video content item 100 may be presented to the user as if the user is located at the center of a sphere representing the spherical video content item 100 and is viewing the spherical video content item 100 through a viewport, and the user may manipulate the viewpoint by rotating the spherical video content item 100 around the viewport. In some such embodiments, manipulation of the viewpoint may be received via any suitable user input such as a mouse click or drag, input from a touch screen associated with a user device, and / or any other suitable user input.

[0055] In some embodiments, the spherical video content item 100 may be divided into a plurality of tiles, such as Figure 1A In some embodiments, tile 102 may have any suitable size. For example, in some embodiments, if a viewer at the center of a sphere representing spherical video content 100 is looking at tile 102, tile 102 may correspond to the size of the viewport, as described below in conjunction with Figure 4Additionally or alternatively, in some embodiments, the tile 102 may be smaller or larger than the size of the viewport.

[0056] In a more specific example, Figure 1B An illustrative example of a sphere is shown that represents a surface on which a spherical video content item 100 may be presented, wherein the surface of the sphere has been divided into a plurality of tiles. It should be noted that the spherical video content item 100 may be divided into tiles of any suitable size (e.g., a size corresponding to the size of the viewport, a default size, etc.). For example, the spherical video content item 100 may be divided into tiles of different sizes, wherein a particular group of tiles is configured as a particular size and another particular group of tiles is configured as a different size. It should also be noted that although Figure 1B The spherical video content item 100 is generally shown divided into tiles having a rectangular shape, which is merely exemplary, and the spherical video content item 100 may be divided into tiles of any suitable shape.

[0057] In some embodiments, referring back to Figure 1A , the content of tile 102 (or any other tile) may be analyzed, for example, to determine whether the content is part of a particular video content item (e.g., a copyrighted video content item and / or any other suitable video content item). Figure 4 More specific techniques for analyzing the content of tile 102 are described further below. In some embodiments, tile 102 may be manipulated in any suitable manner to perform analysis of the content of tile 102. Figure 1C As shown in , the shape of the tile 102 can be modified to create a modified tile 152. As a more specific example, as Figure 1A and 1C As shown, the tiles 102 from the spherical video content 100 may be projected onto a two-dimensional plane to form rectangular modified tiles, such as in combination with Figure 4 Described in more detail.

[0058] return Figure 2 , shows an example of hardware 200 for analyzing spherical video content that can be used according to some embodiments of the disclosed subject matter. As shown, hardware 200 can include one or more servers, such as server 202, communication network 204, and / or one or more user devices 206, such as user devices 208 and 210.

[0059] In some embodiments, server 202 may be any suitable server for storing video content, analyzing video content, sending video content to user device 206, and / or performing any other suitable function. In some embodiments, the video content may include spherical video content, which may include video content recorded from multiple views and stitched together to form video content spanning any suitable spatial extent (e.g., 270 degrees, 360 degrees, and / or any other space), which may be viewed and / or manipulated (e.g., via user input). In some such embodiments, server 202 may perform any suitable function for analyzing spherical video content. For example, in conjunction with Figure 4 As shown and described below, in some embodiments, the server 202 may divide the spherical video content into a plurality of tiles that make up the spherical video content, and may analyze the tiles to determine whether the spherical video content includes a specific video content item. In some embodiments, the server 202 may be omitted so that Figure 4 The method is, for example, executed on the user device 206 instead.

[0060] In some embodiments, the communication network 204 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 204 can include any one or more of the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user device 206 can be connected to the communication network 204 via one or more communication links 212, and the communication network 204 can be linked to the server 202 via one or more communication links (e.g., communication link 216). The communication links 212 and / or 214 can be any communication link suitable for transferring data between the user device 206 and the server 202, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of these links.

[0061] In some embodiments, user device 206 may include one or more computing devices suitable for requesting video content, viewing video content, changing the view of video content, and / or any other suitable function. For example, in some embodiments, user device 206 may be implemented as a mobile device, such as a smart phone, a mobile phone, a tablet computer, a laptop computer, a vehicle (e.g., a car, a boat, an airplane, or any other suitable vehicle) entertainment system, a portable media player, and / or any other suitable mobile device. As another example, in some embodiments, user device 206 may be implemented as a non-mobile device, such as a desktop computer, a set-top box, a television, a streaming media player, a game console, and / or any other suitable non-mobile device.

[0062] Although server 202 is shown as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 202. For example, in some embodiments, the functions performed by server 202 may be performed on a single server. As another example, in some embodiments, multiple devices may be used to implement the functions performed by server 202.

[0063] Despite Figure 2 Two user devices 208 and 210 are shown in FIG. 1 , but in some embodiments, any suitable number of user devices and / or any suitable type of user devices may be used.

[0064] In some embodiments, the server 202 and the user device 206 may be implemented using any suitable hardware. For example, in some embodiments, the server 202 and the user device 206 may be implemented using any suitable general-purpose computer or special-purpose computer. For example, a special-purpose computer may be used to implement the server. Any such general-purpose computer or special-purpose computer may include any suitable hardware. For example, as in Figure 3 As illustrated in example hardware 300 of , such hardware may include a hardware processor 302 , memory and / or storage 304 , an input device controller 306 , an input device 308 , a display / audio driver 310 , display and audio output circuitry 312 , a communication interface 314 , an antenna 316 , and a bus 318 .

[0065] In some embodiments, the hardware processor 302 may include any suitable hardware processor, such as a microprocessor, a microcontroller, a digital signal processor, dedicated logic, and / or any other suitable circuitry for controlling the functions of a general purpose computer or a special purpose computer. In some embodiments, the hardware processor 302 may be controlled by a server program stored in a memory and / or storage 304 of a server (e.g., server 202). For example, the server program may cause the hardware processor 302 to analyze video content stored on the server 202, send the video content to the user device 206, and / or perform any other suitable action. In some embodiments, the hardware processor 302 may be controlled by a computer program stored in a memory and / or storage 304 of the user device 206. For example, the computer program may cause the hardware processor 302 to render the video content, change the view of the video content, and / or perform any other suitable operation.

[0066] In some embodiments, memory and / or storage 304 may be any suitable memory and / or storage for storing programs, data, media content, advertisements, and / or any other suitable information. For example, memory and / or storage 304 may include random access memory, read-only memory, flash memory, hard disk storage, optical media, and / or any other suitable storage.

[0067] In some embodiments, input device controller 306 may be any suitable circuitry for controlling and receiving input from one or more input devices 308. For example, input device controller 306 may be circuitry for receiving input from a touch screen, a keyboard, a mouse, one or more buttons, voice recognition circuitry, a microphone, a camera, an optical sensor, an accelerometer, a temperature sensor, a near-field sensor, and / or any other type of input device.

[0068] In some embodiments, display / audio driver 310 may be any suitable circuit for controlling and driving output to one or more display / audio output devices 312. For example, display / audio driver 310 may be circuitry for driving a touch screen, a flat panel display, a cathode ray tube display, a projector, a speaker or speakers, and / or any other suitable display and / or presentation device.

[0069] The communication interface 314 may be used to communicate with Figure 2 Interface 314 may include any suitable circuitry for interfacing with one or more communication networks of network 204. For example, interface 314 may include network interface card circuitry, wireless communication circuitry, and / or any other suitable type of communication network circuitry.

[0070] In some embodiments, antenna 316 may be any suitable antenna or antennas for wirelessly communicating with a communication network, such as communication network 204. In some embodiments, antenna 316 may be omitted.

[0071] In some embodiments, bus 318 may be any suitable mechanism for communicating between two or more components 302 , 304 , 306 , 310 , and 314 .

[0072] According to some embodiments, any other suitable components may be included in hardware 300 .

[0073] return Figure 4 , shows an example 400 of a process for analyzing spherical video content according to some embodiments of the disclosed subject matter. In some embodiments, blocks of process 400 can be implemented on server 202.

[0074] Process 400 may begin by receiving an indication of a spherical video content item at 402. In some embodiments, the indication may be received in any suitable manner. For example, in some embodiments, process 400 may detect that a spherical video content item has been uploaded to server 202 by a content creator (e.g., using an interface for uploading video content items, using an application programming interface for uploading multiple video content items that may include a spherical video content item, etc.). As another example, in some embodiments, process 400 may use the techniques described in blocks 404-414 to process a set of spherical video content items that include a spherical video content item.

[0075] As described above, the spherical video content item may be any suitable type of video content, including multiple views recorded from any suitable number of cameras that are stitched together to form a video content item that can be manipulated in a viewpoint and browsed by a user. In some embodiments, the spherical video content item may include a still image with multiple views replicated across multiple frames. Note that in some embodiments, the spherical video content item may be encoded as a two-dimensional video, for example, as a projection of the spherical video content onto a rectangular plane.

[0076] Process 400 may identify a frame of the spherical video content item at 404. In some embodiments, process 400 may identify the frame in any suitable manner. For example, in some embodiments, process 400 may begin with the first frame of the spherical video content item. As another example, in some embodiments, process 400 may select a frame from a middle portion of the spherical video content item.

[0077] At 406, process 400 may divide the frame of the spherical video content item into a plurality of tiles that span the frame. It should be noted that the size of each tile may be any suitable size. For example, in some embodiments, the size of the tile may correspond to the size of a viewport corresponding to a viewer's viewpoint located at the center of the spherical video content item viewing the tile. Alternatively, in some embodiments, the size of the tile may be smaller or larger than the size of the viewport. Note that in some embodiments, the plurality of tiles may have different sizes (e.g., the size of the set of tiles in the center of the frame is larger than the size of the set of tiles on the periphery of the frame).

[0078] It should be noted that the visible perspective of the spherical video content item can be represented as a portion of a sphere. For example, process 400 can divide the spherical representation of the spherical video content item into a plurality of tiles, similar to Figure 1B The partitioning shown, wherein a spherical image is mapped to a two-dimensional representation of the spherical image. The mapping may include mapping frames of a spherical video or image to a two-dimensional representation based on a spherical projection. The tiles may, for example, tile the surface of a sphere.

[0079] In some embodiments, process 400 may determine the size of the tile based on any suitable information. For example, in some embodiments, the size may be determined experimentally based on the effectiveness of a tile of a particular size in identifying a particular type of content, and may be modified over time. As a more specific example, in some embodiments, process 400 may use a tile of the maximum size that is effective for detecting a particular type of content (and therefore using the least computing time to analyze the spherical video content item). As another example, in some embodiments, process 400 may use smaller blocks in areas of the spherical video content item that are determined to have a lot of color changes, and use larger blocks in areas of the spherical video content item that are determined to have less color changes. As yet another example, in some embodiments, the size of the tile may be determined based on any other suitable information (such as the type of content associated with the spherical video content item, the identity of the creator of the spherical video content item, and / or any other suitable information).

[0080] At 408, process 400 may determine, for each tile of the frame of the spherical video content item, a probability (i.e., a probability value) that the tile contains content to be further analyzed. For example, in some embodiments, the probability may indicate a probability that the tile contains content in motion, a probability that the tile contains a border or edge that is part of a panel (e.g., a rectangular panel, a square panel, and / or a panel of any other suitable shape) in which a particular video content item is presented, and / or any other suitable probability.

[0081] Process 400 may calculate the probability using any suitable technique or combination of techniques. For example, in some embodiments, process 400 may calculate the probability that the content of a tile includes motion. In some such embodiments, process 400 may determine the probability that the content of a tile includes motion by comparing two or more frames that include the tile. In some embodiments, process 400 may estimate motion using any suitable algorithm or technique. For example, in some embodiments, process 400 may estimate motion using a pixel-based approach (e.g., optical flow, phase correlation, and / or any other suitable technique or combination of techniques). As another example, in some embodiments, process 400 may estimate motion using a feature-based approach. As a more specific example, in some such embodiments, process 400 may identify one or more features present in a tile (e.g., edges, corners, boundaries, specific shapes, and / or any other suitable features) and may compare corresponding features across multiple frames (including tiles).

[0082] As another example, in some embodiments, process 400 may calculate the probability that a tile includes a border and / or edge. As a more specific example, in some embodiments, the border and / or edge may correspond to a border of a rendered video window containing a particular video content item (e.g., a copyrighted video content item). Figure 1D As shown in the specific example of FIG. 1 , process 400 can determine whether tile 162 includes a boundary 164 of a video window in which a video content item is being presented, where the video window is superimposed on the spherical video content item.

[0083] Note that in some embodiments, process 400 may identify the borders of a video window or panel in which the video content item is presented within the spherical video content item in any other suitable manner. For example, in some embodiments, process 400 may calculate the probability that a window of any suitable size (e.g., in pixels, inches, and / or in any other suitable measure) contains an edge or border, and may calculate a probability matrix and / or heat map of the spherical video content item by sliding the window over the spatial extent of the spherical video content item projected on a two-dimensional plane. A more specific example of a probability matrix or heat map may include: [0.0001, 0.0002, ... 0.0123; 0.0001, 0.8, ... 0.7; ...], thereby indicating that this particular frame has a relatively high probability of having a border or edge at a location corresponding to the second row of the matrix. In some such embodiments, process 400 may average the calculated probabilities across multiple frames. Then, process 400 may identify the spatial location within the spherical video content item that is most likely to contain a border or edge (e.g., based on probability, based on average probability, and / or in any other suitable manner).

[0084] As yet another example, in some embodiments, process 400 may calculate a probability that the contents of a tile represent a particular type of object, such as a face, an animal, a logo, a particular actor, text, and / or any other suitable type of object. In some such embodiments, process 400 may use any suitable type of image recognition technology to identify the particular type of object within the tile.

[0085] In some embodiments, in addition to and / or in place of the above-described techniques, any suitable type of classifier or other machine learning technique may be used. For example, in some embodiments, process 400 may use a classifier (e.g., a support vector machine, a logistic regression, a perceptron, and / or any other suitable type of classifier) ​​to calculate the probability that a tile contains content to be further analyzed. As a more specific example, a classifier may be used to calculate the likelihood that a tile and / or window contains a border corresponding to a video player panel in which copyrighted video content is presented, the likelihood that a tile and / or window contains a particular type of object (e.g., a face, a logo, and / or any other suitable type of object), and / or any other suitable classification. In some such embodiments, process 400 may use weights and / or factors associated with a classifier, which are determined based on any suitable training data set consisting of tiles from other spherical video content items.

[0086] Note that in some embodiments, process 400 may calculate, for each tile, a probability that the tile includes content to be further analyzed using a combination of information. For example, in some embodiments, the probability may represent a combination of a first probability that the tile includes content in motion and a second probability that the tile includes a particular type of object (e.g., a face and / or any other suitable type of object). In some such embodiments, multiple sources of information may be combined in any suitable manner, such as a weighted average.

[0087] At 410, the process 400 may determine whether the probability exceeds a predetermined threshold. In some embodiments, the threshold may be any suitable value (e.g., 50%, 70%, 90%, and / or any other suitable value). In some embodiments, the process 400 may set the threshold based on any suitable information. For example, in some embodiments, the process 400 may set the threshold based on the distribution of probability values ​​for all tiles that have been analyzed. As a more specific example, in some embodiments, the process 400 may set the threshold so that less than a predetermined number of tiles (e.g., less than 50, less than ten, and / or any other suitable number) exceed the threshold and / or so that less than a predetermined fraction of the total number of tiles (e.g., less than 10%, less than 20%, and / or any other suitable fraction) exceed the threshold. As another example, in some embodiments, the process 400 may set the threshold based on the size of the tile. As a more specific example, in the case where the size of the tile corresponds to a relatively small portion of the spherical video content item, the process 400 may set the threshold to be higher relative to the case where the size of the tile is a larger portion of the spherical video content item. In some embodiments, the process 400 may modify the threshold over time, for example, based on performance. As a more specific example, in the case where process 400 determines that a certain threshold has allowed tiles that should be marked to not be marked, process 400 can adjust the threshold to a lower number. As another more specific example, in the case where process 400 determines that a certain threshold has marked more than a predetermined number or more than a predetermined proportion of tiles, which are subsequently manually determined to be tiles that should not be marked, process 400 can adjust the threshold to a higher number.

[0088] If it is determined at 410 that the probability does not exceed the threshold ("No" at 410), process 400 may proceed to block 412 and the tile may be discarded. After analyzing any suitable number of tiles identified at block 406 in this manner (e.g., all tiles, a suitable subset of tiles, and / or any other suitable number), process 400 may loop back to block 404 and may identify another frame of the spherical video content item. For example, in some embodiments, process 400 may identify the next frame of the spherical video content item. As another example, in some embodiments, process 400 may skip a predetermined number of frames (e.g., five frames, ten frames, twenty frames, and / or any other suitable number) to identify a different frame of the spherical video content item.

[0089] If it is determined at 410 that the probability does exceed the threshold ("yes" at 410), process 400 can proceed to box 414 and the content of the tile can be further analyzed. For example, in some embodiments, process 400 can perform any appropriate fingerprinting technology (e.g., video fingerprinting, audio fingerprinting, and / or any other appropriate type of fingerprint analysis) to determine whether the content of the tile matches a reference content item. As a more specific example, in some embodiments, process 400 can use fingerprinting technology to determine whether the content of the tile matches any content item within a database of reference content items (e.g., a database of copyrighted videos, copyrighted audio tracks, and / or any other suitable content items). In some embodiments, process 400 can combine audio fingerprinting technology and video fingerprinting technology. In addition, where process 400 utilizes audio fingerprinting technology, process 400 can determine whether the audio content of the tile matches a reference audio content item played at a modified speed (e.g., accelerated, decelerated, and / or modified by any suitable factor).

[0090] In some embodiments, process 400 may perform any suitable processing before analyzing the contents of a tile. For example, in some embodiments, process 400 may determine that only a subset of the spatial region of a tile is to be analyzed. In some such embodiments, process 400 may center the selected subset of the spatial region within a particular viewport, project the selected subset of the spatial region onto a rectangular plane to minimize visual distortion of the selected subset of the spatial region, and / or perform any other suitable processing.

[0091] Note that in some embodiments, if it is determined that the content of the tile matches the reference content, the tile and / or the spherical video content item itself may be marked and / or annotated in any suitable manner. For example, in some embodiments, process 400 may mark an identifier of the spherical video content item so that the spherical video content item is marked for manual review to determine whether the spherical video content item contains a specific video content item (e.g., a specific copyrighted video content item and / or any other suitable type of content). In some such embodiments, the output of process 400 may include any other suitable information, such as a frame number corresponding to the marked tile, an identifier of a reference content item that produces a match with the content of the marked tile, and / or any other suitable information.

[0092] In some embodiments, process 400 may prohibit or prevent a spherical video content item from being presented to other viewers because it may contain a copyrighted video content item. For example, process 400 may cause a warning message or a takedown message to be presented in place of the spherical video content item. In another example, process 400 may modify the spherical video content item (e.g., prevent movement to a particular viewpoint, change video and / or audio content associated with the spherical video content item, etc.).

[0093] Process 400 may then loop back to block 404 and identify different frames of the spherical video content item, as described above.

[0094] Note that in some embodiments, process 400 may terminate before analyzing all frames of the spherical video content item. For example, in some embodiments, process 400 may determine whether a particular stopping criterion is met, and if so, may terminate the analysis of the spherical video content item. As a more specific example, in some embodiments, the stopping criterion may include whether more than a predetermined number of tiles have been marked as including content that matches the reference content item, as described above in connection with block 414.

[0095] According to some embodiments of the disclosed subject matter, a mechanism can be provided for identifying portions of a frame of a spherical video content item that may include an embedded video item (e.g., identifying a partition within a two-dimensional projected area of ​​a selected frame that satisfies a criterion indicating that the area has a likelihood greater than a threshold of including an embedded video content item). Such a spherical fingerprinting mechanism can be used, for example, in response to receiving a request to upload a spherical video content item to a content database. In a more specific example, the mechanism can be used to perform spherical fingerprinting analysis to identify embedded video items within a spherical video content item at upload time (e.g., as opposed to offline processing).

[0096] return Figure 5 , according to some embodiments of the disclosed subject matter, an illustrative example 500 of a process for identifying a portion of a spherical video content item that may include an embedded video item is shown. In some embodiments, blocks of process 500 can be implemented on server 202.

[0097] Process 500 may begin by receiving an indication of a spherical video content item at 502. In some embodiments, the indication may be received in any suitable manner. For example, in some embodiments, process 500 may detect that a spherical video content item has been uploaded by a content creator or has been requested to be uploaded to server 202 by a content creator (e.g., using an interface for uploading video content items, using an application programming interface for uploading multiple video content items that may include a spherical video content item, etc.). As another example, in some embodiments, process 500 may use the techniques described in blocks 504-510 to process a set of spherical video content items that include a spherical video content item.

[0098] As described above, the spherical video content item may be any suitable type of video content, including multiple views recorded from any suitable number of cameras that are stitched together to form a video content item that can be manipulated in a viewpoint and navigated by a user. In some embodiments, the spherical video content item may include a still image with multiple views replicated across multiple frames. Note that in some embodiments, the spherical video content item may be encoded as a two-dimensional video, for example, as a projection of the spherical video content onto a rectangular plane.

[0099] Process 500 may identify a frame of the spherical video content item at 504. In some embodiments, process 500 may identify the frame in any suitable manner. For example, in some embodiments, process 500 may begin with the first frame of the spherical video content item. As another example, in some embodiments, process 500 may select a frame from a middle portion of the spherical video content item.

[0100] At 506, process 500 may project the frame of the spherical video content item using a projection such as a modified equirectangular projection. In some embodiments, the modified equirectangular projection may include any suitable number of rectangles, each representing a specific angular range of the frame of the spherical video content item. For example, in some embodiments, the rectangles included in the modified equirectangular projection may span a specific yaw angle range (e.g., -30 degrees to 30 degrees, -35 degrees to 35 degrees and / or any other suitable yaw range) and a specific pitch angle range (e.g., -25 degrees to 25 degrees, -30 degrees to 30 degrees and / or any other suitable pitch angle range) on the frame of the spherical content item. Note that in some embodiments, the rectangles included in the modified equirectangular projection may span different angles relative to each other. For example, in some embodiments, a rectangle at the top of the modified equirectangular projection representing content near the vertices of the frame of the spherical video content item may span a smaller angular range of the frame of the spherical video content item relative to a rectangle in the middle portion of the modified equirectangular projection.

[0101] exist Figure 6An illustrative example of a modified equirectangular projection 600 of a frame of a spherical video content item is shown in . As shown, projection 600 can be modified to include an extended range of the frame of the spherical video content item. For example, in some embodiments, the extended image of the frame of the spherical video content item can be wrapped around the equator and / or through the poles. As a more specific example, in some embodiments, the modified equirectangular projection 600 can include redundant portions. As a specific example, rectangles 602 and 604 can be redundant portions of the frame of the spherical video content item, and can be located on the left and right sides of the modified equirectangular projection 600, respectively. As another specific example, rectangles 606 and 608 can be redundant portions of the frame of the spherical video content item, which are located at or near the poles of the frame of the spherical video content item, and rectangles 606 and 608 can be located on the top and bottom sides of the modified equirectangular projection 600. Note that although modified equirectangular projection 600 includes redundant rectangles 602 and 604 and redundant rectangles 606 and 608, in some embodiments, modified equirectangular projection 600 may include any suitable number of redundant rectangles (e.g., zero, one, five, ten, and / or any other suitable number).

[0102] It should be noted that the process 500 can project a selected frame of the spherical video content item to a two-dimensional area using a modified equirectangular projection 600 defined by a mapping, according to which adjacent points of the frame are mapped to corresponding adjacent points of the area, and one or more continuous portions of the frame are each mapped to corresponding multiple continuous portions of the area. Continuing with the example, two or more spaced-apart continuous portions of the frame can each be mapped to two corresponding portions of the area. It should also be noted that the two-dimensional area can be spanned by first and second transverse axes, wherein a first continuous portion of the frame can be mapped to two portions of the area having different positions relative to the first axis, and a second continuous portion of the frame can be mapped to two portions of the area having different positions relative to the second axis. It should also be noted that in some embodiments, the process 500 can define a coordinate system for the frame based on latitude and longitude, wherein the longitude corresponds to the first axis, the latitude corresponds to the second axis, and wherein the first continuous portion of the sphere includes a pole according to the coordinate system.

[0103] return Figure 5 At 508, process 500 may identify a rectangle that includes a video content item (e.g., a two-dimensional video content item and / or any other suitable type of video content item) embedded in the spherical video content item in the modified equirectangular projection. For example, process 500 may identify a rectangular partition within the two-dimensional projection area of ​​the selected frame of the spherical video content item that satisfies a criterion indicating that the area has a likelihood greater than a threshold of including a video content item. In some embodiments, process 500 may use any suitable technique or combination of techniques to identify the rectangle.

[0104] For example, in some embodiments, process 500 may use any suitable object recognition techniques to identify specific objects (eg, faces, people, specific characters, logos, and / or any other suitable objects) within the modified equirectangular projection.

[0105] As another example, in some embodiments, process 500 may use any suitable image recognition technique to identify a rectangle included in a modified equirectangular projection that is anomalous with respect to a surrounding portion of the modified equirectangular projection (e.g., where a rectangle or any suitable portion may be considered an anomalous portion by satisfying criteria indicating having at least one characteristic that is different from the surrounding portion of the modified equirectangular projection). As a more specific example, in some embodiments, process 500 may identify a rectangle that includes a color that is substantially different from the surrounding portion of the modified equirectangular projection. As another more specific example, in some embodiments, process 500 may identify edges within a modified equirectangular projection, and may identify a rectangle included in a modified equirectangular projection that includes an edge that is broken or discontinuous with respect to a surrounding portion of the modified equirectangular projection.

[0106] As another example, in some embodiments, process 500 may use any suitable motion vector detection or identification technique to identify rectangles included in an equirectangular projection that may include motion.

[0107] Note that in some embodiments, process 500 may identify rectangles located in any suitable partition within the modified equirectangular projection. Also, note that in some embodiments, process 500 may identify multiple rectangles within the modified equirectangular projection. For example, in the case where the video content item is embedded at or near a pole of a spherical video content item, process 500 may identify a rectangle located in the top portion of the modified equirectangular projection and a redundant rectangle located in the bottom portion of the modified equirectangular projection, as described above in conjunction with Figure 6 As shown and described.

[0108] In some embodiments, process 500 may calculate a probability that the identified rectangle includes an embedded video content item. For example, in some embodiments, the probability may be based on a degree of abnormality of the rectangle relative to a surrounding portion of a modified equidistant rectangular projection (e.g., where the probability may be calculated using parameters indicating a rectangle having at least one attribute that is different from a surrounding portion of a modified equidistant rectangular projection). In some embodiments, in response to determining that the probability exceeds a predetermined threshold (e.g., greater than 0.5, greater than 0.75, and / or any other suitable threshold), process 500 may proceed to block 510. Conversely, in some embodiments, in response to determining that the probability does not exceed the predetermined threshold, process 500 may terminate or loop back to block 504.

[0109] At 510, process 500 may further analyze the identified rectangle. For example, in some embodiments, process 500 may perform any suitable video fingerprinting technique on the content in the identified rectangle to identify the video content item included in the identified rectangle. For example, in some embodiments, the identified video content item may be any suitable video content item, such as a television program, a movie, real-time streaming video content, and / or any other suitable video content item. In some embodiments, process 500 may perform video fingerprinting technique in any suitable manner. For example, in some embodiments, process 500 may calculate any suitable video fingerprint for the identified rectangle, and may compare the video fingerprint to a reference database of fingerprints of known video content items (e.g., a database of copyrighted media content and / or any other suitable database) to identify the corresponding video content item.

[0110] Note that in some embodiments, process 500 may additionally or alternatively perform any suitable audio fingerprinting techniques to, for example, identify copyrighted audio tracks associated with frames of a spherical video content item. In some embodiments, process 500 may combine any suitable video fingerprinting and audio fingerprinting techniques. Furthermore, where process 500 performs audio fingerprinting techniques, process 500 may identify copyrighted audio tracks that have been modified in any suitable manner, such as sped up, slowed down, frequency shifted, and / or modified in any other suitable manner, as described above in conjunction with the above. Figure 4 of 414 are described in more detail.

[0111] In some embodiments, in response to identifying video content items and / or audio content items (e.g., copyrighted video content items, copyrighted audio content items, and / or any other suitable media content items) associated with the identified rectangle of the modified equidistant rectangular projection, process 500 can perform any appropriate action. For example, in some embodiments, process 500 can remove the spherical video content item from the video sharing service to which the spherical video content item is uploaded and / or prevent the presentation of the spherical video content item. As another example, in some embodiments, process 500 can mark the spherical video content item and / or the identified rectangle in any suitable manner. As a more specific example, in some embodiments, process 500 can send an indication of the rectangle and / or spherical video content item to any suitable entity, such as an administrator of the video sharing service to which the spherical video content item is uploaded, a creator of the spherical video content item, and / or any other suitable entity. As yet another example, in some embodiments, process 500 can, for example, store in a server hosting the spherical video content item an indication of a media content item that matches the content in the identified rectangle of the modified equidistant rectangular projection associated with the identifier of the spherical video content item.

[0112] In some embodiments, the following steps may be performed or executed in any order or sequence. Figure 4 and Figure 5 At least some of the above-mentioned blocks, order or sequence of the process are not limited to the order or sequence shown and described in conjunction with the drawings. In addition, Figure 4 and Figure 5 Some of the above blocks in the above blocks may also be executed or performed substantially simultaneously, where appropriate or in parallel to reduce waiting time and processing time. Additionally or alternatively, Figure 4 and Figure 5 Some of the above boxes of the process.

[0113] In some embodiments, any suitable computer-readable medium may be used to store instructions for performing the functions and / or processes of the present invention. For example, in some embodiments, the computer-readable medium may be temporary or non-temporary. For example, non-temporary computer-readable media may include such as magnetic media (such as hard disks, floppy disks, and / or any other suitable magnetic media), optical media (such as compact disks, digital video disks, Blu-ray discs, and / or any other suitable optical media), semiconductor media (such as flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and / or any other suitable semiconductor media), any suitable medium that is not transient or does not have any permanent performance during transmission, and / or any suitable tangible medium. As another example, transient computer-readable media may include signals in networks, wires, conductors, optical fibers, circuits, any suitable medium that is transient and does not have any permanent performance during transmission, and / or any suitable intangible medium. The instruction set may be referred to as a computer program product (e.g., a commodity that can be rented or purchased).

[0114] In cases where the systems described herein collect personal information about a user or utilize personal information, the user may be provided with an opportunity to control whether a program or feature collects information about the user (e.g., information about the user's social network, social activities or events, occupation, the user's preferences, or the user's current location). In addition, certain data may be processed in one or more ways before it is stored or used so that personal information is removed. For example, the user's identity may be processed so that no personal information can be determined for that user, or the user's geographic location may be summarized where location information is obtained (e.g., to a city, zip code, or state level) so that the user's specific location cannot be determined. Thus, the user may control how information about the user is collected and used by the content server.

[0115] Thus, methods, systems, and media are provided for analyzing spherical video content, and more particularly, for detecting misuse of two-dimensional video placed on a sphere in spherical video content by tiling the sphere.

[0116] Although the present invention has been described and illustrated in the foregoing illustrative embodiments, it should be understood that the present disclosure is made by way of example only and that many changes may be made without departing from the embodiments of the present invention. The spirit and scope of the present invention are limited only by the appended claims. The features of the disclosed embodiments may be combined and rearranged in various ways.

Claims

1. A method for analyzing spherical video content, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has a plurality of views; selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the area according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the area; identifying a partition within the region that satisfies a criterion indicating that the region has a likelihood greater than a threshold of including a particular type of content; In response to identifying a partition within the area that meets the criteria, analyzing the identified partition of the area using a video fingerprinting technique; and In response to determining, using the video fingerprinting technique, that content associated with the identified partition of the area matches a reference content item in a plurality of reference content items, generating an indication of the match associated with the identifier of the spherical video content item.

2. The method according to claim 1, wherein: Two or more spaced-apart consecutive portions of the frame are each mapped to two corresponding portions of the region.

3. The method according to claim 2, wherein: The two-dimensional area is spanned by a first horizontal axis and a second horizontal axis, and wherein a first continuous portion of the frame is mapped to two portions of the area having different positions relative to the first horizontal axis, and a second continuous portion of the frame is mapped to two portions of the area having different positions relative to the second horizontal axis.

4. The method according to claim 3, further comprising: A coordinate system is defined for the frame based on latitude and longitude, wherein the longitude corresponds to the first horizontal axis and the latitude corresponds to the second horizontal axis.

5. The method according to claim 4, wherein: The first continuous portion of the frame includes a pole according to the coordinate system.

6. The method according to claim 1, further comprising: In response to determining, using the video fingerprinting technique, that content associated with the identified partition of the modified equirectangular projection matches the reference content item, the spherical video content item is blocked from presentation on a user device.

7. The method according to claim 1, wherein: Regions that meet the criteria are identified based on object recognition techniques applied to the projections, the criteria indicating that the regions have a likelihood greater than a threshold of including the particular type of content.

8. The method according to claim 1, wherein: The particular type of content included in the identified partitions within the area is a two-dimensional video content item.

9. The method according to claim 1, wherein: The plurality of reference content items are a plurality of copyrighted media content items.

10. The method according to claim 1, further comprising: Audio content associated with the global video content item is analyzed using an audio fingerprinting technique.

11. A system for analyzing spherical video content, the system comprising: Memory; and a processor, which, when executing computer-executable instructions stored in the memory, is configured to: receiving an identifier of a spherical video content item, wherein the spherical video content item has a plurality of views; selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the area according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the area; identifying a partition within the region that satisfies a criterion indicating that the region has a likelihood greater than a threshold of including a particular type of content; In response to identifying a partition within the area that meets the criteria, analyzing the identified partition of the area using a video fingerprinting technique; and In response to determining, using the video fingerprinting technique, that content associated with the identified partition of the area matches a reference content item in a plurality of reference content items, generating an indication of the match associated with the identifier of the spherical video content item.

12. The system according to claim 11, wherein: Two or more spaced-apart consecutive portions of the frame are each mapped to two corresponding portions of the region.

13. The system according to claim 12, wherein: The two-dimensional area is spanned by a first horizontal axis and a second horizontal axis, and wherein a first continuous portion of the frame is mapped to two portions of the area having different positions relative to the first horizontal axis, and a second continuous portion of the frame is mapped to two portions of the area having different positions relative to the second horizontal axis.

14. The system according to claim 13, wherein: The processor is further configured to define a coordinate system for the frame based on latitude and longitude, wherein the longitude corresponds to the first horizontal axis and the latitude corresponds to the second horizontal axis.

15. The system of claim 14, wherein: The first continuous portion of the frame includes a pole according to the coordinate system.

16. The system according to claim 11, wherein: The processor is further configured to, in response to determining using the video fingerprinting technique that content associated with the identified partition of the modified equirectangular projection matches the reference content item, prevent the spherical video content item from being presented on a user device.

17. The system of claim 11, wherein: Regions that meet the criteria are identified based on object recognition techniques applied to the projections, the criteria indicating that the regions have a likelihood greater than a threshold of including the particular type of content.

18. The system of claim 11, wherein: The particular type of content included in the identified partitions within the area is a two-dimensional video content item.

19. The system of claim 11, wherein: The plurality of reference content items are a plurality of copyrighted media content items.

20. The system of claim 11, wherein: The processor is further configured to analyze audio content associated with the spherical video content item using an audio fingerprinting technique.

21. A computer program product comprising computer executable instructions which, when executed by a processor, cause the processor to perform a method for presenting video content, the method comprising: receiving an identifier of a spherical video content item, wherein the spherical video content item has a plurality of views; selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional area using a projection defined by a mapping, neighboring points of the first frame being mapped to corresponding neighboring points of the area according to the mapping, and one or more consecutive portions of the frame being each mapped to a corresponding plurality of consecutive portions of the area; identifying a partition within the region that satisfies a criterion indicating that the region has a likelihood greater than a threshold of including a particular type of content; In response to identifying a partition within the area that meets the criteria, analyzing the identified partition of the area using a video fingerprinting technique; and In response to determining, using the video fingerprinting technique, that content associated with the identified partition of the area matches a reference content item in a plurality of reference content items, generating an indication of the match associated with the identifier of the spherical video content item.

Images