Device and method

Abstract

To provide a device and method capable of precisely extracting a similar moving image similar to a moving image from a comparison moving image.SOLUTION: A device 10 comprises: an acquisition part 12 which acquires information related to a moving image and information related to a comparison moving image; a comparison part 13 which compares information related to an object in one image constituting the moving image with information related to an object in arbitrary one image constituting the comparison moving image based upon the information related to the moving image and the information related to the comparison moving image; and an extraction part 14 which extracts a similar moving image similar to the moving image from the comparison moving image based upon the comparison result of the comparison part 13.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an apparatus and a method. [Background technology]

[0002] Patent Document 1 describes a device for extracting similar videos from multiple comparison videos. The device extracts frame-by-frame features from a target video and candidate video clips. The device then determines from the candidate video clips which video clips match the target video based on the degree of matching between the target video and each candidate video clip calculated based on the features of each frame. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-168117 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the device described in Patent Document 1 cannot further extract scenes similar to the target video from the determined candidate video clips. For example, when the target video is a video showing a basketball game, it is possible to determine a candidate video clip showing the basketball game from multiple candidate video clips. However, it is not possible to extract scenes from the candidate video clips that resemble the game situation in the target video. Therefore, it is desirable to accurately extract similar videos that are similar to the target video from comparison videos.

[0005] The present invention has been made in view of the above, and aims to provide an apparatus and method that can accurately extract similar videos that are similar to a video from a comparison video. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the device of the present invention comprises an acquisition unit that acquires information about a video and information about a comparison video, a comparison unit that compares information about an object in an image that constitutes the video with information about an object in any image that constitutes the comparison video based on the information about the video and the information about the comparison video, and an extraction unit that extracts similar videos that are similar to the video from the comparison video based on the comparison result by the comparison unit. [Effects of the Invention]

[0007] In the device and method according to the present invention, similar videos that are similar to a video are extracted from the comparison video based on the results of a comparison between information about an object in an image that constitutes the video and information about an object in any one of the images that constitute the comparison video. According to this configuration, similar videos that are similar to a video are extracted from the comparison video based on the results of a comparison between the appearance of an object in an image that constitutes the video and the appearance of an object in any one of the images that constitute the comparison video. For example, similar videos that are similar to a video are extracted from the comparison video based on the results of a comparison between the positioning of a person in a space captured in an image that constitutes the video and the positioning of a person in a space captured in any one of the images that constitute the comparison video. This allows for accurate extraction of similar videos that are similar to a video from the comparison video. [Brief explanation of the drawings]

[0008] [Figure 1] 1 illustrates an apparatus and method according to an embodiment; [Figure 2] 1 is a block diagram showing an apparatus according to an embodiment of the present invention; [Figure 3] FIG. 10 is a diagram showing an example of a moving image and a comparative moving image. [Figure 4] FIG. 10 is a diagram showing an example of the positions of objects in a still image and a comparison still image. [Figure 5] 10A and 10B are diagrams for explaining a process for calculating the similarity between a still image and a comparison still image. [Figure 6]1 is a flowchart illustrating the operation of the device of the present disclosure. [Figure 7] 7 is a flowchart showing an example of the comparison process in FIG. 6. [Figure 8] 7 is a flowchart showing an example of extraction processing in FIG. 6. [Figure 9] FIG. 1 is a diagram illustrating an example of a hardware configuration of an apparatus according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present disclosure will be described with reference to the accompanying drawings. Whenever possible, the same parts are designated by the same reference numerals and redundant description will be omitted.

[0010] FIG. 1 illustrates an apparatus and method according to this embodiment. In this embodiment, the apparatus extracts similar videos from a comparison video. In this embodiment, the apparatus is a mobile device such as a smartphone or tablet, but may also be a server 20. In the example shown in FIG. 1, members of an amateur basketball team film their team's game using the camera function of the apparatus 10 to obtain a video M1. The apparatus 10 obtains a video showing a game of a professional basketball team as a comparison video M2 from the server 20. The apparatus 10 extracts similar videos from the comparison video M2 that are similar to the video M1. For example, scenes in which the positions of players from the own team and the opposing team are similar to those in the video M1 are extracted from the comparison video M2. The server 20 is, for example, a server to which videos showing games of professional basketball teams are uploaded.

[0011] Next, the function of extracting similar videos will be described in detail. FIG. 2 is a block diagram showing the device according to this embodiment. As shown in FIG. 2, the device 10 includes an imaging unit 11, an acquisition unit 12, a comparison unit 13, and an extraction unit 14. The device 10 may be one element of a search system configured by the device and a server 20.

[0012] The imaging unit 11 captures images of real space to generate a moving image M1. For example, the imaging unit 11 uses a photoelectric conversion element such as a CCD or CMOS and processes a photoelectric conversion signal output from the imaging element to generate an image of real space. The imaging unit 11 acquires two-dimensional images of three colors, R (red), G (green), and B (blue), for example. The imaging unit 11 generates the moving image M1 by continuously capturing the two-dimensional images.

[0013] The acquisition unit 12 acquires information about the video M1 and information about the comparison video M2. The acquisition unit 12 outputs the information about the video M1 and information about the comparison video M2 to the comparison unit 13. Specifically, the acquisition unit 12 acquires information about the video M1 from the imaging unit 11. The information about the video M1 includes information indicating a still image. The video M1 is made up of a plurality of still images. In other words, a still image is one image that makes up the video M1. FIG. 3(a) is a diagram showing an example of the video M1. In the example shown in FIG. 3(a), the video M1 is a video captured by members of an amateur basketball team watching their team's game.

[0014] The acquisition unit 12 acquires information about the comparison video M2 from an external source. For example, the acquisition unit 12 acquires information about the comparison video M2 from a server 20 that is communicatively connected to the device 10. The information about the comparison video M2 includes information indicating a plurality of comparison still images. The comparison video M2 is composed of a plurality of still images. In other words, a comparison still image is one image that constitutes the comparison video M2. FIG. 3(b) is a diagram showing an example of the comparison video M2. In the example shown in FIG. 3(b), the comparison video M2 is a video of a game between professional basketball teams.

[0015] The comparison unit 13 compares information about an object in an image constituting the video M1 with information about an object in an arbitrary image constituting the comparison video M2, based on information about the video M1 and information about the comparison video M2. An object is an object in real space, such as a person or an object. As an example, the object is a player or a ball on a basketball court.

[0016] The information about the object includes information about the position of the object and information about the type of object. For example, the information about the object in the still image includes information about the position of the object in the space captured in the still image. As an example, the information about the object in the still image includes coordinate information indicating the position of the object when the space captured in the still image is viewed vertically. Furthermore, the information about the object in the comparison still image includes information about the position of the object in the space captured in the comparison still image. As an example, the information about the object in the comparison still image includes coordinate information indicating the position of the object when the space captured in the comparison still image is viewed vertically.

[0017] Furthermore, for example, the information about the object in the still image and the information about the object in the comparison still image each include information about the type of the object, such as a type indicating a player on the offensive team, a type indicating a player on the defensive team, or a type indicating a ball.

[0018] Specifically, first, the comparison unit 13 generates information about objects in the still image based on information indicating the still image. In the example shown in FIG. 3(a), the comparison unit 13 selects one still image G1 from multiple still images constituting the video M1. The comparison unit 13 detects objects in the still image G1. The comparison unit 13 extracts bounding boxes of objects belonging to the object type indicating players. The comparison unit 13 generates information about the position of each player (e.g., coordinates A11 to A15, D11 to D15) based on the position of the bounding box. In other words, the information about objects in the still image includes information about the position of the object in the space captured in the still image. Furthermore, the comparison unit 13 generates information indicating that each player (e.g., the player corresponding to coordinates A11 to A15) is an offense player and information indicating that each player (e.g., the player corresponding to coordinates D11 to D15) is a defense player based on the color of the uniform worn by the player in the bounding box.

[0019] FIG. 4(a) is a diagram showing an example of the position of an object in a still image. In the example shown in FIG. 4(a), the comparison unit 13 compares an image showing a basketball court in a planar view with an image showing the basketball court in a still image G1. The comparison unit 13 calculates a matrix for converting coordinates on the basketball court in the still image G1 into coordinates on the basketball court in a planar view. The comparison unit 13 uses the calculated matrix to convert coordinates A11 to A15, D11 to D15 indicating the positions of each player in the still image G1 into coordinates A11 to A15, D11 to D15 on the basketball court in a planar view (coordinates A11 to A15, D11 to D15 shown in FIG. 4(a)). The matrix may be a homography transformation matrix that converts the coordinates of the four corners and the center point of each side of the basketball court in the still image G1 into the coordinates of the four corners and the center point of each side of the basketball court in a planar view, or may be any other known transformation matrix.

[0020] Next, the comparison unit 13 generates information about objects in the comparison still image based on information indicating the comparison still image. In the example shown in FIG. 3(b), the comparison unit 13 selects one comparison still image G2 from the multiple comparison still images that make up the comparison video M2. The comparison unit 13 detects objects in the comparison still image G2. The comparison unit 13 extracts bounding boxes of objects that belong to the object type indicating players. The comparison unit 13 generates information about the position of each player (e.g., coordinates A21 to A25, D21 to D25) based on the position of the bounding box. The comparison unit 13 generates information indicating that each player (e.g., players corresponding to coordinates A21 to A25) is an offense player and information indicating that each player (e.g., players corresponding to coordinates D21 to D25) is a defense player based on the color of the uniform worn by the players in the bounding box.

[0021] 4(b) is a diagram showing an example of the position of an object in a comparison still image. In the example shown in FIG. 4(b), the comparison unit 13 compares an image showing a basketball court in a planar view with an image showing a basketball court in a comparison still image G2. The comparison unit 13 calculates a matrix for converting coordinates on the basketball court in the comparison still image G2 into coordinates on the basketball court in a planar view. The comparison unit 13 uses the calculated matrix to convert coordinates A21 to A25 and D21 to D25 indicating the positions of each player in the comparison still image G2 into coordinates A21 to A25 and D21 to D25 on the basketball court in a planar view. The matrix may be a homography transformation matrix that converts the coordinates of the four corners and the center point of each side of the basketball court in the comparison still image G2 into the coordinates of the four corners and the center point of each side of the basketball court in a planar view, or it may be any other known transformation matrix.

[0022] Next, the comparison unit 13 compares information about objects in the still image with information about objects in the plurality of comparison still images, and calculates the degree of similarity between the still image and each of the plurality of comparison still images as a comparison result. Figures 5(a) and (b) are diagrams for explaining the process of calculating the degree of similarity between a still image and a comparison still image. In the example shown in Figures 5(a) and (b), the comparison unit 13 compares the positions of players on an amateur basketball team with the positions of players on a professional basketball team.

[0023] More specifically, the comparison unit 13 compares coordinates A11-A15 and D11-D15 on the basketball court in a planar view with coordinates A21-A25 and D21-D25 on the basketball court in a planar view. In the example shown in FIG. 5(a), the comparison unit 13 solves an optimal transportation problem between coordinates A11-A15 of offensive players on the basketball court in a planar view and coordinates A21-A25 of offensive players. In the example shown in FIG. 5(b), the comparison unit 13 solves an optimal transportation problem between coordinates D11-D15 of defensive players on the basketball court in a planar view and coordinates D21-D25 of defensive players. The comparison unit 13 calculates the sum of distances obtained as a result of solving the two optimal transportation problems as an index of similarity between the still image and the comparison still image.

[0024] In calculating the similarity, the coordinates of the offensive players and the defensive players in each of the still image and the comparison still image are regarded as two-dimensional matrices. When solving the optimal transportation problem between the matrices, the transportation cost of the transportation method with the lowest transportation cost (the sum of the distances required to move each coordinate) is used as the similarity. In this case, the smaller the sum of the distances (transportation cost), the higher the similarity between the still image and the comparison still image. Note that the method of calculating the similarity is not limited to the above, and other known methods may be used.

[0025] In this way, the comparison unit 13 estimates the coordinates of each player on the basketball court from still image G1 and comparison still image G2 included in the video M1. The comparison unit 13 calculates the similarity between the coordinates of each player corresponding to still image G1 and the coordinates of each player corresponding to comparison still image G2. Note that the comparison unit 13 may also estimate the coordinates of the ball on the basketball court from still image G1 and comparison still image G2 included in the video M1. The comparison unit 13 may calculate the similarity by further taking into account the coordinates of the ball corresponding to still image G1 and the coordinates of the ball corresponding to comparison still image G2.

[0026] The extraction unit 14 extracts similar videos similar to the video M1 from the comparison video M2 based on the comparison result by the comparison unit 13. Specifically, the extraction unit 14 extracts similar videos of a predetermined period from the comparison video M2 that contain similar images, which are comparison still images similar to the still image, based on the comparison result. For example, the extraction unit 14 selects, from among multiple comparison still images included in the comparison video, the comparison still image that has the highest similarity to the still image as the similar image. The extraction unit 14 extracts similar videos of a predetermined period that contain similar images from the comparison video. As an example, the extraction unit 14 sorts all comparison still images (all frames) included in the comparison video M2 in descending order of similarity according to the similarity calculated by the comparison unit 13. The extraction unit 14 extracts several seconds of video from the comparison still image (frame) with the highest similarity as the similar video. In this way, the extraction unit 14 extracts scenes similar to the video from the comparison video.

[0027] Next, the processing executed by the device 10 according to this embodiment will be described using the flowcharts of FIGS. 6 to 8. FIG. 6 is a flowchart showing the operation of the device 10 of the present disclosure. The device 10 may capture an image of real space and generate a video M1 in advance. The device 10 may start the following processing in response to a user operation, or may start the following processing automatically when the video M1 is generated.

[0028] Device 10 acquires information about the video and information about the comparison video (step S01: acquisition step). Based on the information about the video and the information about the comparison video, device 10 compares information about objects in one image that constitutes the video with information about objects in one image that constitutes the comparison video (step S02: comparison step). Based on the comparison result in the comparison step, device 10 extracts similar videos that are similar to the video from the comparison video (step S03: extraction step).

[0029] The processing in step S02 will be described in detail. FIG. 7 is a flowchart showing an example of the comparison processing in step S02. Device 10 randomly extracts one still image from among still images included in a video (step S21). Device 10 acquires each frame of the comparison video as a plurality of comparison still images (step S22). Device 10 calculates the similarity between each of the plurality of comparison still images and the still image (step S23). In step S23, device 10 compares information about objects in a still image generated based on information indicating the still image with information about objects in a plurality of comparison still images generated based on the plurality of comparison still images, and calculates the similarity between the still image and each of the plurality of comparison still images as a comparison result.

[0030] The processing in step S03 will be described in detail. FIG. 8 is a flowchart showing an example of the extraction processing in step S03. First, device 10 selects one comparison still image from multiple comparison still images as a similar image based on the calculated similarity (step S31). Device 10 extracts the appearance time of the similar image in the comparison video (step S32). Device 10 extracts multiple comparison still images before and after the appearance time by an arbitrary time, and selects the extracted multiple comparison still images as a similar video (step S33).

[0031] Next, the effects of the device 10 and method of the present disclosure will be described. The device 10 and method of the present disclosure extract similar videos from the comparison video based on the results of a comparison between information about an object in an image constituting the video and information about an object in any one of the images constituting the comparison video. According to this configuration, similar videos similar to the video are extracted from the comparison video based on the results of a comparison between the appearance of an object in an image constituting the video and the appearance of an object in any one of the images constituting the comparison video. For example, similar videos similar to the video are extracted from the comparison video based on the results of a comparison between the arrangement of people in the space captured in an image constituting the video and the arrangement of people in the space captured in any one of the images constituting the comparison video. As an example, similar videos similar to the video are extracted from the comparison video based on the results of a comparison between the arrangement of players on a basketball court captured in an image constituting a video showing an amateur basketball team's game and the arrangement of players on a basketball court captured in an image constituting a video showing a professional basketball team's game. This allows for accurate extraction of similar videos similar to the video from the comparison video.

[0032] Here, the above-mentioned effects will be explained in detail. Traditionally, in team sports such as basketball, improving team tactics has been as important as improving individual skills. Methods for improving tactics for amateur teams include reading books on basic tactics, seeking instruction from a coach, and imitating the plays of professional teams. However, since reading books incurs learning costs and receiving instruction from a coach incurs financial costs, amateur teams cannot easily adopt these two methods. Therefore, amateur teams often adopt the method of imitating the plays of professional teams and obtain hints for improving their own team's tactics from the plays of professional teams during games. However, it is difficult to search for and extract scenes of desired situations from videos of professional team games.

[0033] Conventional methods for searching for scenes of desired situations from videos showing professional team matches include a method for searching by inputting keywords, and a method for inputting a playbook (a method in which players and the ball are represented by markers and movements are represented by arrows, etc.) and searching for playbooks similar to the input playbook. With the above two methods, the accuracy of the search varies depending on the input keywords and the input playbook. For example, if a user does not have knowledge of keywords and how to write and read a playbook, they will not be able to input appropriate keywords and an appropriate playbook, and will not be able to obtain accurate search results.

[0034] Furthermore, a conventional method for searching for a scene with a desired situation from a video showing a professional team game involves using a sentence describing the type, number, and relationship of objects in the video. However, in a video showing a sports game, it is difficult to express the differences in the situation from scene to scene in a sentence. For example, the sentence "there are 10 people and one ball" corresponds to all scenes in a video showing a basketball game. Therefore, with the above method, it may be difficult for a user to extract the scene they are truly looking for.

[0035] As such, methods have been proposed in the past for searching for scenes similar to a video from a comparison video, but the above-mentioned methods do not take into account the detailed situation of each scene within the video, so when applied to a video showing a sports match, for example, it is thought that it will not be possible to search for scenes with the situation intended by the user from the comparison video.

[0036] In contrast, the device 10 of the present disclosure extracts similar videos from the comparison videos based on the results of a comparison between information about an object in an image constituting the video and information about an object in any image constituting the comparison video. This allows similar videos similar to the video to be extracted from the comparison videos with high accuracy and efficiency. As a result, a player or coach of an amateur team can use the device 10 to accurately and efficiently search for scenes similar to videos showing their team's matches from videos showing professional team matches. For example, simply by filming a video showing an amateur player's match, scenes similar to the filmed video can be extracted from videos showing professional player matches. Therefore, even if a user has no prior knowledge of the device 10, they can intuitively search for scenes similar to videos showing their team's matches from videos showing professional team matches. As a result, more players and teams can improve their tactics more efficiently.

[0037] Furthermore, in the device 10 of the present disclosure, the information about the video may include information indicating a still image, and the information about the comparison video may include information indicating multiple comparison still images. The comparison unit 13 may compare information about objects in a still image generated based on the information indicating the still image with information about objects in multiple comparison still images generated based on the multiple comparison still images, and calculate the similarity between the still image and each of the multiple comparison still images as a comparison result. The extraction unit 14 may extract similar videos of a predetermined period from the comparison video that include similar images, which are comparison still images similar to the still image, based on the similarity between the still image and each of the multiple comparison still images. In this case, similar videos of a predetermined period that include similar images, which are comparison still images similar to the still image, are extracted from the comparison video based on the similarity between the still image and each of the multiple comparison still images. With this configuration, if there is a high similarity between the appearance of an object in a still image and the appearance of an object in a comparison still image, it is possible to extract similar videos of a predetermined period that include the comparison still image from the comparison video. For example, if the position of people in the space captured in a still image is highly similar to the position of people in the space captured in a comparison still image, it is possible to extract a similar video of a predetermined period that includes the comparison still image from the comparison video, thereby enabling more accurate extraction of similar videos that are similar to the video from the comparison video.

[0038] Furthermore, in the device 10 of the present disclosure, the information about an object in a still image may include information about the position of the object in the space captured in the still image, and the information about an object in a comparison still image may include information about the position of the object in the space captured in the comparison still image. In this case, if the position of an object in the space captured in the still image is similar to the position of an object in the space captured in the comparison still image, it is possible to extract a similar video of a predetermined period that includes the comparison still image from the comparison video. This allows for more accurate extraction of similar videos from the comparison video.

[0039] Furthermore, in the device 10 of the present disclosure, the information about the object in the still image and the information about the object in the comparison still image may each include information about the type of object. In this case, the type of each object is taken into consideration when extracting similar videos from the comparison video. This allows for more accurate extraction of similar videos from the comparison video.

[0040] Furthermore, in the device 10 of the present disclosure, the object may be an object in real space. In this case, a similar video that is similar to a video captured in real space can be extracted from the comparison video.

[0041] The device 10 of the present disclosure may also include an imaging unit 11 that captures images of real space and generates a video. The acquisition unit 12 may acquire information about the video from the imaging unit 11 and acquire information about the comparison video from an external source. In this case, after a user captures an image using the device 10, similar videos that are similar to the video captured by the user can be immediately extracted from the comparison videos. As a result, the user's convenience in extracting similar videos can be improved.

[0042] The apparatus 10 and method of the present disclosure include the following components.

[0043] [1] an acquisition unit that acquires information about the video and information about the comparison video; a comparison unit that compares information about an object in one image constituting the video with information about an object in any one image constituting the comparison video based on information about the video and information about the comparison video; An apparatus comprising: an extraction unit that extracts similar videos that are similar to the video from the comparison video based on a comparison result by the comparison unit.

[0044] [2] the information about the moving image includes information indicating a still image, the information about the comparison moving image includes information about a plurality of comparison still images; the comparison unit compares information about objects in the still image generated based on information indicating the still image with information about objects in the plurality of comparison still images generated based on the plurality of comparison still images, and calculates a similarity between the still image and each of the plurality of comparison still images as the comparison result; The extraction unit extracts, from the comparison video, the similar video of a predetermined period that includes a similar image, which is the comparison still image similar to the still image, based on the comparison result. [1] The device described in [1].

[0045] [3] the information about the object in the still image includes information about the position of the object in a space captured in the still image; the information about the object in the comparison still image includes information about the position of the object in a space captured in the comparison still image; [2] The device described in [2].

[0046] [4] the information about the object in the still image and the information about the object in the comparison still image each include information about a type of the object; [2] or [3].

[0047] [5] The object is a physical object in real space. [1] to [4].

[0048] [6] further comprising an imaging unit that captures an image of a real space and generates the video; the acquisition unit acquires information about the video from the imaging unit and acquires information about the comparison video from an external source; [1] to [5].

[0049] [7] an acquisition step of acquiring information about the video and information about the comparison video; a comparison step of comparing information about an object in one image constituting the video with information about an object in any one image constituting the comparison video based on information about the video and information about the comparison video; An extraction step of extracting similar videos that are similar to the video from the comparison video based on the comparison result in the comparison step.

[0050] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, by wire, wirelessly, etc.) and these multiple devices. The functional block may also be realized by combining the single device or the multiple devices with software.

[0051] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, election, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocation, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0052] For example, the device 10 according to an embodiment of the present disclosure may function as a computer that performs processing of the wireless communication method of the present disclosure. Fig. 9 is a diagram illustrating an example of a hardware configuration of the device 10 according to an embodiment of the present disclosure. The device 10 described above may be physically configured as a computer including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

[0053] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configuration of apparatus 10 may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0054] Each function of device 10 is realized by loading specified software (programs) onto hardware such as processor 1001 and memory 1002, causing processor 1001 to perform calculations, control communication via communication device 1004, and control at least one of reading and writing data in memory 1002 and storage 1003.

[0055] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the above-mentioned comparison unit 13 and extraction unit 14 may be realized by the processor 1001.

[0056] The processor 1001 also reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. For example, the comparison unit 13 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and similar implementations may be made for other functional blocks. While the above-described various processes have been described as being executed by one processor 1001, they may also be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may also be transmitted from a network via a telecommunications line.

[0057] The memory 1002 is a computer-readable recording medium and may be configured, for example, by at least one of a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing a wireless communication method according to an embodiment of the present disclosure.

[0058] Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray disc), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other appropriate medium including at least one of memory 1002 and storage 1003.

[0059] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc. to realize at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the above-mentioned acquisition unit 12 may be realized by the communication device 1004. The acquisition unit 12 may be implemented as a transmission unit and a reception unit that are physically or logically separated.

[0060] The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may be integrated into one device (for example, a touch panel).

[0061] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0062] Furthermore, device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, processor 1001 may be implemented using at least one of these pieces of hardware.

[0063] The notification of information is not limited to the aspects / embodiments described in the present disclosure and may be performed using other methods. For example, the notification of information may be performed by physical layer signaling (e.g., Downlink Control Information (DCI), Uplink Control Information (UCI)), higher layer signaling (e.g., Radio Resource Control (RRC) signaling, Medium Access Control (MAC) signaling, broadcast information (Master Information Block (MIB), System Information Block (SIB))), other signals, or a combination thereof. Furthermore, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

[0064] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0065] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0066] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0067] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Furthermore, notification of predetermined information (e.g., notification that "X is true") is not limited to being done explicitly, but may be done implicitly (e.g., by not notifying the predetermined information).

[0068] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0069] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0070] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0071] As used in this disclosure, the terms "system" and "network" are used interchangeably.

[0072] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0073] Note that terms explained in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Furthermore, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0074] Furthermore, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information.

[0075] The names used for the above-described parameters are not intended to be limiting in any way. Furthermore, the mathematical expressions using these parameters may differ from those explicitly disclosed in this disclosure. The various channels (e.g., PUCCH, PDCCH, etc.) and information elements may be identified by any suitable names, and therefore the various names assigned to these various channels and information elements are not intended to be limiting in any way.

[0076] In this disclosure, the terms "Mobile Station (MS)," "user terminal," "User Equipment (UE)," "terminal," etc. may be used interchangeably.

[0077] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terminology.

[0078] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0079] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using one or more wires, cables, and / or printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0080] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0081] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0082] When used in this disclosure, the terms "include," "including," and variations thereof are intended to be inclusive, similar to the term "comprising." Furthermore, when used in this disclosure, the term "or" is not intended to be an exclusive or.

[0083] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0084] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different." [Explanation of symbols]

[0085] 10...device, 12...acquisition unit, 13...comparison unit, 14...extraction unit, 1001...processor, 1002...memory, 1003...storage, 1004...communication device, 1005...input device, 1006...output device.

Claims

1. an acquisition unit that acquires information about the video and information about the comparison video; a comparison unit that compares information about an object in one image constituting the video with information about an object in any one image constituting the comparison video based on information about the video and information about the comparison video; An apparatus comprising: an extraction unit that extracts similar videos that are similar to the video from the comparison video based on a comparison result by the comparison unit.

2. the information about the moving image includes information indicating a still image, the information about the comparison moving image includes information about a plurality of comparison still images; the comparison unit compares information about objects in the still image generated based on information indicating the still image with information about objects in the plurality of comparison still images generated based on the plurality of comparison still images, and calculates a similarity between the still image and each of the plurality of comparison still images as the comparison result; The extraction unit extracts, from the comparison video, the similar video of a predetermined period that includes a similar image, which is the comparison still image similar to the still image, based on the comparison result.

10. The apparatus of claim 1.

3. the information about the object in the still image includes information about the position of the object in a space captured in the still image; the information about the object in the comparison still image includes information about the position of the object in a space captured in the comparison still image; 3. The apparatus of claim 2.

4. the information about the object in the still image and the information about the object in the comparison still image each include information about a type of the object; 3. The apparatus of claim 2.

5. The object is a physical object in real space.

10. The apparatus of claim 1.

6. further comprising an imaging unit that captures an image of a real space and generates the video; the acquisition unit acquires information about the video from the imaging unit and acquires information about the comparison video from an external source; 10. The apparatus of claim 1.

7. an acquisition step of acquiring information about the video and information about the comparison video; a comparison step of comparing information about an object in one image constituting the video with information about an object in any one image constituting the comparison video based on information about the video and information about the comparison video; An extraction step of extracting similar videos that are similar to the video from the comparison video based on the comparison result in the comparison step.

Images