Apparatus and method for generating panoramic image using content information

Abstract

The present invention relates to an apparatus and a method for generating a panoramic image. The apparatus for generating a panoramic image according to an embodiment of the present invention comprises: a reception unit for receiving input information for providing a panoramic image; an image provision unit for providing the panoramic image to a user on the basis of the input information; and a data selection unit for selecting hot spot content highly related to the panoramic image among content information located within a preset range on the basis of location information of the panoramic image, wherein the image provision unit provides the panoramic image and the hot spot content together.

Description

Device and method for generating a panoramic image using content information

[0001] The present invention relates to a device and method for generating a panoramic image using content information, and more particularly, to a device and method for generating a panoramic image for providing hotspot content on a panoramic image based on content information uploaded by users.

[0002] Existing panoramic image generation devices require the photographer to move a certain distance to provide the user with a panoramic image.

[0003] The panoramic image generation device outputs an arrow indicating a location where an image is located within a certain distance on a panoramic image, and when a user clicks on the arrow, a panoramic image corresponding to the location is output, and a panoramic image that exists within a certain distance based on the output panoramic image is displayed as an arrow, providing a road view service that allows the user to take a virtual tour.

[0004] However, the road view service is provided by the map provider directly linking a certain number of panoramic images that have moved a certain distance with linked arrows, but in the user participation map platform service, the locations of multiple contents are not normalized, making it difficult to provide contents within a certain distance based on the panoramic image as a road view.

[0005] Accordingly, an object of the present invention is to provide a panoramic image generation device and method in which content with high importance is output to a panoramic image based on a shooting location according to an embodiment of the present invention.

[0006] According to one aspect of an embodiment of the present invention, a panoramic image generation device comprises: a receiving unit that receives input information for providing a panoramic image; an image providing unit that provides the panoramic image to a user based on the input information; a data selecting unit that selects hotspot content having a high correlation with the panoramic image from among content information located within a preset range based on location information of the panoramic image; and the image providing unit provides the panoramic image and the hotspot content together.

[0007] In addition, the data selection unit may include a data preprocessing unit that calculates an average distance with the content information based on the location information and determines whether the content information is included in a data set that is a selection target of the hotspot content based on the calculated average distance.

[0008] In addition, the data preprocessing unit calculates an average standard deviation from the content information based on the average distance, and sequentially calculates a distance interval between adjacent content information in order from the closest content information before the average distance to the farthest content information after the average distance based on the average distance. However, if the distance interval is greater than the average standard deviation, content information that becomes the target for calculating the distance interval thereafter may not be included in the data set.

[0009] Additionally, the data selection unit may not include a clustering unit that performs density-based clustering on the data set based on Euler angles to generate at least one cluster corresponding to a viewing angle direction.

[0010] Additionally, the clustering unit can determine the number of hotspot contents for each cluster based on the number of content information included in each cluster.

[0011] In addition, the clustering unit can determine the number of hotspot contents per cluster using the following mathematical formula.

[0012] Mathematical formula: (N is an integer)

[0013] (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster)

[0014] Additionally, the minimum number of hotspot contents per cluster may be at least one.

[0015] In addition, the data selection unit may include a data post-processing unit that calculates content importance based on the popularity of the content information, address similarity between the location information and the content information, and the distance between the location information and the content information, and selects the hotspot content in order of high content importance.

[0016] In addition, the data post-processing unit calculates the content importance differently depending on the category that outputs the panoramic image, and when the category is SNS, the weight applied to the popularity is set to be greater than the weight applied to the distance between the location information and the content information, thereby calculating the content importance, and when the category is road view, the weight applied to the distance between the location information and the content information is set to be greater than the weight applied to the popularity, thereby calculating the content importance.

[0017] A method for generating a panoramic image according to an aspect of an embodiment of the present invention comprises: an input information receiving step of receiving input information for providing a panoramic image; a data selection step of selecting hotspot content having a high correlation with the panoramic image from among content information located within a preset range based on location information of the panoramic image; an image providing step of providing the panoramic image to a user based on the input information; and the image providing step may provide the panoramic image and the hotspot content together.

[0018] In addition, in the data selection step, a data preprocessing step may be included that calculates an average distance with the content information based on the location information and determines whether the content information is included in the data set that is the target of selection of the hotspot content based on the calculated average distance.

[0019] In addition, in the data preprocessing step, an average standard deviation with the content information is calculated based on the average distance, and the distance interval between adjacent content information is sequentially calculated from the closest content information before the average distance to the farthest content information after the average distance based on the average distance, but if the distance interval is greater than the average standard deviation, the content information that becomes the target for calculating the distance interval thereafter may not be included in the data set.

[0020] Additionally, the data selection step may include a clustering step of performing density-based clustering on the data set based on Euler angles to generate at least one cluster corresponding to a viewing angle direction.

[0021] Additionally, in the clustering step, the number of hotspot contents per cluster can be determined based on the number of content information included in each cluster.

[0022] Additionally, in the clustering step, the number of hotspot contents per cluster can be determined using the following mathematical formula.

[0023] Mathematical formula: (N is an integer)

[0024] (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster)

[0025] Additionally, the minimum number of hotspot contents per cluster may be at least one.

[0026] In addition, in the data selection step, a data post-processing step may be included for calculating content importance based on the popularity of the content information, address similarity between the location information and the content information, and the distance between the location information and the content information, and selecting the hotspot content in order of high content importance.

[0027] In addition, in the data post-processing step, the content importance is calculated differently depending on the category that outputs the panoramic image, and when the category is SNS, the weight applied to the popularity is set to be greater than the weight applied to the distance between the location information and the content information, thereby calculating the content importance, and when the category is road view, the weight applied to the distance between the location information and the content information is set to be greater than the weight applied to the popularity, thereby calculating the content importance.

[0028] According to the proposed embodiment, there is an effect that allows for selecting hotspot contents with high importance from non-normalized content information by combining various technologies and outputting them on a panoramic image.

[0029] Figure 1 is an exemplary drawing showing a panoramic image generated using conventional content information.

[0030] FIG. 2 is a drawing showing a panoramic image generation system including a panoramic image generation device according to an embodiment of the present invention.

[0031] FIG. 3 is a block diagram schematically illustrating the configuration of a panoramic image generation device among the panoramic image generation systems of FIG. 2.

[0032] Fig. 4 is a flowchart illustrating a method for generating a panoramic image using the panoramic image generating device of Fig. 3.

[0033] FIG. 5 is an exemplary drawing showing a process for selecting content information located within a preset range based on location information by the panoramic image generation device of FIG. 3.

[0034] FIG. 6 is an exemplary drawing showing a process for determining whether a data set of content information is included based on an average distance, by the panoramic image generation device of FIG. 3.

[0035] FIG. 7 is an exemplary drawing showing a plurality of clusters generated by the panoramic image generating device of FIG. 3.

[0036] FIG. 8 is an exemplary diagram showing a process for determining the number of hotspot contents per cluster by the panoramic image generation device of FIG. 3.

[0037] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0038] Although terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, it should be understood that the "first" component referred to below may also be a "second" component within the technical scope of the present invention.

[0039] Identical reference numerals throughout the specification refer to identical components.

[0040] The individual features of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment can be implemented independently of each other or can be implemented together in a related relationship.

[0041] Meanwhile, the tentative effects that can be expected by the technical features of the present invention that are not specifically mentioned in the specification of the present invention are treated as described in the specification, and the present embodiment is provided to more completely explain the present invention to a person having average knowledge in the art, and the contents shown in the drawings may be expressed exaggeratedly compared to the actual implementation of the invention, and a detailed description of a configuration that is judged to unnecessarily obscure the gist of the present invention is omitted or briefly described.

[0042] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0043] Figure 1 is an exemplary drawing showing a panoramic image generated using conventional content information (C).

[0044] Referring to FIG. 1, a user can capture content information (C), which is a panoramic image including captured location information (O), using a user device (100).

[0045] A panoramic image generation device (300) can receive content information (C) from a user device (100), generate a panoramic image based on geographical information, and output the content information (C) as hotspot content (A) on a panoramic image (I).

[0046] At this time, the panoramic image generation device (300) outputs hotspot content on a panoramic image (I) taken at a certain location based on location information (O), so that the user can view content information (C) connected to the hotspot content (A) by selecting the hotspot content (A) corresponding to the viewing angle direction of the panoramic image that the user wants to view using the user device (100).

[0047] However, the conventional user-participatory content-based panoramic image generation device (300) in which users directly shoot and post content, when an unspecified number of people upload content information (C) in the same space, a plurality of hotspot contents (A1, A2, A3, A4, A5, A6, A7, A8, A9, A) are generated in all directions of the panoramic image (I). 10 ) is printed.

[0048] In addition, the content information (C) transmitted on the panoramic image is not normalized, so all hotspot contents (A1, A2, A3, A4, A5, A6, A7, A8, A9, A) on the panoramic image are not normalized. 10) or only content information (C) within a certain distance range based on location information (O) is output as hotspot content (A), which causes a problem in which unrefined content information (C) is scattered on the panoramic image.

[0049] Accordingly, the panoramic image generation device (300) of the present invention can select content information (C) with high importance from non-normalized content information (C) using multiple processing methods, and can provide hotspot content (A) of the selected content information (C) together with a panoramic image.

[0050] In addition, according to the proposed embodiment, by selecting hotspot content (A) using various information separately from the location information (O) of the content information (C), it is possible to provide content information (C) tailored to various platforms utilizing the panoramic image (I).

[0051] FIG. 2 is a drawing showing a panoramic image generation system including a panoramic image generation device according to an embodiment of the present invention.

[0052] Referring to FIG. 2, the panoramic image generation system (10) may include a user device (100) and a panoramic image generation device (300).

[0053] The user device (100) can transmit content information (C) captured by the user and location information (O) of the content information (C) to the panoramic image generation device (300) via a communication network (200).

[0054] At this time, the content information (C) may be information about content generated based on location information (O) such as video and audio, but preferably, the content information (C) may be a panoramic image in which multiple images are aligned.

[0055] A user device (100) can transmit content information (C) to a panoramic image generation device (300), and the panoramic image generation device (300) can provide the content information (C) to the user device (100) so that it can be displayed on a panoramic image in the form of hotspot content (A).

[0056] The user device (100) receives hotspot content (A) and a panoramic image transmitted from a panoramic image generation device (300), outputs the hotspot content (A) on the panoramic image, and can receive a selection from the user of at least one hotspot content (A) output on the panoramic image.

[0057] When a user selects hotspot content (A) output on a panoramic image through a user device (100), the user device (100) requests content information (C) corresponding to the selected hotspot content (A) from a panoramic image generation device (300), and the panoramic image generation device (300) transmits the content information (C) which is a panoramic image to the user device (100), so that the user can view a panoramic image which is content information located in one direction of the panoramic image which was first viewed by the user.

[0058] Accordingly, the user can check content information existing in each location direction based on the panoramic image using the user device (100), and thus use a virtual tour or road view based on the panoramic image received from the panoramic image generation device (300).

[0059] The user device (100) can be connected to a panoramic image generation device (300) and a communication network (200).

[0060] A communication network (200) refers to a connection structure that enables information exchange between each node, such as multiple terminals and servers, and examples of such networks include a local area network (LAN), a wide area network (WAN), the Internet (WWW), a wired and wireless data communication network, a telephone network, a wired and wireless television communication network, etc. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, and DMB (Digital Multimedia Broadcasting) network.

[0061] A panoramic image generation device (300) provides a panoramic image to a user device (100) based on input information input from the user device (100), so that the user can receive a panoramic image associated with geographic information.

[0062] At this time, the panoramic image generation device (300) can select content information based on the location information (O) of the panoramic image and provide the panoramic image and the selected content information, which is hotspot content (A).

[0063] For example, the panoramic image generation device (300) may be a server, but is not limited thereto, and the panoramic image generation device (300) may be a device including a smart device and a control unit capable of providing panoramic images and hotspot content (A).

[0064] In this embodiment, a configuration is disclosed in which a user device (100) requests a panoramic image from a panoramic image generating device (300) using a communication network (200), and the panoramic image generating device (300) provides a panoramic image to the user device (100), but is not limited thereto. In this embodiment, a panoramic image may be provided from a panoramic image database stored in a storage unit of the user device (100), and when the user selects at least one hotspot content (A) based on the panoramic image output through the user device (100), content information (C) corresponding to the hotspot content (A) may be provided to the user.

[0065] FIG. 3 is a block diagram schematically illustrating the configuration of a panoramic image generation device among the panoramic image generation systems of FIG. 2.

[0066] Referring to FIG. 3, a panoramic image generation device (300) may include a receiving unit (310), an image providing unit (320), and a data selection unit (330).

[0067] First, the receiver (310) can receive input information for providing a panoramic image.

[0068] In detail, the receiving unit (310) can receive input information about a panoramic image that the user device (100) wants to receive from the panoramic image generation device (300).

[0069] For example, the receiver (310) may be a communication module capable of receiving data using a communication network (200), but is not limited thereto, and may be one of the input modules capable of receiving input information from a user.

[0070] The above input information may be geographic information or location information (O) displayed on a panoramic image, but is not limited thereto, and the input information may be separate coordinate information for providing a specific panoramic image.

[0071] The image providing unit (320) can provide a panoramic image to the user based on the above input information, and the image providing unit (320) can provide the panoramic image and hotspot content (A) together.

[0072] In detail, the user transmits location information (O) for which a panoramic image is to be provided to the panoramic image generation device (300) through the user device (100), and the receiving unit (310) receives the input information including the location information (O) to be provided, and the image providing unit (320) can provide the user device (100) used by the user with a panoramic image corresponding to the location information (O) and hotspot content (A) to be output on the panoramic image, which will be described later, based on the input information.

[0073] At this time, when the video providing unit (320) provides multiple hotspot contents selected from the same cluster, it can provide the multiple hotspot contents to the user in the form of one hotspot, and when the user selects the hotspot contents provided in the form of one hotspot, it can provide the user with information on multiple contents included in the same cluster.

[0074] The image providing unit (320) can provide hotspot contents included in the same cluster in the form of a single hotspot, thereby simplifying multiple hotspot contents located in a similar direction, thereby reducing the complexity of hotspot contents (A) output on the panoramic image.

[0075] The data selection unit (330) can select hotspot content (A) having a high correlation with the panoramic image from among content information (C) located within a preset range based on the location information (O) of the panoramic image.

[0076] At this time, the data selection unit (330) may include a data preprocessing unit (331), a clustering unit (332), and a data postprocessing unit (333).

[0077] First, the data selection unit (330) may include a data preprocessing unit (331) that calculates an average distance (M) with the content information (C) based on the location information (O) of the panoramic image, and determines whether the content information (C) is included in the data set that is the selection target of the hotspot content (A) based on the calculated average distance (M).

[0078] In detail, the data preprocessing unit (331) calculates an average standard deviation with content information (C) based on an average distance (M), and sequentially calculates a distance interval between adjacent content information in order from the closest content information before the average distance (M) to the farthest content information after the average distance. However, if the distance interval is greater than the average standard deviation, the content information that becomes the target for calculating the distance interval thereafter may not be included in the data set.

[0079] At this time, the data set may be a data set of content information that is the target of selection of hotspot content, or a set of content information that exists within a range from the location information of the panoramic image to the furthest content information that is the target of selection of hotspot content.

[0080] At this time, the average distance (M) can be calculated by the following mathematical formula 1.

[0081]

[0082] That is, the average distance (M) may be a value obtained by dividing the total sum of the distances (r) from the location information of the panoramic image to the content information (C) included within a preset range by the number of content information (n).

[0083] Additionally, the average standard deviation can be calculated using the following mathematical formula 2.

[0084]

[0085] (σ is the mean standard deviation, c is the mean standard deviation coefficient, r is the distance of content information, M is the mean distance, n is the number of content information)

[0086] Additionally, the data selection unit (330) may include a clustering unit (332) that performs density-based clustering on the data set based on Euler angles to generate at least one cluster corresponding to the viewing angle direction.

[0087] At this time, the Euler angle may be at least one value among the angles for yaw, pitch, and roll of each content information (C) based on the position information (O) of the panoramic image.

[0088] At this time, the clustering unit (332) can group dense content information by performing density-based clustering, which is a data clustering algorithm.

[0089] For example, density-based clustering may be one of the algorithms MeanShift, DBSCAN, HDBSCAN, and K-Means, but is not limited thereto. Density-based clustering may be an algorithm that clusters based on the density of data.

[0090] In this embodiment, a configuration is disclosed in which the clustering unit (332) performs density-based clustering to cluster content information, but the present invention is not limited thereto, and clustering of content information densely concentrated in a single area may be performed.

[0091] The clustering unit (332) can determine the number of hotspot contents for each cluster based on the number of content information included in each cluster.

[0092] At this time, the clustering unit (332) can determine the number of hotspot contents per cluster using the following mathematical expression 3.

[0093]

[0094] (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster)

[0095] For example, the clustering unit (332) is set to have a minimum number of hotspot contents per cluster of 1, a ratio of additionally selected content information among the total number of content information per cluster of 0.1, and when the number of content information per cluster is 10, the number of hotspot contents per cluster can be selected as 2, and when the number of content information per cluster is 15, the number of hotspot contents per cluster can be selected as 2, which is the same as when the number of content information per cluster is 10.

[0096] At this time, the minimum number of hotspot contents per cluster may be 0, but the minimum number of hotspot contents per cluster is set to at least one, so that the clustering unit (332) can select at least one hotspot content per cluster, and the panoramic image generation device (300) can provide at least one hotspot content corresponding to the viewing angle direction on the panoramic image.

[0097] In addition, the data selection unit (330) may include a data post-processing unit (333) that calculates content importance based on the popularity of the content information (C), the address similarity between the location information (O) and the content information (C), and the distance between the location information (O) and the content information (C), and selects the hotspot content in order of the high content importance.

[0098] At this time, the popularity of the content information may be at least one of the number of views, favorites, and citations of the content information by users, and is not limited thereto. The popularity of the content information may be a parameter that can determine the measure of popularity among users.

[0099] Additionally, the data post-processing unit (333) can determine the address similarity between location information (O) and content information (C) and calculate an address similarity value.

[0100] The data post-processing unit (333) can calculate an address similarity value through a known string similarity judgment technique, such as a string similarity algorithm or an artificial intelligence algorithm, between an address corresponding to the location information (O) of a panoramic image and an address corresponding to the location of content information (C).

[0101] At this time, the data post-processing unit (333) calculates the content importance based on the address similarity value, so that hotspot content (A) selected to enable movement to a building sharing the same address or a surrounding area with a similar address can be provided to the user.

[0102] The data post-processing unit (333) calculates the content importance differently depending on the category that outputs the panoramic image, and when the category is SNS, the weight applied to the popularity is set to be greater than the weight applied to the distance between the location information (O) and the content information (C), thereby calculating the content importance, and when the category is road view, the weight applied to the distance between the location information (O) and the content information (C) is set to be greater than the weight applied to the popularity, thereby calculating the content importance.

[0103] At this time, the category may be a platform or service that utilizes a panoramic image that the user uses through the user device (100), and for example, the category may be SNS, road view, restaurant application service, etc.

[0104] In the case where the category for outputting the panoramic image is SNS, the data post-processing unit (333) may set the popularity weight for mentions and views among users of SNS as important parameters, and thus the weight for these may be set to be greater than the weights for other parameters.

[0105] In addition, since the data post-processing unit (333) must provide the closest content information (C) from the location information (O) where the panoramic image was shot when the category for outputting the panoramic image is road view, the weight applied to the distance between the location information (O) of the panoramic image and the content information can be set to be large.

[0106] The data post-processing unit (333) calculates the content importance differently according to the category, so that the hotspot content of the panoramic image is provided differently, which has the advantage of providing hotspot content tailored to the platform and service used by the user.

[0107] Fig. 4 is a flowchart illustrating a method for generating a panoramic image using the panoramic image generating device of Fig. 3.

[0108] Referring to FIG. 4, a method for generating a panoramic image using a panoramic image generating device may include a user input step (S100), an input information receiving step (S200), a data selection step (S300), and an image providing step (S400).

[0109] First, in the user input step (S100), the user device (100) can transmit input information for receiving a panoramic image to the panoramic image generation device (300).

[0110] At this time, in the user input step (S100), the user device (100) can receive input information from the user and transmit it to the panoramic image generation device (300), and without limitation, the user can directly input input information into the panoramic image generation device (300) and receive a panoramic image through the user device (100) or the panoramic image generation device (300) in the image provision step (S400) described later.

[0111] In the input information receiving step (S200), the panoramic image generation device (300) can receive input information for providing a panoramic image.

[0112] In the data selection step (S300), the panoramic image generation device (300) can select hotspot content having a high correlation with the panoramic image from among content information located within a preset range based on the location information (O) of the panoramic image.

[0113] In detail, the data selection step (S300) may include a data preprocessing step (S310), a clustering step (S320), and a data postprocessing step (S330).

[0114] First, the data selection step (S300) may include a data preprocessing step (S310) in which the panoramic image generation device (300) calculates an average distance (M) with the content information (C) based on the location information (O) and determines whether the content information (C) is included in the data set that is the selection target of the hotspot content (A) based on the calculated average distance (M).

[0115] In the data preprocessing step (S310), the panoramic image generation device (300) calculates an average standard deviation with the content information (C) based on the average distance (M), and sequentially calculates a distance interval between adjacent content information in order from the closest content information before the average distance (M) to the farthest content information after the average distance (M), but if the distance interval is greater than the average standard deviation, the content information that becomes the target for calculating the distance interval thereafter may not be included in the data set.

[0116] Next, the data selection step (S300) may include a clustering step (S320) in which the panoramic image generation device (300) performs density-based clustering on the data set based on Euler angles to generate at least one cluster corresponding to the viewing angle direction.

[0117] In the clustering step (S320), the panoramic image generation device (300) can determine the number of hotspot contents for each cluster based on the number of content information included in each cluster.

[0118] In the clustering step (S320), the panoramic image generation device (300) can determine the number of hotspot contents per cluster using the following mathematical expression 3.

[0119] (N is an integer)

[0120] (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster)

[0121] At this time, the minimum number of hotspot contents per cluster may be 0, but the minimum number of hotspot contents per cluster may be at least one.

[0122] Next, the data selection step (S300) may include a data post-processing step (S330) in which the panoramic image generation device (300) calculates content importance based on the popularity of the content information, the address similarity between the location information (O) and the content information, and the distance between the location information and the content information, and selects hotspot content (A) in order of high content importance.

[0123] In the data post-processing step (S330), the panoramic image generation device (300) calculates the content importance differently depending on the category that outputs the panoramic image, and when the category is SNS, the weight applied to the popularity is set to be greater than the weight applied to the distance between the location information (O) and the content information (C), thereby calculating the content importance, and when the category is road view, the weight applied to the distance between the location information (O) and the content information (C) is set to be greater than the weight applied to the popularity, thereby calculating the content importance.

[0124] In the image providing step (S400), the panoramic image generating device (300) can provide the panoramic image to the user based on the input information.

[0125] At this time, in the image providing step (S400), the panoramic image generating device (300) can provide the panoramic image and the hotspot content (A) together.

[0126] FIG. 5 is an exemplary drawing showing a process for selecting content information (C) located within a preset range based on location information (O) by the panoramic image generation device (300) of FIG. 3.

[0127] Referring to FIG. 5, the receiving unit (310) of the panoramic image generation device (300) can receive input information for providing a panoramic image.

[0128] At this time, the image providing unit (320) of the panoramic image generating device (300) can provide a panoramic image based on input information, and at this time, the data selection unit (330) can select hotspot content having a high correlation with the panoramic image from among content information (C) located within a preset range (DR) based on the location information (O) of the panoramic image included in the input information.

[0129] At this time, the preset range (DR) may be a range created based on the preset distance (L) to select hotspot content (A) with high relevance to the panoramic image.

[0130] The image providing unit (320) of the panoramic image generating device (300) first selects content information (C) located within a preset range (DR) based on location information (O), thereby selecting hotspot content targeting content information (C) around where the panoramic image was shot, and excluding content information that was shot at a distant location and has low relevance based on the shooting location information (O) of the panoramic image.

[0131] FIG. 6 is an exemplary drawing showing a process for determining whether a data set of content information is included based on an average distance (M) by the panoramic image generation device of FIG. 3.

[0132] Referring to FIG. 6, the data preprocessing unit (331) of the panoramic image generation device (300) calculates an average distance (M) with the content information (C) based on the location information (O) of the panoramic image, and can determine whether the content information is included in the data set that is the selection target of the hotspot content (A) based on the calculated average distance (M).

[0133] At this time, the data preprocessing unit (331) can include content information (C6, C7, C8, C9) located within an average distance (M) based on the location information (O) of the panoramic image in the data set to be selected as the hotspot content, and in the case of content data located outside the average distance (M), it additionally determines whether to include it in the data set to be selected as the hotspot content (A), and excludes data that are relatively far from the location information (O) of the provided panoramic image from the hotspot content (A).

[0134] In detail, the data preprocessing unit (331) calculates the average standard deviation with the content information (C) based on the average distance (M), and calculates the closest content information (C) before the average distance (M) based on the average distance (M). 10 ) from the farthest content information (C) after the above average distance (M) 13 ) can be sequentially calculated as a distance interval (d) between adjacent content information, and if the distance interval (d) is greater than the average standard deviation, the content information that is the target for calculating the distance interval thereafter is not included in the data set, thereby excluding content information that is somewhat distant from the content information after the average distance (M) from the data set, thereby excluding content data that has low correlation with the provided panoramic image from being selected as hotspot content (A).

[0135] For example, the data preprocessing unit (331) of the panoramic image generation device (300) processes the 10th content information (C), which is the closest content information before the average distance (M), based on the average distance (M). 10 ) and adjacent 11th content information (C 11 ) and the distance interval (d) 11 ) is calculated as in the following mathematical formula 4.

[0136]

[0137] (d 11 is the distance interval between the 10th content information and the 11th content information, r 10 is the distance between the 10th content information and the location information, r 11 11) Distance between content information and location information

[0138] Next, the 10th content information (C 10 ) and the 11th content information (C 11 ) distance interval (d) 11 ) and the average standard deviation compared to the 10th content information (C 10 ) and the 11th content information (C 11 ) are included in the data set if the distance interval is not greater than the average standard deviation.

[0139] Next, the 11th content information (C 11 ) and adjacent 12th content information (C 12 ) and the distance interval (d) 12 ) is calculated as in mathematical formula 5 below.

[0140]

[0141] (d 12 is the distance interval between the 11th content information and the 12th content information, r 11 is the distance between the 11th content information and the location information, r 12 is the distance between the 12th content information and location information)

[0142] At this time, the 11th content information (C 11) and the 12th content information (C 12 ) distance interval (d) 12 ) is less than the average standard deviation, the 12th content information (C 12 ) can be included in the data set, but the 11th content information (C 11 ) and the 12th content information (C 12 ) by comparing the distance interval and the average standard deviation of the 11th content information (C 11 ) and the 12th content information (C 12 ) distance interval (d) 12 ) is greater than the average standard deviation, the 12th content information (C 12 ) is the 12th content information (C), which is the target content information for calculating the distance interval thereafter. 12 ) and the 13th content information (C 13 ) may not be included in the data set.

[0143] If the data preprocessing unit (331) determines that the distance between content information is greater than the average standard deviation, the calculation of the distance distance (d) between content information (C) to be included in the subsequent data set can be terminated.

[0144] Accordingly, when the distance interval between content information (C) is determined to be greater than the average standard deviation, the content information (C) that is the target for calculating the subsequent distance interval is excluded from the data set to be selected as hotspot content (A), so that content information having a low distance correlation with the panoramic image including the location information (O) can be excluded from the selection target of the hotspot content output together with the panoramic image.

[0145] FIG. 7 is an exemplary drawing showing a plurality of clusters generated by the panoramic image generating device of FIG. 3.

[0146] Referring to FIG. 7, the data selection unit (330) includes a clustering unit (332) and can perform density-based clustering on the data set based on Euler angles to generate at least one cluster corresponding to the viewing angle direction.

[0147] At this time, the Euler angle may be at least one value among the angles for yaw, pitch, and roll of each content information (C) based on the position information (O), and for example, the Euler angle may be a yaw value, which is a Y-axis rotation angle in the north direction (N) based on the position information (O).

[0148] Based on the location information (O) of the panoramic image, content located in each direction is clustered to create at least one cluster, and content information (C) to be selected as at least hotspot content (A) among the clusters can be selected.

[0149] For example, based on one direction (Y0) of the north direction (N) of the location information (O) of the panoramic image, content information (C) included in the data set (DS) can be clustered according to the direction criteria (Y1, Y2, Y3, Y4, Y5, Y6, Y7) of each viewing angle, so that seven clusters can be created.

[0150] At this time, the direction criterion may be the average value of the Euler angle of each content information (C) based on the position information (O) of the panoramic image, but is not limited thereto, and may be the average direction value of the clustered content information (C) based on the position information (O) of the panoramic image.

[0151] The clustering unit (332) performs clustering on the data set (DS) which is the target content information (C) for selecting the hotspot content (A), thereby selecting some content information among the plurality of content information concentrated in a specific direction as the hotspot content (A), thereby simplifying the proliferation of hotspot content output on the panoramic image that occurs due to the existence of a plurality of content information, and the user can check the surrounding content information that is highly related to the panoramic image through the provision of the hotspot content.

[0152] FIG. 8 is an exemplary drawing showing a process for determining the number of hotspot contents per cluster by the panoramic image generation device of FIG. 3.

[0153] Referring to FIG. 8, the clustering unit (332) of the panoramic image generation device (300) can determine hotspot content among the content information included in the cluster.

[0154] At this time, the clustering unit (332) determines the number of hotspot contents (A) for each cluster based on the number of content information (C1, C2, C3, C4, C5) included in each cluster.

[0155] At this time, the data selection unit (330) may include a data post-processing unit (333) that calculates content importance based on the popularity of the content information, address similarity between the location information and the content information, and the distance between the location information and the content information, and selects the hotspot content (A) in order of high content importance.

[0156] In detail, the data post-processing unit (333) can calculate the content importance based on the popularity of the content information within each cluster, the address similarity between the location information and the content information, and the distance between the location information and the content information.

[0157] For example, within the fourth cluster (CL4), five pieces of content information (C1, C2, C3, C4, C5) are included, and parameters for the popularity of each piece of content information, the address similarity between the location information and the content information, and the distance between the location information and the content information can be expressed as follows.

[0158]

[0159] C1 = {Popularity 20, address similarity 0.56, distance 1.7}

[0160] C2 = {Popularity 25, address similarity 0.56, distance 1.7}

[0161] C3 = {Popularity 10, address similarity 0.56, distance 2.2}

[0162] C4 = {Popularity 30, Address Similarity 0.48, Distance 2.5}

[0163] C5 = {Popularity 10, Address Similarity 0.28, Distance 2.8}

[0164]

[0165] The importance of the content information (C) can be calculated using the following mathematical expression 6 using the parameters of the above content information (C).

[0166]

[0167] (Weight: Content importance of content information, α1: Popularity weight, α2: Address similarity weight, α3: Distance weight)

[0168]

[0169] By applying the above formula and the parameters of each content information, the content importance of each content information can be calculated as follows.

[0170]

[0171] Content importance of the first content information (C1) = {weight: 100}

[0172] Content importance of the second content information (C2) = {weight: 154}

[0173] Content importance of the third content information (C3) = {weight: 92}

[0174] Content importance of the 4th content information (C4) = {weight: 148}

[0175] Content importance of the 5th content information (C5) = {weight: 76}

[0176]

[0177] The above clustering unit (332) can determine the number of hotspot contents per cluster using the following mathematical expression 3.

[0178] (N is an integer)

[0179] (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster)

[0180] At this time, if the minimum number of hotspot contents per cluster is 1, and the ratio of additionally selected content information among the total number of content information per cluster is 0.1, the number of content information (C1, C2, C3, C4, C5) included in the cluster is 5, so the number of hotspot contents per cluster is determined to be 1, and therefore, among the 5 content information (C1, C2, C3, C4, C5) included in the cluster, the second content information (C2) having the highest content importance is selected as the hotspot content, and the hotspot content can be provided to the user device (100) together with the panoramic image.

[0181] The selected hotspot content is provided together with the panoramic image, so that the hotspot content (A) can be provided to the user in the form of a hotspot in the location direction of the content information selected as the hotspot content (A) based on the location information (O) of the panoramic image.

[0182] According to this embodiment, by selecting hotspot content (A) based on highly relevant content information among content information close to the location information (O) of the provided panoramic image, the user has the advantage of being able to check the selected hotspot content (A) in each direction based on the location information (O) of the panoramic image together with the panoramic image.

[0183] Although the preferred embodiments of the present invention have been described above, the present invention is not limited thereto, and various modifications can be made within the scope of the patent claims, the detailed description of the invention, and the attached drawings, and it is obvious that this also falls within the scope of the present invention.

[0184] The mode for carrying out the invention has been described together with the best mode for carrying out the invention above.

[0185] The present invention relates to a device and method for generating a panoramic image using content information, and is applicable to various devices or methods for generating panoramic image content, and has industrial applicability due to its repeatability.

Claims

1. A receiving unit that receives input information for providing a panoramic image; An image providing unit that provides the panoramic image to the user based on the above input information; A data selection unit that selects hotspot content highly related to the panoramic image from among content information located within a preset range based on location information of the panoramic image; The above video provider, A panoramic image generation device characterized by providing the above panoramic image and the above hotspot content together.

2. In paragraph 1, The above data selection unit, Calculate the average distance from the content information based on the above location information, A panoramic image generation device characterized by including a data preprocessing unit that determines whether the content information is included in a data set that is a target for selection of the hotspot content based on the calculated average distance.

3. In paragraph 2, The above data preprocessing unit, Based on the above average distance, the average standard deviation from the above content information is calculated, Based on the above average distance, the distance interval between adjacent content information is sequentially calculated from the closest content information before the above average distance to the farthest content information after the above average distance. A panoramic image generation device characterized in that, if the distance interval is greater than the average standard deviation, content information that is the target for calculating the distance interval thereafter is not included in the data set.

4. In paragraph 2, The above data selection unit, A panoramic image generation device characterized by including a clustering unit that performs density-based clustering based on Euler angles on the above data set to generate at least one cluster corresponding to a viewing angle direction.

5. In paragraph 4, The above clustering part, A panoramic image generation device characterized in that the number of hotspot contents for each cluster is determined based on the number of content information included in each of the above clusters.

6. In paragraph 5, The above clustering part A panoramic image generation device characterized in that the number of hotspot contents per cluster is determined using the following mathematical formula. Mathematical formula: (N is an integer) (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster) 7. In paragraph 6, A panoramic image generation device characterized in that the minimum number of hotspot contents per cluster is at least one.

8. In paragraph 1, The above data selection unit, A panoramic image generation device characterized by including a data post-processing unit that calculates content importance based on the popularity of the content information, address similarity between the location information and the content information, and the distance between the location information and the content information, and selects the hotspot content in order of high content importance.

9. In paragraph 8, The above data post-processing unit, The importance of the content is calculated differently depending on the category that outputs the above panoramic image, If the above category is SNS, the weight applied to the above popularity is set to be greater than the weight applied to the distance between the above location information and the above content information, and the above content importance is calculated. A panoramic image generation device characterized in that, when the above category is a road view, a weight applied to the distance between the location information and the content information is set to be greater than a weight applied to the popularity, thereby calculating the content importance.

10. In a method for generating a panoramic image using a panoramic image generating device, An input information receiving step for receiving input information for providing a panoramic image; A data selection step for selecting hotspot content highly related to the panoramic image from among content information located within a preset range based on the location information of the panoramic image; An image providing step of providing the panoramic image to the user based on the input information; The above video provision step is: A method for generating a panoramic image, characterized in that the panoramic image and the hotspot content are provided together.

11. In paragraph 10, In the above data selection step, Calculate the average distance from the content information based on the above location information, A method for generating a panoramic image, characterized in that it includes a data preprocessing step of determining whether the content information is included in a data set that is a target for selection of the hotspot content based on the calculated average distance.

12. In paragraph 11, In the above data preprocessing step, Based on the above average distance, the average standard deviation from the above content information is calculated, Based on the above average distance, the distance interval between adjacent content information is sequentially calculated from the closest content information before the above average distance to the farthest content information after the above average distance. A method for generating a panoramic image, characterized in that if the distance interval is greater than the average standard deviation, content information that is the target for calculating the distance interval thereafter is not included in the data set.

13. In paragraph 11, In the above data selection step, A method for generating a panoramic image, characterized in that it comprises a clustering step of performing density-based clustering based on Euler angles on the above data set to generate at least one cluster corresponding to a viewing angle direction.

14. In paragraph 13, In the above clustering step, A method for generating a panoramic image, characterized in that the number of hotspot contents for each cluster is determined based on the number of content information included in each of the above clusters.

15. In paragraph 14, In the above clustering step, A method for generating a panoramic image, characterized in that the number of hotspot contents per cluster is determined using the following mathematical formula. Mathematical formula: (N is an integer) (N is the number of hotspot contents for each cluster, a is the minimum number of hotspot contents for each cluster, b is the ratio of additionally selected content information among the total number of content information for each cluster, k is the total number of content information for each cluster) 16. In paragraph 15, A method for generating a panoramic image, characterized in that the minimum number of hotspot contents per cluster is at least one.

17. In paragraph 11, In the above data selection step, A method for generating a panoramic image, characterized in that it includes a data post-processing step of calculating content importance based on the popularity of the content information, the address similarity between the location information and the content information, and the distance between the location information and the content information, and selecting the hotspot content in order of the high content importance.

18. In paragraph 17, In the above data post-processing step, The importance of the content is calculated differently depending on the category that outputs the above panoramic image, If the above category is SNS, the weight applied to the above popularity is set to be greater than the weight applied to the distance between the above location information and the above content information, and the above content importance is calculated. A method for generating a panoramic image, characterized in that when the above category is a road view, the weight applied to the distance between the location information and the content information is set to be greater than the weight applied to the popularity, thereby calculating the content importance.

Images