Information processing device and information processing system

Abstract

The aim is to reduce the effort required for users to identify the location where a scene in the content was filmed. [Solution] The present invention provides an information processing device for providing content to an external device, and is characterized by comprising: receiving means for receiving information corresponding to a scene of the content selected by a user; acquiring means for acquiring geographical information corresponding to the selected scene indicated by the received information corresponding to the scene; and transmitting means for transmitting the acquired geographical information corresponding to the selected scene to the external device.

Description

Technical Field

[0004] ,

[0006] , , , ,

[0005] , , , , ,

[0001] The present invention relates to an information processing apparatus and an information processing system.

Background Art

[0002] Patent Document 1 describes a technique in which links to other images containing products in an image are displayed by an image recognition technique.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

[0007] According to the present invention, the effort required of users to identify the location where a scene in the content was filmed is reduced. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of a system configuration. [Figure 2] A diagram showing an example of the hardware configuration of each device. [Figure 3] A diagram showing an example of a service application screen. [Figure 4] A diagram showing an example of a service application screen. [Figure 5] A diagram showing an example of a map service screen. [Figure 6] A sequence diagram illustrating an example of the processes executed when a content service is provided. [Figure 7] Sequence diagram showing an example of the process when a map of the shooting location is generated. [Figure 8] A flowchart illustrating an example of a process that generates a map of the shooting location. [Figure 9] A flowchart illustrating an example of the process for generating a map of the shooting location. [Figure 10] This diagram shows an example of a screen where the estimated shooting location is confirmed to the user. [Figure 11] A flowchart illustrating an example of a process to improve the accuracy of the shooting location. [Figure 12] A diagram showing an example of a screen where a user reports the location of a photograph. [Figure 13] A flowchart illustrating an example of the process for generating a map of the filming location when there is a reporting history. [Modes for carrying out the invention]

[0009] The best embodiment of the present invention will be described in detail below with reference to the drawings. Note that the following embodiments are not intended to limit the invention as defined in the claims, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] In this application, AI (Artificial Intelligence), also known as artificial intelligence, refers to computer systems and software designed to mimic human intelligence. This AI technology enables functions such as learning, reasoning, problem-solving, perception, and language comprehension. Technologies and methods for realizing AI functions include machine learning, neural networks, natural language processing, robotics, and pattern recognition. These technologies can be applied to various uses such as data analysis, automation tasks, image recognition, and speech recognition.

[0011] Furthermore, in this application, generative AI refers to a technology that automatically generates various types of content such as text, images, music, and videos, similar to those created by humans, by utilizing techniques such as deep learning and machine learning.

[0012] <First Embodiment> Figure 1 shows an example of the system configuration in this embodiment. In this embodiment, as shown in Figure 1(a), client terminals 10-12, data server 20, and AI server 30 are connected via the Internet 50 and are able to communicate with each other. In this system, the number of connected components may be any number, and the system may be composed of multiple combinations. Furthermore, this system constructs an information processing system with these configurations.

[0013] The client terminals 10 to 12 are information processing devices such as smartphones and personal computers (PCs). By executing various programs and applications on the client terminals 10 to 12, the user can utilize various functions. In this embodiment, an example will be described where the user uses the client terminals 10 to 12 to view video content, maps, etc.

[0014] The data server 20 is a type of server, which is an information processing device that communicates with various devices such as other computers and devices through a network such as the Internet 50 and transmits and receives data. In this embodiment, the data server 20 plays a role of providing online content such as videos streamed and played on the client terminals 10 to 12 and services for providing online content. Along with this, the data server 20 also provides applications (service apps) executed on the client terminals 10 to 12 for providing the services.

[0015] <000,0093>The AI server 30 is a type of server, which is also an information processing device that communicates with various devices such as other computers and devices through a network such as the Internet 50 and transmits and receives data. In this embodiment, the AI server 30 provides a generative AI function. Also, the AI server 30 provides a service (map service) that performs image recognition on the image indicated by the received image data, acquires information from an information source (not shown), estimates location information, and generates a map based on the estimated location information.

[0016] The Internet 50 is a type of network and plays a role of mediating communication between connected devices. The connection method may be a wired connection method such as WAN (Wide Area Network) or Ethernet, or a wireless connection method such as Wi-Fi. <000,0098> Alternatively, a system configuration like the one shown in Figure 1(b) may be used. In this example, the data server 20 and the AI ​​server 30 are connected via a network separate from the Internet 50 to which the client terminals 10-12 are connected, such as a LAN (Local Area Network). In this case, the AI ​​server 30 is not connected to the Internet 50. Even with this configuration, it is possible to perform the same communication as in the configuration of Figure 1(a), and in this case, the client terminals 10-12 and the AI ​​server 30 can communicate via the data server 20.

[0018] The functions and services provided by the data server 20 and AI server 30 described in this embodiment may be implemented by sharing the workload among multiple servers. Conversely, if the functions and services provided by the data server 20 and AI server 30 can be implemented by a single server or system, then it is not necessary for the servers to be separated as in this embodiment.

[0019] In particular, the map service provided by the AI ​​server 30 may be configured such that, for example, the AI ​​server 30 generates the map, and another server provides the subsequent services. Note that in this embodiment, the client terminals 10-12 may be referred to as client terminal 10, but this does not mean that only one terminal must be specified.

[0020] Figure 2 shows an example of the hardware configuration of each device in the system of the embodiment. Specifically, it shows an example of the hardware configuration of client terminals 10-12, data server 20, and AI server 30.

[0021] Figure 2(a) shows an example of the hardware configuration of client terminals 10 to 12. In this embodiment, client terminals 10 to 12 consist of a CPU 11, ROM 12, RAM 13, storage 14, communication I / F 15, GPU 16, operation unit 17, and camera 18. CPU stands for Central Control Unit, ROM for Read Only Memory, and RAM for Random Access Memory. I / F stands for Interface, and GPU stands for Graphics Processing Unit.

[0022] The CPU 11 is a central processing unit that performs processing such as control and calculations of various parts based on various programs such as application programs and software programs stored in memory devices such as ROM 12 and storage 14. The CPU 101 controls each part by exchanging control signals and the like with each part that is connected to the CPU 11 via the bus 19. For example, by controlling the communication I / F 15, it enables data exchange between this device and other devices connected to this device.

[0023] The CPU 11 also executes various programs and performs data processing based on instructions input by the user via the operation unit 17. In addition, it performs various processes based on instructions such as control commands received from external devices via the communication interface 15.

[0024] ROM12 is a type of non-volatile memory that stores various programs, such as application programs and software programs, that are read by the CPU11. RAM13 is a type of volatile memory that functions as the main memory, work area, etc., of the CPU11.

[0025] Storage 14 is a type of storage medium on which various data such as various programs, image data, and video data are stored, and is composed of, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Storage 14 has the function of storing (recording and saving) data input to it, and outputting stored data to the outside of Storage 14, for example, by communicating with the CPU 11, and the stored contents can be changed. In this embodiment, Storage 14 stores the program of the service application for executing the service provided by the data server 20.

[0026] Communication I / F15 is an interface that connects various parts of this device, including the CPU11, to external devices via the Internet50, thereby enabling network communication.

[0027] GPU16 is a computing unit composed of image processing processors that can process complex mathematical calculations in parallel at high speed.

[0028] The operation unit 17 consists of, for example, buttons, a keyboard, a touch panel, etc., and user instructions are received by the CPU 11 via the operation unit 17. It may also have a display, and by displaying information on the display or touch panel (display unit), client terminals 10 to 12 can transmit information to the user. In this embodiment, content such as videos provided by the data server 20 is displayed on the operation unit 17, and the user can give instructions such as selecting or playing videos via the operation unit 17.

[0029] The camera 18 is a device that converts light into electrical signals and records them as digital data (image data), and can capture images and videos. The image data obtained through capture can be stored in the storage 14.

[0030] Client terminals 10-12 may also be devices that project light, such as projectors, to display an image on a screen or the like. In this case, client terminals 10-12 have light-emitting parts, lens parts, etc., to perform their functions. In this case, the screen displayed on the operation unit 17 will be projected onto the screen by the projector.

[0031] Figure 2(b) shows an example of the hardware configuration of the data server 20. In this embodiment, the data server 20 consists of a CPU 21, ROM 22, RAM 23, storage 24, and communication I / F 25. The configuration and functions of these parts are the same as those described using Figure 2(a), so their explanation is omitted.

[0032] Figure 2(c) shows an example of the hardware configuration of the AI ​​server 30. In this embodiment, the AI ​​server 30 consists of a CPU 31, ROM 32, RAM 33, storage 34, communication I / F 35, and GPU 36. The configuration and functions of these parts are the same as those described using Figure 2(a), so their explanation is omitted. The GPU 36 of the AI ​​server 30 performs calculations to convert input image and text data using a Large Language Model (LLM) according to control commands given by the CPU 31, for example.

[0033] Figure 3 shows an example of a service application screen provided by the data server 20. Screen 300 shown in Figure 3(a), screen 330 shown in Figure 3(b), and screen 360 shown in Figure 3(c) are examples of screens displayed on the operation unit 17 of the client terminal 10.

[0034] Service applications vary depending on the service provider and the content they handle, so several examples are provided. Here, screen 300 shown in Figure 3(a) is an example of a service application screen used for a service that provides content produced by specialized groups such as filmmakers, drama producers, and anime producers. On the other hand, screen 330 shown in Figure 3(b) is an example of a service application screen used for a service that provides content such as videos that can be shared by individuals with other individuals, regardless of the genre. In such services, the service vendor issues accounts to users, who can then post content using those accounts.

[0035] Furthermore, even for screens provided by the same service application from the same vendor, the display can differ depending on the type of client terminal 10 displaying the screen and the orientation of the client terminal 10 (operation unit 17), so several examples are shown. Here, screen 330 shown in Figure 3(b) is an example of a service application screen displayed on the operation unit 17 of a client terminal 10 such as a smartphone or a small tablet device. On the other hand, screen 360 shown in Figure 3(c) is an example of a service application screen displayed on the operation unit 17 of a client terminal 10 such as a PC, a large tablet device, or a projector.

[0036] Figure 3(a) shows a first example of a service application screen provided by the data server 20. Screen 300 is a screen that includes various objects and areas, which will be described later. Screen 300 is also a screen that is displayed when the user makes a selection on a content selection screen (not shown), and it displays information about the selected content.

[0037] Area 301 is an image commonly known as a thumbnail, which represents the content or displays an image of a part of the content. Note that area 301 is not limited to still images; it may also display, for example, a video (a medium that appears to have continuous movement visually through the switching of images).

[0038] The title field 302 is an area where a string of text indicating the title of the content displayed on screen 300 is displayed. The format of the title displayed in the title field 302 is not limited to a string of text. If the content title is represented by an area with design elements, such as a logo, the title field 302 may be configured to display the image.

[0039] Object 303 is an object commonly referred to as a play button, and when object 303 is pressed, content provision begins. Specifically, with screen 300 displayed on the operation unit 17 of the client terminal 10, content is displayed / played within area 301. Alternatively, a screen like the one shown in Figure 4 is displayed on one side of the operation unit 17 of the client terminal 10, and content is displayed / played. In this embodiment, content display / playback occurs when the data server 20 and the client terminal 10 are connected. On the other hand, when the data server 20 and the client terminal 10 are not connected, display / playback will not occur (it will stop).

[0040] Object 304 is an object commonly referred to as a download button. When object 304 is pressed, the content displayed on screen 300 is recorded in the storage 14 of the client terminal 10. This allows the content stored in storage 14 to be displayed / played on the client terminal 10 even when the data server 20 and the client terminal 10 are not connected, as described in object 303.

[0041] Description field 305 is an area where a string of text describing the content displayed on screen 300 is shown. This string of text could include, for example, the content's synopsis, cast, release date, number of views, keywords, tags, number of ratings, and reviews. It is not necessary for the entire description to be displayed in description field 305; a portion of the description may be displayed in description field 305, and the description that was not originally displayed may be revealed when description field 305 is selected.

[0042] Object 306 is a commonly known "list registration button," which is used to save the content displayed on screen 300 to a list corresponding to the user account. Object 306 may also be a favorites button or flag, and pressing it will similarly save the content to the list. This improves usability because users can access the content from the list if they intended to view it later or want to view it repeatedly.

[0043] Object 307 is an object commonly known as a rating button, used to rate the content displayed on screen 300. Object 307 may use an icon resembling a thumbs-up, for example. If a user likes the content, selecting object 307 will result in a positive rating. Alternatively, object 307 may use an icon resembling a thumbs-down; in this case, selecting object 307 will result in a negative rating. Multiple objects 307 may exist.

[0044] Object 308 is an object commonly referred to as a share button, and it is used to send information about the content displayed on screen 300 to other users. When object 308 is pressed, other functions and applications available on the client terminal 10 are displayed, and the user can send the content information by selecting or entering the function or application to use and the destination.

[0045] Object 309 is an object used to display location information related to the content, such as the location where the content displayed on screen 300 was filmed. In this embodiment, it is tentatively named and displayed as a "Visit" button, and when object 309 is selected, a map like the one described later is ultimately generated and displayed.

[0046] Figure 3(b) shows a second example of a service application screen provided by the data server 20. Screen 330 is a screen that includes various objects and areas, which will be described later. Screen 330 is also a screen that is displayed when the user makes a selection on a content selection screen (not shown), and it displays information about the selected content.

[0047] Area 331 is the same as area 301. Title field 332 is the same as title field 302. Object 333 is the same as object 303. Object 337 is the same as object 307, and in the example in Figure 3(b), object 337 shows the received evaluation numerically. Object 338 is the same as object 308. Object 339 is the same as object 309.

[0048] Object 340 is commonly referred to as a report button. It is used to report to the vendor if the content displayed on screen 330 violates the terms and conditions set by the service application or is otherwise inappropriate for public display. When object 340 is selected, information about the content displayed on screen 330 is sent to the vendor, allowing the vendor to review the content and take appropriate action, such as making the content private.

[0049] Object 341 is an object commonly referred to as a registration button, which allows users to register the account that provided (uploaded) the content displayed on screen 330. The difference between object 341 and object 306 in Figure 3(a) is that while pressing object 306 saves the content information, pressing object 341 saves (registers) the information of the account that provided the content.

[0050] This allows users to receive notifications, for example, when a registered account provides new content. Information about the account that provided the content will be explained in section 344 below.

[0051] Object 342 is an object commonly referred to as an ellipsis, and it is used to execute other functions provided by the service application. When object 342 is selected, an unillustrated menu that is not shown when screen 330 is displayed appears, and the user can select the function they want to execute from the menu, thereby executing that function.

[0052] Area 343 is one of the areas that displays information about the content shown on screen 330. In this embodiment, it displays the number of times the content has been played, the date the content was provided, and keywords used for searching, known as hashtags. However, the displayed content is not limited to these.

[0053] Area 344 is an area where information about the account that provided (posted) the content displayed on screen 330 is displayed. In this embodiment, the account icon, account name, and number of registered users are displayed. When area 344 is selected, a screen displaying the account's profile is displayed, for example.

[0054] A registered user is defined as an account that registered by clicking on object 341 as mentioned above, or an account that selected the same registration button on the profile screen. The number of registered users is the total number of these accounts.

[0055] Area 345 is generally referred to as the comment section, and it displays a list of comments posted in response to the content displayed on screen 330. In this embodiment, the total number of posted comments and some of the comments are displayed. By selecting area 345, the user can also view comments that were not displayed when screen 330 was displayed.

[0056] Icon 346 represents the user icon of the user who posted text 347, which will be explained next. Text 347 is a comment posted by the account corresponding to icon 346, and the string "3:33" indicates that it is a comment about the scene at 3 minutes and 33 seconds in the video content.

[0057] Area 348 is the area where content related to the content displayed on screen 330, or content recommended for the user who viewed screen 330, is displayed. Based on the content information, content of the same or similar genre may be displayed, or based on the analysis results of users using the service app, content that is likely to be viewed next may be displayed.

[0058] Area 349 is an image commonly known as a thumbnail, which displays an image that represents the content of the related content or an image of a part of the content. Note that area 301 is not limited to still images; it may also display, for example, a video (a medium that appears to have continuous movement visually through the switching of images). The title field 350 is an area where a string of text indicating the title of the related content is displayed.

[0059] Figure 3(c) shows a third example of a service application screen provided by the data server 20. Screen 360 is a screen that includes various objects and areas, which will be explained later. Screen 360 is also a screen that is displayed when the user makes a selection on a content selection screen (not shown), and it displays information about the selected content.

[0060] Area 361 is the same as area 331. Title field 362 is the same as title field 332. Object 363 is the same as object 333. Description field 365 is the same as description field 305. Area 373 is the same as area 343. Area 374 is the same as area 344. Area 375 is the same as area 345. Area 378 is the same as area 348. Area 379 is the same as area 349. Title field 380 is the same as title field 350. Area 381 is an area where information about the content indicated in title field 380 is displayed, and its display content is the same as description field 365.

[0061] Figure 4 shows an example of a service application screen provided by the data server 20, where content is displayed and played. Screens 400-460 are displayed on the operation panel 17 of the client terminal 10 when, for example, objects 303, 333, and 363 are pressed. Four patterns of these screens will be explained below as examples.

[0062] In the examples in screens 400, 420, and 440, video content playback is temporarily paused and each object (menu) is displayed. On the other hand, the example in screen 460 is an example diagram showing a moment when video content is playing after object 401 is selected, and in an actual service application, the content will continue without user intervention after object 401 is selected. Note that the following examples explained in screens 400, 420, and 440 do not necessarily have to be performed while the content is paused; they may also be performed while the content is playing, as explained in screen 460.

[0063] In the screen 400 shown in Figure 4(a), object 401 is an object commonly referred to as a play button, and when object 401 is pressed, content provision begins. Specifically, with screen 400 displayed on the operation unit 17 of the client terminal 10, the content is displayed / played.

[0064] Object 402 is commonly known as a seek bar and is used together with object 403. For example, if the content is a video, it serves to display the time.

[0065] Object 403 is a slider, commonly known as a slider, and when used in conjunction with Object 402, it serves to display things like elapsed time in the case of video content, or progress in the case of games. Object 403 performs its function by moving left and right above Object 402.

[0066] Object 404 is an object commonly referred to as a flag, and when selected, it records information about the content displayed at the time of selection. In other words, it is an object that accepts the user's selection of a scene. As will be described later, when a location that the user wants to visit is displayed (scene) from the displayed / played content, the user can select object 404, and the location displayed at that time will be identified. Specifically, a map is generated by the AI ​​server 30 corresponding to the content information stored when object 404 is selected. Note that the act of the user selecting object 404 is sometimes referred to as setting a flag, saving, or clipping.

[0067] Figure 4(b) is a diagram illustrating in detail the state when object 404 is selected. In this diagram, object 404 is selected by the user and is displayed in a different state than object 404 in Figure 4(a). Specifically, object 404 in Figure 4(b) is distinguishable from object 404 displayed in Figure 4(a) because its color is different, allowing the user to recognize that object 404 has been selected.

[0068] Object 421 is an object that indicates that object 404 has been selected, and is also a type of object 403. Object 421 is displayed on top of object 402, which indicates the elapsed playback time of video content, for example, when object 404 is selected by the user. Furthermore, even as the content progresses, object 421 continues to be displayed at the position indicating the time object 404 was selected. In this case, the position of object 421 corresponds to the elapsed time of the content.

[0069] In the example shown in screen 420, the content corresponding to the time point of the rightmost object 421 is displayed. As will be explained later, specifically, screen 420 is an example showing the scene at 3 minutes and 33 seconds in a video content that is 5 minutes long.

[0070] This allows the user to recognize whether object 404 was selected by object 421, and further, if the content is a video, to recognize when and how many times object 404 was selected.

[0071] Figures 4(c) and 4(d) show examples of how information about the position of the currently displayed scene of the content being played is displayed, and will be explained in detail in the second embodiment.

[0072] Figure 5 shows an example of a map service screen that displays location information for content, generated by the AI ​​server 30. Screen 500 shown in Figure 5(a) and screen 550 shown in Figure 5(b) are examples of screens displayed on the operation unit 17 of the client terminal 10. These screens are map screens that contain geographical information generated based on flags set by the user.

[0073] For example, the content displayed on the map service screen may differ depending on the size of the control panel 17 of the client terminal 10 displaying the screen, so several examples are shown. Note that the content displayed on the map service screen may be the same regardless of which client terminal 10 it is displayed on.

[0074] Here, the screen 500 shown in Figure 5(a) is an example of a map service screen displayed on the operation unit 17 of a client terminal 10, such as a smartphone or a small tablet device. On the other hand, the screen 550 shown in Figure 5(b) is an example of a map service screen displayed on the operation unit 17 of a client terminal 10, such as a PC, a large tablet device, or a projector.

[0075] Figure 5(a) shows a first example of the map service screen provided by the AI ​​server 30. Screen 500 is a screen that includes various objects and areas, which will be explained later, and is displayed when the user selects objects 309, 339, and 369. Also, when objects 309, 339, and 369 are selected, screen 500 displays a map of the location information corresponding to the scene where object 404 was selected and flagged in the content displayed on each screen.

[0076] Object 501 is an object representing a map, and on screen 500, a map of Japan is displayed. While the map is shown abstractly in this application, in practice, it is preferable to use a more accurate shape to display a more accurate map. Specifically, when the map is generated, it is preferable that the AI ​​server 30 obtains map data from a database or service (not shown), and uses the obtained map data to generate the map on this screen. An abstract map is also acceptable.

[0077] In this embodiment, object 501 abstractly represents a map of Japan as a whole, but is not limited to this. For example, the location where object 503, described later, is displayed corresponds to Hokkaido. For example, if all of the objects 502, described later, displayed within object 501 are displayed on a map of Hokkaido, object 501 may be configured to display a map of Hokkaido locally.

[0078] Furthermore, in the example shown in screen 500, object 502 is displayed only on land, but object 502 may also be displayed on rivers, oceans, glaciers, icebergs, etc. For example, if a user flags a shooting scene at sea, such as on a ship, and a map (object 501) is generated, object 502 will be displayed at the location on the map where that scene is estimated to have been taken.

[0079] This can be achieved, for example, by having distinctive land features such as lighthouses, coastlines, or breakwaters visible in the background from the ship. The estimation method will be described later. Similarly, even if a scene was filmed underwater, if an underwater database is available and the AI ​​server 30 can identify the filming location when generating the map, object 502 will be displayed at the identified location on the map.

[0080] Object 502 is an object commonly referred to as a pin, which is displayed at a corresponding location on a map to visually convey location information to the user. In this embodiment, as shown in Figure 4(b), location information such as the shooting location in each scene corresponding to the flags (object 421) set by the user in advance is displayed on the map. Since eight flags have been set in Figure 4(b), eight corresponding objects 502 are displayed on screen 500 in Figure 5 (including object 503). Note that the number is not limited to eight; if the user sets a predetermined number of flags, that predetermined number of objects 502 will be displayed.

[0081] In other words, object 502 is displayed at the location of object 501 corresponding to all scenes that the user has flagged. As will be described later, if the estimation of the location of a flagged scene fails due to reasons such as the image recognition processing by the AI ​​server 30 failing, object 502 corresponding to the location that was successfully estimated will be displayed.

[0082] Furthermore, object 502 corresponds to area 504 (including area 505), which will be described later, and detailed location information for each of the objects 502 is displayed in area 504. By selecting a pin for which the user wants to see detailed location information, the selected object 502 is displayed in a way that makes it distinguishable from other pins, like object 503. In addition, location information corresponding to the selected pin (object 503) is displayed in a way that makes it distinguishable from other areas 504, like area 505.

[0083] Object 503 is one of the objects 502 (pins), and when any object 502 is selected by the user, it is displayed (highlighted) in a way that makes it distinguishable from other pins by changing its color and size. In this embodiment, an example is shown where the distinction is made by changing the color and size, but this is not the only way. For example, the distinction may be made by changing either the color or the size, or by adding a border or background around the object 502.

[0084] When the user then selects another pin (object 502), that pin will be highlighted as shown in object 503, and the pin that was originally selected and highlighted will be displayed in its state before highlighting.

[0085] Area 504 is an area that displays detailed location information for the locations corresponding to the positions of object 502 displayed on the map of object 501. In other words, area 504 is an area that displays location information corresponding to the scenes of content that have been flagged by the user. In the example of screen 500, object 501 displays a map of Japan, and information for each location corresponding to the position where object 502 is displayed is displayed in area 504. Area 504 may display, for example, addresses, place names, and facility names, but not all of these are required, and it may also be configured to display information such as postal codes, latitude, and longitude.

[0086] Region 504 is generated in the same number as object 502. In the example screen 500 shown in Figure 5(a), eight objects 502 (including object 503) are displayed, so eight regions 504 are also generated. However, due to the size of the operation unit 17, only four of these regions 504 are displayed in this example. The region 504 that is not currently displayed will become visible when the user, for example, selects a corresponding object 502 or scrolls the screen downwards.

[0087] Region 505 is one of the regions 504 and shows an example of displaying position information corresponding to the selected object 503. It is displayed (highlighted) in a way that makes it distinguishable from the other regions 504. In this embodiment, region 505 is displayed larger than the other regions and its border is thicker to make it distinguishable, but this is not the only way. For example, it may be distinguishable by changing either the size or the border width, or by changing the color of the border or background of region 505.

[0088] As a result, on screen 500, the information of the flag set by the user when selecting object 404, the position where object 503 is displayed, and the highlighted area 505 are displayed in correspondence.

[0089] When the user selects another pin, the area showing the location information corresponding to that pin will be highlighted as shown in area 505, while the area showing the location information corresponding to the originally selected pin will be displayed in its pre-highlighted state.

[0090] In other words, on screen 500, as the object 502 selected by the user changes, the highlighted object 502 (object 503) and the highlighted area 504 (area 505) change in conjunction.

[0091] Text 510 is a string that displays the title, indicating which content's filming location map is being displayed. In this embodiment, the example shows a title displayed according to the rule "content title string" + "of" + "filming location map".

[0092] Figure 5(b) shows a second example of the map service screen provided by the AI ​​server 30. Screen 550 is a screen that includes various objects and areas, which will be explained later, and is displayed when the user selects objects 309, 339, and 369. Additionally, when objects 309, 339, and 369 are selected, screen 500 displays a map of the location information corresponding to the scene where object 404 was selected and flagged in the content displayed on each screen.

[0093] Object 551 is an object that represents a map, and on screen 550, a world map is displayed. Its other characteristics are the same as those of object 501, so the explanation is omitted.

[0094] Object 552 is the same as object 502. Object 553 is the same as object 503.

[0095] Region 554 is similar to Region 504. In the example on screen 550, object 551 shows a world map, and information for each location corresponding to the location where object 552 is displayed is shown in Region 554. In other words, Region 554 is a region that shows location information corresponding to the content scene that has been flagged by the user. In addition to the example in Region 504, Region 554 may also display an image of the location (Region 556), the country name, a URL showing information such as a homepage related to the location, etc. However, not all of these are mandatory, and the configuration may also display information such as latitude and longitude.

[0096] Region 555, like region 505, is one of the regions 554 and shows an example of displaying location information corresponding to the selected object 553. It is displayed (highlighted) in a way that makes it distinguishable from the other regions 554.

[0097] Area 556 is the area where a photograph of the location corresponding to the position where object 552 is displayed is shown.

[0098] Area 557 is the area where the content scene is displayed when the user flags it in the service application. In other words, the example in Figure 5(b) is an example where the same scene as the scene displayed on screen 420 in Figure 4(b) is displayed. The scene corresponding to the location of area 558, which will be described later, is displayed.

[0099] Area 558 is the area where information such as the duration of video content and the completion status of game content is displayed. In the example in Figure 5(b), the content is video content, and the string "3:33 / 5:00" is displayed, indicating that a scene from 3 minutes and 33 seconds of a 5-minute video is being shown.

[0100] Object 559 is a slider, similar to object 421. In the example on screen 550, it is displayed corresponding to the currently highlighted object 552 (object 553) and the currently highlighted area 554 (area 555).

[0101] In other words, on screen 550, as the slider selected by the user changes, the highlighted object 552 (object 553), the highlighted area 554 (area 555), and the scene displayed in area 557 change in conjunction. Similarly, on screen 550, as the object 552 selected by the user changes, the highlighted object 552 (object 553), the highlighted area 554 (area 555), and the scene displayed in area 557 change in conjunction.

[0102] Text 560 is the same as Text 510.

[0103] Figure 6 is a sequence diagram showing an example of the processes performed by both the data server 20 and the client terminal 10 when the data server provides content services to the client terminal 10. This process begins when the user launches the service application on the client terminal 10. Specifically, the CPU 11 of the client terminal 10 detects the user's operation and launches the service application by loading the program stored in the storage 14 into the RAM 13, thereby initiating this process. The CPU 11 launches the service application and displays various screens necessary for the user to use the service on the operation unit 17.

[0104] It should be noted that, as long as the effects of the present invention can be achieved, it is not essential that all of the following processes be performed, and the order of the processes may be changed or additional processes may be added.

[0105] In S601, the CPU 11 of the client terminal 10 displays an authentication information input screen (not shown) on the operation unit 17, and the user enters the authentication information necessary to use the service application. At this time, the user enters the authentication information using the keyboard on the operation unit 17 or software keys displayed on the operation unit 17. As a result, the CPU 11 receives the authentication information necessary to use the service application and to log in to the account corresponding to the user via the operation unit 17.

[0106] Authentication information includes, for example, an ID (identification), email address, password, and passcode corresponding to the account. Alternatively, other functions available on the client terminal 10 may be used, such as biometric authentication methods including fingerprint recognition, facial recognition, vein recognition, and iris recognition.

[0107] In S602, CPU 11 transmits the authentication information received in S601 and the authentication request (an instruction to execute the authentication process) to data server 20 via communication I / F 15. CPU 21 of data server 20 receives this via communication I / F 25. At this time, because the transmitted information is highly confidential, it is preferable to transmit the authentication information using an encrypted communication protocol.

[0108] In S603, the CPU 21 of the data server 20 executes the authentication process using the received authentication information, in accordance with the authentication request received in S602. Specifically, for example, authentication information such as account IDs and passwords is pre-stored as a database in the storage 24, and in S603, the CPU 21 searches the database for the received authentication information and executes the authentication process.

[0109] At this time, the CPU 21 determines that authentication was successful if it finds that the received authentication information is stored in the database, and that authentication was unsuccessful if it finds that the received authentication information is not stored in the database. This sequence diagram illustrates an example of the process executed when authentication is successful. Furthermore, in subsequent communications, an access token corresponding to the user account issued as a result of successful authentication may be exchanged with each communication.

[0110] In S604, CPU21 begins providing services to client terminal 10. CPU21 may also send a message at this time indicating that the authentication process was successful.

[0111] Providing a service means providing the services offered by the data server 20. For example, in the case of a video content distribution service, this refers to processing such as displaying a screen on the client terminal 10 showing a list of content available for viewing by the user. At this time, the CPU 21 provides the service by transmitting not only the information on the screen, but also all information corresponding to the content, such as the content title, length, number of views, and number of ratings.

[0112] Furthermore, the provision of services is not one-way from the data server 20, but rather bi-directional between the data server 20 and the client terminal 10.

[0113] For example, CPU 21 sends a screen displaying a list of content to client terminal 10, and when the user selects the desired content from the displayed screen, client terminal 10 sends a request to send information about that content. Upon receiving this, CPU 21 of the data server sends information about the specified (selected) content, or a screen of the specified content as shown in Figure 3, to client terminal 10. In this way, CPU 21 provides services to client terminal 10.

[0114] In S605, the CPU 11 of the client terminal 10 displays various information and screens provided by the data server 20 in S604 on the operation unit 17. As a result, for example, the screen shown in Figure 3 is displayed on the operation unit 17, and the user can use the service.

[0115] In S606, CPU11 detects the selection of the play button. Specifically, CPU11 detects that object 333 or object 363 in Figure 3 has been selected and accepts user instructions.

[0116] In S607, CPU 11, having detected the selection of the play button in S606, requests data server 20 to send content information. Specifically, CPU 11 sends a request (an instruction to send content information) along with the content information to data server 20 via communication interface 15, and CPU 21 of data server 20 receives it via communication interface 25. At this time, CPU 11 sends information such as the content name and ID, which allows data server 20 to identify the content.

[0117] In S608, the CPU 21 of the data server 20, having received the request in S607, sends information about the content corresponding to the received request to the client terminal 10 via the communication interface 25. If the content is video, the video data is sent from the data server 20 to the client terminal 10 in S608.

[0118] The CPU 11 of the client terminal 10, which has received content information (such as video data) via the communication interface 15, displays, for example, the screen shown in Figure 4 on the operation unit 17.

[0119] Figure 7 is a sequence diagram showing an example of the process performed when a shooting location map is generated, involving the client terminal 10, data server 20, and AI server 30. This process is performed after the CPU 11 of the client terminal 10 displays content on the operation unit 17 in S608, as described in Figure 6.

[0120] It should be noted that, as long as the effects of the present invention can be achieved, it is not essential that all of the following processes be performed, and the order of the processes may be changed or additional processes may be added.

[0121] In S701, the CPU 11 of the client terminal 10 detects that a flag has been set. Specifically, at any point when the content displayed to the user is shown (played), the CPU 11 detects the object 404 displayed on the operation unit 17 of the client terminal 10 when the user selects this object.

[0122] In this case, for example, with video content, the user selects object 404 and sets a flag when a scene appears in the video content that they want to know the filming location of. A scene refers to a specific moment within the content.

[0123] When object 404 is selected, CPU 11 stores flag information in RAM 13 or storage 14, associating it with the content's timing (duration) and scene. Specifically, for example, if object 404 is selected by the user when a scene at 3 minutes and 33 seconds of a 5-minute video content is displayed, CPU 11 stores the flag information associating it with the timing (duration) information of 3 minutes and 33 seconds.

[0124] In S702, the CPU 11 detects that the "Visit" button, i.e., object 309, object 339, and object 369, has been selected. At this time, the CPU 11 retrieves the flag information stored in S701, such as the number of flags set and the information stored corresponding to each flag.

[0125] In S703, the CPU 11 sends the content information and the flag information obtained in S702 to the data server 20 via the communication interface 15. The data server 20 receives this information sent by the CPU 11.

[0126] In S704, the CPU 21 of the data server 20 acquires a scene image based on the content information and flag information received in S703. Specifically, the CPU 21 identifies the scene corresponding to the content ID (content information) received in S703 and the timing of its receipt (flag information), and acquires an image of that scene (a single frame of video data).

[0127] In S705, the CPU 21 sends the acquired scene image, a request to generate a shooting location map (an instruction to execute the map generation process), and information about the account (client terminal 10) that sent the information in S703 to the AI ​​server 30 via the communication interface 25. The AI ​​server 30 receives this information sent by the CPU 21.

[0128] In S706, the CPU 31 of the AI ​​server 30 performs image recognition on the scene image received in S705. This process is not mandatory. The image recognition performed by the CPU 31 in S706 involves, for example, extracting characteristic landscapes or structures, identifying which parts of the scene image are necessary for estimating location, and recognizing only the necessary areas. In addition, the CPU 31 may convert the scene image into a natural language string that represents the scene image using, for example, an LLM (Large Language Model).

[0129] At this time, for example, if the image includes a sign with a place name written on it, processing that text using OCR (Optical Character Reader) and searching for it greatly contributes to estimating the location where the image was taken. In S706, CPU31 performs what could be called pre-processing, which is necessary for the estimation process in the following S707.

[0130] In S707, CPU31 estimates the location where the scene image received in S705 was taken. Specifically, CPU31 uses the image and text preprocessed in S706 to obtain necessary information by searching various search engines (not shown) and various databases (not shown) to estimate the location where the scene image was taken. At this time, it obtains various information related to the estimated location, such as images, addresses, postal codes, place names / facility names, related websites, and telephone numbers.

[0131] In S708, CPU31 generates a map of the shooting location based on various information acquired in S707 that corresponds to the scene image. At this time, CPU31 acquires information necessary for map generation, such as map information for displaying object 501 and object 551, and generates a map of the shooting location as shown in Figure 5.

[0132] In S709, the CPU 31 sends the map information of the shooting location generated in S708 to the client terminal 10, which is logged in with the account corresponding to the account information received in S705, and the client terminal 10 receives it. As a result, the AI ​​server 30 provides the map service. Note that the provision of the service is not one-way from the AI ​​server 30, but may be two-way between the AI ​​server 30 and the client terminal 10.

[0133] In the case of the configuration shown in Figure 1(b), the system works as follows: in S709, the CPU 31 transmits the map information of the shooting location generated in S708 to the data server 20, and then the CPU 21 of the data server 20 transmits this information to the client terminal 10.

[0134] In S710, the CPU 11 of the client terminal 10 displays the map information of the shooting location received in S709 on the operation unit 17. Through this process, the user can utilize the map service. Note that the map service screen, as shown in Figure 5, may be displayed by a service application provided by the data server 20, or it may be displayed as a different service, such as a browser.

[0135] If possible, the CPU 11 of the client terminal 10 may acquire the scene image using S703. In this case, the content information and flag information do not need to be transmitted, and their transmission may be omitted. Furthermore, in this case, the subsequent processing of S704 may not be executed. Alternatively, the client terminal 10 and the AI ​​server 30 may communicate directly.

[0136] Furthermore, in this embodiment, it is assumed that the scene image acquired in S704 is a single image (one frame). However, in the case of video content, for example, the CPU 21 may be configured to acquire images before and after this image (additional images). In this case, the acquired additional images are sent to the AI ​​server 30 in S705 and used by the CPU 31 in S707 as reference information for estimating the shooting location. This is expected to improve the accuracy of location estimation.

[0137] Figure 8 is a flowchart illustrating an example of a process in which the data server 20 provides a service and requests the generation of a map of the shooting location. As explained in S602, this process is initiated when the data server 20 receives authentication information and an authentication request from the client terminal 10, and is an example of a process executed by the CPU 21 of the data server 20.

[0138] It should be noted that, as long as the effects of the present invention can be achieved, it is not essential that all of the following processes be performed, and the order of the processes may be changed or additional processes may be added.

[0139] In S801, CPU21, similar to S603, executes authentication processing using the received authentication information in response to the authentication request. Specifically, authentication information such as account ID and password is pre-stored as a database in storage24, and in S801, CPU21 searches the database for the received authentication information and executes authentication processing.

[0140] In S802, CPU21 determines whether the authentication process in S801 was successful. Specifically, CPU21 searches a database pre-stored in storage24. If the search results indicate that the received authentication information is stored in the database, it determines that authentication was successful and proceeds to S803. In subsequent communications, an access token corresponding to the user account issued upon successful authentication may be exchanged with each communication. On the other hand, if CPU21 determines that the received authentication information is not stored in the database, it determines that authentication was unsuccessful and proceeds to S809.

[0141] In S809, CPU21 generates a message and screen to notify the user that the authentication process has failed, and sends these to the client terminal via communication I / F25 to notify the user. Once this process is complete, CPU21 terminates the process shown in this flowchart.

[0142] In S803, CPU21, similar to S604, begins providing services to client terminal 10. CPU21 may also send a message at this time indicating that the authentication process was successful.

[0143] Providing a service means providing the services offered by the data server 20. For example, in the case of a video content distribution service, this refers to processing such as displaying a screen on the client terminal 10 showing a list of content available for viewing by the user. At this time, the CPU 21 provides the service by transmitting not only the information on the screen, but also all information corresponding to the content, such as the content title, length, number of views, and number of ratings.

[0144] Furthermore, the provision of services is not one-way from the data server 20, but rather bi-directional between the data server 20 and the client terminal 10.

[0145] For example, CPU 21 sends a screen displaying a list of content to client terminal 10, and when the user selects the desired content from the displayed screen, client terminal 10 sends a request to send information about that content. Upon receiving this, CPU 21 of the data server sends information about the specified (selected) content, or a screen of the specified content as shown in Figure 3, to client terminal 10. In this way, CPU 21 provides services to client terminal 10.

[0146] In S804, CPU21 receives a request for content information from client terminal 10, similar to S607. At this time, by receiving information such as the content name and ID, CPU21 can identify the content in the subsequent S805.

[0147] In S805, CPU21, similar to S608, receives a request in S804 and, in accordance with that, transmits information about the content corresponding to the received request to the client terminal 10 via the communication interface 25. For example, if the content is a video, CPU21 transmits the video data to the client terminal 10.

[0148] In S806, CPU21 receives content information and flag information from client terminal 10, similar to S703.

[0149] In S807, CPU21, similar to S704, acquires a scene image based on the content information and flag information received in S807. Specifically, CPU21 identifies the scene corresponding to the received content ID (content information) and the timing of the reception (flag information), and acquires an image of that scene (a single frame of video data).

[0150] S808, like S705, sends the scene image acquired in S807, a request to generate a shooting location map (an instruction to execute the map generation process), and information about the account (client terminal 10) that sent the information in S806 to the AI ​​server 30. Once this process is complete, CPU21 terminates this process.

[0151] Figure 9 is a flowchart illustrating an example of the process by which the CPU 31 of the AI ​​server 30 generates a map of the shooting location. As explained in S705, this process is initiated when the AI ​​server 30 receives the scene image, a request to create a map, and account information from the data server 20, and is an example of a process executed by the CPU 31 of the AI ​​server 30.

[0152] It should be noted that, as long as the effects of the present invention can be achieved, it is not essential that all of the following processes be performed, and the order of the processes may be changed or additional processes may be added.

[0153] In S901, CPU31 performs image recognition on the received scene image, similar to S706. This process is not mandatory. The image recognition performed by CPU31 in S901 involves, for example, extracting characteristic landscapes or structures, identifying which parts of the scene image are necessary for location estimation, and recognizing only the necessary areas. In addition, CPU31 may convert the scene image into a natural language string representing the scene image using LLM or similar methods.

[0154] At this time, for example, if the image includes a sign with a place name written on it, performing OCR processing on that text and searching for that string greatly contributes to improving the accuracy of estimating the location where the photo was taken. In S901, CPU31 performs what could be called the preliminary preparations (preprocessing) necessary for the estimation process in the following S902.

[0155] In S902, CPU31, similar to S707, estimates the location where the received scene image was taken. Specifically, CPU31 estimates the location where the scene image was taken by searching various search engines (not shown) and various databases (not shown) using the image and text preprocessed in S901. At this time, it obtains various information related to the estimated location, such as images, addresses, postal codes, place names / facility names, related websites, and telephone numbers.

[0156] In S903, CPU31, similar to S708, generates a map of the shooting location based on various information acquired in S902 that corresponds to the scene image. At this time, CPU31 acquires information necessary for map generation, such as map information for displaying object 501 and object 551, and generates a map of the shooting location as shown in Figure 5.

[0157] In S904, the CPU 31, similar to S709, sends the map information of the shooting location generated in S903 to the client terminal 10 based on the received account information, and the client terminal 10 receives it. As a result, the AI ​​server 30 provides the map service. When S904 is complete, the CPU 31 terminates this process.

[0158] The service may be provided not only unilaterally from the AI ​​server 30, but also bidirectionally between the AI ​​server 30 and the client terminal 10. Once this process is complete, the CPU 31 terminates the process. At this time, the CPU 31 may log in the storage 34 that the location of the content scene has been estimated and transmitted.

[0159] As described above, in this embodiment, when a user is viewing (watching) content, if they want to know the filming location of a particular scene in that content, they can flag it and select the button they want to visit, thereby viewing information about the filming location of that scene in map format. At this time, the user selects object 404 and flags it, and then selects objects 309, 339, and 369, and a map of the filming locations is generated and displayed without the user having to search or operate anything further.

[0160] Furthermore, as shown in Figures 4 and 5, after a single user operation (selection of objects 309, 339, and 369), a screen like the one shown in Figure 5 will be displayed without the user having to perform any further operations. Also, if the user flags eight locations as in this embodiment, the geographical information for all eight shooting locations will be displayed with this single operation.

[0161] Furthermore, if the S902 cannot estimate the shooting location, the shooting location for that scene does not need to be displayed on the map. In that case, the system may display an error message or error message indicating that the shooting location could not be estimated.

[0162] With conventional technology, users had to take screenshots of scenes from content, search for the locations using search engines, and then research information about those locations themselves. However, this invention, which utilizes AI, significantly reduces that effort. As a result, users who are fans of the content can more easily visit the various filming locations, thus enhancing the user experience.

[0163] Furthermore, by using AI and an accumulated database, the present invention can estimate the filming location not only for television programs, dramas, and movies filmed in the real world, but also for content such as games, animations, and comics. In this embodiment, the filming location refers to the location that served as the theme for depicting animations, CG, etc.

[0164] Furthermore, if the data server 20 manages or publishes information about the filming locations of each scene in the content, and therefore the AI ​​server 30 does not need to estimate the filming locations, the above-mentioned map of filming locations may be generated without this estimation.

[0165] Furthermore, while this embodiment shows an example where geographical information of the shooting locations corresponding to scenes flagged by the user is displayed on the map, it is also possible to configure the system so that geographical information of all scenes or major scenes is displayed on the map without user selection. In this case, by sending the entire content data to the generation AI, the generation AI can acquire information on the shooting locations corresponding to these scenes by, for example, selecting scenes at equal intervals or selecting scenes with a high number of plays / views. Moreover, it is also possible to configure the system so that the shooting locations of scenes not selected by the user are displayed on the map together with the shooting locations of scenes selected by the user.

[0166] In this system, for example, servers such as data server 20 and AI server 30 may be located outside of Japan, while client terminals 10 (hereinafter referred to as "terminal devices"), which are terminal devices, may be located within Japan. Even in this case, files and data can be sent from each server to the terminal devices, and the terminal devices can receive those files and data.

[0167] Even if each server is located outside of Japan, the transmission and reception (sending and receiving) of files and data within this system are performed as a single integrated process. Furthermore, since the system functions by receiving these files and data on terminal devices located within Japan, it can be considered that such transmission and reception took place domestically.

[0168] In this system, even if, for example, the server is located outside of Japan and the terminal device is located within Japan, the terminal device can perform the main functions of the system, and the effects of those functions can be realized within Japan. For example, even if the server is located outside of Japan, if the terminal device constituting this system is located within Japan, it is possible to use the system within Japan using that terminal device. Furthermore, the use of such a system may affect the economic interests of, for example, the patent holder.

[0169] <Second Example> The following describes a second embodiment of the present invention. The second embodiment is an example in which, while content is being displayed (played) on the client terminal 10, information about the shooting location of the scene of the displayed content is displayed on the operation unit 17.

[0170] In this embodiment, there is no concept of setting a flag (object 404); instead, each time the object 441 corresponding to the button to visit is selected, the processes equivalent to S702 and S703 are executed. As a result, the client terminal 10 receives and displays information about the shooting location of the scene that was displayed when the button to visit was selected. This description of the embodiment will focus particularly on the differences from the previously described embodiment.

[0171] In the screen 440 shown in Figure 4(c), object 441 corresponds to objects 309, 339, and 369. Information 442 indicates the shooting location of the scene when object 441 is selected. In reality, a delay in processing is expected, but assuming that processing does not take time, the shooting location information of the scene displayed on the operation unit 17 will be displayed when the user selects object 441. Objects 401 to 403 are the same as those described above.

[0172] When object 441 is selected, the CPU 11 of the client terminal detects, in response to S702, that the Visit button has been selected. Next, in response to S703, the CPU 11 sends the content information and the scene information when object 441 was selected to the data server 20. Next, similar to S704, the CPU 21 of the data server 20 retrieves the scene image. Next, in response to S705, the CPU 21 sends the scene image and account information to the AI ​​server 30.

[0173] Next, similar to S706 and S707, the CPU 31 of the AI ​​server 30 performs image recognition processing and estimation of the scene's shooting location. Then, corresponding to S709, the CPU 31 sends the scene's shooting location information to the corresponding account (client terminal). Finally, the CPU 11 of the client terminal 10 displays the received scene's shooting location information as shown in information 442.

[0174] On the other hand, in the example of screen 460 shown in Figure 4(d), the information 442 is the same as in the explanation in Figure 4(c). Unlike the example in Figure 4(c), the example in Figure 4(d) shows a case where the shooting location information of the scene displayed on the operation unit 17 is always displayed, even if the user does not select an object 441.

[0175] In this case, even if object 441 is not selected, the CPU 11 of the client terminal 10 responds to S703 and sends the content information and the scene information displayed on the operation unit 17 to the data server 20. From there, the process is executed as described in Figure 4(c), and while the content is being displayed (played), the information of the shooting location of the displayed scene is always displayed on the operation unit 17.

[0176] By executing the processing in this embodiment, users can view content while simultaneously learning about the filming location of the scenes displayed with that content.

[0177] <Third Embodiment> The following describes a third embodiment of the present invention. In the third embodiment, when a map of the shooting location is generated, the user is shown the estimation result according to the number of times the CPU 31 of the AI ​​server 30 has generated it up to that point, thereby improving the accuracy of the estimation of the shooting location. In describing this embodiment, we will focus particularly on the differences from the previously described embodiment.

[0178] Figure 10 shows an example of a screen generated by the AI ​​server 30 to confirm the estimated shooting location to the user. Screen 1000 shown in Figure 5(a) and screen 1050 shown in Figure 5(b) are examples of screens displayed on the operation unit 17 of the client terminal 10.

[0179] For example, the content displayed on the map service screen may differ depending on the size of the control panel 17 of the client terminal 10 displaying the screen, so several examples are shown. Note that the content displayed on the map service screen may be the same regardless of which client terminal 10 it is displayed on.

[0180] Here, the screen 1000 shown in Figure 10(a) is an example of a map service screen displayed on the operation unit 17 of a client terminal 10, such as a smartphone or a small tablet device. On the other hand, the screen 1050 shown in Figure 10(b) is an example of a map service screen displayed on the operation unit 17 of a client terminal 10, such as a PC, a large tablet device, or a projector.

[0181] Objects 402, region 558, object 559, and region 577 in screens 1000 and 1050 are the same as those described above, so their explanation is omitted.

[0182] Figure 10(a) is an example of a map service screen provided by the AI ​​server 30, and shows a first example of a screen for confirming the estimated shooting location. Screen 1000 is a screen that includes various objects and areas, which will be explained later. When screen 1000 is displayed on the operation unit 17 of the client terminal 10, and the user responds, the CPU 31 of the AI ​​server 30 receives the response, the map of shooting locations is updated, and the information is used as reference information for estimating shooting locations in the future.

[0183] Text 1001 is a string that indicates the title of the content corresponding to the generated map of the shooting location.

[0184] Text 1002 is a question to the user, a string that asks (confirms with the user) whether the estimated location is correct. In the example shown in this embodiment, text 1002 displays information (name) of the estimated shooting location. Note that text 1002 may also display other shooting location information such as an address in addition to the name.

[0185] Area 1003 is an area where information such as images related to the estimated location is displayed. For example, it displays images used to generate the shooting map, which are stored in a database such as a web server and show images of the estimated location.

[0186] Object 1004 is a button for the user to answer the question in text 1002. In the example in screen 1000, a string indicating affirmation (that the estimated shooting location is correct) is displayed on object 1004. However, the content displayed on object 1004 is not limited to affirmative statements, nor is it limited to strings. For example, symbols such as ○, ×, or checkmarks may be displayed on object 1004.

[0187] In this embodiment, the user checks the scene and text 1002 of the content displayed in area 577 and the content displayed in area 1003, determines whether the shooting location is correct, and selects object 1004 if it is correct. Once object 1004 is selected by the user, a response is sent from the client terminal 10 to the AI ​​server 30, as described in S1005 below.

[0188] Object 1005 is a button that allows the user to suggest alternative locations to those indicated by, for example, text 1002 or area 1003. The example in screen 1000 shows an instance where location information (such as place name and address) and an image of that location are displayed in object 1005. Note that it is not mandatory for object 1005 to display all of this information, and it may also display other types of information.

[0189] In this embodiment, the user checks the scene and text 1002 of the content displayed in area 577 and the content displayed in area 1003, determines whether the shooting location is correct, and selects object 1005 if it is incorrect. In particular, if the user determines that it is incorrect and then determines that the candidate shown in object 1005 is correct, the user selects the corresponding object 1005. Once object 1005 is selected by the user, the response is sent from the client terminal 10 to the AI ​​server 30, as described in S1005 below.

[0190] In this embodiment, the shooting location indicated by text 1002 and area 1003 is different from the shooting location indicated by object 1005; that is, object 1005 is an object for suggesting other shooting locations to the user. However, the candidate shooting locations displayed by object 1005 may be the same as the shooting locations indicated by text 1002 and area 1003, for example, by displaying a location where the first side of a building is visible and a location where a second side different from the first side is visible.

[0191] Object 1006 is a button that allows the user to suggest shooting locations other than those indicated by text 1002 and area 1003, and those indicated by object 1005, as shooting locations for the scene displayed in area 577.

[0192] In this embodiment, the user checks the scene and text 1002 of the content displayed in area 577 and the content displayed in area 1003 to determine whether the shooting location is correct. If it is determined to be incorrect, the user selects object 1006. In particular, if the user determines that it is incorrect and also determines that there is no suitable shooting location among the candidates shown in object 1005, the user selects object 1005.

[0193] When object 1006 is selected, an input field for entering location information is displayed, and the user can enter the correct shooting location information via the control unit 17. Once the user enters the shooting location information, the contents are sent as a response from the client terminal 10 to the AI ​​server 30, as described in S1005 below.

[0194] Furthermore, the system may be configured such that when object 1006 is selected by the user, another candidate location (for example, object 1005) is displayed, or the user is able to upload an image. In S1005, in the former case, it is the same as when object 1005 is selected, and in the latter case, the uploaded image is sent and used in S1007 and S1008.

[0195] Figure 10(b) is an example of a map service screen provided by the AI ​​server 30, and shows a second example of a screen for confirming the estimated shooting location. Screen 1050 is a screen that includes various objects and areas, which will be explained later. When screen 1050 is displayed on the operation unit 17 of the client terminal 10, and the user responds, the CPU 31 of the AI ​​server 30 receives the response, the map of shooting locations is updated, and the information is used as reference information for estimating shooting locations in the future.

[0196] Text 1051 is the same as Text 1001. Text 1052 is the same as Text 1002.

[0197] Area 1053 is the same as area 1003. The example of screen 1050 shown in Figure 10(b) shows an example where, in addition to the example of images displayed on screen 1000, information such as the content of literature and articles, and URLs indicating the location where they are displayed (managed) is also displayed. Furthermore, the system may be configured so that detailed information is displayed when the user selects an image or URL.

[0198] Object 1054 is the same as object 1004. Object 1055 is the same as object 1005. Object 1056 is the same as object 1006.

[0199] Text 1057 is an example of a message used to inform the user that the shooting location indicated in text 1052 has been estimated based on the content displayed in area 1053. Note that displaying text 1057 is not mandatory.

[0200] Text 1058 is an example of a message that guides the user to select object 1055 or object 1056 to suggest alternative shooting locations when the estimated shooting location does not match the content's shooting location. Note that displaying text 1058 is not mandatory.

[0201] Figure 11 is a flowchart illustrating an example of a process that improves the accuracy of the estimated shooting location by the CPU 31 of the AI ​​server 30. Similar to Figure 9, this process is initiated when the AI ​​server 30 receives a scene image, a map creation request, and account information from the data server 20, as explained in S705, and is an example of a process executed by the CPU 31 of the AI ​​server 30. The processes S901 to S904 in the first half of the flowchart are the same as those explained in Figure 9, so their explanation is omitted.

[0202] It should be noted that, as long as the effects of the present invention can be achieved, it is not essential that all of the following processes be performed, and the order of the processes may be changed or additional processes may be added.

[0203] In S1101, the CPU 31 estimates the location of the content in that scene and determines whether the number of times it has been sent exceeds a certain threshold. In this embodiment, the threshold is set to 10 times, but it is not limited to this. Specifically, the CPU 31 searches the logs recorded in the storage 34, and if the search results indicate that the number of times it has been sent exceeds the threshold, this process is terminated. If the search results indicate that the number of times it has been sent is less than the threshold, the process proceeds to S1102.

[0204] In S1102, the CPU 31 acquires other candidate shooting locations. Specifically, for example, the CPU 31 acquires information on shooting locations that were not selected as final candidates from among the multiple shooting locations found when the shooting location estimation was performed in S902. In this embodiment, the CPU 31 records this information (search results) in the storage 34 during the shooting location estimation process in S902. Alternatively, the system may perform another location acquisition (search) based on the received scene image.

[0205] In S1103, the CPU 31 generates a confirmation screen, for example, as shown in Figure 10. Specifically, the CPU 31 generates text 1002 and text 1052 based on the shooting location estimated in S902, and generates the display content for area 1003 and area 1053 based on images and documents obtained through the search. The CPU 31 also generates object 1005 and object 1055 based on information about other candidate shooting locations acquired in S1102. In addition, it generates area 577 based on the image of the received content.

[0206] Furthermore, when the AI ​​server 30 obtains information such as flags entered into the client terminal 10, the CPU 31 can generate object 402, region 558, and object 559 on screen 1050 based on this information.

[0207] In S1104, CPU31 sends the confirmation screen information generated in S1103 to client terminal10 based on the received account information.

[0208] As a result, screens 1000 and 1050 shown in Figure 10 are displayed on the operation panel 17 of the client terminal 10. When these confirmation screens are displayed, the user selects one of the following: object 1004, object 1005, object 1006, object 1054, object 1055, or object 1056.

[0209] In S1105, CPU31 receives a response from client terminal 10 (corresponding account). If the user selects object 1004 or object 1054, CPU31 receives information such as a command indicating that the shooting location estimated in S902 is correct.

[0210] Furthermore, if the user selects object 1005 or object 1055, CPU 31 receives the ID corresponding to each object, as well as location information such as place name and address. The corresponding ID is the ID defined by CPU 31 in S1104 for object 1005 or object 1055, and is an ID that can identify location information.

[0211] Furthermore, if the user selects object 1006 or object 1056, the CPU 31 receives the information entered by the user as a result of selecting these objects.

[0212] In S1106, CPU31 determines whether the shooting location estimated in S902 was correct, based on the user's response received in S1105. Specifically, CPU31 determines that the shooting location estimated in S902 was correct if the information received in S1105 is information indicating that the shooting location is correct, such as a command. If CPU31 determines that it was correct, this process ends. On the other hand, if the information received in S1105 is not the above information such as an ID, shooting location information, or information entered by the user, CPU31 determines that the shooting location estimated in S902 was incorrect. If CPU31 determines that it was incorrect, the process proceeds to S1107.

[0213] In S1107, CPU 31 retrieves and stores information about candidate locations based on the user's response received in S1105. Specifically, CPU 31 analyzes the response received in S1105 and retrieves information about the shooting location (including information corresponding to the ID). Alternatively, if information entered by the user is received in S1105, it performs a search based on this information and retrieves information about the shooting location. CPU 31 also stores this retrieved shooting location information in storage 34.

[0214] At this time, the CPU 31 stores information about the shooting location, associating it with the image and flag information of the scene. This allows for improved accuracy in estimating the shooting location for the same scene in the future by using this stored information.

[0215] In S1108, CPU31 generates and modifies a map of shooting locations based on the candidate location information (shooting location information) obtained in S1107, similar to S903.

[0216] Note that the processing from S1101 onwards is performed in accordance with each scene. That is, as described in Example 1, when the user has set eight flags, eight shooting locations are displayed on the map. In other words, in S1101, the CPU 31 checks the number of transmissions for each of these eight scenes. Therefore, if information is received in S1105 indicating that only a few of the eight locations do not match the shooting locations, in S1108 only the shooting locations of those few locations need to be corrected and the map generated.

[0217] In S1109, CPU31 sends the map information of the shooting locations corrected in S1108 to the client terminal 10 based on the received account information. At this time, only the information about the shooting locations corrected in S1108 may be sent, or information about shooting locations that do not require correction (where the scene and shooting location match) may also be sent. Once S1109 is completed, CPU31 terminates this process.

[0218] When executing the processing in this embodiment, if the information from S1105 and S1107 is stored in storage 34, the CPU 31 considers this information in S902 to estimate the location. This improves the accuracy of the map of shooting locations generated by the CPU 31 of the AI ​​server 30.

[0219] Note that the order of each step in the process shown in the flowchart of Figure 11 can be changed, or the process can be configured to perform only the necessary steps. For example, by performing a threshold determination process like S1101 at the beginning of the process, it is possible to determine whether the scene shown in the received image has been estimated in the past. If the scene has been estimated in the past, the shooting location can be identified without having to perform the process of estimating the shooting location again. Furthermore, if the scene is stored in association with information that object 1004 or object 1054 was selected by the user, it will contribute even more to improving the accuracy of the map.

[0220] <Fourth Embodiment> Notification System The following describes a fourth embodiment of the present invention. The fourth embodiment is an example in which, if it is undesirable for location information to be made public by the generated map of shooting locations, the user can report this fact. In describing this embodiment, we will focus particularly on the differences from the previously described embodiments.

[0221] Figure 12 shows an example of a screen generated by the AI ​​server 30 for a user to report a shooting location. This reporting screen 1200 is for users who decide that it is undesirable for the shooting location to be made public, and who wish to report that some or all of the information about the reported location be prevented from being posted on the map of shooting locations.

[0222] The notification screen 1200 is a screen that can be accessed from the map service screen in Figure 10, and is described as a screen that is displayed on the operation unit 17 of the client terminal 10 when an unillustrated menu or other item included in area 504 or area 554 is selected. Screen 1000 is a screen that includes various objects and areas, which will be described next.

[0223] Text 1201 is an example of a message indicating that screen 1200 is a reporting screen, and may also be the title of the screen.

[0224] Area 1202 is an area that displays information about the location where the user intends to report the incident. In this embodiment, an example is shown where an image indicating the location is displayed.

[0225] Text 1203 indicates that the location displayed in area 1202 has been estimated as the shooting location and provides information about the shooting location. In the example shown in this embodiment, text 1002 displays information (name) about the estimated shooting location. Note that text 1002 may also display other information about the shooting location, such as an address, in addition to the name.

[0226] Text 1204 is a question posed to the user, and is an example of a message asking which of the publicly available location information the user wants to report.

[0227] Object 1205 is a commonly known check button, and it is displayed according to the type of information being made public. Users can select the object 1205 that corresponds to the information of the location they wish to report. In the example shown in Figure 12, the screen shows the state where the user has decided that the phone number of this filming location is not suitable for public disclosure and has selected the object 1205 corresponding to the phone number.

[0228] Area 1206 is an input field for users to enter the reason for reporting. Area 1207 is an input field for users to enter their contact information so that the service administrator can contact the user who made the report.

[0229] Note that the display of areas 1206 and 1207 is not mandatory. For example, if the service administrator can contact the user based on account information, area 1207 does not need to be displayed.

[0230] Object 1208 is a button for the user to submit a report. When object 1208 is selected, the information of the selected object 1205, the information of the entered region 1206, and the information of the entered region 1207 are sent to the AI ​​server 30. This allows the user to submit a report.

[0231] When a report is made, the CPU 31 of the AI ​​server 30 stores the fact that a report has been made and its contents in the storage 34, associating them with the location where the image was taken. The service administrator can also take measures such as suspending the disclosure of information if necessary, based on the content of the report.

[0232] Figure 13 is a flowchart illustrating an example of the process for generating a map of the shooting location when there is a reporting history. Similar to Figure 9, this process is initiated when the AI ​​server 30 receives the scene image, the map creation request, and the account information from the data server 20, as explained in S705, and is an example of a process executed by the CPU 31 of the AI ​​server 30. The processes S901 to S904 in the first half of the flowchart are the same as the process explained in Figure 9, so their explanation is omitted.

[0233] Furthermore, this process assumes that if object 1208 has been selected and reported in the past, information indicating that it has been reported is stored in correspondence with the location information of the shooting. In addition, it is not necessary for all of the following processes to be executed as long as the effects of the present invention can be achieved, and the order of the processes may be changed or additional processes may be added.

[0234] In S1301, the CPU 31 searches the reporting history for the location estimated in S902. Specifically, the CPU 31 searches the storage 34 and checks whether the reporting history is stored corresponding to the location estimated in S904, and whether there is a record of the service administrator taking measures such as suspending the effect. In this embodiment, an example is described in which the disclosure of information is suspended based on the report when one is reported, regardless of the judgment of the service administrator.

[0235] In S1302, CPU 31 determines whether or not a report history is stored in the location estimated in S904 in S1301. If CPU 31 determines that a report history is stored, it proceeds to S1303; otherwise, it proceeds to S903.

[0236] In S1303, CPU31 executes a process to exclude locations that have been previously reported from the map generation process in S903, based on the content of the report history searched in S1301. If the reported content is part of the publicly available information on the filming location, CPU31 executes a process to exclude that part of the information from the map generation process in S903.

[0237] As a result of the above process, information about filming locations that were deemed inappropriate to disclose based on past user reports will no longer be made public. This applies to all users of this service.

[0238] In the explanation of Figure 13, an example was shown in which a process is performed to omit reported locations or information before the map is generated. However, it is also possible to generate the map with respect to reported locations or information and hide the relevant items to prevent the information from being made public.

[0239] This means that if a user does not want location information to be made public through the generated map of filming locations, they can report it, and the publicly available location information will be removed.

[0240] <Other examples> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

Claims

1. An information processing device that provides content to an external device, A receiving means for receiving information corresponding to a scene of the aforementioned content selected by the user, An acquisition means for acquiring geographical information corresponding to the selected scene indicated by the information corresponding to the received scene, The acquired information includes a transmission means for transmitting geographical information corresponding to the selected scene to the external device. An information processing device characterized by having the following features.

2. An information processing device that provides content, A receiving means that receives information from a first external device that corresponds to a scene of the aforementioned content selected by the user, The system includes a transmission means for transmitting information indicated by the selected scene, which is indicated by the information corresponding to the received scene, to a second external device. The second external device acquires geographical information corresponding to the scene indicated by the information indicated by the scene selected by the user in the transmitted information, and the acquired geographical information is displayed on the first external device. An information processing device characterized by the following:

3. A means of displaying content, A means for receiving the user's selection of a scene from the displayed content, A transmission means for transmitting information corresponding to the selected scene to an external device, Information obtained by the transmission means transmitting information corresponding to the scene, and a receiving means that receives geographical information corresponding to the selected scene, The display means for displaying the received geographical information and An information processing device characterized by having the following features.

4. The information processing apparatus according to any one of claims 1 to 3, characterized in that the geographical indication is the address of the selected scene.

5. The information processing device according to any one of claims 1 to 3, characterized in that the geographical indication is the place name of the selected scene.

6. The information processing apparatus according to any one of claims 1 to 3, characterized in that the aforementioned content is a video.

7. The information processing device according to claim 6, characterized in that the information corresponding to the selected scene is the elapsed time of the content.

8. The information processing apparatus according to any one of claims 1 to 3, characterized in that the information corresponding to the selected scene is image data showing the scene.

9. The information processing apparatus according to any one of claims 1 to 3, characterized in that the external device is an information processing apparatus separate from the information processing apparatus.

10. The information processing apparatus according to claim 2, characterized in that the second external device is an information processing apparatus having a generation AI function.

11. The information processing device according to any one of claims 1 to 3, characterized in that the aforementioned geographical information is displayed on a map.

12. The information processing device according to any one of claims 1 to 3, characterized in that the geographical information is displayed when the user selects an object, even if no other operations are performed after the object is selected.

13. The information processing apparatus according to any one of claims 1 to 3, characterized in that a predetermined number of geographical information is displayed when the user selects a predetermined number of objects.

14. The information processing apparatus according to claim 13, characterized in that the object is selected by the user while the content is displayed.

15. An information processing system comprising a first information processing device, a second information processing device, and a third information processing device, The first information processing device is A means for providing content to the second information processing device, Receiving means for receiving information from the second information processing device that corresponds to a scene of the content selected by the user, The system includes a transmission means for transmitting information indicated by the selected scene, which is indicated by the information corresponding to the received scene, to the third information processing device. The aforementioned second information processing device is A display means for displaying the content provided by the first information processing device, A means for receiving the user's selection of a scene from the displayed content, A transmission means for transmitting information corresponding to the selected scene to the first information processing device, A receiving means that receives geographical information acquired by the third information processing device as a result of the transmitting means transmitting information corresponding to the scene, The system includes the display means for displaying the received geographical information, The previous third information processing device is Receiving means for receiving information representing the selected scene from the first information processing device, An acquisition means for acquiring geographical information corresponding to the selected scene from the received information, The system includes a transmission means for transmitting the acquired geographical information to the second information processing device. An information processing system characterized by the following:

Images