PROGRAM, INFORMATION PROCESSING METHOD, INFORMATION PROCESSING APPARATUS, AND SYSTEM

The program allows real-world races to be displayed as virtual races with multiple viewpoints, improving user satisfaction by dynamically changing perspectives based on user input.

JP7581132B2Active Publication Date: 2024-11-12COLOPL
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2021100031
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-06-16
Publication Date
2024-11-12
Estimated Expiration
2041-03-10

AI Technical Summary

Technical Problem

Existing technologies do not allow users to watch real-world races in a simulated manner from multiple viewpoints, limiting the user's satisfaction during virtual race experiences.

Method used

A program that generates virtual objects representing racetracks and contestants based on real-world race information, allowing superimposition on a user's surroundings and enabling viewpoint changes based on user input, thereby displaying a virtual race from multiple perspectives.

Benefits of technology

Enables the display of real-world races as virtual races from various viewpoints, enhancing user satisfaction and realism through dynamic viewpoint adjustments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007581132000001
    Figure 0007581132000001
  • Figure 0007581132000002
    Figure 0007581132000002
  • Figure 0007581132000003
    Figure 0007581132000003
Patent Text Reader

Abstract

To display a real-world race as a virtual race by using virtual objects linked to the real-world race, and to display the virtual race from a plurality of viewpoints. [Solution] The program causes the processor to execute the steps of receiving first information about a real-world race, generating virtual objects including a racecourse object 501 and a moving object 502 based on the first information, and superimposing and displaying the virtual objects on a real image. The first information includes position information of participants or moving objects during the race and time information corresponding to the position information, and the displaying step includes moving the moving object 502 based on the first information to display a virtual race corresponding to the real-world race. The program further causes the processor to execute the step of changing the viewpoint of the virtual race based on a user's operational input.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing method, an information processing device, and a system. [Background technology]

[0002] Patent Documents 1 and 2 disclose technologies related to AR (Augmented Reality). Specifically, Patent Document 1 discloses replacing a landmark captured from a user's current location with a virtual object corresponding to the landmark and displaying the replaced landmark, and changing the viewpoint from which the virtual object is displayed in response to a user's operation. Patent Document 2 also discloses an approach to synthesizing multiple streamed videos so that streaming content viewable in AR or VR can be viewed from an angle selected by the user. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “ShootAR: Proposal of an operation style for mobile AR considering user posture”, [online], [Retrieved February 8, 2021], Internet<https: / / rekimotolab.files.wordpress.com / 2011 / 03 / interaction2011_shimada.pdf> [Non-patent document 2] “Apple files patent application for AR / VR video recording with multi-angle playback”, [online], [Retrieved February 8, 2021], Internet <https: / / thebridge.jp / 2020 / 02 / apple-seeks-patent-for-ar-vr-video-recording-with-multi-angle-playback> Summary of the Invention [Problem to be solved by the invention]

[0004] In the real world, various races such as boat races and horse races are held. Some people actually go to race tracks to watch these races, but sometimes it is impossible to actually go to a race track due to time or geographical constraints.

[0005] In light of the above situation, if a real-world race could be displayed as a virtual race using virtual objects linked to the real-world race, it would be beneficial to be able to virtually watch the race without actually going to the race track. Furthermore, if the virtual race could be viewed from multiple viewpoints, user satisfaction would increase. However, the technologies disclosed in Patent Documents 1 and 2 did not originally enable virtual viewing of an actual race.

[0006] One aspect of the present disclosure aims to display a real-world race as a virtual race using virtual objects linked to the real-world race, and to enable the virtual race to be displayed from multiple viewpoints. [Means for solving the problem]

[0007] According to one embodiment shown in the present disclosure, A program executed on a first computer including a processor and an imaging unit, The program causes the processor to: receiving first information from a second computer regarding a predetermined race in the real world; generating, based on the first information, virtual objects including a racetrack object representing a racetrack and moving objects representing participants or moving objects in the predetermined race; and displaying the virtual object superimposed on a real image of the surroundings of the first computer captured by the imaging unit, the first information includes location information of the contestants or the moving objects during the execution of the predetermined race and time information corresponding to the location information; the displaying step includes moving the moving object on the racetrack object based on the first information and displaying a virtual race corresponding to the predetermined race; The program further causes the processor to changing a viewpoint of the displayed virtual race based on an operation input by a user of the first computer; Programs are offered. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, a real-world race can be displayed as a virtual race using virtual objects linked to the real-world race, and the virtual race can be displayed from multiple viewpoints. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a system configuration according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of a user terminal according to an embodiment. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of a server according to an embodiment. [Figure 4] FIG. 1 is a schematic diagram illustrating an example of a real-world racetrack in accordance with one embodiment. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a virtual object displayed on a user terminal according to an embodiment. [Figure 6] 10 is a flowchart illustrating an example of an operation process according to an embodiment. [Figure 7] FIG. 2 is a schematic diagram showing an example of a real image captured by an imaging section according to an embodiment. [Figure 8] FIG. 10 is a schematic diagram showing an example of a screen displaying a virtual object superimposed on a real image according to an embodiment; [Figure 9]FIG. 10 is a schematic diagram showing an example of a screen displaying a virtual object superimposed on a real image according to an embodiment; [Figure 10] FIG. 10 is a schematic diagram showing an example of a display screen displaying a virtual race according to an embodiment. [Figure 11] FIG. 10 is a schematic diagram showing an example of a display screen displaying a virtual race according to an embodiment. [Figure 12] FIG. 10 is a schematic diagram for illustrating a change of viewpoint in a virtual race according to an embodiment. [Figure 13] FIG. 10 is a schematic diagram for illustrating a change of viewpoint in a virtual race according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same elements will be given the same reference numerals, and duplicate explanations will be omitted as appropriate. In one or more embodiments shown in this disclosure, elements included in each embodiment can be combined with each other, and the combined result also forms part of the embodiment shown in this disclosure.

[0011] (System configuration) Fig. 1 is a diagram showing the configuration of system 1 according to the present embodiment. System 1 is capable of displaying, for example, a predetermined race held in the real world as a virtual race using virtual objects on an information processing device used by a user. In this specification, the term "predetermined race" is not particularly limited as long as it is a race held in the real world, and examples include boat races (real races and exhibition races), horse racing, bicycle racing, auto racing, car races such as F1, drone racing, dog racing, marathons, and relay races.

[0012] As shown in FIG. 1, the system 1 includes a plurality of user terminals 10, such as user terminal 10A, user terminal 10B, and user terminal 10C, which are information processing devices (first computers) used by each user (hereinafter, user terminals such as user terminals 10A, 10B, and 10C are collectively referred to as "user terminals 10"), a first server device (second computer) 20, a second server device 40, and a network 30.

[0013] The user terminal 10A and the user terminal 10B connect to the network 30 by communicating with a wireless base station 31. The user terminal 10C connects to the network 30 by communicating with a wireless router 32 installed in a facility such as a house. The user terminal 10 is, for example, a portable terminal equipped with a touch screen, and may be a smartphone, a phablet, a tablet, or the like.

[0014] The user terminal 10 executes, for example, a program installed via a platform that distributes apps or the like, or a program including pre-installed software for browsing websites, etc. By executing the program, the user terminal 10 communicates with the first server device 20 and transmits and receives data related to a specific race, data related to the user, etc. to and from the first server device 20, thereby enabling the user terminal 10 to display a virtual race.

[0015] The first server device 20 receives data related to a predetermined race from the second server device 40. The first server device 20 transmits the data related to the predetermined race to the user terminal 10 as appropriate. The first server device 20 stores and manages the data related to the predetermined race and data related to each user.

[0016] The first server device 20 includes, as its hardware configuration, a communication IF (Interface) 22, an input / output IF 23, a memory 25, a storage 26, and a processor (second processor) 29, which are connected to each other via a communication bus.

[0017] The communication IF 22 is compatible with various communication standards such as LAN (Local Area Network) standards, and functions as an interface for transmitting and receiving data to and from the user terminal 10, the second server device 40, and the like.

[0018] The input / output IF23 functions as an interface for accepting input of information to the first server device 20 and outputting information to the outside of the first server device 20. The input / output IF23 may include an input accepting unit that accepts connection of an information input device such as a mouse or a keyboard, and an output unit that accepts connection of an information output device such as a display for displaying images, etc.

[0019] The memory 25 is a storage device for storing data used in processing. For example, the memory 25 provides the processor 29 with a working area for temporary use when the processor 29 performs processing. The memory 25 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and a It is composed of storage devices such as a USB flash drive (USB 2.0), a USB flash drive (USB 3 ...

[0020] The storage 26 is a storage device for storing various programs and data to be read and executed by the processor 29. Information stored in the storage 26 includes data related to a predetermined race, data related to each user, and the like. The storage 26 may be configured to include a storage device such as an HDD (Hard Disk Drive) or flash memory. Note that the storage is not limited to being included in a server device, and cloud services may also be used.

[0021] The processor 29 controls the operation of the first server device 20 by reading and executing programs stored in the storage 26. The processor 29 may be, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like. The ROM may be configured to include a ROM (ROM Processing Unit) and the like.

[0022] The second server device 40 stores and manages data related to a predetermined race. The second server device 40 is, for example, a server device managed by the organizer of the predetermined race or a server device managed by an organization that disseminates information about the predetermined race to the outside (such as a publisher of a racing magazine). The second server device 40 transmits data related to the predetermined race to the first server device 20 as appropriate. In some aspects, the second server device 40 may transmit data related to the predetermined race to the user terminal 10. The hardware configuration of the second server device 40 may be the same as that of the first server device 20, as long as no contradictions arise.

[0023] (user terminal) Fig. 2 is a block diagram showing an example of the functional configuration of the user terminal 10. As shown in Fig. 2, the user terminal 10 includes an antenna 110, a wireless communication IF 120, a touch screen 130, an input / output IF 140, a storage unit 150, an audio processing unit 160, a microphone 161, a speaker 162, an imaging unit 170, and a control unit (first processor) 190.

[0024] The antenna 110 radiates the signal emitted by the user terminal 10 into space as a radio wave. The antenna 110 also receives the radio wave from space and provides the received signal to the wireless communication IF 120.

[0025] The wireless communication IF 120 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 110 etc. so that the user terminal 10 can communicate with other communication devices. The wireless communication IF 120 is a communication module for wireless communication that includes a tuner, a high-frequency circuit etc., and performs modulation / demodulation and frequency conversion of wireless signals transmitted and received by the user terminal 10, and provides the received signals to the control unit 190.

[0026] Touch screen 130 accepts input from the user and outputs information to the user on display 132. Touch screen 130 includes a touch panel 131 for accepting user operation input, and display 132. Touch panel 131 uses, for example, a capacitance type to detect the approach of a user's finger or the like. Display 132 is realized by, for example, an LCD (Liquid Crystal Display), an organic EL (Electroluminescence) or other display device.

[0027] The input / output IF 140 functions as an interface for accepting input of information to the user terminal 10 and outputting information to the outside of the user terminal 10.

[0028] The storage unit 150 is configured with a flash memory, a HDD, etc., and stores programs used by the user terminal 10, various data received by the user terminal 10 from the first server device 20, etc.

[0029] The audio processing unit 160 modulates and demodulates audio signals. The audio processing unit 160 modulates a signal provided from the microphone 161 and provides the modulated signal to the control unit 190. The audio processing unit 160 also provides the audio signal to the speaker 162. The audio processing unit 160 is realized, for example, by a processor for audio processing. The microphone 161 functions as an audio input unit that receives an input of an audio signal and outputs it to the control unit 190. The speaker 162 functions as an audio output unit that outputs the audio signal to the outside of the user terminal 10.

[0030] The image capturing unit 170 is a camera that captures a real image of the surroundings of the user terminal 10. The image captured by the image capturing unit 170 is subjected to image processing by the control unit 190 and output to the display 132.

[0031] The control unit 190 controls the operation of the user terminal 10 by reading and executing programs stored in the storage unit 150. The control unit 190 is realized by, for example, an application processor.

[0032] A more detailed description will be given of the process in which control unit 190 executes application program 151. Storage unit 150 stores application program 151, application information 152, and user information 153.

[0033] The user terminal 10, for example, downloads an application program 151 from the first server device 20 and stores it in the storage unit 150. In addition, the user terminal 10 communicates with the first server device 20 to transmit and receive various data such as application information 152 and user information 153 to and from the first server device 20.

[0034] The application program 151 is a program for displaying a virtual race on the user terminal 10. The application information 152 includes various types of data referenced by the application program 151. The application information 152 includes first information 152A.

[0035] The first information 152A is information about a predetermined race transmitted from the first server device 20 or the second server device 40. The first information 152A includes, for example, location information of participants or moving objects (hereinafter also referred to as "participants, etc.") in a predetermined race during the execution of the predetermined race, and time information corresponding to the location information. In some aspects, the first information 152A may include race time information indicating the race time of the participants, etc. (the time required from the start to the finish line).

[0036] In this specification, the term "participant" refers not only to humans but also to animals such as horses and dogs. Furthermore, a "mobile object" refers to the main moving object in a given race, such as an animal or vehicle ridden by a participant or a vehicle remotely controlled by a participant. In marathons, dog races, etc., the "participant" and the "mobile object" are the same.

[0037] In addition to the above, the first information 152A may also include, for example, the name of the specified race, the date and time of the race, racecourse data, participant data, mobile data, odds information, race predictions, race entry list, last-minute race information, pit reports, race results, race video, still images of the race, past race information, and other information that may be published in information magazines or information sites about the specified race.

[0038] The user information 153 includes information about the user of the user terminal 10. The user information 153 may include, for example, information for identifying the user, location information of the user terminal 10, the user's race purchase history and winning rate (for example, in the case of a boat race, the history of boat tickets purchased and the winning rate of the purchased boat tickets), etc.

[0039] By reading and executing the application program 151, the control unit 190 performs the functions of an operation input reception unit 191, a transmission / reception unit 192, an object generation unit 193, a display control unit 194, a detection unit 195, and a viewpoint change unit 196.

[0040] The operation input receiving unit 191 receives an operation input from the user based on the output of the touch screen 130. Specifically, the operation input receiving unit 191 detects that the user's finger or the like has come into contact with or approached the touch panel 131 as coordinates in a coordinate system consisting of the horizontal and vertical axes of the surface that constitutes the touch screen 130.

[0041] The operation input receiving unit 191 determines a user operation on the touch screen 130. The operation input receiving unit 191 determines a user operation such as, for example, an "approach operation," a "release operation," a "tap operation," a "double tap operation," a "long press operation (long touch operation)," a "drag operation (swipe operation)," a "move operation," a "flick operation," a "pinch in operation," and a "pinch out operation."

[0042] The operation input receiving unit 191 may receive, as an operation input, the movement of the user terminal 10 detected by an acceleration sensor, a gyro sensor, or the like mounted on the user terminal 10.

[0043] The transmitting / receiving unit 192 transmits and receives various information to and from external communication devices such as the first server device 20 and the second server device 40 via the wireless communication IF 120 and the network 30. The transmitting / receiving unit 192 receives, for example, first information 152A from the first server device 20 or the second server device 40. In addition, the transmitting / receiving unit 192 transmits, for example, information corresponding to an operation input received by the operation input receiving unit 191, information stored in the user information 153, etc. to the first server device 20 or the second server device 40.

[0044] The object generation unit 193 generates, as virtual objects, a racetrack object representing the racetrack and moving objects representing the contestants, etc. The object generation unit 193 may generate a virtual display board for displaying information about the race in text. In addition to the above objects, the object generation unit 193 may also generate a virtual screen for displaying information about the race as an image, various building objects, landscape objects that make up the landscape such as trees, objects that serve as the user's avatar, etc.

[0045] The display control unit 194 displays on the display 132 an image (hereinafter also referred to as a "superimposed image") in which the virtual object generated by the object generation unit 193 is superimposed on a real image of the surroundings of the user terminal 10 captured by the imaging unit 170. The display control unit 194 moves the moving object on the racecourse object based on the position information of predetermined participants, etc. during the execution of the predetermined race and the time information corresponding to the position information, which are included in the first information 152A, and displays a virtual race that virtually reproduces the predetermined race on the display 132. In some aspects, the display control unit 194 may reproduce the virtual race based on race time information.

[0046] The display control unit 194 displays various menu screens and GUIs (Graphical User Interfaces) on the display 132 and changes the display contents of the display 132 in accordance with the operation input received by the operation input receiving unit 191.

[0047] The detection unit 195 detects flat surfaces in the real image captured by the imaging unit 170. The detection of flat surfaces is achieved by a conventionally known image recognition technology. For example, when a user performs an operation to select a flat surface detected by the detection unit 195, a superimposed image in which a racetrack object is placed on the flat surface is displayed on the display 132.

[0048] The flat surface is preferably a horizontal surface. The angle between the flat surface and the bottom surface constituting the racetrack object may be 0 degrees, but is preferably an acute angle, for example, in the range of 15 degrees to 45 degrees. The angle may be preset in the user terminal 10, or may be changeable by the user. The angle may also be changed by the viewpoint change unit 196. Even if there is a protrusion on part of the flat surface in the real world, or if there is an object on the flat surface, the surface may be detected as a flat surface on which a racetrack object can be placed, as long as the protrusion or the object is small enough to be hidden by the racetrack object.

[0049] The viewpoint change unit 196 is responsible for processing on the user terminal 10 related to changing the viewpoint of the virtual race, and changes the viewpoint of the virtual race displayed by the display control unit 194 based on operational input from the user. The line of sight direction at the changed viewpoint may be predetermined, or may be changeable based on operational input from the user. There are no particular limitations on the user's operational input for changing the viewpoint or line of sight direction, but examples include pressing a physical button, input to the touch screen 130, and movement of the user terminal 10 detected by an acceleration sensor, gyro sensor, etc.

[0050] By providing the viewpoint change unit 196, the user can select one viewpoint from a plurality of viewpoints, such as a viewpoint positioned to the side of the racecourse object, a viewpoint positioned above, and a viewpoint positioned below, and view the virtual race from that viewpoint. The user may also be able to change the line of sight from the selected viewpoint, for example.

[0051] The viewpoint changing unit 196 may also change the viewpoint of the virtual race to a viewpoint determined based on the progress of the virtual race. For example, the viewpoint changing unit 196 changes the viewpoint of the virtual race to a viewpoint appropriate for the progress of the virtual race at that time. Definitions of the progress of the virtual race and information on the viewpoint corresponding to each progress status may be stored in the storage unit 150, for example.

[0052] The "progress status of the virtual race" refers to the status when the progress of the virtual race is divided into predetermined sections based on a predetermined criterion, such as immediately after the start of the virtual race, in the middle of the virtual race, at the end of the virtual race, near the end of the virtual race, and after the moving object has reached the finish line. The predetermined criterion is not particularly limited and may be determined as appropriate by an administrator of the first server device 20 depending on the type of race, for example, and may be a criterion such as a situation where the moving object is located at the first corner of the first lap, or a situation where the moving object is located on the straight line after the first corner of the first lap.

[0053] Furthermore, when the viewpoint changing unit 196 is set to automatically switch the viewpoint of the virtual race by user input, the viewpoint changing unit 196 automatically switches sequentially between at least two or more pre-set viewpoints according to the progress of the virtual race, or changes the viewpoint continuously from a first pre-set viewpoint to a second pre-set viewpoint.

[0054] Furthermore, the viewpoint changing unit 196 may change the viewpoint in accordance with the movement of a moving object selected based on a predetermined selection criterion. The predetermined selection criterion is not particularly limited, but examples include a moving object selected by the user, a moving object corresponding to a contestant for whom the user has purchased a betting ticket, or a contestant who is attracting attention because he or she is the most popular, etc. The viewpoint changing unit 196 may change the viewpoint so as to follow the movement of the moving object selected based on the predetermined selection criterion.

[0055] The viewpoint changing unit 196 may also change the viewpoint of the virtual race to that of a moving object selected based on the above-mentioned predetermined selection criteria, or may change the viewpoint of the virtual race to that of an arbitrary position on the racecourse object.

[0056] It is preferable that the viewpoint change unit 196 calculates the placement position of a virtual object such as a moving object at the changed viewpoint based on, for example, the position information of the contestants, etc. included in the first information 152A and the time information corresponding to the position information, and performs the viewpoint change as described above.

[0057] The calculation of the placement position of the virtual object from the changed viewpoint may be performed by a viewpoint change unit 293 on the first server device 20, which will be described later. In such a configuration, the user terminal 10 transmits information regarding the viewpoint and line of sight desired by the user to the first server device 20, receives information for displaying the virtual race after the viewpoint change from the first server device 20, and displays the virtual race based on that information.

[0058] (First server device) Fig. 3 is a block diagram showing the functional configuration of the first server device 20. The detailed configuration of the first server device 20 will be described with reference to Fig. 3. The first server device 20 performs the functions of a communication unit 220, a storage unit 250, and a control unit 290 by operating in accordance with a program.

[0059] The communication unit 220 functions as an interface for the first server device 20 to communicate with external communication devices such as the user terminal 10 and the second server device 40 via the network 30.

[0060] Storage unit 250 stores various programs and data for implementing system 1. In one aspect, storage unit 250 stores program 251, race information 252, and user information 253.

[0061] The program 251 is a program that enables the first server device 20 to communicate with the user terminal 10 and the second server device 40 to realize the system 1. When executed by the control unit 290, the program 251 causes the first server device 20 to perform processes such as sending and receiving data to and from the user terminal 10 and the second server device 40, processes according to operations performed by the user of the user terminal 10, and processes that update the race information 252 and the user information 253.

[0062] The race information 252 includes various data related to a predetermined race. The race information 252 includes, for example, first information 252A. The first information 252A is information that is the source of the first information 152A, and the first information 152A can be a part of the first information 252A. The first information 252A is, for example, information acquired from the second server device 40.

[0063] The user information 253 is information about the user of the user terminal 10. The user information 253 includes a user management table 253A. The user management table 253A stores, for each user, information identifying the user, terminal location information of the user terminal 10, the user's race purchase history and hit rate, etc.

[0064] The control unit 290 is realized by the processor 29, and by executing the program 251, it performs the functions of a transmission / reception unit 291, a first information acquisition unit 292, a viewpoint change unit 293, a data management unit 294, and a timing unit 295.

[0065] The transmitting / receiving unit 291 transmits and receives various information to and from external communication devices such as the user terminal 10 and the second server device 40 via the communication unit 220 and the network 30. The transmitting / receiving unit 291 transmits, for example, at least a part of the first information 252A to the user terminal 10. The transmitting / receiving unit 291 also receives, for example, the first information 252A from the second server device 40. The first information acquisition unit 292 acquires the first information 252A from the second server device 40 via the transmitting / receiving unit 291.

[0066] The viewpoint changing unit 293 is responsible for processing on the side of the first server device 20 related to changing the viewpoint of the virtual race. When the viewpoint changing unit 196 described above is configured to calculate the placement position of the virtual object from the changed viewpoint, the first server device 20 does not need to be equipped with the viewpoint changing unit 293.

[0067] The data management unit 294 performs a process of updating various data stored in the storage unit 250 according to the processing results of the first information acquisition unit 292 etc. The timing unit 295 performs a process of measuring time. Based on the time measured by the timing unit 295, various times (e.g., the time until the start of a race) displayed on the user terminal 10 can be controlled.

[0068] (Example of operation) Next, an example of operation of the system 1 will be described with reference to Figures 4 to 13. The following description will mainly focus on the case where the predetermined race is a boat race, but the following description can also be applied to cases where the predetermined race is another race. Furthermore, the following description will be given assuming that data is transmitted and received between the user terminal 10 and the first server device 20, and between the first server device 20 and the second server device 40, but in some aspects, the system may be configured so that data is transmitted and received directly between the user terminal 10 and the second server device 40.

[0069] 4 is a schematic diagram showing an example of a real-world boat racing track. Two turn marks 403 are set up at a boat racing track 401, and a race is held between boats 402a-402f, each carrying a boat racer. When the race is over, race time information indicating the race time of each of the boats 402a-402f is transmitted from the second server 40 to the first server 20, and then from the first server 20 to the user terminal 10.

[0070] The boat racing venue 401 is provided with imaging devices (cameras) 404a-404b. The imaging device 404a views the boats 402a-402f from above the boat racing venue 401. The imaging device 404b views the boats 402a-402f from the side of the boat racing venue 401. Images of the boats 402a-402f captured by the imaging devices 404a-404b are transmitted to the second server 40. The second server 40, for example, performs image analysis on each image and calculates position information indicating the position of each of the boats 402a-402f at the time each image was captured. The calculated position information and time information relating to the capture time corresponding to the position information are transmitted to the first server 20, and then transmitted from the first server 20 to the user terminal 10. Note that the calculation of the position information may be performed by the first server 20.

[0071] Furthermore, instead of or in addition to the imaging devices 404a to 404b, position sensors such as GPS sensors may be installed on the boats 402a to 402f. The position information of the boats 402a to 402f acquired by the position sensors and time information indicating the time when the position information was acquired are ultimately transmitted to the user terminal 10.

[0072] Fig. 5 is a schematic diagram showing an example of virtual objects displayed on the user terminal 10. In the example of Fig. 5, a racetrack object 501, moving objects 502a to 502f, two turn mark objects 503, and a virtual display board 505 are shown as virtual objects.

[0073] The racecourse object 501 is an object that virtually displays the boat racing track 401. The racecourse object 501 and the turn mark object 503 are preferably created based on racecourse data such as course information of the boat racing track 401, and preferably have a shape corresponding to the boat racing track 401.

[0074] The moving objects 502a to 502f are objects that virtually represent the boats 402a to 402f, respectively, and have shapes that resemble boats. The moving objects 502a to 502f move on the racetrack object 501 based on race time information, position information of the boats 402a to 402f, and time information corresponding to the position information. In other words, a real-world race is displayed as a virtual race on the user terminal 10 by the racetrack object 501 and the moving objects 502a to 502f.

[0075] It is also possible to display a virtual race using only the race time information, even if the position information and time information of the boats 402a to 402f are not available. In this case, however, although the final finishing order will be the same as in the real-world race, it is difficult to reproduce the race progress, such as the ranking during the race.

[0076] The virtual display board 505 is an object that displays text information. The virtual display board 505 is, for example, an object that does not have a corresponding object in the boat racing venue 401. The text information displayed on the virtual display board 505 is not particularly limited and may be, for example, ranking information, odds information, etc. Furthermore, the text information displayed on the virtual display board 505 may be changeable based on a user's operational input.

[0077] 6 is a flowchart showing an example of a display control process. The order of the processes constituting the flowchart described below is random as long as no contradictions or inconsistencies occur in the process content. Furthermore, the processes executed by each device may be executed by another device as long as no contradictions occur.

[0078] 6 is realized by control unit 190 executing application program 151 and control unit 290 executing program 251. First, in step S610, control unit 190 starts up imaging unit 170, which is a camera. The imaging unit 170 captures a real image of the surroundings of user terminal 10.

[0079] In step S620, the control unit 190 detects a flat surface in the image captured by the imaging unit 170. In step S630, the control unit 190 places a virtual object on the detected flat surface.

[0080] Here, the processing of steps S610, S620, and S630 will be specifically described with reference to Figures 7 to 9. Figure 7 is a schematic diagram showing an example of a real image captured by the imaging unit 170. In the example of Figure 7, a keyboard 702 and a monitor device 703 are placed on a flat desk 701.

[0081] When imaging unit 170 is activated in step S610, a real image captured by imaging unit 170 is displayed on display 132. Next, in step S620, control unit 190 detects a flat surface within the image captured by imaging unit 170, i.e., within the image displayed on display 132.

[0082] 7, an area 704 is detected as a flat surface. A keyboard 702 is present within the area 704, but the keyboard 702 is small enough to be hidden by the racetrack object 501, so the control unit 190 detects the area 704 as a flat surface.

[0083] 7, if the position captured by the imaging unit 170 is changed, the position of the area 704 can also be changed. The area 704 is displayed on the display 132, for example, by adding a predetermined color to the area 704 so that it can be distinguished from other parts. When the user performs a tap operation or the like on the area 704, the control unit 190 places a virtual object, such as the racetrack object 501, on the area 704 in step S630.

[0084] 8 is a schematic diagram showing an example of a screen displaying a virtual object superimposed on a real image. In FIG. 8, the area marked with a dot pattern including the monitor device 703 is the real image, and the other areas are areas where virtual objects are displayed. In areas where virtual objects are not displayed, for example, an advertising image may be displayed.

[0085] 8, the following virtual objects are displayed: a racetrack object 501, multiple moving objects 502, two turn mark objects 503, a large monitor object 506, building objects 507a to 507b, and numerous other objects not labeled (such as tree objects and clock objects). These objects are created based on, for example, first information 152A received from first server device 20.

[0086] 8, the racetrack object 501 is placed on the area 704 so that the angle between the flat surface included in the area 704 and the bottom surface of the racetrack object 501 is 0 degrees. In this case, for example, areas in the depth direction such as area X1 or overlapping moving objects 502 may be difficult to see. Also, blind spots that make the moving objects 502 invisible may occur in areas such as the area on the depth side of the large monitor object 506.

[0087] However, in the state shown in FIG. 8 , the user can change the viewpoint of the superimposed image displayed on the display 132, for example, by using the viewpoint changing unit 196. For example, when the user performs an operation input such as a pinch-in operation or a pinch-out operation, the viewpoint position of the imaging unit 170 may move closer to or farther away from the racetrack object 501. Also, when the user moves toward the depth or front of the display 132 while holding the user terminal 10, the viewpoint position of the imaging unit 170 may move closer to or farther away from the racetrack object 501. Also, when the user performs an operation to specify a predetermined position on the display 132, the viewpoint position may move to the specified position. Changing the viewpoint in this way may solve the visibility problem described above.

[0088] The above-mentioned visibility problem can also be alleviated by arranging the racetrack object 501 so that the angle θ formed between the flat surface and the bottom surface that constitutes the racetrack object 501 is an acute angle.

[0089] Fig. 9 is a schematic diagram showing an example of a screen in which a virtual object is superimposed on a real image, and shows another aspect of the racetrack object 501 shown in Fig. 8. Specifically, Fig. 9 shows an example in which the predetermined race is a horse race.

[0090] 9, the area marked with a dot pattern including monitor device 703 is the real image, and the other area is the area where virtual objects are displayed. In Fig. 9, a racetrack object 911, multiple moving objects 912, a large monitor object 913, a pond object 914, and multiple tree objects 915 are displayed as virtual objects on display 132. These objects are also created based on, for example, first information 152A received from first server device 20.

[0091] The racetrack object 911, large monitor object 913, pond object 914, and multiple tree objects 915 are preferably created based on racetrack data such as course information of a specific racetrack in the real world. The multiple moving objects 912 are, for example, objects that virtually represent the horses and jockeys participating in the race.

[0092] Returning to the description of the flowchart in Fig. 6, in step S640, the control unit 190 acquires position information of the boats 402a to 402f at the boat racing track 401 in the real world. That is, when a race between the boats 402a to 402f starts in the real world, the control unit 190 acquires position information and time information of the boats 402a to 402f from the first server device 20. The method of acquiring the position information and time information is as described using Fig. 4.

[0093] In step S650, the control unit 190 controls the position information acquired in step S640 to be linked with the moving objects. Specifically, using time information and position information, the control unit 190 controls the movement of each of the moving objects 502a to 502f on the racecourse object 501 to be similar to the movement of the boats 402a to 402f on the boat racing track 401.

[0094] In step S660, if control unit 190 receives an operational input to change the viewpoint of the virtual race (Yes in step S660), control unit 190 changes the viewpoint to one that corresponds to the user's operational input and displays the virtual race. On the other hand, in step S660, if control unit 190 does not receive an operational input to change the viewpoint of the virtual race (No in step S660), the process of step S670 is not executed. The processes of steps S660 and S670 will be described in detail in later paragraphs.

[0095] The processes of steps S640 to S670 are repeated at least from the start to the end of the race in the real world, but may also be repeated before the start and after the end of the race in the real world. The control unit 190 ends the series of display control processes in response to, for example, receiving an operation input to end the application program 151.

[0096] If the race to be displayed as a virtual race is a past race, position information and the like of the boats 402a to 402f from the start to the finish of the race may be acquired all at once before the start of the virtual race. Alternatively, the virtual race may be displayed by acquiring only race time information without acquiring position information and the like. The race to be displayed as a virtual race may be an exhibition race.

[0097] The processing of steps S660 and S670 in Fig. 6 will be described in detail below. Fig. 10 and Fig. 11 are schematic diagrams showing an example of a display screen displaying a virtual race according to an embodiment. Specifically, Fig. 10 is an example of displaying the virtual race from a viewpoint located to the side of the racecourse object 501, and Fig. 11 is an example of displaying the virtual race from a viewpoint located above the racecourse object 501.

[0098] 10 and 11, each moving object 502 moves on a racecourse object 501 based on, for example, the position information and time information of the corresponding boat 402. In addition to the already-described virtual objects such as the racecourse object 501, a large clock object 1001 and viewpoint change buttons 1002a to 1002d are also displayed on the display 132. The large clock object 1001 is a virtual object corresponding to a large clock placed at the boat racing course 401.

[0099] The viewpoint change buttons 1002a to 1002d are buttons for changing the viewpoint of the virtual race displayed on the display 132. The viewpoint change button 1002a is a button for changing the viewpoint of the virtual race to a viewpoint located above the racecourse object 501.

[0100] The viewpoint change button 1002b is a button for changing the viewpoint of the virtual race to a viewpoint positioned to the side of the racecourse object 501. In Fig. 10, the viewpoint change button 1002b is displayed in a different manner from the other viewpoint change buttons 1002a and 1002c to 1002d. This indicates that the virtual race displayed in Fig. 10 is viewed from a viewpoint positioned to the side.

[0101] It is preferable that the user can fine-tune the position of the viewpoint and the direction of the line of sight in the state shown in Fig. 10. Specifically, it is preferable that the user can change the direction of the line of sight when an operation such as tilting the user terminal 10 is detected in the state shown in Fig. 10, or can change the position of the viewpoint when an operation such as moving the position of the user terminal 10 is detected. Note that the fine adjustment of the position of the viewpoint and the direction of the line of sight may be performed by an operation such as input to the touch screen 130.

[0102] In the state shown in Fig. 10, for example, when an operation (e.g., a tap operation) is performed to select viewpoint change button 1002a, the state changes to the state shown in Fig. 11, that is, a state in which the virtual race is displayed from a viewpoint located above racecourse object 501. Note that while display of virtual objects such as building object 507a is omitted in Fig. 11, these omitted virtual objects may be hidden in the state shown in Fig. 11. In other words, the type of virtual object displayed may differ depending on the position of the viewpoint.

[0103] The viewpoint change button 1002d is a button for changing the viewpoint of the virtual race in a manner different from that of the viewpoint change buttons 1002a to 1002c. When an operation is performed to select the viewpoint change button 1002d, for example, a viewpoint change method different from that of the viewpoint change buttons 1002a to 1002c is presented to the user, and when the user selects one of the viewpoint change methods, the viewpoint is changed in accordance with that viewpoint change method.

[0104] Examples of viewpoint change methods other than those used by the viewpoint change buttons 1002a to 1002c include changing the viewpoint of the virtual race to a viewpoint located in a direction different from that of the viewpoint change buttons 1002a and 1002b (for example, below the racecourse object 501), changing the viewpoint to that from a moving object 502 selected based on the above-mentioned predetermined selection criteria, and changing the viewpoint to that from a predetermined position on the racecourse object 501 (for example, points A1 to A3 shown in FIG. 10). Note that the above-mentioned predetermined position does not have to be a preset position such as points A1 to A3, but may be any position selected by the user by a tap operation or the like.

[0105] The viewpoint change button 1002c is a button for automatically changing the viewpoint of the virtual race in accordance with the progress of the virtual race. When the viewpoint change button 1002c is pressed, the viewpoint is changed in accordance with the progress of the virtual race, for example. Specifically, control is performed such that the viewpoint is automatically switched between two or more viewpoints in sequence, or the viewpoint is changed continuously from a first viewpoint to a second viewpoint, in accordance with the progress of the virtual race.

[0106] 12 and 13 are schematic diagrams illustrating viewpoint changes in a virtual race according to an embodiment. Specifically, FIG. 12 shows an example of automatically and continuously changing the viewpoint from a first viewpoint to a second viewpoint, and FIG. 13 shows an example of automatically sequentially switching between two or more viewpoints. When the viewpoint change button 1002c shown in FIGS. 10 and 11 is selected, whether the viewpoint change is as shown in FIG. 12 or as shown in FIG. 13 may be preset depending on the type of race, or may be selectable by the user. The viewpoint change mode shown in FIG. 12 and the viewpoint change mode shown in FIG. 13 may also be combined.

[0107] 12(a) shows, for example, a situation immediately after the start of a virtual race. FIG. 12(b) shows, for example, a situation immediately after the moving object 502 has passed a corner on the first lap. FIG. 12(c) shows, for example, a situation in which the moving object 502 is proceeding in a straight line after passing a corner on the first lap.

[0108] In the example of Fig. 12, the viewpoint changes continuously from Fig. 12(a) to Fig. 12(b) as the virtual race progresses. Similarly, the viewpoint changes continuously from Fig. 12(b) to Fig. 12(c) as the virtual race progresses. In other words, in the example of Fig. 12, the viewpoint changes so that the viewpoint moves along a preset trajectory as the virtual race progresses.

[0109] In the example of Fig. 12, the viewpoint is changed so that the moving object 502a is positioned near the center. The moving object 502a is selected based on the above-mentioned predetermined selection criteria, for example. The example of Fig. 12 is also an example in which the viewpoint is continuously changed to follow the moving object 502a.

[0110] Figure 13(a) shows, for example, a situation where the leading moving object 502a is approaching a corner on the second lap. Figure 13(b) shows, for example, a situation after passing a corner on the second lap. In Figure 13(b), the viewpoint position has been changed so that the moving objects 502a and 502b currently ranked first and second are displayed in close-up.

[0111] (c) of Fig. 13 shows, for example, the situation after passing a corner on the second lap, which is the situation immediately after (b) of Fig. 13. In (c) of Fig. 13, the viewpoint position has been changed so that the moving objects 502c to 502f currently ranked third and below are displayed. (d) of Fig. 13 shows, for example, the situation proceeding on a straight line after passing a corner on the second lap. In (d) of Fig. 13, the viewpoint has been changed to a position where the moving objects 502a to 502f proceeding on the far side of the center pole object 1301 can be seen.

[0112] (a) to (d) of Fig. 13 are examples in which a virtual race is displayed based on a viewpoint that is preset for each progress status. In the example of Fig. 13, the viewpoint switches sequentially in the order of (a), (b), (c), and (d) as the virtual race progresses. That is, in the example of Fig. 13, as the virtual race progresses, the viewpoint instantaneously switches from the current viewpoint to another predetermined viewpoint that is discontinuous with the current viewpoint.

[0113] The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

[0114] [Additional Notes] The contents of the present disclosure are listed as follows:

[0115] (Item 1) A program executed on a first computer including a processor and an imaging unit, The program causes the processor to: receiving first information from a second computer regarding a predetermined race in the real world; generating, based on the first information, virtual objects including a racetrack object representing a racetrack and moving objects representing participants or moving objects in the predetermined race; and displaying the virtual object superimposed on a real image of the surroundings of the first computer captured by the imaging unit, the first information includes location information of the contestants or the moving objects during the execution of the predetermined race and time information corresponding to the location information; the displaying step includes moving the moving object on the racetrack object based on the first information and displaying a virtual race corresponding to the predetermined race; The program further causes the processor to changing a viewpoint of the displayed virtual race based on an operation input by a user of the first computer; program. This makes it possible to display a real-world race as a virtual race using virtual objects linked to the real-world race, and to display the virtual race from multiple viewpoints.

[0116] (Item 2) the step of changing includes changing the viewpoint of the virtual race to a viewpoint determined based on a progress status of the virtual race. Item 1. The program according to item 1. This makes it possible to display the virtual race based on an appropriate viewpoint according to the progress of the virtual race, thereby increasing the sense of realism of the virtual race and the excitement of the user.

[0117] (Item 3) the step of changing includes automatically and sequentially switching between at least two or more preset viewpoints in accordance with the progress of the virtual race. The program according to item 1 or 2. This makes it possible to dynamically change the viewpoint depending on the progress of the virtual race, thereby further increasing the sense of realism of the virtual race and the excitement of the user.

[0118] (Item 4) the changing step includes automatically and continuously changing the viewpoint from a first viewpoint to a second viewpoint that is set in advance in accordance with the progress of the virtual race. Item 3. The program according to any one of items 1 to 3. This allows the viewpoint to be gradually changed according to the progress of the virtual race, which results in a viewpoint change similar to that seen when watching a race at a real racetrack, further enhancing the sense of realism and excitement of the virtual race.

[0119] (Item 5) the changing step includes changing the viewpoint in accordance with movement of the moving object selected based on a predetermined criterion; Item 4. The program according to any one of items 1 to 4. This allows the virtual race to be displayed with a focus on, for example, contestants for whom the user has purchased a betting ticket, or contestants who are the most popular and attracting a lot of attention, thereby increasing user convenience and satisfaction.

[0120] (Item 6) the step of changing includes changing the viewpoint from the moving object selected based on the predetermined criteria; Item 5. The program according to item 5. This allows the user to experience the virtual race from the perspective of the participants, further increasing the sense of realism of the virtual race and the excitement of the user.

[0121] (Item 7) the step of changing includes changing the viewpoint from a predetermined position on the racetrack object. 7. The program according to any one of items 1 to 6. This makes it possible to watch the virtual race from various viewpoints, such as the user's favorite location, a paid seat location, or locations that are inaccessible in reality, such as underwater or locations where spectators are prohibited from entering, thereby increasing user satisfaction and the appeal of the virtual race.

[0122] (Item 8) An information processing method executed in a first computer including a processor and an imaging unit, The information processing method includes: receiving first information from a second computer regarding a predetermined race in the real world; generating, based on the first information, virtual objects including a racetrack object representing a racetrack and moving objects representing participants or moving objects in the predetermined race; a step of superimposing and displaying the virtual object on a real image of the surroundings of the first computer captured by the imaging unit; and causing the the first information includes location information of the contestants or the moving objects during the execution of the predetermined race and time information corresponding to the location information; the displaying step includes moving the moving object on the racetrack object based on the first information and displaying a virtual race corresponding to the predetermined race; The information processing method further comprises: and executing a step of changing a viewpoint of the displayed virtual race based on an operation input by a user of the first computer. Information processing methods. This makes it possible to display a real-world race as a virtual race using virtual objects linked to the real-world race, and to display the virtual race from multiple viewpoints.

[0123] (Item 9) An information processing device including a processor and an imaging unit, The processor: receiving first information from a second computer regarding a predetermined race in the real world; generating virtual objects including a racetrack object representing a racetrack and a moving object representing a participant or a moving object in the predetermined race based on the first information; the virtual object is displayed superimposed on a real image of the surroundings of the information processing device captured by the imaging unit, the first information includes location information of the contestants or the moving objects during the execution of the predetermined race and time information corresponding to the location information; the superimposing and displaying includes moving the moving object on the racetrack object based on the first information and displaying a virtual race corresponding to the predetermined race, The processor further comprises: changing a viewpoint of the displayed virtual race based on an operation input by a user of the information processing device; Information processing device. This makes it possible to display a real-world race as a virtual race using virtual objects linked to the real-world race, and to display the virtual race from multiple viewpoints.

[0124] (Item 10) A system implemented in a first computer including a first processor and an imaging device, and a second computer including a second processor and communicatively connectable to the first computer, The second processor obtaining first information regarding a given race in the real world; sending the first information to a first computer; The first processor receiving the first information from a second computer; generating virtual objects including a racetrack object representing a racetrack and a moving object representing a participant or a moving object in the predetermined race based on the first information; the virtual object is displayed superimposed on a real image of the surroundings of the first computer captured by the imaging unit, the first information includes location information of the contestants or the moving objects during the execution of the predetermined race and time information corresponding to the location information; the superimposing and displaying includes moving the moving object on the racetrack object based on the first information and displaying a virtual race corresponding to the predetermined race, The first processor further changing a viewpoint of the displayed virtual race based on an operation input by a user of the first computer; system. This makes it possible to display a real-world race as a virtual race using virtual objects linked to the real-world race, and to display the virtual race from multiple viewpoints. [Explanation of symbols]

[0125] 1: System, 10: User terminal, 20: First server device, 30: Network, 40: Second server device, 130: Touch screen, 150: Storage unit (of user terminal), 190: Control unit (of user terminal), 250: Storage unit (of first server device), 290: Control unit (of first server device)

Claims

1. A program executed on a computer, The program causes the computer to A receiving means for receiving first information related to a predetermined race in the real world; a generation means for generating virtual objects including a racecourse object representing a racecourse and a moving object representing a participant or a moving object of the predetermined race based on the first information; a display unit that displays an image including the virtual object; The first information includes location information of the contestant or the moving object and time information corresponding to the location information, the display means moves the moving object on the racetrack object based on the first information, and displays a virtual race corresponding to the predetermined race; The program further causes the computer to when a user's operation input for changing a viewpoint of the virtual race is received during the progress of the virtual race, the change means changes the viewpoint of the virtual race that is being displayed in accordance with the movement of the moving object that is automatically selected based on the progress of the virtual race; program.

2. the changing means changes the viewpoint to that from the moving object selected based on the progress status. The program according to claim 1.

3. An information processing device including a processor, The processor, receiving first information relating to a given race in the real world; generating virtual objects including a racetrack object representing a racetrack and a moving object representing a participant or a moving object of the predetermined race based on the first information; An image including the virtual object is displayed, The first information includes location information of the contestant or the moving object and time information corresponding to the location information, displaying the image includes moving the moving object on the racecourse object based on the first information and displaying a virtual race corresponding to the predetermined race; The processor further comprises: when a user's operation input for changing a viewpoint of the virtual race is received during the progress of the virtual race, the viewpoint of the displayed virtual race is changed in accordance with the movement of the moving object that is automatically selected based on the progress of the virtual race. Information processing device.

4. A system implemented in a first computer having a first processor and a second computer having a second processor and communicatively connectable to the first computer, The second processor Obtaining first information regarding a given race in the real world; Sending the first information to a first computer; The first processor, receiving the first information from a second computer; generating virtual objects including a racetrack object representing a racetrack and a moving object representing a participant or a moving object of the predetermined race based on the first information; An image including the virtual object is displayed, The first information includes location information of the contestant or the moving object and time information corresponding to the location information, displaying the image includes moving the moving object on the racecourse object based on the first information and displaying a virtual race corresponding to the predetermined race; The first processor further comprises: when a user's operation input for changing a viewpoint of the virtual race is received during the progress of the virtual race, the viewpoint of the displayed virtual race is changed in accordance with the movement of the moving object that is automatically selected based on the progress of the virtual race. system.

Citation Information

Patent Citations

  • Betting system and computer readable recording medium recorded with program

    JP2002133007A

  • Game system

    JP2008149173A

  • Gaming machine capable of displaying some characters of multiple characters participating in competition race game in three-dimensional image based on specified viewpoint, and its control method

    JP2009045336A

  • Poultry carcass breast meat cut-making device

    JP2020000002A