Display terminal, display method, program, and server
The display terminal and method simulate the sensation of touching an object by moving images on a touch panel with delay and deformation, effectively conveying tactile sensations, addressing the limitations of existing touch panels in conveying tactile feedback.
Patent Information
- Application Number
- PCT/JP2024/016924
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-21
AI Technical Summary
Existing touch panels lack effectiveness in conveying tactile sensations to users, as described in Patent Document 1, and there is a need to transmit the feeling of touching an object using general-purpose display devices like touch panels, displays, and projectors.
A display terminal, method, program, and server that simulate the sensation of touching an object by displaying images on a touch panel, where the image of a hand and fingers moves with a delay relative to a moving object, and the viscosity and volatility of the object are considered to deform and evaporate the image accordingly, mimicking the tactile experience.
The solution effectively conveys the tactile sensation of an object to the user through visual and touch feedback, enhancing the user experience by simulating the interaction with various substances like liquids and solids.
Smart Images

Figure JP2024016924_21112024_PF_FP_ABST
Abstract
Description
Display terminal, display method, program and server
[0001] The present disclosure relates to a display terminal, a display method, a program, and a server.
[0002] Technologies for transmitting tactile sensations via touch panels have been studied. For example, Patent Literature 1 describes a touch panel equipped with a tactile feedback actuator that allows a user to sense tactile sensations through vibrations.
[0003] JP 2012-226743 A
[0004] Touch panels equipped with actuators such as those described in Patent Document 1 have not been widely adopted. Therefore, there is a demand for transmitting tactile sensations to users using general-purpose display devices (e.g., touch panels, displays, projectors, etc.).
[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a display terminal, a display method, a program, and a server that enable the sensation of touching an object to be conveyed to a user via an image.
[0006] The display terminal of the embodiment has a display unit, a display control unit that displays a screen on the display unit that includes a first image and a moving object for moving the first image, and a reception unit that receives a movement operation for moving the first image, and is characterized in that the display control unit moves the first image to follow the moving object in accordance with the movement operation.
[0007] Preferably, the screen further includes a second image arranged at a position where it overlaps with the first image due to the movement of the first image.
[0008] In addition, it is preferable that the display terminal further has an acquisition unit that acquires tracking parameters that are set according to the object indicated by the second image, and the display control unit moves the first image to follow the moving object based on the tracking parameters.
[0009] Furthermore, it is preferable that the display control unit sets a moving speed of the first image based on the following parameter so as to lag behind the moving object, and moves the first image at the set moving speed.
[0010] Furthermore, it is preferable that the display control unit sets a delay time for the first image based on the tracking parameter, and moves the first image so as to track the moving object with a delay of the set delay time.
[0011] Furthermore, it is preferable that the object is a liquid or a slurry, the acquisition unit acquires a viscosity parameter set according to the viscosity of the object, and the display control unit deforms the second image based on the viscosity parameter when the first image and the second image overlap.
[0012] Preferably, the object is a solid object, and the tracking parameters are set according to the feel of the object.
[0013] In addition, it is preferable that the display terminal further has an update unit that updates the tracking parameters based on the time since the movement of the first image began or the movement history of the first image, and the display control unit moves the first image so as to follow the moving object with a delay based on the updated tracking parameters.
[0014] Furthermore, it is preferable that the display control unit displays a third image showing the coating film at a location where the first image has passed.
[0015] Furthermore, it is preferable that the screen further includes a second image arranged at a position where it overlaps with the first image due to the movement of the first image, and that the display control unit displays the third image after the first image and the second image overlap.
[0016] In addition, it is preferable that the display terminal further has an acquisition unit that acquires a volatility parameter that is set according to the volatility of the object shown by the second image, and the display control unit reduces the opacity of the third image based on the elapsed time since the third image was displayed and the volatility parameter.
[0017] Preferably, the first image is an image of fingers.
[0018] Preferably, the display device is a touch panel, and the movement operation is a swipe operation on the touch panel.
[0019] The display method according to the embodiment is a display method executed by a display terminal having a display unit, and is characterized in that it includes displaying a screen on the display unit that includes a first image and a moving object for moving the first image, accepting a movement operation for moving the first image, and moving the first image to follow the moving object in accordance with the movement operation.
[0020] The program of the embodiment is a program for a computer having a display unit, and is characterized in that it causes the computer to execute the following: displaying a screen on the display unit that includes a first image and a moving object for moving the first image; accepting a movement operation for moving the first image; and moving the first image to follow the moving object in accordance with the movement operation.
[0021] The server according to the embodiment is a server that communicates with a display terminal having a display unit, and has a display control unit that displays a screen on the display unit that includes a first image and a moving object for moving the first image, and a reception unit that receives a movement operation for moving the first image, and the display control unit moves the first image to follow the moving object in accordance with the movement operation.
[0022] The display terminal, display method, program, and server make it possible to convey to a user, via an image, the sensation felt when touching an object.
[0023] 1 is an exemplary functional block diagram of a display terminal 1. FIG. 1 is a diagram showing an example of an object selection screen G1. FIG. 2 is a diagram showing an example of screen transitions on a sensation transfer screen G2. FIG. 3 is a diagram showing an example of the data structure of an object table T1. FIG. 4 is a graph showing the ratio Fy / Fz of a force Fy in the direction of a swipe operation applied to a user's finger from the surface of the touch panel 12 when the user performs a swipe operation to a normal force Fz. FIG. 5 is a diagram showing an example of the data structure of an update table T2. FIG. 6 is a flow diagram showing an example of the flow of sensation transfer processing. FIG. 7 is a flow diagram showing an example of the flow of a screen update processing. FIG. 8 is a diagram showing an example of screen transitions on a sensation transfer screen G3 when the object is a solid. FIG. 9 is a diagram showing an example of screen transitions on a sensation transfer screen G4 including an image of lipstick. FIG. 10 is a diagram showing an example of a sensation transfer screen G5 including an image of a puff. FIG. 11 is an exemplary functional block diagram of a display system 2.
[0024] Various embodiments will be described below with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions set forth in the claims and their equivalents.
[0025] FIG. 1 is an exemplary functional block diagram of a display terminal 1 according to an embodiment of the present invention. The display terminal 1 is a computer such as a PC (Personal Computer), a smartphone, a mobile phone, a tablet terminal, or a portable game console. The display terminal 1 has a storage unit 11, a touch panel 12, and a processing unit 13. The display terminal 1 conveys to the user the sensation felt when touching an object via an image displayed on the touch panel 12 in response to the user's operation on the touch panel 12. The object is a liquid such as lotion, emulsion, cream, serum, sunscreen, cleanser, or oil. For example, the display terminal 1 conveys to the user the sensation felt when applying the object to the skin surface with the fingers.
[0026] The storage unit 11 is configured to store programs and data, and includes, for example, a semiconductor memory. The programs stored in the storage unit 11 include an operating system program, driver programs, and application programs used in processing by the processing unit 13. The programs are installed into the storage unit 11 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc Read Only Memory). The programs may also be stored in a storage medium owned by a predetermined server and installed via a network.
[0027] Touch panel 12 is configured to display images and accept user operations on display terminal 1. Touch panel 12 includes an integrated display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and an operation device which is a capacitive or pressure-sensitive touchpad. The display device displays images based on image data supplied from processing unit 13. The operation device generates an operation signal corresponding to the accepted operation and supplies it to processing unit 13. The operation signal is a signal indicating the position of a contact point detected on the touchpad.
[0028] The processing unit 13 is configured to comprehensively control the operation of the display terminal 1 and includes one or more processors and their peripheral circuits. For example, the processing unit 13 includes a central processing unit (CPU). The processing unit 13 may also include a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like. The processing unit 13 performs various processes based on programs stored in the storage unit 11 and operation signals supplied from the touch panel 12.
[0029] The processing unit 13 includes, as functional blocks, a selection unit 131, an acquisition unit 132, a display control unit 133, an update unit 134, and a reception unit 135. Each of these units is a functional module realized by a program executed by the processing unit 13. Each of these units may be implemented in the display terminal 1 as a dedicated arithmetic circuit.
[0030] 2 is a diagram showing an example of an object selection screen G1 displayed on the touch panel 12. The object selection screen G1 is a screen for selecting an object that will convey a sensation when applied to the skin surface. The object selection screen G1 is displayed in response to execution of an application program for conveying to the user a sensation when the object is applied to the skin surface.
[0031] The object selection screen G1 includes a selection object G11 and a decision object G12. The selection object G11 is an object for selecting an object that transmits a sensation when applied to the skin surface from among multiple objects. In the example shown in FIG. 2 , the selection object G11 is a radio button. The decision object G12 is an object for determining the object. When an operation to select the decision object G12 is performed, the object indicated by the selection object G11 is determined as the object that transmits a sensation when applied to the skin surface.
[0032] 3(A)-(D) and 4(A)-(B) are diagrams for explaining screen transitions of the sensation transmission screen G2 displayed on the touch panel 12. The sensation transmission screen G2 is a screen for transmitting to the user the sensation felt when the object is applied to the skin surface. The sensation transmission screen G2 is displayed when the decision object G12 is selected on the object selection screen G1.
[0033] 3A is a diagram showing an example of the sensory transmission screen G2 that is initially displayed. The sensory transmission screen G2 includes information G20 about the selected object, an image G21 of the object, an image G22 of a hand or finger, and a moving object G23.
[0034] The object information G20 includes information that allows the type of the selected object to be recognized. For example, the object information G20 includes an image of the object's packaging, the product name, etc. The object image G21 shows the object selected on the object selection screen G1. Because the object is a liquid, it is illustrated as a circle to show a droplet of the object dropped on the skin surface, as shown in FIG. 3(A). The object image G21 is an example of a second image.
[0035] The image G22 of fingers moves along a predetermined direction D in response to a user operation. The image G22 of fingers is positioned such that the fingertips are at a position a predetermined distance away from the image G21 of the object along the direction D. The image G22 of fingers is an example of a first image.
[0036] The moving object G23 is an object for moving the image G22 of fingers. In the example shown in FIG. 3A , the moving object G23 is a scroll bar and is arranged at approximately the same position as the fingers in the movement direction D. A user can move the moving object G23 along the direction D so as to follow the swipe operation by performing a swipe operation to displace a contact point with the touch panel 12 while touching the touch panel 12. When the moving object G23 moves along the direction D, the image G22 of the fingers moves along the direction D so as to follow the moving object G23. The swipe operation is an example of a movement operation. In the following description, the image G22 of the fingers follows the moving object G23 with a delay, but may follow the moving object G23 without a delay.
[0037] 3(B) is a diagram showing an example of the sensory transmission screen G2 in a state in which a swipe operation is accepted in the state shown in FIG. 3(A) and the moving object G23 is moving in a direction D1. For example, the image G22 of fingers moves so as to follow the moving object G23 with a delay. Therefore, when the moving object G23 is moving, the position of the moving object G23 in the direction D differs from the position of the fingers. In FIG. 3(B), the position of the fingers is separated from the position of the moving object G23 by a distance L1 in a direction D2 opposite to the direction D1, and the image G22 of the fingers is shown to follow the moving object G23 with a delay.
[0038] 3(C) is a diagram showing the sensory transmission screen G2 in a state in which the image G22 of the fingers has moved further from the state shown in FIG. 3(B) so that the image G22 of the fingers and the image G21 of the object overlap. When a liquid object comes into contact with the fingers, it deforms so as to spread in a direction along the surface of the fingers depending on the viscosity, surface tension, and the like of the object. Therefore, as shown in FIG. 3(C), when at least a portion of the image G22 of the fingers and the image G21 of the object overlap and the object comes into contact with the fingers, the image G21 of the object deforms so as to spread in a direction along the surface of the fingers. In the example shown in FIG. 3(C), the direction along the surface of the fingers is a direction perpendicular to the movement direction D.
[0039] 3(D) is a diagram showing an example of the sensory transmission screen G2 in a state after the image G22 of the fingers and the image G21 of the object overlap and then the image G22 of the fingers moves. When the object comes into contact with the fingers and the fingers move further, the object attached to the fingers spreads on the skin surface, forming a coating film of the object. In the example shown in FIG. 3(D), an image G24 of the coating film of the object is displayed in the opposite direction D2 of the direction D1 of the movement of the fingers relative to the image G22 of the fingers. Furthermore, because the object has been spread, the image G21 of the object located on the side of the movement direction D1 of the fingers becomes smaller relative to the image G22 of the fingers. The image G24 of the coating film is an example of a third image.
[0040] FIG. 4(A) is a diagram showing an example of the sensory transmission screen G2 after time has passed since the state shown in FIG. 3(D). The painted object evaporates. Therefore, as shown in FIG. 4(A), the opacity of the coating film image G24 decreases over time. Furthermore, the coating film image G24 becomes completely transparent over time. In other words, the coating film image G24 is erased after a predetermined time has passed. The example shown in FIG. 4(A) shows that the opacity of the coating film image G24 has decreased compared to FIG. 3(D). Furthermore, in the example shown in FIG. 4(A), the opacity increases toward the direction D1 in which the finger is moving. In other words, because the opacity decreases over time, the further the area displayed in the coating film image G24 is, the less opaque it becomes.
[0041] Fig. 4(B) is a diagram showing an example of the sensory transmission screen G2 in a state in which a swipe operation is accepted in the state shown in Fig. 4(A) and the moving object G23 is moving in the direction D2. As in the example shown in Fig. 3(D), in Fig. 4(B), an image G24 of the coating film of the object is displayed in the direction D1 opposite to the direction D2 in which the fingers are moving relative to the image G22 of the fingers. In addition, because the object has been spread, the image G21 of the object located on the side of the finger movement direction D2 has become even smaller relative to the image G22 of the fingers.
[0042] Some objects, such as lotions or emulsions, change their feel while being applied. For example, some objects change from a fresh feel to a thick feel while being applied. The feel of the object is expressed by the time the image G22 of the fingers lags behind the moving object G23. The longer the time the image G22 of the fingers lags behind the moving object G23, the more the user feels that the object is thick, sticky, sticky, heavy, etc. Furthermore, the shorter the time the image G22 of the fingers lags behind the moving object G23, the more the user feels that the object is fresh, smooth, silky, silky, light, etc.
[0043] In the example shown in Figure 4(B), the position of the fingers is separated from the position of the moving object G23 by a distance L2 in the direction D1 opposite to the direction D2. Distance L2 is longer than distance L1 shown in Figure 3(B). Therefore, Figure 4(B) shows that the delay between the image G22 of the fingers and the moving object G23 increases while the object is being applied. In other words, Figure 4(B) shows that the object has changed from a moist feel to a thick feel.
[0044] 5 is a diagram showing an example of the data structure of the object table T1 stored in the storage unit 11. The object table T1 stores parameters related to the display mode of the sensory transmission screen G2 for each of a plurality of objects. In the example shown in FIG. 5, the object table T1 stores parameters related to the droplets on the object, parameters related to the coating film on the object, and tracking parameters related to the tracking ability of the finger.
[0045] The droplet-related parameters include parameters related to the amount of deformation, the shrinkage speed, and the initial size of the droplet. The parameter related to the amount of deformation indicates the degree of deformation of the droplet when the finger comes into contact with the object, i.e., when the image G22 of the finger and the image G21 of the object overlap. The larger the parameter, the larger the deformation of the droplet, and the smaller the parameter, the smaller the deformation of the droplet. The parameter related to the shrinkage speed indicates how quickly the droplet located on the side of the image G22 of the finger in the direction of finger movement shrinks after the finger comes into contact with the object, i.e., after the image G22 of the finger and the image G21 of the object overlap. The larger the parameter, the faster the droplet shrinks, and the smaller the parameter, the slower the droplet shrinks. The parameter related to the initial size indicates the size of the droplet when the sensory transmission screen G2 is initially displayed. The larger the parameter, the larger the size of the droplet, and the smaller the parameter, the smaller the size of the droplet.
[0046] The droplet parameters are set in advance according to the properties of the target object. For example, the parameters for the deformation amount and shrinkage speed are set according to the viscosity and surface tension of the target object. The parameters for the initial size are set according to the amount of the target object.
[0047] The parameters related to the coating film include parameters related to the darkness, thickness, and volatilization resistance of the coating film. The parameter related to darkness indicates the magnitude of the difference between the color of the coating film and the background color. The larger the parameter, the greater the difference between the color of the coating film and the background color, and the smaller the parameter, the smaller the difference between the color of the coating film and the background color. The parameter related to thickness indicates the width of the image G24 of the coating film in a direction perpendicular to the direction of movement of the finger. The larger the parameter, the greater the width, and the smaller the parameter, the smaller the width. The parameter related to volatilization resistance indicates the period from when the image G24 of the coating film is displayed to when it is erased. The larger the parameter, the longer the period until the coating film is erased, and the smaller the parameter, the shorter the period until the coating film is erased. The parameter related to volatilization resistance is an example of a volatility parameter.
[0048] The parameters related to the coating film are set in advance according to the properties of the object. For example, the parameters related to the thickness are set according to the color and transparency of the object. The parameters related to the thickness are set according to the viscosity and amount of the object. The parameters related to the volatility resistance are set according to the volatility of the object.
[0049] The tracking parameter indicates the magnitude of delay of the image G22 of the fingers relative to the moving object G23. The larger the tracking parameter, the higher the tracking ability and therefore the shorter the delay, and the smaller the tracking parameter, the lower the tracking ability and therefore the longer the delay. In other words, the larger the tracking parameter, the smaller the difference between the moving speed of the image G22 of the fingers and the moving speed of the moving object G23, and the smaller the parameter, the larger the difference between the moving speed of the image of the fingers and the moving speed of the moving object. The tracking parameter is set depending on the viscosity, surface tension, etc. of the object.
[0050] The lower the tracking ability of the image G22 of the hand or finger, i.e., the greater the delay relative to the moving object G23, the more slowly the user tends to perform a swipe operation to prevent the image G22 of the hand or finger from lagging behind. When performing a swipe operation slowly, the user tends to press harder on the touch panel 12 compared to when performing a swipe operation quickly. The stronger the pressure on the touch panel 12, the greater the frictional force the user feels between the touch panel 12 and the object. Therefore, when the object feels supple and the tracking parameter is set small, the user feels a greater frictional force between the touch panel 12 and the object. Furthermore, when the object feels fresh and moist and the tracking parameter is set small, the user feels a smaller frictional force between the touch panel 12 and the object. In other words, by displaying the sensory transmission screen G2, the display terminal 1 can transmit the texture of the object not only visually but also through the sense of touch.
[0051] FIG. 6 is a graph showing the ratio Fy / Fz of the force Fy in the swipe direction applied to the user's finger from the surface of the touch panel 12 when the user performs a swipe operation to the normal force Fz. The horizontal axis of the graph shows the value of the tracking parameter. As shown in FIG. 6, the ratio Fy / Fz was large when the tracking parameter was 50 or 100, and the ratio Fy / Fz was small when the tracking parameter was 20 or 10. In other words, it was confirmed that the higher the tracking ability of the finger image G22, the smaller the normal force Fz, and the user felt a smaller frictional force. Furthermore, the p-value between the tracking parameter and the ratio Fy / Fz of the data shown in FIG. 6 was less than 0.05, confirming a significant correlation between the two.
[0052] That is, the tracking parameters are set according to the frictional force, etc., that the user's fingers performing the swipe operation receive from the touch panel, and assist in transmitting the tactile sensation of the object.
[0053] 7 is a diagram showing an example of the data structure of the update table T2 stored in the storage unit 11. The update table T2 stores, for each of a plurality of objects, information regarding changes in the feel of the object while the object is being applied. In the example shown in FIG. 7, the update table T2 stores parameter update conditions and update contents.
[0054] The parameter update conditions are information indicating the conditions for updating the parameters stored in the object table T1. The update conditions are, for example, a condition related to the time since the movement of the image G22 of the fingers started or a condition related to the movement history of the image G22 of the fingers. FIG. 7 shows that for cosmetic A, the parameters related to volatilization resistance and the tracking parameters are updated when 10 seconds have elapsed since the movement of the image of the fingers started. Also, for cosmetic B, it shows that the parameters are updated when the image G22 of the fingers makes one round trip. One round trip of the image of the fingers means that the image of the fingers starts moving, stops, starts moving in the opposite direction, and stops again. The condition related to the movement history may be a condition related to the distance moved since the movement of the image G22 of the fingers started.
[0055] The parameter update conditions and update contents are set in advance based on the change in the feel of the object while the object is being applied. For example, the update contents for an object that changes from a thick feel to a fresh feel while being applied are set so that the tracking parameter is updated from a small value to a large value and the parameter related to volatilization resistance is updated from a large value to a small value. Note that for objects whose feel does not change while being applied, the update conditions and update contents may not be set and the parameters may not be updated.
[0056] 8 is a flow diagram showing an example of the flow of the sensory transmission process executed by the processing unit 13. The sensory transmission process is a process for transmitting to the user, via an image, the sensation experienced when a target substance is applied to the skin surface. For example, the sensory transmission process is executed in response to the acceptance of an operation to launch a predetermined application program. The sensory transmission process is realized by the processing unit 13 working in cooperation with other components of the display terminal 1 based on a program stored in the memory unit 11.
[0057] First, the selection unit 131 selects an object that will transmit a sensation when applied to the skin surface in response to a user operation (step S101). For example, the selection unit 131 displays the object selection screen G1 on the touch panel 12 by supplying display data for the object selection screen to the display device of the touch panel 12. When the decision object G12 is selected, the selection unit 131 selects the object indicated by the selection object G11 as the object that will transmit a sensation when applied to the skin surface.
[0058] Next, the acquisition unit 132 acquires parameters related to the selected object (step S102). The acquisition unit 132 acquires the droplet parameters, the coating film parameters, and the tracking parameters associated with the selected object in the object table T1.
[0059] Next, the display control unit 133 displays the sensation transmission screen G2 on the touch panel 12 (step S103). For example, the display control unit 133 displays the sensation transmission screen G2 by supplying display data of the sensation transmission screen G2 to the display device of the touch panel 12.
[0060] Next, the display control unit 133 determines whether the screen update time has arrived (step S104). The screen update time is the time when the sensory transmission screen G2 is displayed or a predetermined time after the previous screen update time. The predetermined time is the frame update interval of the display device of the touch panel 12, and is, for example, 1 / 60 seconds.
[0061] If the screen update time has not arrived (step S104: No), the sensation transmission process returns to step S104. That is, the display control unit 133 waits until the screen update time arrives.
[0062] If the screen update time has arrived (step S104—Yes), the display control unit 133 executes a screen update process (step S105). The screen update process is a process for updating the sensory transmission screen G2. The details of the screen update process will be described later.
[0063] Next, the update unit 134 determines whether the update conditions are satisfied (step S106). The update unit 134 acquires the update conditions associated with the object displayed on the sensory transmission screen G2 from the update table T2. The update unit 134 determines whether the acquired update conditions are satisfied.
[0064] If the update condition is satisfied (step S106—Yes), the update unit 134 updates the parameters acquired in step S102 (step S107). The update unit 134 refers to the update table T2 and updates the parameters according to the update content associated with the satisfied update condition.
[0065] After step S107, or if the update condition is not satisfied in step S106 (No in step S106), the display control unit 133 determines whether a termination condition is satisfied (step S108). The termination condition may be, for example, that an operation to terminate a predetermined application program has been performed, or that a swipe operation has not been accepted for a predetermined period of time or more.
[0066] If the termination condition is not satisfied (step S108-No), the sensation transmission process returns to step S104, and the display control unit 133 waits until the next screen update time arrives.
[0067] If the termination condition is met (step S108-Yes), the sensation transmission process ends.
[0068] 9 is a flowchart showing an example of the flow of the screen update process, which is executed in step S105 of the sensory transmission process.
[0069] First, the reception unit 135 receives a swipe operation on the touch panel 12 (step S201). The reception unit 135 determines whether a swipe operation has been performed based on an operation signal supplied from an operation device of the touch panel 12. For example, the reception unit 135 determines whether a contact point is continuously detected based on the operation signal. If a contact point is continuously detected, the reception unit 135 acquires the amount of displacement of the most recently detected contact point along the movement direction D relative to the contact point detected immediately before that. If the magnitude of the acquired amount of displacement is equal to or greater than a threshold, the reception unit 135 determines that a swipe operation has been performed.
[0070] When the swipe operation is accepted (step S201—Yes), the display control unit 133 sets the movement speed of the moving object G23 (step S202). For example, the display control unit 133 sets the value obtained by dividing the displacement amount acquired in step S201 by the screen update interval as the movement speed of the moving object G23.
[0071] Next, the display control unit 133 sets the moving speed of the image G22 of the fingers (step S203). The display control unit 133 sets the moving speed of the image G22 of the fingers based on the positions and tracking parameters of the nearest moving object G23 and the image G22 of the fingers. For example, the display control unit 133 sets the moving speed vf of the image G22 of the fingers so that the image G22 follows the moving object G23 with a delay. The moving speed vf is set according to the following formula: Here, vobj is the movement speed of the nearest moving object G23, k is the tracking parameter, xf(n) is the position of the nearest finger image G22, xobj(n) is the position of the nearest moving object G22, and Δt is the interval between screen update times.
[0072] Next, the display control unit 133 determines whether the image G22 of the fingers and the image G21 of the object already overlap each other (step S204). For example, when a contact flag (described later) is set, the display control unit 133 determines that the image G22 of the fingers and the image G21 of the object already overlap each other.
[0073] If the image G22 of the fingers and the image G21 of the object already overlap (step S204—Yes), the display control unit 133 generates an image G24 of the coating film (step S205). The display control unit 133 specifies an area that is included in the image G22 of the nearest fingers but is not included in the image G22 of the fingers after the screen update as an area in which the image G24 of the coating film is to be displayed. The display control unit 133 generates an image G24 of the coating film having a width specified by a parameter related to the thickness of the coating film in the specified area.
[0074] Next, the display control unit 133 reduces the image G21 of the object (step S206). For example, the display control unit 133 reduces the image of the object located on the side of the image G22 of the finger in the direction of movement of the image G22 of the finger by a ratio based on a parameter related to the reduction speed of the droplet.
[0075] If the image G22 of the hand and the image G21 of the object do not yet overlap (step S204—No), the display control unit 133 determines whether the image G22 of the hand and the image G21 of the object overlap for the first time as a result of the currently executed screen update process (step S207). The display control unit 133 determines whether the area of the image G21 of the object and the area of the image G22 of the hand overlap after the screen update based on the positions of the image G21 of the object and the image G22 of the hand after the screen update and the shapes of the image G21 of the object and the image G22 of the hand. If there is overlap, the display control unit 133 determines that the image G22 of the hand and the image G21 of the object overlap for the first time as a result of the currently executed screen update process.
[0076] When the image G22 of the hand and the image G21 of the object overlap for the first time (step S207—Yes), the display control unit 133 deforms the image G21 of the object (step S208). For example, the display control unit 133 compresses the image G21 of the object along the movement direction D of the image G22 of the hand and the image G22 at a ratio based on a parameter related to the deformation amount of the droplet, and expands the image G21 of the object along a direction perpendicular to the movement direction D at a ratio based on a parameter related to the deformation amount.
[0077] If a swipe operation is not accepted in step S201 (step S201-No), if the image G22 of the finger and the image G21 of the object do not overlap in step S207 (step S207-No), or after step S206 or S208, the display control unit 133 determines whether the image G24 of the coating film is displayed on the sensory transmission screen G2 (step S209).
[0078] If the image G24 of the coating film is displayed (step S209—Yes), the display control unit 133 reduces the opacity of the image G24 of the coating film (step S210). The display control unit 133 calculates the transparency of the image G24 of the coating film after the screen update so that the opacity decreases as the time elapsed since the image G24 of the coating film is displayed increases.
[0079] Note that the display control unit 133 may maintain the opacity constant until a predetermined time has elapsed since the image G24 of the coating film is displayed, in order to prevent the image G24 of the coating film from being erased too quickly and becoming invisible to the user. The display control unit 133 may prevent the opacity from falling below a predetermined threshold until a predetermined time has elapsed since the image G24 of the coating film is displayed. The predetermined threshold is, for example, 0.2. The predetermined time may be a default value or may be set based on a parameter related to volatilization resistance. For example, the display control unit 133 calculates the opacity α of the image G24 of the coating film after the screen update according to the following formula: Here, αth is a predetermined threshold value for opacity, α0 is the initial value of opacity, x is a parameter related to the volatilization resistance of the coating film, and N is the number of screen update processes performed since the image G24 of the coating film was displayed.
[0080] If the coating film image G24 is not displayed in step S209 (step S209—No), or after step S210, the display control unit 133 updates the sensory transmission screen G2 (step S211). The display control unit 133 moves the moving object G23 and the finger image G22 at the movement speeds set in steps S202 and S203, respectively, arranges the coating film image G24 generated in step S205, arranges the object image generated in step S206 or S208, and generates image data for the sensory transmission screen G2 in which the opacity calculated in step S210 is set for the coating film image G24. The display control unit 133 updates the sensory transmission screen G2 by supplying the generated image data to the display device of the touch panel 12. This completes the screen update process.
[0081] As described above, the display control unit 133 of the display terminal 1 moves the moving object G23 to follow the swipe operation, and moves the image G22 of the finger to follow the moving object G23. This enables the display terminal 1 to convey to the user the sensation of touching an object via the image. In particular, the display terminal 1 enables the user to convey the feel of the object to the touch.
[0082] Furthermore, the display control unit 133 moves the image of the hand or finger so as to follow the moving object with a time delay based on the following parameters set for the object. This enables the display terminal 1 to convey to the user, via the image, the sensation felt when touching each of the multiple objects.
[0083] The object is a liquid, and the display control unit 133 deforms the image of the object based on the viscosity parameter when the image of the finger and the image of the object overlap, thereby enabling the display terminal 1 to convey to the user, via the image, the sensation of the object deforming when it touches the skin surface.
[0084] Furthermore, after the image G22 of the fingers and the image G23 of the object overlap, the display control unit 133 displays the image G24 of the coating film at the location where the image G22 of the fingers passed through. This enables the display terminal 1 to convey to the user, via the image, the sensation of spreading when the object is applied.
[0085] Furthermore, the display control unit 133 erases the image of the spread object when a time based on the volatility parameter has elapsed since the image G24 of the coating film was displayed, thereby enabling the display terminal 1 to convey to the user, via the image, the volatility sensation felt when the object is applied.
[0086] Furthermore, the update unit 134 updates the tracking parameters based on the time since the movement of the image of the hand and fingers started or the movement history of the image of the hand and fingers, and the display control unit 133 moves the image G22 of the hand and fingers so that it follows the moving object G23 with a time delay based on the updated tracking parameters. This enables the display terminal 1 to convey to the user, via the image, changes in the feel of the object while it is being applied.
[0087] The following modifications may be applied to the display terminal 1.
[0088] In the above-described embodiment, the object is a liquid. However, the object is not limited to this example, and may be a slurry such as foundation. In this case, the display terminal 1 can also convey the sensation of touching the object to the user via an image. When the object is a slurry, the display control unit 133 does not need to reduce the opacity of the image G24 of the coating film. The object may also be lipstick, eyeliner, eyebrow pencil, etc.
[0089] The object may also be a solid object such as a cosmetic compact case. The object may also be a container for lotion, emulsion, serum, facial cleanser, etc. The container may be a bottle or a tube.
[0090] 10(A)-(B) are diagrams illustrating an example of screen transitions on the sensory transmission screen G3 when the object is a solid. In the example shown in FIG. 10(A), the object is a cosmetic compact case. The sensory transmission screen G3 when the object is a solid is a screen for transmitting the tactile sensation of the object's surface to the user. As shown in FIG. 10(A), when the object is a solid, the sensory transmission screen G3 includes an image G31 of the object, an image G32 of fingers, and a moving object G33, just as when the object is a liquid or slurry. As shown in FIG. 10(B), the moving object G33 moves to follow the swipe operation, and the image G32 of the fingers moves to follow the moving object G33 with a delay.
[0091] If the object is solid, it will not be deformed even if it comes into contact with a finger. Therefore, as shown in Figures 10(A)-(B), the image G31 of the object will not be deformed even if it comes into contact with a finger. Also, if the object is solid, the image of the coating film will not be displayed.
[0092] As described above, the display control unit 133 of the display terminal 1 moves the moving object G33 to follow the swipe operation, and also moves the image G32 of the finger to follow the moving object G33 after a predetermined time delay. This allows the display terminal 1 to convey to the user the sensation of touching an object via the image. In particular, the display terminal 1 allows the user to convey the tactile sensation of a solid object.
[0093] In the above-described embodiment, the sensory transmission screen G2 includes the image G22 of fingers, and the image G24 of the applied film is displayed at the location where the image G22 of the fingers passed only after the image G22 of the fingers and the image G21 of the object overlap. However, the sensory transmission screen G2 may include, as an example of the first image, an image of lipstick or an image of a puff for applying foundation instead of the image G22 of fingers.
[0094] 11A and 11B are diagrams showing an example of the screen appearance of the sensory transmission screen G4 when a lipstick image is displayed instead of an image of fingers. As shown in Fig. 11A, the sensory transmission screen G4 includes a lipstick image G41, a moving object G42, and an image G43 of lips to be applied. The moving object G42 moves to follow the swipe operation, and the lipstick image G41 moves to follow the moving object G42.
[0095] 11(B), the display control unit 133 displays an image G44 of an applied film of lipstick, which shows the spread lipstick, in the area through which the lipstick image G41 passed. That is, in the example shown in Fig. 3, the image of the applied film was displayed only after the image of the object and the image of the finger overlapped, but in the example shown in Fig. 11(B), the image of the applied film is displayed regardless of whether the image of the object and the image of the finger overlap.
[0096] 11A and 11B show the action of applying lipstick to lips. However, the present invention is not limited to this example, and the action of applying skin care products to arms or face may also be displayed. Furthermore, the images of the lips, arms, and face to be applied may be illustrations or photographs.
[0097] 12 is a diagram showing an example of a sensory transmission screen G5 that includes an image of a puff applying foundation. The sensory transmission screen G5 includes a puff image G51 and a moving object G52. The moving object G52 moves to follow the swipe operation, and the puff image G51 moves to follow the moving object G52. The display control unit 133 displays an image of an applied film that shows the foundation spread in the area through which the puff image G51 has passed. As shown in FIG. 12, the moving direction of the moving object G52 and the puff image G51 may be vertical.
[0098] (Description of Another Embodiment) In the above-described embodiment, the display terminal 1 has the touch panel 12 that includes an integrated display device and operation device. However, the present invention is not limited to this example, and the display terminal 1 may have a display unit and an operation unit that is separate from the display unit. The operation unit is, for example, a mouse, a keyboard, or a keypad. In this case, the reception unit 135 receives a drag operation, a key input operation, or the like as an operation for moving the moving object G23.
[0099] In the above-described embodiment, the display terminal 1 accepts user operations on the display terminal 1 via the touch panel 12. This example is not limiting, and the display terminal 1 may accept user operations on the display terminal 1 via a gesture input system or a gesture recognition system. The gesture input system or the gesture recognition system acquires, for example, information indicating the shape or movement of the user's body detected by a spatial sensor such as an imaging device or a distance measuring device, or output from an acceleration sensor worn on the user's body. The gesture input system or the gesture recognition system recognizes a user's gesture based on the information or output acquired from the above sensor, and accepts the user's operation on the display terminal 1 based on the gesture.
[0100] In the above-described embodiment, the object table T1 stores parameters for each of a plurality of objects. However, the present invention is not limited to this example, and the object table T1 may store parameters for only one object. In this case, step S101 of the sensory transmission process may be omitted, and an object may not be selected.
[0101] In the above-described embodiment, the processing unit 13 generates a sensory transmission screen G2 that includes information G20 of a selected object, an image G21 of the object, an image G22 of a hand, and a moving object G23. This example is not limiting, and the processing unit 13 may generate a sensory transmission screen G2 that includes the image G21 of the object but does not include at least one of the information G20 of the object, the image G22 of the hand, and the moving object G23. The processing unit 13 can provide the user with various augmented reality (AR) experiences or virtual reality (VR) experiences by controlling at least one of (a) the presence or absence of various images or objects arranged on the sensory transmission screen G2 and (b) the procedure for generating the images or objects.
[0102] In the above-described embodiment, the sensory transmission screen G2 includes the moving object G23. However, this is not a limitation, and the sensory transmission screen G2 may not include the moving object G23. In this case, the image G22 of the hand and fingers moves to follow the user's input operation. The image G22 of the hand and fingers may follow the input operation without delay, or may follow the input operation with a delay. Examples of the input operation include a swipe operation, a drag operation, a tap operation, and a click operation. As described above, the input operation may be input via a touch panel, a mouse, a keyboard, or a keypad, or may be input via a gesture input system or a gesture recognition system.
[0103] In one embodiment, the position of the image G22 of the fingers on the sensory transmission screen G2 is determined, for example, so that a predetermined position or region of the image G22 of the fingers is located at an input position or input coordinates (sometimes referred to as an indicated position) on the screen indicated by a user's input operation. In another embodiment, the position of the image G22 of the fingers on the sensory transmission screen G2 is determined, for example, so that the predetermined position or region of the image G22 of the fingers is located at a position offset by a predetermined distance or relative position from the indicated position.
[0104] In yet another embodiment, the position of the image G22 of the fingers on the sensory transmission screen G2 is determined, for example, so that a predetermined position or area of the image G22 of the fingers changes within a predetermined area on the sensory transmission screen G2 in a manner similar to the change in the indicated position described above. In this case, the predetermined position or area of the image G22 of the fingers may change with a delay relative to the change in the indicated position. In yet another embodiment, the position of the image G22 of the fingers on the sensory transmission screen G2 is determined, for example, so that the movement of the predetermined position or area of the image G22 of the fingers follows the movement of the indicated position described above. As described above, the degree of tracking or delay can be set arbitrarily. The tactile sensation of the object can be reproduced depending on the degree of tracking or delay.
[0105] In the above-described embodiment, the sensory transmission screen G2 includes an image G21 of an object placed on the skin surface, an image G22 of fingers, and a moving object G23. The sensory transmission screen G2 is not limited to this example, and may include an image or object of an object (sometimes referred to as a support) on which the object is placed, such as the skin in the above example. The procedure for generating the image or object of the object is not particularly limited, and the image or object may be generated using an image acquired by a spatial sensor such as an imaging device or a distance measuring device, or may be generated by computer graphics.
[0106] In the above-described embodiment, the sensory transmission screen G2 includes an image G21 of an object placed on the skin surface, an image G22 of fingers, and a moving object G23. This example is not limited to this example, and the sensory transmission screen G2 does not need to include the image G22 of fingers. For example, when the sensory transmission screen G2 is displayed on the touch panel 12 and a user inputs the above-described designated position by moving the user's fingers on the touch panel 12, the sensory transmission screen G2 is generated so that the shape, pattern, or color of the object shown in the image G21 changes in accordance with the movement of the user's fingers. In this case, the shape, pattern, and / or color of the object shown in the image G21 may change with a delay relative to the change in the designated position. Depending on the degree of the delay, the tactile sensation of the object may be reproduced.
[0107] In the above-described embodiment, the parameters stored in the object table T1 are set based on the properties or texture of the object. This example is not limiting, and these parameters may be set based on the user's skin condition. For example, if the user has dry skin, the object will penetrate quickly and feel moisturized. Therefore, the tracking parameters may be set large. On the other hand, if the user has oily skin, the object will feel sticky. Therefore, the tracking parameters may be set small.
[0108] Furthermore, when the sensory transmission process is executed multiple times, the parameters related to the object may be set based on whether the sensory transmission process was executed immediately before or the type of object selected in the sensory transmission process executed immediately before. For example, the properties and tactile sensation of a lotion change when a user applies lotion to bare skin compared to when a user applies lotion over a skin toner. Therefore, the parameters related to the object may differ depending on whether the sensory transmission process was executed immediately before and another object was selected.
[0109] In the above-described embodiment, in the screen update process, the display control unit 133 sets the movement speed of the image G22 of the fingers based on the tracking parameters and moves the image G22 of the fingers at the set movement speed. This example is not limited to this example. The display control unit 133 may also set a delay time for the image G22 of the fingers based on the tracking parameters. In this case, the display control unit 133 moves the image G22 of the fingers so that it follows the moving object G23 with a delay of the set delay time. That is, the display control unit 133 places the image G22 of the fingers at a position along the direction D of the moving object G23 at a time just before the delay time. In this manner, the display terminal 1 can still convey the sensation of touching an object to the user via an image.
[0110] In the above-described embodiment, in step S208 of the screen update process, the display control unit 133 deforms the image of the object when the image G22 of the fingers and the image G21 of the object overlap. The present invention is not limited to this example. Alternatively, the display control unit 133 may move the image G21 of the object together with the image G22 of the fingers without deforming it when the image G22 of the finger and the image G21 of the object overlap. Furthermore, the display control unit 133 may erase the image G21 of the object when the image G22 of the finger and the image G21 of the object overlap.
[0111] In the above-described embodiment, in step S205 of the screen update process, the display control unit 133 displays the image G24 of the coating film at a location where the image G22 of the fingers passed after the image G22 of the finger and the image G21 of the object overlap. The present invention is not limited to this example, and the display control unit 133 does not necessarily have to display the image G24 of the coating film.
[0112] In the above-described embodiment, in the screen update process, the display control unit 133 erases the image of the coating film when a predetermined time has elapsed since the image G24 of the coating film was displayed. The present invention is not limited to this example, and the display control unit 133 does not have to erase the displayed image G24 of the coating film. For example, the display control unit 133 may prevent the opacity of the image G24 of the coating film from falling below a predetermined threshold regardless of the elapsed time since the image G24 of the coating film was displayed. Furthermore, the display control unit 133 may maintain the opacity of the image G24 of the coating film constant without decreasing it depending on the elapsed time since the image G24 of the coating film was displayed.
[0113] 13 is an exemplary functional block diagram of a display system 2 according to another embodiment. The display system 2 has a display terminal 3 and a server 4 that communicate with each other via a network N. The display terminal 3 differs from the display terminal 1 in that it further has a communication unit 34 and that it has a receiving unit 334 and a transmitting unit 335 as functional blocks of the processing unit 13. The other components of the display terminal 3 are the same as the corresponding components of the display terminal 1, so the same reference numerals are used and descriptions thereof will be omitted.
[0114] The communication unit 34 is a component that enables the display terminal 3 to communicate with the server 4, and includes a communication interface circuit. The communication interface circuit included in the communication unit 34 is a communication interface circuit for a wireless LAN (Local Area Network) or a wired LAN. The communication unit 34 transmits data supplied from the processing unit 13 to the server 4, and also supplies data received from the server 4 to the processing unit 13.
[0115] The server 4 includes a storage unit 41 , a communication unit 42 , and a processing unit 43 .
[0116] The storage unit 41 is configured to store programs and data, and includes, for example, a semiconductor memory. The programs stored in the storage unit 41 include an operating system program, driver programs, and application programs used in processing by the processing unit 43. The programs are installed into the storage unit 41 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM or DVD-ROM. The programs may also be stored in a storage medium owned by the display terminal 3 or another server, and installed via a network.
[0117] The communication unit 42 is a component that enables the server 4 to communicate with the display terminal 3, and includes a communication interface circuit. The communication interface circuit included in the communication unit 42 is a communication interface circuit for a wireless LAN, a wired LAN, or the like. The communication unit 42 transmits data supplied from the processing unit 43 to the display terminal 3, and also supplies data received from the display terminal 3 to the processing unit 43.
[0118] The processing unit 43 is configured to comprehensively control the operation of the server 4 and includes one or more processors and their peripheral circuits. For example, the processing unit 43 includes a CPU. The processing unit 43 may also include a GPU, a DSP, an FPGA, an ASIC, or the like. The processing unit 43 executes various processes based on the programs stored in the storage unit 41.
[0119] The processing unit 43 includes, as functional blocks, a selection unit 431, an acquisition unit 432, a display control unit 433, an update unit 434, and a reception unit 435. Each of these units is a functional module realized by a program executed by the processing unit 43. Each of these units may be implemented in the server 4 as a dedicated arithmetic circuit.
[0120] The selection unit 431, acquisition unit 432, display control unit 433, update unit 434 and reception unit 435 perform sensory transmission processing in the same manner as the selection unit 131, acquisition unit 132, display control unit 133, update unit 134 and reception unit 135 of the display terminal 1.
[0121] At this time, instead of supplying the display data of the object selection screen G1 to the touch panel 12 in step S101 of the sensory transmission processing, the selection unit 431 transmits the display data to the display terminal 3 via the communication unit 42. The receiving unit 334 of the display terminal 3 supplies the received display data to the touch panel 12. In this way, the selection unit 431 displays the object selection screen G1 on the touch panel 12. Furthermore, the transmitting unit 335 of the display terminal 3 transmits information about the object indicated by the selection object G11 to the server 4 via the communication unit 34 when the decision object G12 is selected.
[0122] Furthermore, in step S103 of the sensory transmission processing and step S211 of the screen update processing, the display control unit 433 transmits the display data of the sensory transmission screen G2 to the display terminal 3 via the communication unit 42, instead of supplying the display data to the touch panel 12. The receiving unit 334 of the display terminal 3 supplies the received display data to the touch panel 12. In this way, the selection unit 431 displays the sensory transmission screen G2 on the touch panel 12.
[0123] In addition, in step S201 of the screen update process, instead of acquiring the operation signal from the touch panel 12, the reception unit 435 receives information about the operation signal transmitted by the transmission unit 335 of the display terminal 3 via the communication unit 42.
[0124] As described above, the display system 2, like the display terminal 1, makes it possible to convey to the user, via an image, the sensation felt when touching an object.
[0125] The various embodiments and modifications described above may be combined as appropriate and implemented. Furthermore, the various processes described above may be executed in a different order as appropriate.
[0126] REFERENCE SIGNS LIST 1 display terminal 11 storage unit 12 touch panel 131 selection unit 132 acquisition unit 133 display control unit 134 update unit 135 reception unit
Claims
1. A display terminal comprising: a display unit; a display control unit that displays a screen on the display unit including a first image and a moving object for moving the first image; and a reception unit that receives a movement operation for moving the first image, wherein the display control unit moves the first image to follow the moving object in response to the movement operation.
2. The display terminal according to claim 1, wherein the screen further includes a second image arranged at a position where it overlaps with the first image by moving the first image.
3. The display terminal of claim 2, further comprising an acquisition unit that acquires tracking parameters that are set according to the object shown in the second image, and the display control unit moves the first image so as to track the moving object based on the tracking parameters.
4. The display terminal according to claim 3, wherein the display control unit sets a moving speed of the first image so as to lag behind the moving object based on the tracking parameters, and moves the first image at the set moving speed.
5. The display terminal according to claim 3, wherein the display control unit sets a delay time for the first image based on the tracking parameters, and moves the first image so as to track the moving object with a delay of the set delay time.
6. The display terminal of claim 3, wherein the object is a liquid or a slurry, the acquisition unit acquires a viscosity parameter set according to the viscosity of the object, and the display control unit deforms the second image based on the viscosity parameter when the first image and the second image overlap.
7. The display terminal according to claim 3, wherein the object is a solid, and the tracking parameters are set according to the tactile feel of the object.
8. A display terminal as described in claim 3, further comprising an update unit that updates the tracking parameters based on the time since movement of the first image began or based on the movement history of the first image, and the display control unit moves the first image so as to follow the moving object with a delay based on the updated tracking parameters.
9. The display terminal according to claim 1, wherein the display control unit displays a third image showing a coating film at a location where the first image has passed.
10. The display terminal of claim 9, wherein the screen further includes a second image arranged at a position where it overlaps with the first image by moving the first image, and the display control unit displays the third image after the first image and the second image overlap.
11. A display terminal as described in claim 10, further comprising an acquisition unit that acquires a volatility parameter that is set according to the volatility of the object represented by the second image, and the display control unit reduces the opacity of the third image based on the elapsed time since the third image was displayed and the volatility parameter.
12. The display terminal according to claim 1, wherein the first image is an image of a finger.
13. The display terminal according to claim 1, wherein the display unit is a touch panel, and the movement operation is a swipe operation on the touch panel.
14. A display method executed by a display terminal having a display unit, comprising: displaying on the display unit a screen including a first image and a moving object for moving the first image; accepting a movement operation for moving the first image; and moving the first image to follow the moving object in response to the movement operation.
15. A program for a computer having a display unit, the program causing the computer to execute the following: displaying on the display unit a screen including a first image and a moving object for moving the first image; accepting a movement operation for moving the first image; and moving the first image to follow the moving object in response to the movement operation.
16. A server that communicates with a display terminal having a display unit, the server having: a display control unit that displays on the display unit a screen including a first image and a moving object for moving the first image; and a reception unit that receives a movement operation for moving the first image, the display control unit moving the first image to follow the moving object in response to the movement operation.
Citation Information
Patent Citations
Electronic game device, method therefor and storage medium
JP2002000939A
Control system and control method
JP2009070416A
Game system, control method, and program
JP2017029577A
Game program, game device, and game control method
JP2022087933A
