Information processing device and storage medium

Abstract

An information processing device and a storage medium are provided. The information processing device has a processor. The processor generates an object in a virtual space corresponding to an object in a real space, associates an associated object in a virtual space displayed corresponding to the object with at least a part of the object, and moves the associated object while maintaining a relative positional relationship between the associated object and the object when movement of the object is detected in the virtual space.

Description

Technical Field

[0001] The present invention relates to an information processing device and a storage medium storing an information processing program. Background Art

[0002] Patent document 1 discloses the following display device, which is a transmissive display device that allows a user to visually confirm an image as a virtual image, and is characterized in that it has: an object detection unit that detects the coordinate position and shape of a real object in the real space; a display control unit that displays an image object with a set coordinate position in the real space so that visual confirmation can be performed corresponding to the coordinate system of the real space; and a display unit that displays the image object, and the display control unit controls the image object according to the coordinate position and shape of the real object.

[0003] Patent document 2 discloses the following information processing device, which outputs a composite image obtained by synthesizing a real image representing a real space and a virtual object image in a virtual space to a display device for visual confirmation by a user, and is characterized in that it has: a real object position determination unit, which determines the position of a real object in the real space; a movement determination unit, which determines whether the real object has moved based on information on the position of the real object determined by the real object position determination unit; and an output unit, which outputs a composite image obtained by synthesizing the real image and the virtual object image to be displayed on the display device when the movement determination unit determines that the real object has moved, so as to be able to identify whether there is a real object.

[0004] Patent document 3 discloses the following information processing device, which comprises: a virtual space identification unit, which analyzes the three-dimensional space structure of the real space to identify the virtual space and obtain the position information of the device in the virtual space; a storage unit, which stores the objects configured in the virtual space and the position information of the objects; a display unit, which displays the objects configured in the virtual space on a display device; a detection unit, which detects the device information of the display device; and an execution unit, which performs prescribed processing on the object according to the device information.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2015-026286

[0008] Patent Document 2: Japanese Patent Application Publication No. 2016-122392

[0009] Patent Document 3: Japanese Patent Application Publication No. 2011-186536 Summary of the invention

[0010] Problem that the invention aims to solve

[0011] In an information processing device that generates mixed reality, a setting is required for displaying an associated object in the virtual space (e.g., an error / message) that should be displayed in association with an object (e.g., a digital printing press) when the object moves in the real space, at a position in the virtual space corresponding to the position of the object after the movement in the real space.

[0012] One non-limiting embodiment of the present disclosure provides an information processing device and a storage medium storing an information processing program, which can reduce the setting work when displaying an associated object that should be displayed in association with the object at a position in the virtual space corresponding to the position of the object after moving in the real space, even when the object moves in the real space.

[0013] Means used to solve problems

[0014] [1] According to one embodiment of the present disclosure, there is provided an information processing device, the information processing device having a processor,

[0015] The processor performs the following processing: generates an object in a virtual space corresponding to an object in a real space; associates an associated object in the virtual space displayed corresponding to the object with at least a portion of the object; and when movement of the object is detected in the virtual space, moves the associated object while maintaining a relative positional relationship between the associated object and the object.

[0016] [2] Alternatively, in the information processing device of [1], when the processor detects that the object in the real space has moved, the processor configures the position of the object in the virtual space to the position of the object in the real space after the movement.

[0017] [3] Alternatively, in the information processing device of [1] or [2], the associated object is an arrow object indicating a direction from a starting point toward an end point, and the processor determines the position of the starting point of the arrow object based on the position of the object in the virtual space, and the processor determines the position of the end point of the arrow object based on the positions of other object objects in the virtual space corresponding to other objects in the real space.

[0018] [4] Alternatively, in the information processing device of [1] or [2], the processor determines the relationship between the object object in the virtual space and the associated object based on a processing device that serves as the object in the real space and a processing step performed by the processing device in the real space.

[0019] [5] Alternatively, in the information processing device of [4], the associated object is an arrow object indicating a direction from a starting point toward an end point, and the processor determines the position of the starting point of the arrow object based on attribute information of a target object in the virtual space corresponding to a target object in the real space for the processing device to perform the processing step, and the processor determines the position of the end point of the arrow object based on the position of other object objects in the virtual space corresponding to other objects that perform other processing steps in the real space that may be performed subsequent to the processing step.

[0020] [6] Alternatively, in the information processing device of [5], the processor performs the following processing: determining the other processing steps based on the processing steps performed by the object on the target object in the real space; determining the position of the starting point of the arrow object based on the position of the target object in the virtual space; and determining the position of the end point of the arrow object based on the positions of other object objects in the virtual space corresponding to other objects in the real space that perform the other processing steps determined.

[0021] [7] Alternatively, in the information processing device of [5], the processor enters the following processing: in response to the processing step performed by the object on the target object in the real space, accepting other processing steps from the user; determining the position of the starting point of the arrow object based on the position of the target object in the virtual space; and determining the position of the end point of the arrow object based on the positions of other object objects in the virtual space corresponding to other objects in the real space that perform the accepted other processing steps.

[0022] [8] Alternatively, in the information processing device of [1] or [2], the processor determines the position of the associated object based on information possessed by the object object related to the movement path of a target object for the object to perform a processing step in the real space.

[0023] [9] Alternatively, in the information processing device of [8], the associated object is a dynamic display object of the moving path in the virtual space corresponding to the dynamic display of the moving path of the target object in the real space, and the processor determines the corresponding position of the starting point and the end point of the dynamic display object of the moving path based on the position of one of the starting point and the end point of the moving path of the target object indicated by information related to the moving path of the target object.

[0024]

[10] According to another embodiment of the present disclosure, a storage medium is provided, which stores an information processing program for causing a computer to perform the following processing: generating an object in a virtual space corresponding to an object in a real space; associating an associated object in the virtual space displayed corresponding to the object with at least a portion of the object; and when movement of the object is detected in the virtual space, moving the associated object while maintaining a relative positional relationship between the associated object and the object.

[0025] Effects of the Invention

[0026] According to the information processing device of [1] and the storage medium of

[10] , it is possible to reduce the setting work when displaying the associated object that should be associated with the object at a position in the virtual space corresponding to the position of the object after moving in the real space even when the object moves in the real space.

[0027] According to the information processing device of [2], the position of the object for determining the position of the related object to be displayed in association with the object can be linked to the movement of the object.

[0028] According to the information processing device of [3], the position of the start point and the end point of the arrow object can be determined without obtaining the position of the object in the real space and the position of other objects.

[0029] According to the information processing device of [4], it is possible to determine the relationship between the associated object and the physical object in the virtual space without being affected by the situation in the virtual space.

[0030] According to the information processing device of [5], the direction indicated by the arrow object can reflect the attribute information of the target object and the positions of other possible objects.

[0031] According to the information processing device of [6], the direction indicated by the arrow object can reflect the position of the target object and the positions of other determined objects.

[0032] According to the information processing device of [7], the direction indicated by the arrow object can reflect the position of the target object and the positions of other objects specified by the user.

[0033] According to the information processing device of [8], the position of the related object can be set in a manner along the movement path of the target object.

[0034] According to the information processing device of [9], it is possible to make either the starting point or the end point of the dynamic display object coincide with the corresponding one of the starting point or the end point of the movement path of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The configuration of an information processing device according to the first exemplary embodiment is shown.

[0036] Figure 2 This is a functional block diagram of the information processing device according to the first exemplary embodiment.

[0037] Figure 3 An example of the real space according to the first exemplary embodiment is shown.

[0038] Figure 4 An example of a virtual space according to the first exemplary embodiment is shown.

[0039] Figure 5 An example of a mixing space according to the first exemplary embodiment is shown.

[0040] Figure 6 This is a flowchart showing the operation of the information processing device according to the first exemplary embodiment.

[0041] Figure 7 An example of the real space (before movement) according to the first exemplary embodiment is shown.

[0042] Figure 8 An example of the real space (after movement) according to the first exemplary embodiment is shown.

[0043] Fig. 9 An example of the virtual space (before movement) according to the first exemplary embodiment is shown.

[0044] Fig.10 An example of the virtual space (after movement) according to the first exemplary embodiment is shown.

[0045] Fig.11 An example of the mixing space (after movement) according to the first exemplary embodiment is shown.

[0046] Fig.12 An example of the real space according to the second exemplary embodiment is shown.

[0047] Fig.13 An example of a virtual space according to the second exemplary embodiment is shown.

[0048] Fig.14 An example of a mixed space according to the second exemplary embodiment is shown.

[0049] Fig.15 This is a flowchart showing the operation of the information processing device according to the second exemplary embodiment.

[0050] Fig.16 An example of the real space (before movement) according to the second exemplary embodiment is shown.

[0051] Fig.17 An example of the real space (after movement) according to the second exemplary embodiment is shown.

[0052] Fig.18 An example of a virtual space (before movement) according to the second exemplary embodiment is shown.

[0053] Fig.19 An example of the virtual space (after movement) according to the second exemplary embodiment is shown.

[0054] Fig. 20 An example of the mixing space (after movement) according to the second exemplary embodiment is shown.

[0055] Fig.21 This is a functional block diagram of an information processing device according to the third exemplary embodiment.

[0056] Fig. 22 A table stored in the storage medium according to the third exemplary embodiment is shown.

[0057] Fig.23 This is a flowchart showing the operation of the information processing device according to the third exemplary embodiment.

[0058] Fig.24 An example of the real space (before processing) according to the third exemplary embodiment is shown.

[0059] Fig.25 An example of the real space (after processing) according to the third exemplary embodiment is shown.

[0060] Fig.26 An example of a virtual space (before processing) according to the third exemplary embodiment is shown.

[0061] Fig. 27 An example of a virtual space (first half after processing) according to the third exemplary embodiment is shown.

[0062] Fig.28 An example of a virtual space (second half after processing) according to the third exemplary embodiment is shown.

[0063] Fig.29 An example of a mixed space (after processing) according to the third exemplary embodiment is shown.

[0064] Fig.30 This is a flowchart showing the operation of the information processing device according to the fourth exemplary embodiment.

[0065] Fig.31 An example of a virtual space (after processing) according to the fourth exemplary embodiment is shown.

[0066] Fig.32An example of the mixed space (the first half after processing) according to the fourth exemplary embodiment is shown.

[0067] Fig.33 An example of the mixed space (the second half after processing) according to the fourth exemplary embodiment is shown.

[0068] Fig.34 An example of a virtual space (after selection) according to the fourth exemplary embodiment is shown.

[0069] Fig.35 An example of a mixed space (after selection) according to the fourth exemplary embodiment is shown.

[0070] Fig.36 This is a flowchart showing the operation of the information processing device according to the fifth exemplary embodiment.

[0071] Fig.37 An example of the real space according to the fifth exemplary embodiment is shown.

[0072] Fig.38 An example of a virtual space according to the fifth exemplary embodiment is shown.

[0073] Fig.39 An example of a mixed space according to the fifth exemplary embodiment is shown. DETAILED DESCRIPTION

[0074] <First exemplary embodiment>

[0075] Hereinafter, a first exemplary embodiment of the information processing device of the present invention will be described.

[0076] <Structure of the first exemplary embodiment>

[0077] Figure 1 The structure of the information processing device JS of the first exemplary embodiment is shown. The information processing device JS of the exemplary embodiment is, for example, a MR (Mixed Reality) device which is a type of head mounted display. The information processing device JS converts an object in the real space GK, such as a digital printing machine RT ( Figure 3 ), and an object object (object) corresponding to the object in the virtual space KK, such as a digital printing machine object RTobj ( Figure 4 Thus, the information processing device JS generates a mixed space FK ( Figure 5 In order to generate the mixed space FK, the information processing device JS Figure 1 As shown, it includes an input unit 1, a CPU 2 (Central Processing Unit), an output unit 3, a storage medium 4, and a memory 5.

[0078] The input unit 1 is composed of, for example, an image sensor, a camera, or a touch panel. The CPU 2 is an example of a processor, and is the core of a well-known computer that operates hardware according to software. The output unit 3 is composed of, for example, a transmissive display. The storage medium 4 is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), or a ROM (Read Only Memory). The memory 5 is composed of, for example, a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory).

[0079] Storage medium 4 such as Figure 1 As shown, a program PR is stored. The program PR is a command group that defines the content of a process to be executed by the information processing device JS.

[0080] Figure 2 1 is a functional block diagram of the information processing device JS according to the embodiment of this example. Figure 2 As shown, the information processing device JS of the embodiment shown in this example includes an acquisition unit 11, a detection unit 12, a generation unit 13, an update unit 14, a mixing unit 15, a control unit 16, and a storage unit 17.

[0081] Regarding the relationship between the hardware structure and the functional structure in the information processing device JS, in terms of hardware, the CPU 2 uses the memory 5 (implementing the functions of another part of the storage unit 17) and executes the program PR stored in the storage medium 4 (implementing the functions of a part of the storage unit 17), and controls the operation of the input unit 1 and the output unit 3 as the control unit 16 as needed, thereby realizing the functions of each unit of the acquisition unit 11, the detection unit 12, the generation unit 13, the update unit 14, and the mixing unit 15. The functions of each unit will be described later.

[0082] Regarding the realization of each unit by the CPU 2, it may be realized by, for example, an external server (not shown) instead of being realized in the information processing device JS.

[0083] Figure 3 An example of the real space GK according to the present exemplary embodiment is shown. Figure 4 An example of a virtual space KK according to the present exemplary embodiment is shown. Figure 5 An example of the mixed space FK according to the present exemplary embodiment is shown.

[0084] The real space GK is the actual space. In the real space GK, Figure 3 As shown, for example, there are a digital printing press RT and a plurality of paper rolls RS1 to RS3 as objects.

[0085] The virtual space KK is a virtual space generated by the information processing device JS. In the virtual space KK, Figure 4 As shown, there is a virtual object generated by the information processing device JS corresponding to the digital printer RT in the real space GK, namely, the digital printer object RTobj as an object object, and an associated object as a virtual object, namely, the message / object MGobj, which should be displayed in association with the digital printer RT in the real space GK.

[0086] As described later, the digital printer object RTobj is used by the information processing device JS to calculate the coordinate position of the message / object MGobj, so there is no need to enable the user of the information processing device JS to visually confirm the digital printer object RTobj, and the digital printer object RTobj is not visually confirmed.

[0087] On the other hand, it is desirable that the message / object MGobj can be visually confirmed for operation by the user of the information processing device JS. Moreover, the coordinate position of the message / object MGobj in the virtual space KK is determined based on the relative relationship with the coordinate position of the digital printer object RTobj in the virtual space KK.

[0088] For example, assuming that the coordinate position of the digital printer object RTobj in the virtual space KK is (xa, yb, zc) (not shown), the coordinate position of the message / object MGobj in the virtual space KK is determined by (xa, yb+m, zc). Here, m is an arbitrary value. In the following description, it is assumed that m=5.

[0089] In the following description, an object whose outline is drawn with a “dotted line” means that it is “not visually recognizable”.

[0090] The information processing device JS processes the digital printing press RT and the plurality of paper rolls RS1 to RS3 ( Figure 3 ), and the digital printing machine object RTobj and message / object MGobj in the virtual space KK ( Figure 4 ), i.e., they are made to coincide, thereby generating a hybrid space FK ( Figure 5 ). The hybrid space FK is a space recognized by the user through the output unit 3, i.e., the transmissive display. In the generated hybrid space FK, there are the digital printer RT, the plurality of paper rolls RS1 to RS3, and the message / object MGobj that can be visually recognized by the user of the information processing device JS. That is, in the hybrid space FK, the digital printer object RTobj is not visually recognized by the user of the information processing device JS.

[0091] <Operation of the First Exemplary Embodiment>

[0092] Figure 6 1 is a flowchart showing the operation of the information processing device JS according to the present exemplary embodiment. Figure 7 An example of the real space (before movement) in the embodiment of this example is shown. Figure 8 An example of the real space (after movement) in the embodiment of this example is shown. Fig. 9 An example of a virtual space (before movement) according to the present exemplary embodiment is shown. Fig.10 An example of a virtual space (after movement) according to the present exemplary embodiment is shown. Fig.11 An example of the mixed space (after movement) according to the embodiment of this example is shown.

[0093] Below, refer to Figure 6 Flowchart and Figure 7 to Figure 11 , the operation of the information processing device JS according to this exemplary embodiment is described.

[0094] For ease of explanation and understanding, it is assumed that only the digital printer RT moves in the real space GK. In addition, it is assumed that the user has specified in advance that the message / object MGobj to be displayed in association with the digital printer RT, in other words, to be displayed near the digital printer RT, should be displayed near the digital printer object RTobj in the hybrid space FK (with respect to the y-axis direction, the coordinate position of +5).

[0095] Step S11: In the information processing device JS, the CPU 2 ( Figure 1 ) as the acquisition unit 11 ( Figure 2 ), by a known method, scanning Figure 7 The real space GK shown is captured, for example, by the input unit 1 as a camera. More specifically, the CPU 2 obtains the shapes (outer shapes) and coordinate positions of the digital printing press RT and the plurality of paper rolls RS1 to RS3 in the real space GK along the Z-axis direction.

[0096] Here, the CPU 2 obtains, for example, the shape of the digital printing press RT, which is "long in the X-axis direction". In addition, the CPU 2 obtains, for example, the coordinate position (X10, Y10, Z10) of the digital printing press RT in the real space GK. The CPU 2 also obtains the shapes (outer shapes) and coordinate positions of the plurality of paper rolls RS1 to RS3 in the same manner as the digital printing press RT.

[0097] Step S12: After obtaining the shape "long in the X-axis direction" and coordinate position (X10, Y10, Z10) of the digital printing press RT and the shapes and coordinate positions of the plurality of rolls RS1 to RS3 in step S11, the CPU 2 as the generating unit 13 ( Figure 2 Figure ), such as Fig. 9As shown, a virtual space KK is generated.

[0098] The virtual space KK includes objects corresponding to all objects acquired by scanning in step S11. More specifically, the virtual space KK includes a digital printing press object RTobj corresponding to the digital printing press RT and a plurality of roll paper objects RS1obj to R3obj corresponding to the plurality of roll paper RS1 to RS3.

[0099] The digital printing machine object RTobj corresponds to the digital printing machine RT, and has information such as the shape (including similar shapes), coordinate position, IP address, and model name of the digital printing machine RT.

[0100] The plurality of paper roll objects RS1obj to R3obj also correspond to the plurality of paper rolls RS1 to RS3 , respectively, and have information such as the shapes (including similar shapes), coordinate positions, and product names of the plurality of paper rolls RS1 to RS3 .

[0101] In addition, the CPU 2, as the generating unit 13, generates a message / object MGobj ( Fig. 9 An association is formed by associating the digital printing machine object RTobj with at least a portion of the digital printing machine object RTobj.

[0102] Step S13: After the virtual space KK is generated in step S12, the CPU 2 as the detection unit 12 ( Figure 2 ), detects whether the objects in the real space GK, namely the digital printing machine RT and the plurality of paper rolls RS1 to RS3, have moved. Figure 7 and Figure 8 As shown by the hollow arrow, the digital printing press RT has moved. As a result, the CPU 2 detects the fact that the shape "long in the X-axis direction" has moved, in other words, the fact that the digital printing press RT has moved is detected, and thus enters "Yes".

[0103] Step S14: After the movement of the digital printing press RT in the real space GK is detected in step S13, Fig.10 As shown, the CPU 2 functions as the acquisition unit 11 to acquire the coordinate position (x11, y11, z11) of the digital printing press object RTobj in the virtual space KK.

[0104] Here, the digital printer object RTobj is moved in conjunction with the movement of the digital printer RT by the CPU 2. The digital printer object RTobj may be arranged at a position desired by the user through a manual operation of the user, such as input of a coordinate position, instead of being moved in conjunction with the movement of the digital printer RT as described above. In addition, the digital printer object RTobj may be arranged at a position reflecting the user's wishes according to the movement of the digital printer RT by combining the above-mentioned movement in conjunction with the movement of the digital printer RT and the manual adjustment of the coordinate position by the user.

[0105] Step S15: After the coordinate position (x11, y11, z11) of the digital printing press object RTobj in the virtual space KK is detected in step S14, the CPU 2 as the updating unit 14 ( Figure 2 ), such as Fig.10 As shown, in the virtual space KK, the coordinate position of the message / object MGobj is updated or determined according to the coordinate position (x11, y11, z11) of the digital printing press object RTobj after the movement. Thus, the CPU 2 obtains the coordinate position (x11, y11+5, z11) of the message / object MGobj. In other words, the CPU 2 configures the message / object MGobj at the coordinate position (x11, y11+5, z11) in the virtual space KK. In other words, the CPU 2 moves the message / object MGobj while maintaining the relative positional relationship between the digital printing press object RTobj and the message / object MGobj.

[0106] Here, regarding the message / object MGobj, a manual operation by a user, such as input of a coordinate position, may be performed instead of or together with the update of the coordinate position by the CPU 2. Thus, the above-mentioned coordinate position (x11, y11+5, z11) may be replaced or the above-mentioned coordinate position (x11, y11+5, z11) may be adjusted, for example, the message / object MGobj may be arranged at the coordinate position (x11, y11+7, z11) or (x11-3, y11+5, z11).

[0107] Step S16: After obtaining the coordinate position (x11, y11+5, z11) of the message / object MGobj in the virtual space KK in step S15, the CPU 2 as the mixing unit 15 ( Figure 2 ), the digital printing machine RT and multiple paper rolls RS1 to RS3 ( Figure 8 ), as well as the digital printing machine object RTobj and the message / object MGobj ( Fig.10 (shown in the figure) to mix and generate Fig.11 Blend Space FK shown.

[0108] In the generated mixed space FK, there are a digital printer RT that can be visually confirmed, a plurality of rolls of paper RS1 to RS3, and a message / object MGobj, and a digital printer object RTobj that cannot be visually confirmed (in Fig.11 In the figure, the digital printing machine object RTobj which cannot be visually confirmed is not shown. ).

[0109] In the first exemplary embodiment, as described above, the setting work when displaying the message / object MGobj in the virtual space KK when the digital printer RT moves in the real space GK is reduced. That is, the coordinate position of the message / object MGobj in the virtual space KK is immediately determined based on the coordinate position of the digital printer object RTobj in the virtual space KK.

[0110] <Second exemplary embodiment>

[0111] An information processing device according to a second exemplary embodiment will be described.

[0112] <Structure of the Second Exemplary Embodiment>

[0113] The structure of the information processing device JS in this exemplary embodiment is similar to Figure 1 and Figure 2 The structure of the information processing device JS of the first exemplary embodiment shown is the same.

[0114] <Operation of the Second Exemplary Embodiment>

[0115] The information processing device JS of the present exemplary embodiment is different from the information processing device JS of the first exemplary embodiment in that instead of displaying the "message / object MGobj" (for example, Figure 4 ), but rather displays the "arrow object YJobj" as an associated object (e.g., Fig.13 (see figure in the figure).

[0116] Fig.12 An example of a real space in the embodiment of this example is shown. Fig.13 An example of a virtual space according to the present exemplary embodiment is shown. Fig.14 An example of a mixed space in the embodiment of this example is shown.

[0117] In the real space GK of this exemplary embodiment, Fig.12 As shown, there are a digital printing machine RT as an object and a processing machine KA as another object.

[0118] like Fig.13As shown, in the virtual space KK of the illustrated embodiment, there exist: a virtual object generated by the information processing device JS and corresponding to the digital printing machine RT, namely, a digital printing machine object RTobj as an object object; a virtual object generated by the information processing device JS and corresponding to the processing machine KA, namely, a processing machine object KAobj as another object object; and an arrow object YJobj generated by the information processing device JS and as an associated object.

[0119] The digital printer object RTobj and the paper roll object RSobj are used by the information processing device JS to calculate the coordinate position of the arrow object YJobj. Therefore, there is no need for the user of the information processing device JS to visually confirm the digital printer object RTobj and the paper roll object RSobj, and the digital printer object RTobj and the paper roll object RSobj are not visually confirmed.

[0120] The arrow object YJobj is an object pointing from the digital printing machine RT as the starting point in the real space GK toward the processing machine KA as the end point in the real space GK. For the convenience of the user of the information processing device JS, it is desirable that the arrow object YJobj can be visually confirmed. The starting point of the arrow object YJobj is determined based on the coordinate position of the digital printing machine object RTobj, while the end point of the arrow object YJobj is determined based on the coordinate position of the processing machine object KAobj.

[0121] The information processing device JS processes the digital printing machine RT and the processing machine KA in the real space GK ( Fig.12 ), as well as the digital printing machine object RTobj, processing machine object KAobj and arrow object YJobj ( Fig.13 ) to generate a hybrid space FK ( Fig.14 In the mixed space FK ( Fig.14 In the mixed space FK, there are digital printer RT, processing machine KA, and arrow object YJobj that can be visually confirmed, and digital printer object RTobj and processing machine object KAobj that cannot be visually confirmed. That is, in the mixed space FK, the digital printer object RTobj and processing machine object KAobj are not visually confirmed by the user of the information processing device JS.

[0122] Fig.15 1 is a flowchart showing the operation of the information processing device JS according to the present exemplary embodiment. Fig.16 An example of the real space (before movement) in the embodiment of this example is shown. Fig.17 An example of the real space (after movement) in the embodiment of this example is shown. Fig.18 An example of a virtual space (before movement) according to the present exemplary embodiment is shown. Fig.19An example of a virtual space (after movement) according to the present exemplary embodiment is shown. Fig. 20 An example of the mixed space (after movement) according to the embodiment of this example is shown.

[0123] Below, refer to Fig.15 Flowchart and Figures 16 to 20 The operation of the information processing device JS according to the present exemplary embodiment will be described.

[0124] For ease of explanation and understanding, it is assumed that the digital printing machine RT and the processing machine KA move in the real space GK, and the arrow object YJobj also needs to move along with the movement of the digital printing machine RT and the processing machine KA.

[0125] Step S21: In the information processing device JS, the CPU 2 ( Figure 1 ) as the acquisition unit 11 ( Figure 2 As shown in the figure, the step S11 of the first exemplary embodiment is similar to the step S12 of the first exemplary embodiment. Fig.16 The real space GK shown is scanned. More specifically, the CPU 2 obtains the shapes (outer shapes) and coordinate positions of the digital printing machine RT and the processing machine KA in the real space GK along the Z-axis direction. Here, the CPU 2 obtains, for example, the shape "long in the X-axis direction" and the coordinate position (X20, Y20, Z20) for the digital printing machine RT. The CPU 2 obtains, for example, the shape "long in the X-axis direction" and the coordinate position (X30, Y30, Z30) for the processing machine KA.

[0126] Step S22: After obtaining the shape "long in the X-axis direction" and coordinate position (X20, Y20, Z20) of the digital printing machine RT and the shape "long rectangular parallelepiped in the X-axis direction" and coordinate position (X30, Y30, Z30) of the processing machine KA in step S22, the CPU 2 as the generating unit 13 ( Figure 2 ), such as Fig.18 As shown, a virtual space KK is generated.

[0127] The virtual space KK includes objects corresponding to all objects acquired by scanning in step S21. More specifically, the virtual space KK includes a digital printing press object RTobj corresponding to the digital printing press RT and a processing machine KAobj corresponding to the processing machine object KA.

[0128] Step S23: After the virtual space KK is generated in step S22, the CPU 2 as the detection unit 12 ( Figure 2), similarly to step S12 of the first exemplary embodiment, it is detected whether at least one of the digital printing machine RT and the processing machine KA has moved in the real space GK. Here, as described above, in addition, Fig.16 and Fig.17 As shown by the two hollow arrows, both the digital printing machine RT and the processing machine KA have moved. As a result, the CPU 2 detects the fact that the shape "long in the X-axis direction" has moved, in other words, the fact that the digital printing machine RT has moved. Similarly, the CPU 2 detects the fact that the shape "long in the X-axis direction" has moved, in other words, the fact that the processing machine KA has moved. By detecting the above two facts, the process goes to "Yes".

[0129] Step S24: After the movement of the digital printing machine RT and the processing machine KA in the real space GK is detected in step S23, the CPU 2 generates the following information in the acquisition unit 11: Fig.19 As shown, the coordinate position (x21, y21, z21) of the digital printing machine object RTobj and the coordinate position (x31, y31, z31) of the processing machine object KAobj in the virtual space KK are obtained.

[0130] Step S25: After obtaining the coordinate position (x21, y21, z21) of the digital printing machine object RTobj and the coordinate position (x31, y31, z31) of the processing machine object KAobj in the virtual space KK in step S24, the CPU 2 as the updating unit 14 ( Figure 2 ), such as Fig.19 As shown, in the virtual space KK, the coordinate position of the arrow object YJobj is updated according to the coordinate position (x21, y21, z21) of the digital printing machine object RTobj and the coordinate position (x31, y31, z31) of the processing machine object KAobj. Thus, the CPU 2 obtains the starting point ST (x21, y21, z21) and the end point EN (x31, y31, z31) of the arrow object YJobj. In other words, the CPU 2 configures the arrow object YJobj in the virtual space KK, the starting point ST of the arrow object YJobj is the coordinate position (x21, y21, z21), and the end point EN is (x31, y31, z31).

[0131] Step S26: After obtaining the coordinate position (x21, y21, z21) of the start point ST and the coordinate position (x31, y31, z31) of the arrow object YJobj in the virtual space KK in step S25, the CPU 2 as the mixing unit 15 ( Figure 2 Figure 2), the digital printing machine RT and processing machine KA ( Fig.17), as well as the digital printing machine object RTobj, processing machine object KAobj and arrow object YJobj ( Fig.19 ) are mixed to generate Fig. 20 In the generated mixed space FK, there are digital printing machine RT, processing machine KA and arrow object YJobj that can be visually confirmed, and digital printing machine object RTobj and processing machine object KAobj that cannot be visually confirmed (in Fig. 20 In the figure, the digital printing machine object RTobj and the processing machine object KAobj that cannot be visually confirmed are not shown).

[0132] In the second exemplary embodiment, as described above, the coordinate position of the start point ST and the coordinate position of the end point EN of the arrow object YJobj can be determined without acquiring the coordinate positions of the digital printing press RT and the processing machine KA in the real space GK.

[0133] <Third exemplary embodiment>

[0134] An information processing device according to a third exemplary embodiment will be described.

[0135] <Structure of the third exemplary embodiment>

[0136] The structure of the information processing device JS of the third exemplary embodiment is Figure 1 The structure of the information processing device JS of the first exemplary embodiment shown is the same.

[0137] Fig.21 It is a functional block diagram of the information processing device JS according to the third exemplary embodiment.

[0138] According to the first exemplary embodiment Figure 2 Compared with the third exemplary embodiment Fig.21 As can be seen from the comparison, the information processing device JS of the present exemplary embodiment includes an acquisition unit 11, a detection unit 12, a generation unit 13, an update unit 14, a mixing unit 15, a control unit 16, and a storage unit 17, similarly to the first exemplary embodiment. In addition, unlike the first exemplary embodiment, the information processing device JS further includes an addition unit 18 and a search unit 19. The functions of the addition unit 18 and the search unit 19 will be described later.

[0139] Fig. 22 The table TB stored in the storage medium 4 of the present exemplary embodiment is shown.

[0140] The storage medium 4 ( Figure 1 The table TB is stored in the figure. Fig. 22 As shown, Table TB shows the digital printing machine RT (for example, Fig.24), i.e., the "printing content" (what is printed), and the processing machines KA1 to KA3 (for example, Fig.24 ) and the corresponding relationship between the "processing operation" performed by the processing machines KA1 to KA3 that should actually perform the "processing operation". In Table TB, the processing machine numbers "1", "2", and "3" respectively mean processing machine KA1, processing machine KA2, and processing machine KA3. Here, "printing content" is an example of a "processing process".

[0141] <Operation of the Third Exemplary Embodiment>

[0142] The information processing device JS of the present exemplary embodiment is different from the information processing device JS of the first exemplary embodiment, and similarly to the information processing device JS of the second exemplary embodiment, displays the "arrow object YJobj" (for example, Fig.28 (see figure in the figure).

[0143] Fig.23 1 is a flowchart showing the operation of the information processing device JS according to the present exemplary embodiment. Fig.24 An example of a real space (before processing) in the present exemplary embodiment is shown. Fig.25 An example of the real space (after processing) in the embodiment of this example is shown. Fig.26 An example of a virtual space (before processing) according to the present exemplary embodiment is shown. Fig. 27 An example of a virtual space (first half after processing) according to the present exemplary embodiment is shown. Fig.28 An example of a virtual space (second half after processing) according to the present exemplary embodiment is shown. Fig.29 An example of a mixed space (after processing) according to the embodiment of this example is shown.

[0144] Below, refer to Fig.23 Flowchart and Figure 24 to Figure 29 , the operation of the information processing device JS according to this exemplary embodiment is described.

[0145] For ease of explanation and understanding, assume that in the real space GK, the digital printing press RT prints the print content "booklet" on a roll paper RS ​​of coated paper such as coated paper, matte coated paper, premium paper, special paper, etc.

[0146] Here, the roll paper RS ​​is an example of a "target object". The roll paper object RSobj described later is an example of a "target object". Paper materials such as coated paper, matte coated paper, high-quality paper, and special paper, and printed contents such as "booklets" and "single-volume books" ( Fig. 20 ) is an example of "attribute information".

[0147] Step S31: In the information processing device JS, the CPU 2 ( Figure 1 ) is similar to step S11 of the first exemplary embodiment, as the acquisition unit 11 ( Fig.21 ), scan Fig.24 More specifically, the CPU 2 Fig.24 As shown, along the Z-axis direction in the real space GK, the shapes and coordinate positions of the digital printing press RT, the plurality of processing machines KA1 to KA3 , and the roll paper in the real space GK are acquired.

[0148] Here, the CPU 2 obtains the shape "long shape in the X-axis direction" and the coordinate position (X40, Y40, Z40) for the digital printing machine RT. In addition, the CPU 2 obtains the shape "rectangular block" and the coordinate position (X51, Y51, Z51), the shape "rectangular block" and the coordinate position (X52, Y52, Z52), and the shape "rectangular block" and the coordinate position (X53, Y53, Z53) for the multiple processing machines KA1 to KA3. The CPU 2 also obtains the shape "cylinder" and the coordinate position (X60, Y60, Z60) for the roll paper RS.

[0149] Step S32: After obtaining the shapes and coordinate positions of the digital printing machine RT, the plurality of processing machines KA1 to KA3, and the roll paper RS ​​in the real space GK in step S31, the CPU 2 as the generating unit 13 ( Fig.21 Figure ), such as Fig.26 As shown, a virtual space KK is generated.

[0150] The virtual space KK includes objects corresponding to all objects acquired by scanning in step S31. More specifically, the virtual space KK includes a digital printing press object RTobj corresponding to the digital printing press RT, a plurality of processing machine objects KA1obj to KA3obj corresponding to the plurality of processing machines KA1 to KA3, and a roll paper object RSobj corresponding to the roll paper RS.

[0151] Step S33: After the virtual space KK is generated in step S32, the CPU 2 as the detection unit 12 ( Fig.21 ), to check whether the digital printing machine RT has printed certain contents on the roll paper RS ​​( Fig. 22 In addition, as mentioned above, Fig.24 and Fig.25 As shown by the hollow arrow in , the roll paper RS ​​is supplied to the digital printing press RT, and the digital printing press RT prints the print content “booklet” on the roll paper RS, so the process proceeds to “Yes”.

[0152] Step S34: After detecting in step S33 that the digital printing press RT has printed the print content "booklet" on the roll paper RS, the CPU 2 as the acquisition unit 11 ( Fig.21 The coordinate position (x61, y61, z61) of the paper roll object RSobj is obtained. The paper roll object RSobj is used by the information processing device JS to calculate the arrow object YJobj ( Fig.28 , shown in FIG29 ), therefore, there is no need to enable the user of the information processing device JS to visually confirm the roll paper object RSobj, and the roll paper object RSobj is not visually confirmed.

[0153] Step S35: After obtaining the coordinate position (x61, y61, z61) of the roll paper object RSobj in step S34, the CPU 2 as the adding unit 18 ( Fig.21 ), the printed content "Booklet" ( Fig. 22 The virtual object (shown in the figure) is the print content / object PCobj.

[0154] Here, the print content / object PCobj is used by the information processing device JS to calculate the processing step of printing the print content "booklet" which is then performed by the digital printing machine RT, and is a processing step that should be performed by any one of the processing machines KA1 to KA3. Therefore, there is no need for the user of the information processing device JS to visually confirm the print content / object PCobj, and the print content / object PCobj is not visually confirmed.

[0155] Step S36: After the print content / object PCobj is added to the coordinate position (61x, y61+5, z61) near the roll paper object RSobj in the virtual space KK in step S35, the CPU 2 as the search unit 19 ( Fig.21 Figure), refer to Table TB ( Fig. 22 ). The CPU 2 retrieves the processing job and the processing machine number corresponding to the print content "Booklet" printed by the digital printing press RT on the roll paper RS ​​in the real space GK through this reference. Thus, the CPU 2 obtains the processing job "Saddle Stitching" and the processing machine number "3".

[0156] Step S37: After acquiring the processing operation "saddle stitching" and the processing machine number "3" in step S36, the CPU 2 as the acquisition unit 11 ( Fig.21 Figure ), such as Fig.28 As shown, the coordinate position (x53, y53, z53) of the processing machine object KA3obj corresponding to the processing machine KA3 with the processing machine number "3" is obtained.

[0157] Here, the processing machine object KA3obj is used by the information processing device JS to calculate the coordinate position of the arrow object YJobj, and therefore does not need to be visually confirmed by the user of the information processing device JS and is not visually confirmed.

[0158] Step S38: After obtaining the coordinate position (x53, y53, z53) of the processing machine object KA3obj in the virtual space KK in step S37, the CPU 2 as the generating unit 13 ( Fig.21 In the figure), an arrow object YJobj is generated in the virtual space KK. In more detail, Fig.28 As shown, the CPU 2 determines the coordinate position of the arrow object YJobj based on the coordinate position (x61, y61, z61) of the roll paper object RSobj and the coordinate position (x53, y53, z53) of the processing machine object KA3obj in the virtual space KK. Thus, the CPU 2 determines the coordinate position (x61, y61, z61) of the start point ST and the coordinate position (x53, y53, z53) of the end point EN of the arrow object YJobj.

[0159] In other words, CPU 2 determines the relationship between the digital printing machine object RTobj as the object object and the arrow object YJobj as the associated object through the digital printing machine RT as the object and the processing steps performed by the digital printing machine RT, namely the printing content “booklet”. More specifically, the relationship is determined based on the coordinate position of at least the roll paper object RSobj specified by the digital printing machine RT as the object and the processing steps performed by the digital printing machine RT, namely the printing content “booklet”.

[0160] Step S39: After obtaining the coordinate position (x61, y61, z61) of the start point ST and the coordinate position (x53, y53, z53) of the end point EN of the arrow object YJobj in step S38, the CPU 2 as the mixing unit 15 ( Fig.21 ), the digital printing machine RT, multiple processing machines KA1~KA3 and paper roll RS ( Fig.25 ), as well as the paper roll object RSobj, print content / object PCobj, processing machine objects KA1obj~KA3obj and arrow object YJobj ( Fig.28 ) are mixed to generate Fig.29 Blend Space FK shown.

[0161] In the generated mixed space FK, there are digital printing machine RT, paper roll RS, processing machines KA1 to KA3, and arrow object YJobj that can be visually confirmed, as well as processing machine objects KA1obj to KA3obj, paper roll object RSobj, and print content / object PCobj that cannot be visually confirmed (in Fig.29 In the figure, the print content / object PCobj, the roll paper object RSobj, and the processing machine objects KA1obj to KA3ob that cannot be visually confirmed are not shown. ).

[0162] In the third exemplary embodiment, as described above, the relationship between the arrow object YJobj and the digital printing press object RTobj in the virtual space KK can be determined without being affected by other conditions in the virtual space KK.

[0163] That is, the direction indicated by the arrow object YJobj reflects the attribute information of the roll paper object RSobj and the position of the possible processing machine object KA3obj. In addition, the direction indicated by the arrow object YJobj also reflects the position of the roll paper object RSobj.

[0164] <Fourth exemplary embodiment>

[0165] An information processing device according to a fourth exemplary embodiment will be described.

[0166] <Structure of the fourth exemplary embodiment>

[0167] The configuration of the information processing device JS of the fourth exemplary embodiment is similar to the configuration of the information processing device JS of the first exemplary embodiment ( Figure 1 Same as the figure in the figure.

[0168] The functional blocks of the information processing device JS of the fourth exemplary embodiment are similar to the functional blocks of the information processing device JS of the third exemplary embodiment ( Fig.21 Same as the figure in the figure).

[0169] On the other hand, the information processing device JS of the fourth exemplary embodiment is different from the information processing device JS of the third exemplary embodiment in that the selection of any one of the processing machines KA1 to KA3 is performed by the user of the information processing device JS instead of the information processing device JS, and therefore, the table TB ( Fig. 22 (see figure in the figure).

[0170] <Operation of the Fourth Exemplary Embodiment>

[0171] The information processing device JS according to the fourth exemplary embodiment displays the arrow object YJobj similarly to the information processing device JS according to the third exemplary embodiment.

[0172] On the other hand, the information processing device JS of the present exemplary embodiment is different from the information processing device JS of the third exemplary embodiment in that, instead of the print content / object PCobj that cannot be visually confirmed, a print content / object PCobj that can be visually confirmed is added to facilitate the user of the information processing device JS to make the above-mentioned selection.

[0173] Fig.30 1 is a flowchart showing the operation of the information processing device JS according to the present exemplary embodiment. Fig.31 An example of a virtual space (after processing) according to the present exemplary embodiment is shown. Fig.32 An example of the mixed space (the first half after processing) of the embodiment shown in this example is shown. Fig.33 An example of the mixed space (the second half after processing) of the embodiment shown in this example is shown. Fig.34 An example of a virtual space (after selection) according to the present exemplary embodiment is shown. Fig.35 An example of a mixed space (after selection) according to the embodiment of this example is shown.

[0174] Below, refer to Fig.30 Flowchart of the third exemplary embodiment Fig.24 , Fig.25 , Fig.26 , and the embodiment of this example Figure 31 to Figure 35 , the operation of the information processing device JS according to this exemplary embodiment is described.

[0175] For ease of explanation and understanding, unlike the third exemplary embodiment, it is assumed that in the real space GK, the digital printing press RT prints the print content "single volume" instead of the print content "booklet" on a roll paper RS ​​of coated paper such as coated paper, matte coated paper, high-quality paper, special paper, etc.

[0176] Step S41: Similar to step S31 of the third exemplary embodiment, in the information processing device JS, the CPU 2 ( Figure 1 ) as the acquisition unit 11 ( Fig.21 ), scan Fig.24 The real space GK shown in FIG. Fig.24 As shown, for the digital printing machine RT, the shape "longer in the X-axis direction" and the coordinate position (X40, Y40, Z40) are obtained.

[0177] In addition, CPU 2 obtains the shape of a "rectangle" and its coordinate position (X51, Y51, Z51), the shape of a "rectangle" and its coordinate position (X52, Y52, Z52), and the shape of a "rectangle" and its coordinate position (X53, Y53, Z53) for multiple processing machines KA1~KA3 respectively.

[0178] The CPU 2 also obtains the shape "cylinder" and the coordinate position (X60, Y60, Z60) of the paper roll RS.

[0179] Step S42: After obtaining the shapes and coordinate positions of the digital printing machine RT, the plurality of processing machines KA1 to KA3, and the roll paper RS ​​in the real space GK in step S41, the CPU 2 as the generating unit 13 ( Fig.21 Figure ), such as Fig.26 As shown, a virtual space KK is generated.

[0180] Step S43: After the virtual space KK is generated in step S42, the CPU 2 as the detection unit 12 ( Fig.21 ), detect whether the digital printing machine RT has printed certain print contents on the roll paper RS ​​( Fig. 22 In addition, as mentioned above, Fig.24 and Fig.25 As shown by the hollow arrow in , the roll paper RS ​​is provided to the digital printing press RT, and the digital printing press RT prints the print content “single-sheet” on the roll paper RS, so the process proceeds to “Yes”.

[0181] Step S44: After detecting in step S43 that the digital printing press RT has printed the print content "single volume" on the roll paper RS, the CPU 2 as the acquisition unit 11 ( Fig.21 ), and obtains the coordinate position (x61, y61, z61) of the roll paper object RSobj. As in the third exemplary embodiment, the roll paper object RSobj is used by the information processing device JS to calculate the arrow object YJobj (for example, Fig.34 Therefore, it is not necessary to enable the user of the information processing device JS to visually confirm the roll paper object RSobj, and the roll paper object RSobj is not visually confirmed.

[0182] Step S45: After obtaining the coordinate position (x61, y61, z61) of the roll paper object RSobj in step S44, the CPU 2 as the adding unit 18 ( Fig.21 In the figure), the printed content "Single Volume" ( Fig. 22 The virtual object (shown in the figure) is the print content / object PCobj.

[0183] Regarding the print content / object PCobj, in the third exemplary embodiment, it is used for the information processing device JS to determine the processing step of printing the print content "single volume" that is subsequently performed by the digital printing machine RT and should be performed by any one of the processing machines KA1 to KA3. Here, unlike the third exemplary embodiment, the print content / object PCobj is used for the user of the information processing device JS to select, and therefore, it is desired to enable the user of the information processing device JS to visually confirm it.

[0184] Step S46: In step S45, after adding the visually identifiable print content / object PCobj to the coordinate position (61x, y61+5, z61) near the roll paper object RSobj in the virtual space KK, the CPU 2 as the mixing unit 15 ( Fig.21 ), the digital printing machine RT, multiple processing machines KA1~KA3 and paper roll RS ( Fig.25 ), and the roll paper object RSobj and print content / object PCobj ( Fig.31 ) are mixed to generate Fig.32 Blend Space FK shown.

[0185] In the generated mixed space FK, there are the digital printing machine RT, the paper roll RS, the processing machines KA1 to KA3, and the print content / object PCobj that can be visually recognized, and the paper roll object RSobj that cannot be visually recognized (in Fig.32 In the figure, the roll paper object RSobj that cannot be visually confirmed is not shown).

[0186] Step S47: After the mixed space FK is generated in step S46, the user of the information processing device JS visually confirms the print content / object PCobj of the print content "single volume", and selects which of the processing machines KA1 to KA3 is appropriate to use for the process after the printing of the digital printing machine RT. The CPU 2, as the acquisition unit 11, accepts the user's selection. Here, it is assumed that the user selects the processing operation "gluing and binding" that is desired to be performed on the roll paper RS ​​printed with the print content "single volume", or the "processing machine KA2" that performs the processing operation "gluing and binding".

[0187] Here, in Fig.33 In the figure, the two dotted arrows, the MR device imitating the glasses, and the disk do not exist in the mixed space FK. They schematically show the communication of the print content "single volume" from the print content / object PCobj to the user (icon of glasses) and the user's selection of the processing machine KA2.

[0188] Step S48: After accepting the selection of the processing job "gluing and binding" or the processing machine KA2 in step S47, the CPU 2 as the acquisition unit 11 acquires the coordinate position (x52, y52, z52) of the processing machine object KA2obj in the virtual space KK corresponding to the selected processing machine KA2.

[0189] Step S49: After obtaining the coordinate position (x52, y52, z52) of the machining machine object KA2obj in the virtual space KK in step S48, the CPU 2 as the generating unit 13 generates the arrow object YJobj in the virtual space KK.

[0190] More specifically, if Fig.34 As shown, the CPU 2 updates or obtains the coordinate position of the arrow object YJobj based on the coordinate position (x61, y61, z61) of the roll paper object RSobj and the coordinate position (x52, y52, z52) of the processing machine object KA2obj in the virtual space KK. Thus, the CPU 2 determines the coordinate position (x61, y61, z61) of the start point ST and the coordinate position (x52, y52, z52) of the end point EN of the arrow object YJobj.

[0191] Step S50: After obtaining the coordinate position (x61, y61, z61) of the start point ST and the coordinate position (x52, y52, z52) of the end point EN of the arrow object YJobj in step S49, the CPU 2 as the mixing unit 15 performs a combination of the digital printing press RT, the plurality of processing machines KA1 to KA3, and the roll paper RS ​​in the real space GK. Fig.25 ), as well as the paper roll object RSobj, print content / object PCobj, processing machine objects KA1obj~KA3obj and arrow object YJobj ( Fig.34 ) are mixed to generate Fig.35 Blend Space FK shown.

[0192] In the generated mixed space FK, there are visually identifiable digital printing machine RT, paper roll RS, processing machines KA1 to KA3, print content / object PCobj and arrow object YJobj, as well as visually identifiable paper roll object RSobj and processing machine objects KA1obj to KA3obj. (In Fig.35 In the figure, the roll paper object RSobj and processing machine objects KA1obj to KA3obj) which cannot be visually confirmed are not shown.

[0193] As described above, in the third exemplary embodiment, the direction indicated by the arrow object YJobj reflects the coordinate position of the processing machine object KA3obj determined by the information processing device JS. Instead, in the fourth exemplary embodiment, the direction indicated by the arrow object YJobj reflects the coordinate position of the processing machine object KA2obj determined by the user.

[0194] <Fifth exemplary embodiment>

[0195] An information processing device according to a fifth exemplary embodiment will be described.

[0196] <Structure of the fifth exemplary embodiment>

[0197] The configuration and functional blocks of the information processing device JS of the fifth exemplary embodiment are similar to those of the information processing device JS of the first exemplary embodiment ( Figure 1 , Figure 2 Same as the figure in the figure).

[0198] On the other hand, in the information processing device JS of the fifth exemplary embodiment, unlike the information processing device JS of the first exemplary embodiment, the digital printer object RTobj (for example, Fig.38 ) has a digital printing press RT (e.g., Fig.37 Figure 2) for the roll paper RS ​​(for example, Fig.37 ) is “information related to the moving path of the roll paper RS” when printing. The information related to the moving path of the roll paper RS ​​is information about the moving path of the roll paper RS ​​( Fig.38 The hollow arrows are used to illustrate the coordinate position of the starting point and the coordinate position of the end point. Fig.37 As shown, it is assumed that in the real space GK, the coordinate position of the start point STT of the paper roll RS is (X70, Y70, Z70), and the coordinate position of the end point END is (X71, Y71, Z71). Here, the paper roll RS is an example of a "target object".

[0199] <Operation of the Fifth Exemplary Embodiment>

[0200] Fig.36 It is a flowchart showing the operation of the information processing device JS according to the fifth exemplary embodiment. Fig.37 An example of a real space in the embodiment of this example is shown. Fig.38 An example of a virtual space according to the present exemplary embodiment is shown. Fig.39 An example of a mixed space in the embodiment of this example is shown.

[0201] Below, refer to Fig.36 Flowchart and Figure 37 to Figure 39The operation of the information processing device JS according to the present exemplary embodiment will be described.

[0202] Step S51: In the information processing device JS, the CPU 2 as the acquisition unit 11 ( Figure 2 Figure ), according to the digital printing machine RT ( Fig.37 The digital printing machine object RTobj ( Fig.38 ), obtains information related to the moving path of the paper roll RS when the digital printer RT prints on the paper roll RS in the real space GK. More specifically, the CPU 2 obtains the coordinate position (X70, Y70, Z70) of the starting point STT and the coordinate position (X71, Y71, Z71) of the end point END of the paper roll RS. The digital printer object RTobj is an object used by the information processing device JS to obtain information related to the moving path of the paper roll RS. Therefore, it is not necessary for the user of the information processing device JS to visually confirm the digital printer object RTobj, and the digital printer object RTobj is not visually confirmed.

[0203] Step S52: After obtaining the coordinate position (X70, Y70, Z70) of the start point STT and the coordinate position (X71, Y71, Z71) of the end point END of the paper roll RS in the real space GK in step S51, the CPU 2 as the generating unit 13 ( Figure 2 Figure ), such as Fig.38 As shown, a roll object RSobj and a reproduction button / object BTobj corresponding to the roll RS in the real space GK are generated in the virtual space KK. Here, the roll object RSobj is an example of a "dynamic display object" that displays the movement path of the roll RS in the real space GK in the form of animation. In addition, the reproduction button / object BTobj is an object for instructing the reproduction and stop of the dynamic display object, that is, the roll object RSobj.

[0204] Step S53: After generating the roll paper object RSobj and the reproduction button / object BTobj in step S52, the CPU 2 as the updating unit 14 ( Figure 2 The CPU 2 updates the position of the paper roll object RSobj in the virtual space KK based on the information related to the moving path of the paper roll RS, i.e., the coordinate position (X70, Y70, Z70) of the starting point STT and the coordinate position (X71, Y71, Z71) of the end point END of the paper roll RS in the real space GK. Thus, the CPU 2 obtains the coordinate position (x70, y70, z70) of the starting point ST and the end point EN (x71, y71, z71) of the paper roll object RSobj in the virtual space KK.

[0205] Step S54: After the CPU 2 obtains the coordinate position (x70, y70, z70) of the start point ST and the coordinate position (x71, y71, z71) of the end point EN of the roll paper object RSobj in step S54, the CPU 2 as the mixing unit 15 ( Figure 2 Figure 2), the digital printing machine RT and paper roll RS in the real space GK ( Fig.37 ), and the roll paper object RSobj and the reproduction button / object BTobj ( Fig.38 ) are mixed to generate Fig.39 Blend Space FK shown.

[0206] In the generated mixed space FK, there exist the digital printing press RT, the roll paper RS, the roll paper object RSobj, and the reproduction button / object BTobj, which can be visually confirmed.

[0207] The user of the information processing device JS visually confirms the mixed space FK, thereby being able to recognize how the roll paper RS ​​moves by virtually and dynamically displaying the moving path of the roll paper RS ​​by the roll paper object RSobj in the virtual space KK, even if the digital printing press RT does not print the roll paper RS ​​in the real space GK.

[0208] In the embodiment of this example, as described above, the coordinate position of the roll paper object RSobj in the virtual space KK is set to be along the moving path of the roll paper RS ​​in the real space GK. That is, the starting point ST and the end point EN of the roll paper object RSobj as a dynamic display object in the virtual space KK are consistent with the starting point STT and the end point END of the moving path of the roll paper RS ​​in the real space GK.

[0209] 〈Supplementary explanation of processor and program〉

[0210] In the above-mentioned embodiment, the processor refers to a processor in a broad sense, which includes not only general-purpose processors (e.g., CPU: Central Processing Unit, etc.) but also special-purpose processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic devices, etc.).

[0211] In the above-mentioned embodiments, the operations of the processors may be implemented by one processor or by cooperation of a plurality of processors. In addition, the order of the operations of the processors is not limited to the order in the above-mentioned embodiments and may be changed as appropriate.

[0212] In the above-mentioned embodiment, the program PR can also be provided by being recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory) and a USB (Universal Serial Bus) memory instead of being pre-stored (installed) on the storage medium 4, and can also be downloaded from an external device via a network.

[0213] Embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed form. Obviously, many modifications and variations will be apparent to those skilled in the art. The embodiments are selected and described in order to best explain the principles of the present invention and its practical application, so that other persons skilled in the art can understand that the present invention is used for various embodiments and has various modifications suitable for the specific use expected. The scope of protection of the present invention is limited by the appended claims and their equivalents.

Claims

1. An information processing device, wherein: The information processing device has a processor. The processor performs the following processing: Generate an object in the virtual space corresponding to an object in the real space; associating an associated object in a virtual space displayed in correspondence with the object with at least a portion of the object, the associated object being an arrow object indicating a direction from a starting point toward an end point; determining a relationship between the object in the virtual space and the associated object based on a processing device that is the object in the real space and a processing step performed by the processing device in the real space; determining the position of the starting point of the arrow object according to the attribute information of the target object in the virtual space corresponding to the target object in the real space for the processing device to perform the processing step; determining the position of the end point of the arrow object according to the position of another object in the virtual space corresponding to another object that performs another processing step in the real space that is likely to be performed subsequent to the processing step; as well as When movement of the physical object is detected in the virtual space, the related object is moved while maintaining a relative positional relationship between the related object and the physical object.

2. The information processing device according to claim 1, wherein: When the processor detects that the object in the real space has moved, the processor disposes the position of the object in the virtual space at the position of the object after the movement in the real space.

3. The information processing device according to claim 1 or 2, wherein: The processor performs the following processing: Determining the other processing steps according to the processing steps performed by the object on the target object in the real space; Determining the position of the starting point of the arrow object according to the position of the target object in the virtual space; The position of the end point of the arrow object is determined based on the position of another object in the virtual space corresponding to another object in the real space that performs the determined other processing step.

4. The information processing device according to claim 1 or 2, wherein: The processor performs the following processing: accepting another processing step from a user in response to the processing step performed by the object on the target in the real space; Determining the position of the starting point of the arrow object according to the position of the target object in the virtual space; as well as The position of the end point of the arrow object is determined based on the position of another object in the virtual space corresponding to another object in the real space that performs the other processing step received.

5. A storage medium storing an information processing program for causing a computer to execute the following processing: Generate an object in the virtual space corresponding to an object in the real space; associating an associated object in a virtual space displayed in correspondence with the object with at least a portion of the object, the associated object being an arrow object indicating a direction from a starting point toward an end point; determining a relationship between the object in the virtual space and the associated object based on a processing device that is the object in the real space and a processing step performed by the processing device in the real space; determining the position of the starting point of the arrow object according to the attribute information of the target object in the virtual space corresponding to the target object in the real space for the processing device to perform the processing step; determining the position of the end point of the arrow object according to the position of another object in the virtual space corresponding to another object that performs another processing step in the real space that is likely to be performed subsequent to the processing step; as well as When movement of the physical object is detected in the virtual space, the related object is moved while maintaining a relative positional relationship between the related object and the physical object.

Images