Program, information processing device, and method for controlling information processing device

Abstract

To provide a mechanism that can easily select an object that is difficult to be select by tapping.SOLUTION: An information processing device 1001 equipped with a touch display 14 displays a poster editing screen 201 on the touch display 14. If it is determined that an operation for selecting an object displayed on the poster editing screen 201 is not easy, the information processing device 1001 displays a selection editing assistance item 90 that assists in operation to edit the object and brings the object into a selected state.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a program, an information processing device, and a control method for an information processing device. [Background technology]

[0002] There is layout software (hereafter referred to as layout software) that creates designs by arranging objects such as shapes, images, and text boxes. In layout software, objects can be edited on a screen (hereafter referred to as editing screen) where designs are created and edited. To edit objects, an input device attached to a terminal such as a PC or smartphone is used to select the object (hereafter referred to as selection operation) and place the object in a selected state (hereafter referred to as selected state). To place an object in a selected state, an input device attached to the terminal is used to select an area that accepts input for placing the object in a selected state (hereafter referred to as selection area). The selection area is generally a rectangle circumscribing the object or an area of ​​the same shape as the object.

[0003] On terminals such as PCs, an object is selected when an input device such as a mouse is used to operate the mouse cursor and click the selection area of ​​the object to be selected. On the other hand, on terminals such as smartphones, an input device such as a finger or a touch pen is used to tap the selection area of ​​the object to be selected on the touch display (hereinafter referred to as a tap operation), and the object is selected. When an object is selected, it becomes possible to edit information such as the fill, border, and size of the object (hereinafter referred to as attribute information). When creating a design using multiple objects, the multiple objects may be placed in an overlapping state. In this state, when the editing screen is set to the XY plane, the overlapping order of the objects is managed by the Z axis. When objects overlap, the object on the lower layer is displayed as if it is hidden by the object on the upper layer. To select an object on a lower layer, if there is a selection area of ​​the object on the lower layer that is visible from the top layer, that area is selected. When the selection area of ​​the lower layer object is entirely hidden by the upper layer object or when the selection area of ​​the lower layer object visible from the top layer is extremely small, the upper layer object is temporarily placed in another position so that the lower layer object can be selected.

[0004] The technology of Patent Document 1 has been proposed as a technology related to the selection operation of multiple overlapping objects. In Patent Document 1, when multiple objects are overlapping, performing a predetermined operation on the selection area of ​​the overlapping objects switches the object to be selected from the multiple overlapping objects. This is a function that assists in selecting an object placed in a lower layer. Meanwhile, PowerPoint provided by Microsoft Corporation has a function that displays a list of placed objects and selects any object. Also, Illustrator provided by Adobe Corporation has a function that allows the object to be selected to be recognized by highlighting the border of the object when the mouse cursor is placed over the object (mouse over).

[0005] In addition, in object detection such as Convolutional Neural Network (hereafter referred to as CNN) and You Only Look Once (hereafter referred to as YOLO), when an object to be searched for is detected in an image, a technique is known in which the detected part is given a rectangular frame or the edge of the detected part is used as a boundary to fill in the detected part, and the image is output. The detected part may be treated as an object and a selection operation may be performed. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-197024 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in terminals such as smartphones, fine editing operations may be required depending on the settings of an object, and object selection operations may be difficult. One example of when object selection operations become difficult is when the size of an object is small. Since the selection area becomes smaller with the size of the object, fine selection operations are required to select the object. In terminals such as PCs, it is possible to realize the selection operation of a small object by operating a mouse and making fine adjustments to the mouse cursor, but fine selection operations are difficult in terminals such as smartphones where tap operations are the norm. In addition, in the technology of Patent Document 1 described above, it is necessary to tap the selection area to select a small object, and fine selection operations are difficult in terminals such as smartphones.

[0008] Another example of a case where the object selection operation becomes difficult is when the color setting of the attribute information of a first object, such as the fill, is set to a similar color to the color setting of the attribute information of an adjacent second object, reducing the visibility of the first object (hereinafter referred to as object assimilation). When object assimilation occurs, it becomes difficult to recognize the placement position of the object. The above-mentioned PowerPoint provides a function for displaying all objects in a list, but since the display size of a terminal such as a smartphone is relatively small, it is not possible to provide a list display of objects. In addition, the above-mentioned Illustrator provides a function for highlighting the frame line by hovering the mouse over an object, but since the mouse cursor cannot be operated on a terminal such as a smartphone, it is not possible to highlight the object by hovering the mouse over it. Furthermore, in the technology of the above-mentioned Patent Document 1, in order to select an object in which object assimilation has occurred, it is necessary to tap the selection area of ​​the object. When object assimilation has occurred, the placement position of the object cannot be visually recognized, making it difficult to select the object.

[0009] Another example of a case where an object selection operation becomes difficult is when the selection area of ​​an object is mostly hidden by an object in an upper layer. In a terminal such as a smartphone, the area of ​​the selection area of ​​a lower layer object visible from the top layer becomes small, making it difficult to perform a tap operation to select the lower layer object. In the technology of Patent Document 1 described above, the lower layer object can be selected by performing a predetermined selection operation on the overlapping selection area, but if the lower layer object is located in a position that cannot be seen from the top layer, the overlapping selection area cannot be recognized, making it difficult to select the object. In this way, in the past, it was not possible to easily select an object that is difficult to select by tapping.

[0010] An object of the present invention is to provide a mechanism for easily selecting an object that is difficult to select by a tap operation. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the program of the present invention is characterized in that it causes an information processing device having a touch display that accepts user operations to execute the steps of: displaying an object on the touch display in accordance with the user operations accepted by the touch display; determining whether or not the object displayed on the touch display is easy to select; and, if it is determined that the object displayed on the touch display is not easy to select, displaying an auxiliary UI that assists in editing the object and setting the object to a selected state. Effect of the Invention

[0012] According to the present invention, it is possible to easily select an object that is difficult to select by a tap operation. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of an information processing system including an information processing device according to an embodiment of the present invention. [Diagram 2] 1. FIG. 4 is a diagram showing an example of a poster editing screen displayed on a touch display of the information processing device in FIG. [Diagram 3] 4 is a flowchart showing the steps of a poster creation process executed by the information processing device of FIG. 1. [Figure 4] 1. FIG. 4 is a diagram showing an example of a poster editing screen displayed on a touch display of the information processing device in FIG. [Diagram 5] 4 is a flowchart showing the procedure of the determination process in S305 of FIG. 3. [Figure 6] 6 is a flowchart showing the procedure of the process of determining a color difference in S503 of FIG. 5. [Figure 7]FIG. 6 is a diagram for explaining the determination of color difference in S503 of FIG. 5. [Figure 8] FIG. 7 is a diagram for explaining the determination of the area of ​​the visible region in S505 of FIG. 5. [Figure 9] 2 is a diagram showing an example of a selection editing support item displayed on a touch display of the information processing device of FIG. 1. [Figure 10] 12 is a diagram for explaining display control in the second edit control process of FIG. 11. FIG. [Figure 11] 13 is a flowchart showing the procedure of a second edit control process executed by the information processing device of FIG. [Figure 12] 13 is a flowchart showing the procedure of a third edit control process executed by the information processing device of FIG. [Figure 13] 11 is a diagram showing a state in which a user performs a swipe operation on the poster editing screen using a pointing device. FIG. [Figure 14] 13 is a flowchart showing the procedure of a fourth edit control process executed by the information processing device of FIG. [Figure 15] 13 is a flowchart showing the procedure of a fifth edit control process executed by the information processing device of FIG. [Figure 16] 11A to 11C are diagrams for explaining display control of a selected editing support item in the present embodiment. [Figure 17] 13 is a flowchart showing the procedure of a sixth edit control process executed by the information processing device of FIG. [Figure 18] 18 is a diagram for explaining display control in the sixth edit control process of FIG. 17. FIG. [Figure 19] 13 is a flowchart showing the procedure of a seventh edit control process executed by the information processing device of FIG. [Figure 20] 20 is a diagram for explaining display control in the seventh edit control process in FIG. 19. FIG. [Figure 21] 13 is a flowchart showing the procedure of an eighth edit control process executed by the information processing device of FIG. [Figure 22]FIG. 22 is a diagram for explaining the display control in S2102 of FIG. [Figure 23] 11A to 11C are diagrams for explaining display control of a selected editing support item in the present embodiment. [Figure 24] 13 is a flowchart showing the procedure of a ninth edit control process executed by the information processing device of FIG. [Diagram 25] FIG. 25 is a diagram for explaining display control in the ninth edit control process of FIG. 24. [Figure 26] 13 is a flowchart showing the procedure of a tenth edit control process executed by the information processing device of FIG. 1. [Figure 27] FIG. 27 is a diagram for explaining the process of S2602 in FIG. 26. [Figure 28] FIG. 2 is a diagram showing an example of a UI diagram of the object detection software according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. Furthermore, the present invention is applicable to applications that perform object selection operations, in addition to general layout software such as poster creation applications, photo album editing applications, and presentation creation applications. In this embodiment, a poster creation application will be described as an example of layout software.

[0015] 1 is a block diagram showing an example of the configuration of an information processing system 1000 including an information processing device 1001 according to this embodiment. In FIG. 1, the information processing system 1000 is made up of the information processing device 1001 and an external server 1002.

[0016] First, the configuration of the information processing device 1001 will be described. In Fig. 1, the information processing device 1001 includes, as a hardware configuration, a CPU 10, a ROM 11, a RAM 12, and a touch display 14. The information processing device 1001 also includes an input / output interface (not shown) for connecting to a network 20 such as the Internet. The information processing device 1001 is, for example, a mobile terminal such as a smartphone or a tablet terminal. In the information processing device 1001, a selection area of ​​an object to be selected is tapped using an input device such as a finger or a touch pen.

[0017] The CPU 10 is a central processing unit, and controls the entire information processing device 1001 by executing an OS 13 stored in the ROM 11 or RAM 12. The CPU 10 also performs calculations based on input data by executing a Web browser 15, a native application 16 that is executed directly on the information processing device 1001, and other programs stored in the ROM 11 or RAM 12. This allows data to be processed and output, and each piece of hardware to be controlled, thereby realizing each function of the information processing device 1001.

[0018] The ROM 11 is a read-only memory that stores programs for operating the layout software. The RAM 12 is a random access memory that is used as a work memory for the CPU 10. If the RAM 12 is a non-volatile RAM, it can also store programs.

[0019] The touch display 14 includes a display 141 that displays a screen and a touch sensor 142 that detects a touch operation. On the touch display 14, a user directly touches the screen with a finger or the like to perform an input operation, and the processing result is displayed on the screen. In the above-mentioned layout software, a user can select an object arranged on the screen by directly touching it with a finger, and can move the position of the object by sliding the finger while keeping the finger touched. In addition, a user can perform a zoom operation in which the object is enlarged by placing two fingers on an object and spreading them apart, and can input characters by touching a software keyboard displayed on the screen.

[0020] The Web browser 15 browses Web sites on the Internet and executes Web applications. When executing a Web application, the information processing device 1001 receives a client-side program from an external server 1002 connected via the Internet. The Web browser 15 of the information processing device 1001 interprets script languages ​​such as HTML and JavaScript to display the Web site and operate the Web application.

[0021] The native application 16 is an application that becomes operational by being installed in the OS 13. The native application 16 may operate without coordinating with the external server 1002, or may operate in coordination with the external server 1002. Furthermore, the native application 16 has a function of assisting in processing that is difficult to execute with a Web application. The above-mentioned layout software may be provided as a Web application.

[0022] Next, a description will be given of the configuration of the external server 1002. In Fig. 1, the external server 1002 includes a program execution server 30, a print server 32, a storage server 33, and an SNS / cloud service 34. The external server 1002 also includes an input / output interface (not shown) for connecting to a network 20 such as the Internet.

[0023] The program execution server 30 executes a server-side program 301 which is a program that runs on the external server 1002. The print server 32 has a function of causing a printer 21 connected on the network to print a design image created by the layout software in the information processing device 1001.

[0024] The storage server 33 can be used as an auxiliary device for the RAM 12, and stores, for example, layout data output by the layout software in the information processing device 1001, processed images, and the like.

[0025] The SNS / cloud service 34 accesses the provider of the SNS or cloud service, and controls downloading images of layout materials and uploading layout data to a storage service on the cloud. In the present embodiment, the external server 1002 includes the program execution server 30, the print server 32, the storage server 33, and the SNS / cloud service 34, but the present invention is not limited to this configuration. For example, the program execution server 30, the print server 32, the storage server 33, and the SNS / cloud service 34 may be realized as physically separate devices.

[0026] The printer 21 is connected to the information processing device 1001 via a wired connection, via the network 20, or via a wireless interface (not shown) such as BlueTooth or IrDA, and prints a design image created by the layout software in the information processing device 1001.

[0027] 2 is a diagram showing an example of a poster editing screen 201 displayed on the touch display 14 of the information processing device 1001 in FIG. 1. The poster editing screen 201 is a screen for a user to create a poster. A background image 202 of the poster is arranged on the poster editing screen 201. The background image 202 is graphics data or image data. In addition to the background image 202, objects such as a graphics data object 203, a text data object 204, and an image data object 205 are arranged at arbitrary positions on the poster editing screen 201.

[0028] In the poster editing screen 201, the user performs an editing operation of the object 203 by, for example, inputting text data 210, selecting a fill color 211 for filling the object 203, and selecting a line type and a line color of a frame line 212 of the object 203. When editing the object 203, the user needs to select the object 203. When the user taps the selection area of ​​the object to be selected with a finger or touches it with a pointing device such as a touch pen, the touch display 14 detects the coordinates and selects the object at the top layer on the coordinates. For example, when the object 203 transitions to the selected state, the object 203 can be edited by pressing a button 209, and the object 203 is deleted by pressing a button 206. Also, a new object is added by pressing a button 208. When a button 207 is pressed, a design image of the poster created on the poster editing screen 201 is transferred to the print server 32 of the external server 1002 via the OS 13.

[0029] 3 is a flowchart showing the procedure of the poster creation process executed by the information processing device 1001 of FIG. 1. FIG. 3(a) is a diagram showing the procedure of the poster creation process executed by the information processing device 1001. The poster creation process is realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The poster creation process is executed when the user performs a predetermined operation such as an operation for adding a new object to the poster editing screen 201 displayed on the touch display 14 or an operation for editing an object already placed on the poster editing screen 201. In this embodiment, a case will be described as an example in which the poster creation process is started in response to the user pressing the button 208 for adding a new object to the poster editing screen 201.

[0030] In FIG. 3(a), first, in response to pressing of button 208, CPU 10 causes poster editing screen 201 to additionally display a new object (S301).

[0031] Next, in S302, the CPU 10 accepts a selection operation of an object displayed on the poster editing screen 201 from the user. Specifically, the CPU 10 accepts a tap operation on this object from the user. The CPU 10 puts this object into a selected state. Note that the tap operation may be not only a tap with a finger, but also a touch or click with a pointing device such as a touch pen.

[0032] Next, in S303, CPU 10 accepts an edit operation of the selected object from the user. This edit operation is, for example, an operation of moving, resizing, changing the fill color of the selected object, etc. For example, when an edit operation of moving the selected object is accepted, CPU 10 acquires the coordinates of the selected object after it is moved. Also, when an edit operation of text data (for example, text data 210) or fill color (for example, fill color 211) constituting the selected object is accepted, CPU 10 acquires the change content of the selected object.

[0033] Next, in S304, CPU 10 causes poster editing screen 201, in which the editing operation accepted in S303 is reflected in the selected object, to be displayed on touch display 14. For example, when an editing operation to move the selected object is accepted in S303, CPU 10 causes poster editing screen 201, in which the position of the selected object has been changed in accordance with the editing operation accepted in S303, to be displayed on touch display 14. Thereafter, the poster creation process ends.

[0034] In such an object editing operation, if multiple objects are arranged so as to overlap, the user cannot easily select the desired object. For example, as shown in Fig. 4, if an object 401 is arranged on top of an object 203, the selection area for the object 203 becomes narrow, and the user cannot easily select the object 203.

[0035] In contrast, in this embodiment, if it is determined that the selection operation of an object displayed on the poster editing screen 201 is not easy, a selection editing support item 90 (support UI) described later is displayed to support the editing operation of the object.

[0036] Fig. 3(b) is a flowchart showing the procedure of the first edit control process executed by the information processing device 1001 of Fig. 1. Like the above-mentioned poster creation process, the first edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The first edit control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208.

[0037] In FIG. 3(b), first, CPU 10 executes the determination process of FIG. 5 to determine whether or not the selection operation of an object arranged on poster editing screen 201 is easy (S305).

[0038] FIG. 5 is a flowchart showing the procedure of the determination process in S305 of FIG.

[0039] 5, CPU 10 executes the processes of S502 to S507 for an object arranged on poster editing screen 201. If multiple objects are arranged on poster editing screen 201, CPU 10 executes the processes of S502 to S507 for each of these multiple objects. Hereinafter, an object for which processing is being performed is referred to as a processing target object.

[0040] In S502, CPU 10 judges whether the size of the object to be processed is equal to or larger than a predetermined size. Specifically, CPU 10 judges whether the height of the object to be processed is equal to or larger than a height threshold and the width of the object to be processed is equal to or larger than a width threshold. For example, CPU 10 sets the height threshold and the width threshold based on the pixel value of the object to be processed drawn on touch display 14 and the actual size of the object to be processed printed by printer 21, and performs the judgment in S502. If it is judged that the height of the object to be processed is equal to or larger than the height threshold and the width of the object to be processed is equal to or larger than the width threshold, the judgment process in FIG. 5 proceeds to S503.

[0041] In S503, the CPU 10 determines whether the color difference in the boundary area between the object to be processed and other areas is equal to or greater than a predetermined value. A method for determining this color difference will be described with reference to FIGS.

[0042] FIG. 6 is a flowchart showing the procedure of the process of determining the color difference in S503 of FIG.

[0043] In Fig. 6, CPU 10 acquires a boundary area between the object to be processed and other areas. Here, an example will be described in which a background image 71, an object 72, and an object to be processed 73 are arranged on poster editing screen 201 as shown in Fig. 7(a). In this case, there are two boundary areas between object to be processed 73 and other areas as shown in Fig. 7(b). The first area is boundary area 74 between object 72 and object to be processed 73, and the second area is boundary area 75 between background image 71 and object to be processed 73.

[0044] Next, CPU 10 executes the processes of S603 to S605 for each of boundary regions 74, 75 acquired in S601. In S603, CPU 10 acquires the configuration of the boundary region. For example, Fig. 7(c) is an enlarged view of boundary region 74. In S603, for example, CPU 10 acquires that the configuration of boundary region 74 is the fill color of object 72 and the fill color of processing target object 73.

[0045] Next, in S604, the CPU 10 calculates the color difference from the configuration of the boundary region acquired in S603. For example, the CPU 10 calculates the color difference of the boundary region 74 from the fill color of the object 72 and the fill color of the processing target object 73. In calculating the color difference of the boundary region 74, for example, the CPU 10 may acquire from the OS 13 the RGB values ​​for displaying the object 72 on the touch display 14 and the RGB values ​​for displaying the processing target object 73, respectively, and calculate the Euclidean distance between these RGB values.

[0046] Next, in S605, the CPU 10 determines whether the color difference calculated in S604 is equal to or greater than a predetermined value. Data indicating "true" as the determination result for a boundary region where it is determined that the color difference calculated in S604 is equal to or greater than the predetermined value is stored in the ROM 11 or RAM 12. Also, data indicating "false" as the determination result for a boundary region where it is determined that the color difference calculated in S604 is less than the predetermined value is stored in the ROM 11 or RAM 12. When the CPU 10 has completed the processes of S603 to S605 for all boundary regions, it performs the process of S606.

[0047] In S606, the CPU 10 obtains the logical product of all the determination results stored in the ROM 11 or the RAM 12. Next, in S607, the CPU 10 determines whether the logical product of S606 is "true". If the logical product of S606 is determined to be "true", that is, if the color differences calculated in all boundary regions are equal to or greater than a predetermined value, the CPU 10 determines that the color difference in the boundary region of the object to be processed is equal to or greater than a predetermined value (S608). On the other hand, if the logical product of S606 is determined to be "false", that is, if the color difference calculated in any boundary region is less than the predetermined value, the CPU 10 determines that the color difference in the boundary region of the object to be processed is not equal to or greater than a predetermined value (S609).

[0048] Returning to the explanation of S503 in Fig. 5, if it is determined that the color difference in the boundary region of the processing target object is equal to or greater than the predetermined value, the determination process in Fig. 5 proceeds to S504.

[0049] In S504, the CPU 10 determines whether the object to be processed is located in the topmost layer. If it is determined that the object to be processed is not located in the topmost layer, but in a layer other than the topmost layer, the determination process in FIG. 5 proceeds to S505.

[0050] In S505, CPU 10 judges whether the area of ​​the visible area of ​​the object to be processed is equal to or larger than a predetermined value. The visible area is an area of ​​the object to be processed that is not overlapped with an object above the object to be processed and is visible. The judgment of the area of ​​the visible area will be described with reference to Figs. 8(a) to (d).

[0051] FIG. 8(a) is a diagram showing a case where a processing target object 81 and an object 82 are arranged on a poster editing screen 201. The visible area of ​​the processing target object 81 in such a case is an area 84 in FIG. 8(c) where no object 82 above the processing target object 81 exists. FIG. 8(b) is a diagram showing a case where a processing target object 81 and an object 83 are arranged on a poster editing screen 201. The visible area of ​​the processing target object 81 in such a case is an area 85 in FIG. 8(d) where no object 83 above the processing target object 81 exists. In S505, the CPU 10 acquires the area of ​​the area 84 or the area of ​​the area 85 as the area of ​​the visible area, and determines whether the area of ​​the acquired visible area is equal to or larger than a predetermined value. Here, the CPU 10 may acquire the proportion of the visible area in the processing target object, and determine whether the proportion of the acquired visible area is equal to or larger than a predetermined proportion. Furthermore, the size and resolution of touch display 14, the display magnification of poster editing screen 201, etc. may be used as a threshold value such as a predetermined value or a predetermined ratio used in the determination in S505.

[0052] If it is determined in S505 that the area of ​​the visible region of the processing target object is equal to or larger than a predetermined value, the determination process in Fig. 5 proceeds to S506. Also, if it is determined in S504 that the processing target object is arranged in the top layer, the determination process in Fig. 5 proceeds to S506. In S506, CPU 10 determines that the selection operation of the processing target object is easy.

[0053] On the other hand, if it is determined in S505 that the area of ​​the visible region of the object to be processed is less than the predetermined value, the determination process in FIG. 5 proceeds to S507. Also, if it is determined in S502 that the size of the object to be processed is less than the predetermined size, that is, the height of the object to be processed is not equal to or greater than the height threshold or the width of the object to be processed is not equal to or greater than the width threshold, the determination process in FIG. 5 proceeds to S507. Furthermore, if it is determined in S503 that the color difference in the boundary region of the object to be processed is not equal to or greater than the predetermined value, the determination process in FIG. 5 proceeds to S507. In S507, the CPU 10 determines that the selection operation of the object to be processed is not easy. In this way, in this embodiment, the determination process in FIG. 5 is performed on all objects arranged on the poster editing screen 201.

[0054] Returning to the explanation of S305 in Fig. 3(b), if CPU 10 determines in S305 that the selection operation of an object arranged on poster editing screen 201 is easy, the first edit control process ends. On the other hand, if CPU 10 determines in S305 that the selection operation of an object arranged on poster editing screen 201 is not easy, the first edit control process proceeds to S306.

[0055] In S306, the CPU 10 displays an item (hereinafter referred to as a selection editing support item) that supports the selection and editing of an object for which it has been determined that the selection operation is not easy. For example, of the objects 203 and 401 arranged on the poster editing screen 201 shown in Fig. 4, the selection editing support item 90 shown in Fig. 9(a) corresponding to the object 203 for which it has been determined in S305 that the selection operation is not easy is displayed. The selection editing support item 90 not only indicates that the selection editing support item 90 has put the object 203 in a selected state, but also has some of the functions of editing the object 203.

[0056] The selection editing support item 90 includes items 91, 92, and 93. The item 91 has a function of resizing the object 203. For example, when the item 91 is dragged toward the outside of the object 203, the object 203 is enlarged. When the item 91 is dragged toward the inside of the object 203, the object 203 is reduced. The item 92 has a function of rotating the object 203. For example, when a circle is drawn on the item 92, the object 203 is rotated in the direction of the circle. The item 93 has a function of moving the object 203. For example, when a direction is drawn on the item 93, the object 203 is moved in the direction of the drag. Note that the configuration of the selection editing support item 90 shown in FIG. 9(a) is one example, and other configurations may be used. FIG. 9(b) is a diagram showing an example of the poster editing screen 201 on which the selection editing support item 90 of the object 203 determined to be difficult to select in S305 is displayed.

[0057] When the process of S306 is completed, the CPU 10 selects the object 203 on which the selected edit support item 90 is displayed (S307). This allows the user to perform an edit operation on the object 203 using the selected edit support item 90. Thereafter, the first edit control process ends.

[0058] According to the embodiment described above, when it is determined that an object displayed on the poster editing screen 201 is not easy to select, the selection editing assistant item 90 that assists in the editing operation of the object is displayed, and the object is placed in a selected state. This makes it possible to easily select an object that is difficult to select by a tap operation.

[0059] In the above embodiment, when it is determined that the size of an object displayed on poster editing screen 201 is less than a predetermined size, it is determined that the selection operation of the object is not easy. This makes it possible to easily select an object that is difficult to select by tapping due to its small size.

[0060] Furthermore, in the above-described embodiment, if it is determined that the color difference in the boundary area of ​​an object displayed on poster editing screen 201 is not equal to or greater than a predetermined value, it is determined that the selection operation of the object is not easy. This makes it possible to easily select an object that is difficult to select by tapping because the color difference with other areas is small and the placement position is difficult to visually recognize.

[0061] In the above embodiment, if it is determined that an object displayed on poster editing screen 201 is not the topmost layer and the area of ​​the viewable region of the object is less than a predetermined value, it is determined that the object is not easy to select. This makes it possible to easily select an object that is difficult to select by tapping because it overlaps with an object in an upper layer.

[0062] In the above-described embodiment, the selection and editing assistant item 90 has an object rotation function, a movement function, and a resizing function, so that the user can easily perform editing operations such as rotation, movement, and resizing on an object that is difficult to select by a tap operation.

[0063] In the embodiment described above, whether or not the object arranged on poster editing screen 201 is easy to select is determined in response to a new object being arranged on poster editing screen 201 in response to pressing of button 208. This makes it possible to easily select an object that has become difficult to select by a tap operation due to the arrangement of the new object.

[0064] In the present embodiment, a configuration has been described in which the first editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208, but the present invention is not limited to this configuration. For example, the first editing control process may be executed in response to the content of an editing operation by the user being reflected in an object placed on the poster editing screen 201. This makes it possible to easily select an object that has become difficult to select by a tap operation due to the content of an editing operation by the user being reflected in an object placed on the poster editing screen 201.

[0065] In addition, in this embodiment, the selected editing support item 90 may be controlled to be hidden at a predetermined timing. Here, for example, when the user moves the object 31 from the position in Fig. 9(b) to the position in Fig. 10(a), the area of ​​the visible region of the object 203 to which the selected editing support item 90 is added becomes equal to or larger than a predetermined value. In such a case, the object 203 can be easily selected and operated, and the selected editing support item 90 becomes unnecessary.

[0066] In contrast, in this embodiment, when it is determined that the selection operation of an object to which a selection editing support item 90 is assigned is easy as the content of the editing operation performed by the user is reflected in the object placed on the poster editing screen 201, the selection editing support item 90 assigned to the object is hidden.

[0067] Fig. 11 is a flowchart showing the procedure of the second edit control process executed by the information processing device 1001 of Fig. 1. Like the above-mentioned first edit control process, the second edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The second edit control process is executed when S307 of the above-mentioned first edit control process is completed, a selection edit support item 90 for any object is displayed on the poster edit screen 201, and the object is edited according to an edit operation by the user.

[0068] 11, first, the CPU 10 judges whether the object to which the selection edit support item 90 has been added in S306 described above is easy to select (S1101). For example, the CPU 10 performs the process of S1101 on the object 203 to which the selection edit support item 90 has been added in FIG. 10(a). Specifically, the CPU 10 performs the same processes as S502 to S507 described above on the object 203 to judge whether the selection operation of the object 203 is easy. If it is judged in S1101 that the selection operation of the object 203 is not easy, the second edit control process ends, and the display of the selection edit support item 90 of the object 203 continues.

[0069] In S1101, when it is determined that the selection operation of the object 203 is easy, the CPU 10 makes the selected editing support item 90 invisible (S1102), for example, as shown in Fig. 10(b). In addition, the CPU 10 cancels the selection state of the object 203. Thereafter, the second editing control process ends.

[0070] In the above-mentioned second editing control process, when it is determined that the selection operation of the object 203 to which the selection editing support item 90 is attached is easy as the content of the editing operation by the user is reflected in the object arranged on the poster editing screen 201, the selection editing support item 90 of the object 203 is hidden. This makes it possible to prevent the selection editing support item 90 that has become unnecessary due to the editing operation by the user from continuing to be displayed, thereby reducing the operability for the user.

[0071] Incidentally, there are cases where the user wishes to change the content displayed on the touch display 14 by performing a swipe operation on the poster editing screen 201. For example, when an object is enlarged and displayed over the entire area of ​​the poster editing screen 201, the user may perform a swipe operation on the poster editing screen 201 in order to display another object on the touch display 14. In addition, when a poster with multiple pages is being edited on the poster editing screen 201, the user may perform a swipe operation on the poster editing screen 201 in order to display another page on the touch display 14. If the selection editing support item 90 continues to be displayed while such a swipe operation is being performed, the user may feel bothered because he or she must be conscious of not accidentally tapping the selection editing support item 90 when performing the operation.

[0072] In contrast, in this embodiment, when the start of a swipe operation is detected in a state in which the selected editing support item 90 is displayed on the poster editing screen 201, the selected editing support item 90 is hidden.

[0073] Fig. 12 is a flowchart showing the procedure of a third edit control process executed by the information processing device 1001 of Fig. 1. Like the first and second edit control processes described above, the third edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The third edit control process is executed when the CPU 10 receives a signal corresponding to a user operation from the touch sensor 142 after S307 of the first edit control process described above is completed and a selection edit support item 90 of any object is displayed on the poster editing screen 201.

[0074] 12, the CPU 10 determines whether or not a signal received from the touch sensor 142 is a signal indicating a swipe operation (S1201). Fig. 13 is a diagram showing a state in which a user performs a swipe operation on the poster editing screen 201 using a pointing device such as the touch pen 131. Note that the user may perform the swipe operation with, for example, a finger other than the touch pen 131.

[0075] If it is determined in S1201 that the signal received from the touch sensor 142 is not a signal indicating a swipe operation, the third edit control process ends, and the display of the selected edit support item 90 continues. If it is determined in S1201 that the signal received from the touch sensor 142 is a signal indicating a swipe operation, the CPU 10 hides the selected edit support item 90 (S1202). As a result, for example, the selected edit support item 90 that has been assigned to the object 203 as shown in FIG. 9(b) is hidden as shown in FIG. 13. Thereafter, the third edit control process ends.

[0076] In the above-described third editing control process, when the start of a swipe operation is detected in a state in which the selected editing support item 90 is displayed on the poster editing screen 201, the selected editing support item 90 is hidden. That is, when the start of a swipe operation is detected, the selected editing support item 90 is immediately hidden. This can reduce the inconvenience of the selected editing support item 90 during a swipe operation.

[0077] Furthermore, in this embodiment, when an object other than the object to which the selection edit support item 90 is added is selected by a user operation, the selection edit support item 90 may be hidden.

[0078] 14 is a flowchart showing the procedure of the fourth editing control process executed by the information processing device 1001 of FIG. 1. The fourth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12, similarly to the first, second, and third editing control processes described above. The fourth editing control process is executed when the information processing device 1001 receives a user operation after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201. For example, when a selection operation of an object other than the object to which the selection editing support item 90 is added in S306 is received from the user, the object selected by the user is selected on the poster editing screen 201.

[0079] In FIG. 14, the CPU 10 determines whether the selected object is an object other than the object to which the selected edit support item 90 was added in S306 (S1401).

[0080] If it is determined in S1401 that the selected object is the object to which the selected edit support item 90 was added in S306, the fourth edit control process ends and the display of the selected edit support item 90 continues.

[0081] If it is determined in S1401 that the selected object is an object other than the object to which the selected edit support item 90 was added in S306, the CPU 10 makes the selected edit support item 90 invisible (S1402) and ends this process.

[0082] In the above-mentioned fourth editing control process, when an object other than the object to which the selection editing support item 90 is added is selected by a user operation, the selection editing support item 90 is hidden. This makes it possible to prevent the selection editing support item 90 of the object whose selection state has been released from continuing to be displayed, thereby preventing a decrease in user operability.

[0083] Furthermore, in this embodiment, when it is determined that the operation of selecting an object to which the selection edit support item 90 is attached is not easy, the selection edit support item 90 may be temporarily hidden.

[0084] Fig. 15 is a flowchart showing the procedure of a fifth edit control process executed by the information processing device 1001 of Fig. 1. Like the first, second, third and fourth edit control processes described above, the fifth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The fifth edit control process is executed after S307 of the first edit control process described above is completed and the selection edit support item 90 of any object is displayed on the poster edit screen 201.

[0085] 15, the CPU 10 determines whether the object to which the selected edit support item 90 has been added in S306 described above is easy to select (S1501). In S1501, the same processes as those in S502 to S507 described above are performed on the object to which the selected edit support item 90 has been added in S306. If it is determined in S306 that the object to which the selected edit support item 90 has been added is not easy to select, the fifth edit control process proceeds to S1502.

[0086] In S1502, the CPU 10 temporarily hides the selected editing support item 90. Next, the CPU 10 determines whether or not a predetermined condition is met (S1503). The CPU 10 waits until the predetermined condition is met. For example, if a predetermined time has elapsed since the selected editing support item 90 was hidden in S1502, or if pressing of the button 207, which is a print instruction, is detected, it is determined that the predetermined condition is met. In this case, the CPU 10 redisplays the selected editing support item 90 that was hidden in S1502 (S1504), and ends this process.

[0087] If it is determined in S1501 that the selection operation of the object to which the selection edit support item 90 has been added in S306 is easy, the CPU 10 makes the selection edit support item 90 invisible (S1505) and ends this processing.

[0088] In the fifth editing control process described above, if it is determined that the selection operation of the object to which the selection editing support item 90 is applied is not easy, the selection editing support item 90 is temporarily hidden. This makes it possible to prevent the user from mistaking the selection editing support item 90 for part of the design, and thus allows the user to properly check the finished design without impairing the operability of the editing operation.

[0089] In this embodiment, the Web browser 15 or the native application 16 may be configured to be able to switch whether or not to execute the first, second, third, fourth, and fifth edit control processes.

[0090] Furthermore, in this embodiment, when other objects are arranged so as to cover the selected edit support item 90, the selected edit support item 90 may be controlled to be displayed in the uppermost layer.

[0091] Fig. 16(a) is a diagram showing a case where object 161 is arranged so as to cover object 203. When objects 161 and 203 are arranged in this manner, the above-mentioned editing control process causes selected editing support items 90 for object 203 to be displayed as shown in Fig. 16(b). However, some items of selected editing support items 90, for example item 91, are hidden by object 161, and the user cannot see item 91.

[0092] In contrast, in this embodiment, when another object is arranged so as to cover at least a part of the selected edit support item 90, the selected edit support item 90 is controlled to be displayed in the uppermost layer.

[0093] Fig. 17 is a flowchart showing the procedure of a sixth edit control process executed by the information processing device 1001 of Fig. 1. Like the first, second, third, fourth and fifth edit control processes described above, the sixth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The sixth edit control process is executed after S307 of the first edit control process described above is completed and a selected edit support item for any object is displayed on the poster edit screen 201.

[0094] In FIG. 17, the CPU 10 determines whether at least some of the items included in the selected editing support items 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more (S1701). If it is determined that at least some of the items included in the selected edit support item 90 displayed in S306 do not overlap with an object above the object corresponding to the selected edit support item 90 by a predetermined ratio or more, the sixth edit control process ends. In other words, the display of the selected edit support item 90 continues as is.

[0095] If it is determined in S1701 that at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more, the sixth editing control process proceeds to S1702.

[0096] In S1702, CPU 10 displays selected edit support item 90 in the top layer of poster edit screen 201. As a result, for example, item 93 hidden by object 181 as in Fig. 18(a) is displayed on poster edit screen 201 as in Fig. 18(b). Thereafter, the sixth edit control process ends.

[0097] In the sixth editing control process described above, when other objects are arranged so as to cover at least a part of the selected editing support item 90, the selected editing support item 90 is displayed in the top layer. This allows the user to view all items constituting the selected editing support item 90 regardless of the arrangement positions of the other objects.

[0098] In addition, in this embodiment, if another object is positioned so as to cover at least a portion of the selected editing support item 90, the selected editing support item 90 may be controlled to be displayed in a position that does not overlap with the other object.

[0099] Fig. 19 is a flowchart showing the procedure of a seventh edit control process executed by the information processing device 1001 of Fig. 1. Like the first, second, third, fourth, fifth and sixth edit control processes described above, the seventh edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12. The seventh edit control process is executed after S307 of the first edit control process described above is completed and a selection edit support item 90 for any object is displayed on the poster edit screen 201.

[0100] In FIG. 19, the CPU 10 determines whether at least some of the items included in the selected editing support items 90 displayed in S306 overlap with objects above the object corresponding to the selected editing support items 90 by a predetermined percentage or more (S1901). If it is determined that at least some of the items included in the selected edit support item 90 displayed in S306 do not overlap with an object above the object corresponding to the selected edit support item 90 by a predetermined ratio or more, the seventh edit control process ends. In other words, the display of the selected edit support item 90 continues as is.

[0101] If it is determined in S1901 that at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more, the seventh editing control process proceeds to S1902.

[0102] In S1902, the CPU 10 corrects the display position of the selected edit support item 90 to a position that is visible on the poster editing screen 201. Specifically, the CPU 10 corrects the display position of the selected edit support item 90 to a position where no object is located above the object corresponding to this selected edit support item 90. Thereafter, the seventh edit control process ends.

[0103] In the seventh editing control process described above, when another object is arranged so as to cover at least a part of the selected editing support item 90, the selected editing support item 90 is displayed at a position where it does not overlap with the other object. This makes it possible to suppress a decrease in operability of the selected editing support item 90 caused by the arrangement position of the other object.

[0104] In addition, in the above-mentioned S1902, the display position of the selected editing support item 90 may be corrected based on the user's handedness information. The user's handedness information is information indicating the user's handedness (right-handed or left-handed), and is stored in the ROM 11 or the RAM 12. For example, when the user's handedness information indicates right-handedness, items 902 and 903 corresponding to items 92 and 93 included in the selected editing support item 90 are displayed at the right end of the poster editing screen 201 as shown in FIG. 20(a). When the user's handedness information indicates left-handedness, items 902 and 903 corresponding to items 92 and 93 included in the selected editing support item 90 are displayed at the left end of the poster editing screen 201 as shown in FIG. 20(b).

[0105] By controlling in this way, the operability of the selected editing auxiliary item 90 can be improved.

[0106] In addition, in this embodiment, if there are multiple objects that are determined to be difficult to select in S305 described above, a selection editing support item 90 for the lowest object among the multiple objects that are determined to be difficult to select may be displayed.

[0107] Fig. 21 is a flowchart showing the procedure of an eighth edit control process executed by the information processing device 1001 of Fig. 1. The eighth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12, similar to the first, second, third, fourth, fifth, sixth, and seventh edit control processes described above. The eighth edit control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208 after S307 of the first edit control process described above is completed and the selection edit support item 90 of any object is displayed on the poster editing screen 201.

[0108] 21, CPU 10 performs the determination process of Fig. 5 to determine whether or not there are multiple objects that are difficult to select on poster editing screen 201 (S2101). If it is determined that there are not multiple objects that are difficult to select on poster editing screen 201, the eighth edit control process ends.

[0109] If it is determined in S2101 that there are a plurality of objects on the poster editing screen 201 that are difficult to select, the eighth editing control process proceeds to S2102.

[0110] In S2102, the CPU 10 displays the selection and editing support item 90 for the object at the bottom layer among the plurality of objects determined to be difficult to select. For example, in Fig. 22, the selection and editing support item 90 for the object 203 at the bottom layer among the objects 203 and 221 determined to be difficult to select is displayed.

[0111] Next, the CPU 10 selects the object corresponding to the selected edit support item 90 displayed in S2102 (S2103), and ends this process.

[0112] In the above-mentioned eighth editing control process, when there are a plurality of objects determined to be difficult to select, a selection editing support item 90 for the object at the bottom layer among the plurality of objects determined to be difficult to select is displayed. The object at the bottom layer is also placed in a selected state. This makes it possible to easily select the object at the bottom layer that is the most difficult to select when a plurality of objects that are difficult to select are arranged on the poster editing screen 201.

[0113] In addition, in this embodiment, when there are a plurality of objects determined to be difficult to select, an operation button for switching the object to which the selection editing support item 90 is to be added, for example, the button 231 in FIG. 23(a) may be displayed. For example, in FIG. 23(a), the selection editing support item 90 of the object 203 is displayed by the above-mentioned S2102. When the user presses the button 231 in this state, the CPU 10 displays the selection editing support item 90 of the object 221 out of the objects 203 and 221 determined to be difficult to select in S305, as shown in FIG. 23(b). In this way, in this embodiment, the object to which the selection editing support item 90 is to be added can be switched according to the pressing of the button 231. Note that the switching of the object to which the selection editing support item 90 is to be added may be switched according to the stacking order, for example, from the lowest layer.

[0114] By controlling in this manner, when multiple objects that are difficult to select are placed on the poster editing screen 201, the user can easily switch between these objects to select the object to which the selection editing support item 90 is to be applied.

[0115] In this embodiment, the selection and editing support item 90 may be displayed for each of all objects that are determined to be difficult to select.

[0116] Fig. 24 is a flowchart showing the procedure of a ninth edit control process executed by the information processing device 1001 of Fig. 1. The ninth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12, similar to the first, second, third, fourth, fifth, sixth, seventh, and eighth edit control processes described above. The ninth edit control process is executed when a new object is placed on the poster edit screen 201 in response to the user pressing the button 208 after S307 of the first edit control process described above is completed and the selection edit support item 90 of any object is displayed on the poster edit screen 201.

[0117] 24, similarly to S2101 described above, CPU 10 determines whether or not there are multiple objects that are difficult to select on poster editing screen 201 (S2401). If it is determined that there are not multiple objects that are difficult to select on poster editing screen 201, the ninth editing control process ends.

[0118] If it is determined in S2401 that there are multiple objects on the poster editing screen 201 that are difficult to select, the ninth editing control process proceeds to S2402. For example, in FIG. 25(a), objects 251 and 252 of the same color as the background image 202 are arranged on the background image 202. Note that although the borders of the objects 251 and 252 are shown by dotted lines in FIG. 25(a), these borders are not actually displayed. In such a case, it is determined in S2401 that there are multiple objects on the poster editing screen 201 that are difficult to select.

[0119] In S2402, the CPU 10 displays a selection edit support item for each of all objects determined to be difficult to select. For example, as shown in Fig. 25(b), items 911, 912, and 913 are displayed as the selection edit support items 90 for the object 251 determined to be difficult to select. Also, items 921, 922, and 923 are displayed as the selection edit support items 90 for the object 252 determined to be difficult to select.

[0120] Next, the CPU 10 selects the lowest object among the objects determined to be difficult to select (S2403). For example, in FIG. 25(b), the lowest object 251 among the objects 251 and 252 determined to be difficult to select is selected. At this time, the colors of the items 911, 912, and 913 may be changed to items 931, 932, and 933 in FIG. 25(c), respectively, so as to clearly indicate that the object 251 is selected. Furthermore, the CPU 10 may switch the selected object from the object 251 to the object 252 in response to the user pressing the button 231. At this time, the colors of the items 931, 932, and 933 are changed to their original colors (the colors of the items 911, 912, and 913), respectively, and the colors of the items 921, 922, and 923 are changed to items 931, 932, and 933, respectively. When the process of S2403 is completed, the ninth edit control process ends.

[0121] In the above-mentioned ninth editing control process, the selection editing assistant item 90 is displayed for each of all objects determined to be difficult to select, and the object at the bottom of the objects determined to be difficult to select is selected. This allows the user to easily edit the object at the bottom that is the most difficult to select, and also allows the user to easily grasp the arrangement positions of other objects that are difficult to select.

[0122] In addition, in this embodiment, when a plurality of objects determined to be difficult to select are included in a predetermined area, a selection assistance UI 272 (described later) may be displayed to assist in the selection of the plurality of objects.

[0123] Fig. 26 is a flowchart showing the procedure of a tenth edit control process executed by the information processing device 1001 of Fig. 1. The tenth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or the RAM 12, similar to the first, second, third, fourth, fifth, sixth, seventh, eighth, and ninth edit control processes described above. The tenth edit control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208 after S307 of the first edit control process described above is completed and the selection edit support item 90 of any object is displayed on the poster editing screen 201.

[0124] 26, similarly to S2101 and S2401 described above, CPU 10 determines whether or not there are multiple objects that are difficult to select on poster editing screen 201 (S2601). If it is determined that there are not multiple objects that are difficult to select on poster editing screen 201, the tenth editing control process ends.

[0125] If it is determined in S2601 that there are a plurality of objects on the poster editing screen 201 that are difficult to select, the tenth editing control process proceeds to S2602.

[0126] In S2602, CPU 10 judges whether or not there are multiple objects that are difficult to select in a predetermined area corresponding to the user's touch position. For example, as shown in Fig. 27, object 203 and object 221 are placed in a position that is judged to be difficult to select, and the user taps area 271 as an editing operation. In S2602, CPU 10 judges whether or not there are multiple objects that are difficult to select in area 271.

[0127] If it is determined in S2602 that there are not multiple objects that are difficult to select within the predetermined area, the tenth edit control process ends. On the other hand, if it is determined in S2602 that there are multiple objects that are difficult to select within the predetermined area, the tenth edit control process proceeds to S2603.

[0128] In S2603, the CPU 10 displays the selection assistant UI 272 for selecting an object. Specifically, the CPU 10 acquires information on all objects in the area 271, and displays a list of buttons indicating attribute information of each object (such as the fill color of the object and text information associated with the object) in the assistant UI 272. For example, if the user has input text data of "Summer Sale" to the object 203, the CPU 10 displays "Summer Sale" as the button 274 corresponding to the object 203. Also, if the user has set the fill color of the object 222 to blue, the CPU 10 displays the color name as the button 275 corresponding to the object 222. When the user taps the button 274, the CPU 10 selects the object 203. Also, when the user taps the button 275, the CPU 10 selects the object 222. When the process of S2603 is completed, the tenth editing control process ends.

[0129] In the above-mentioned tenth editing control process, when a plurality of objects determined to be difficult to select are included in a predetermined area, a selection assistance UI 272 that assists the user in selecting the plurality of objects is displayed. This allows the user to easily select the plurality of objects determined to be difficult to select.

[0130] In the above-mentioned embodiment, the layout software is described as an example, but the present invention may be applied to software using object detection technology such as CNN or YOLO (hereinafter referred to as object detection software). Specifically, the UI diagram of FIG. 28 is used for explanation. When the object detection software detects the pressing of the button 286, it starts a process of detecting the search target 285 from the source image 281. The object detection software uses the object detection technology to display the detection areas 282 to 284 in a rectangle on the source image 281, but the user may select the detection area 282 using the object detection software. If the object detection software determines in the above-mentioned S305 that the selection operation of the detection area 282 is not easy, a selection editing assistant item is displayed. As a result, the object detection software can also achieve the same effects as those of the above-mentioned embodiment.

[0131] Needless to say, the object of the present invention can also be achieved by supplying a recording medium on which is recorded the program code of software that realizes the functions of the above-mentioned embodiments to a system or device, and having the computer (or CPU or MPU) of the system or device read and execute the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is stored constitutes the present invention.

[0132] Examples of storage media for supplying the program code include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, DVDs, and the like.

[0133] Furthermore, it goes without saying that the functions of the above-mentioned embodiments are not only realized by the computer executing the program code it has read, but also includes cases where an Operating System (OS) running on a computer performs some or all of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments are realized through that processing.

[0134] Furthermore, it goes without saying that this also includes cases where the program code read from the storage medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to the computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

[0135] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0136] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A program that causes an information processing device having a touch display that accepts user operations to execute the following steps: displaying an object on the touch display in accordance with the user operations accepted by the touch display; determining whether the object displayed on the touch display is easy to select; and, if it is determined that the object displayed on the touch display is not easy to select, displaying an auxiliary UI that assists in editing the object and setting the object to a selected state. (Configuration 2) The program described in configuration 1, characterized in that in the judgment process, if the size of an object displayed on the touch display is less than a predetermined size, it is judged that the selection operation of the object is not easy. (Configuration 3) The program described in configuration 1 or 2, characterized in that in the judgment process, if the color difference in the boundary area between an object displayed on the touch display and an area other than the object is not a predetermined value or more, it is judged that the selection operation of the object is not easy. (Configuration 4) A program described in any one of configurations 1 to 3, characterized in that in the judgment process, if an object displayed on the touch display is not in the top layer and the area of ​​the visible area of ​​the object displayed on the touch display is less than a predetermined value, it is judged that the object is not easy to select. (Configuration 5) The program described in any one of configurations 1 to 4, wherein the auxiliary UI has a function for rotating, moving, and resizing the object. (Configuration 6) The program according to any one of configurations 1 to 5, characterized in that the program causes the information processing device to perform the determination step when a new object is displayed on the touch display in accordance with the user operation. (Configuration 7) A program described in any one of configurations 1 to 5, characterized in that the program causes the information processing device to perform the judgment process in accordance with the editing content corresponding to the user operation being reflected in an object displayed on the touch display. (Configuration 8) The program described in Configuration 7, characterized in that when it is determined that the object for which the auxiliary UI is displayed is easy to select in accordance with the editing content corresponding to the user operation being reflected in the object displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI and canceling the selection state of the object. (Configuration 9) A program described in any one of configurations 1 to 8, characterized in that, after the auxiliary UI is displayed on the touch display, if it is determined that the selection operation of an object corresponding to the auxiliary UI is not easy, the program causes the information processing device to execute a process of temporarily hiding the auxiliary UI. (Configuration 10) A program described in any one of configurations 1 to 9, characterized in that when the start of a swipe operation for changing the display content of the touch display is detected while the auxiliary UI is displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI. (Configuration 11) The program described in any one of configurations 1 to 10, characterized in that when an object other than an object corresponding to the auxiliary UI is selected in accordance with the user operation, the program causes the information processing device to execute a process of hiding the auxiliary UI. (Configuration 12) A program described in any one of configurations 1 to 11, characterized in that when another object is placed so as to cover at least a portion of the auxiliary UI, the program causes the information processing device to execute a process of displaying the auxiliary UI in the top layer. (Configuration 13) A program described in any one of configurations 1 to 11, characterized in that, when another object is placed so as to cover at least a portion of the auxiliary UI, the program causes the information processing device to execute a step of displaying the auxiliary UI in a position that does not overlap the other object. (Configuration 14) A program described in any one of configurations 1 to 13, characterized in that the program causes the information processing device to execute a process of controlling the display position of the auxiliary UI based on information indicating the dominant hand of a user operating the information processing device. (Configuration 15) A program described in any one of configurations 1 to 14, characterized in that when it is determined that selection operation of multiple objects among the objects displayed on the touch display is not easy, the program causes the information processing device to execute a process of displaying the auxiliary UI of the lowest object among the multiple objects determined to be difficult to select and selecting the lowest object. (Configuration 16) The program described in Configuration 15, characterized in that the information processing device is caused to execute a process of switching an object to which the auxiliary UI is to be provided from among a plurality of objects determined to be difficult to select in accordance with a user operation. (Configuration 17) A program described in any one of configurations 1 to 14, characterized in that when it is determined that selection operation of multiple objects among the objects displayed on the touch display is not easy, the program causes the information processing device to execute a process of displaying the auxiliary UI of all objects for which the selection operation is determined to be not easy, and selecting the lowest object among the multiple objects for which the selection operation is determined to be not easy. (Configuration 18) A program described in any one of configurations 1 to 17, characterized in that when a predetermined area corresponding to the touch position of the touch display by the user includes multiple objects for which the selection operation is determined to be difficult, the program causes the information processing device to execute a step of displaying another auxiliary UI different from the auxiliary UI, which assists in the selection operation of the multiple objects. [Explanation of symbols]

[0137] 10 CPU 14 Touch Display 31 Objects 71 Background Images 72 Objects 73 Processing Objects 74 Boundary area 75 Boundary area 81 Processing Objects 82 Objects 83 Objects 90 Selection editing support items 91 items 92 items 93 items 161 Objects 181 Objects 203 Objects 221 Objects 222 Objects 231 Button 251 Objects 252 Objects 272 Selection Assistance UI 401 Object 902 items 903 items 911 items 912 items 913 items 931 items 932 items 933 items 1001 Information processing equipment

Claims

1. An information processing device having a touch display that accepts user operations, displaying an object on the touch display in accordance with a user operation received on the touch display; a determination step of determining whether or not a selection operation of an object displayed on the touch display is easy; When it is determined that the selection operation of an object displayed on the touch display is not easy, a program that executes a process of displaying an auxiliary UI that assists in editing operations of the object and setting the object in a selected state.

2. 2. The program according to claim 1, wherein in the determining step, if a size of an object displayed on the touch display is less than a predetermined size, it is determined that the object is not easy to select.

3. The program according to claim 1, characterized in that in the judgment process, if the color difference in the boundary area between an object displayed on the touch display and an area other than the object is not a predetermined value or more, it is judged that the object is not easy to select.

4. The program according to claim 1, characterized in that in the judgment process, if an object displayed on the touch display is not in the top layer and the area of ​​the visible area of ​​the object displayed on the touch display is less than a predetermined value, it is judged that the object is not easy to select.

5. 2. The program according to claim 1, wherein the auxiliary UI has a function of rotating, moving and resizing the object.

6. 2. The program according to claim 1, further comprising causing the information processing device to perform the determining step when a new object is displayed on the touch display in accordance with the user's operation.

7. 2. The program according to claim 1, further comprising causing the information processing device to perform the determining step in response to an edit content corresponding to the user operation being reflected in an object displayed on the touch display.

8. The program described in claim 7, characterized in that when it is determined that the object for which the auxiliary UI was displayed is easy to select in accordance with the editing content corresponding to the user operation being reflected in the object displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI and canceling the selection state of the object.

9. The program according to claim 1, further comprising a step of causing the information processing device to execute a process of temporarily hiding the auxiliary UI when it is determined that an object corresponding to the auxiliary UI is not easy to select after the auxiliary UI is displayed on the touch display.

10. The program according to claim 1, characterized in that when the start of a swipe operation for changing the display content of the touch display is detected while the auxiliary UI is displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI.

11. 2. The program according to claim 1, further comprising: making the information processing device execute a step of hiding the auxiliary UI when an object other than the object corresponding to the auxiliary UI is selected in accordance with the user operation.

12. 2. The program according to claim 1, further comprising: displaying the auxiliary UI in the uppermost layer when another object is arranged so as to cover at least a part of the auxiliary UI.

13. The program according to claim 1, characterized in that, when another object is arranged so as to cover at least a portion of the auxiliary UI, the program causes the information processing device to execute a process of displaying the auxiliary UI in a position that does not overlap the other object.

14. 2. The program according to claim 1, further comprising causing the information processing device to execute a step of controlling a display position of the auxiliary UI based on information indicating a dominant hand of a user who operates the information processing device.

15. The program according to claim 1, characterized in that when it is determined that it is not easy to select multiple objects among the objects displayed on the touch display, the program causes the information processing device to execute a process of displaying the auxiliary UI of the lowest object among the multiple objects determined to be difficult to select and selecting the lowest object.

16. The program according to claim 15, characterized in that the information processing device is caused to execute a process of switching an object to which the auxiliary UI is to be assigned from among a plurality of objects determined to be difficult to select in accordance with the user operation.

17. The program described in claim 1, characterized in that when it is determined that selection operation of multiple objects among the objects displayed on the touch display is not easy, the program causes the information processing device to execute a process of displaying the auxiliary UI for all objects determined to be difficult to select and selecting the lowest object among the multiple objects determined to be difficult to select.

18. The program described in claim 1, characterized in that when a predetermined area corresponding to a touch position on the touch display by a user includes multiple objects for which the selection operation is determined to be difficult, the program causes the information processing device to execute a process of displaying another auxiliary UI different from the auxiliary UI, which assists in the selection operation of the multiple objects.

19. An information processing device having a touch display that accepts user operations, means for displaying an object on the touch display in accordance with a user operation received on the touch display; a means for determining whether an object displayed on the touch display can be easily selected; An information processing device characterized in that, when it is determined that the selection operation of an object displayed on the touch display is not easy, an auxiliary UI is displayed to assist in the editing operation of the object, and the object is placed in a selected state.

20. A method for controlling an information processing device having a touch display that accepts user operations, comprising: displaying an object on the touch display in accordance with a user operation received on the touch display; determining whether an object displayed on the touch display can be easily selected; A control method for an information processing device, comprising the steps of: when it is determined that the selection operation of an object displayed on the touch display is not easy, displaying an auxiliary UI that assists in editing operations of the object and setting the object in a selected state.

Images