Programs and Electronic Devices

Abstract

To provide a program and an electronic device in which an image processing device efficiently performs preliminary action for hastening completion of specific action using the image processing device.SOLUTION: An electronic device 1 comprises a millimeter wave radar 15 and a user IF 13. The electronic device causes the user IF 13 to display a specific object which receives an input of execution indication of specific action, and causes a printer 3 to execute the specific action when operation to the specific object is received. In addition the electronic device 1 acquires a position in a screen of the specific object displayed on the user IF 13, acquires the position of a fingertip based on a waveform signal outputted from the millimeter wave radar 15, determines whether or not the acquired position of the fingertip is within a predetermined range based on the position of the specific object, and transmits to the printer 3 a preliminary action command which performs instruction of executing preliminary action for hastening completion of the specific action when determining that it is within the predetermined range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technical field disclosed in this specification relates to a program and an electronic device that controls an image processing apparatus. [Background technology]

[0002] As a technology for an image processing device, a technology is known in which the image processing device performs a preparatory operation to hasten the completion of a specific operation before an instruction to execute the specific operation using the image processing device is input. For example, Patent Document 1 discloses a configuration for an image processing device having a power saving mode, in which when a human presence sensor of the image processing device detects an object during the power saving mode, the power saving mode is released and the device is returned to a normal power mode as a preparatory operation to hasten the completion of image processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-057919 A Summary of the Invention [Problem to be solved by the invention]

[0004] If the power-saving mode is cancelled in response to a person approaching the image processing device as in Patent Document 1, the power-saving mode will be cancelled even in cases where it is not necessary to cancel the power-saving mode, such as when a person happens to pass by the image processing device for some other reason than using the image processing device. This results in a large amount of wasteful preparatory operations.

[0005] This specification discloses a technique that enables an image processing device to efficiently perform a preparatory operation for speeding up the completion of a specific operation using the image processing device. [Means for solving the problem]

[0006] A program devised for the purpose of solving this problem is a program executable by a computer of an electronic device equipped with an electromagnetic wave sensor and a display, the electromagnetic wave sensor being a sensor that receives electromagnetic waves with wavelengths in millimeters or units smaller than millimeters and outputs a waveform signal based on the received electromagnetic waves, the program causes the computer to display an object that accepts input of an instruction to perform a specific operation on a screen of the display, and when an operation on the object is accepted, causes the computer to execute a process to perform the specific operation, the specific operation including an operation to control an image processing device, and further causes the computer to display an object displayed on the screen of the display, The display device executes a first acquisition process to acquire a position of an object on the screen; a second acquisition process to acquire a position of an indicator based on the waveform signal output from the electromagnetic wave sensor at least during a period in which the object is displayed on the display; a judgment process to determine whether the position of the indicator acquired by the second acquisition process is within a predetermined range based on the position of the object acquired by the first acquisition process; and a preliminary process to transmit, when it is determined in the judgment process that the position of the indicator is within the predetermined range, a preliminary action command to the image processing device that instructs the image processing device to execute a preparatory action to hasten the completion of the specific action.

[0007] By executing the program disclosed in this specification, the electronic device acquires the position of an object that accepts input of an instruction to execute a specific action and the position of a pointer, and transmits a preparatory action command to the image processing device when the position of the pointer is within a predetermined range. When the pointer is in a range close to the object, there is a possibility that an operation will be performed on the object, so it is expected that the completion of the specific action will be accelerated by instructing the image processing device to execute the preparatory action and starting the preparatory action before the object is operated. Furthermore, in a situation where an object is displayed on the electronic device and the pointer is in a range close to the object, there is a high possibility that an instruction to execute the specific action will be input, so there is a low possibility that the preparatory action will be wasted.

[0008] An electronic device for implementing the functions of the above-mentioned program, a control method for implementing the functions of the program, and a computer-readable storage medium for storing the program are also novel and useful. Effect of the Invention

[0009] The technology disclosed in this specification realizes a technology that enables an image processing device to efficiently perform a preparatory operation for hastening the completion of a specific operation using the image processing device. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic configuration diagram of an electronic device according to an embodiment. [Diagram 2] FIG. 11 is a sequence diagram showing an example of an operation procedure when printing is executed. [Diagram 3] FIG. 13 is an explanatory diagram showing an example of an editing screen. [Figure 4] FIG. 11 is an explanatory diagram showing an example of a print execution screen. [Diagram 5] 10 is a flowchart showing a procedure of a printing process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of a program incorporated in an electronic device will be described in detail with reference to the accompanying drawings. This embodiment discloses an application program (hereinafter, referred to as an "app") incorporated in an electronic device equipped with a display screen, such as a smartphone.

[0012] As shown in Fig. 1, the electronic device 1 of this embodiment includes a controller 10 including a CPU 11 and a memory 12. Furthermore, the electronic device 1 includes a user interface (hereinafter referred to as "user IF") 13, a communication interface (hereinafter referred to as "communication IF") 14, and a millimeter wave radar 15, which are electrically connected to the controller 10. The electronic device 1 is, for example, a device capable of executing various applications for causing the printer 3 to print. Note that the controller 10 in Fig. 1 is a collective term for hardware and software used to control the electronic device 1, and does not necessarily represent a single piece of hardware actually present in the electronic device 1.

[0013] The CPU 11 executes various processes according to a program read from the memory 12 and based on a user's operation. The CPU 11 is an example of a computer. The memory 12 includes a ROM, a RAM, and a non-volatile memory such as a HDD and a flash memory, and stores various programs and data.

[0014] The user IF 13 includes a display screen 131, which is hardware having a display function. The user IF 13 is an example of a display. The user IF 13 of the electronic device 1 of the present embodiment is a touch panel having both a display function and an operation reception function. The electronic device 1 of the present embodiment can display objects for receiving user operations on the display screen 131, and receives operations by contacting or pressing the displayed object with a pointer such as a fingertip.

[0015] The communication IF 14 includes hardware for communicating with an external device such as the printer 3. The communication method of the communication IF 14 is Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, etc., and may be wireless or wired, or may include a plurality of methods. Note that the electronic device 1 of this embodiment may have a function of connecting to the Internet via the communication IF 14.

[0016] The millimeter wave radar 15 includes hardware that transmits electromagnetic waves, receives reflected waves, and outputs a waveform signal based on the received reflected waves. The millimeter wave radar 15 is an example of an electromagnetic wave sensor. The millimeter wave radar 15 transmits, for example, a frequency modulated signal having a wavelength in millimeters or units smaller than a millimeter and a frequency of about 60 GHz, and receives a superposition of reflected waves. The millimeter wave radar 15 outputs a waveform signal based on the received reflected waves, and the electronic device 1 of this embodiment can detect, for example, the position and movement of an object in the vicinity of the electronic device 1 in millimeter units based on the waveform signal.

[0017] 1, an operating system (hereinafter, referred to as "OS") 21 and a print application 22 are built into the memory 12 of the electronic device 1 of this embodiment. The OS 21 is a multitasking OS that can process multiple tasks in parallel by managing and switching between multiple tasks, and is, for example, any one of iOS (registered trademark), Android (registered trademark), Windows (registered trademark), macOS (registered trademark), and Linux (registered trademark).

[0018] The print application 22 is an application for creating a printed matter using the printer 3. The print application 22 is an example of a program. For example, when the print application 22 of the present embodiment receives a print execution instruction from a user, the print application 22 generates a print command and executes a process for sending the generated print command to the printer 3. Note that the print application 22 of the present embodiment may be a program that can be executed independently based on an execution instruction from a user, or may be a program that is called and executed from another program while the other program is being executed.

[0019] The printer 3 of this embodiment has at least a printing function and a communication function, and performs an operation corresponding to a command received from the electronic device 1 or the like. For example, when the printer 3 receives a print command, it executes a print operation based on the received print command to generate a printed matter. The printer 3 is an example of an image processing device. The print operation by the printer 3 is an example of image processing, and the print command is an example of an image processing command.

[0020] 1, the printer 3 of this embodiment includes a controller 30 including a CPU and memory, a user IF 33, a communication IF 34, an image forming unit 35, and a power control unit 36. The printer 3 of this embodiment is, for example, a so-called label printer. The image forming unit 35 includes, for example, a thermal transfer print head, and creates labels by printing images on label paper that is wound up and stored in a roll and by transporting the label paper.

[0021] The power control unit 36 ​​controls the supply of power to the image forming unit 35. Specifically, the printer 3 goes into a sleep state when a state where no command is accepted continues for more than a predetermined time. In the sleep state, the power control unit 36 ​​reduces power consumption by stopping the supply of power to the image forming unit 35. In the sleep state, the printer 3 accepts operations to the user IF 33 and receives commands via the communication IF 34, but cannot form printed matter. When the printer 3 receives a print command in the sleep state, the printer 3 performs a recovery operation to switch from the sleep state to a non-sleep state before starting the execution of the print operation, and starts the print operation after the recovery operation is completed. As a recovery operation, the printer 3 starts the supply of power to the image forming unit 35 by the power control unit 36, and preheats the image forming unit 35 as necessary, thereby becoming in a state where the print operation is possible.

[0022] The printer 3 also accepts commands other than print commands from external devices such as the electronic device 1, and performs an operation corresponding to the accepted command. For example, when the printer 3 receives a search command, it transmits a predetermined response command to the sender of the command. The search command is a command transmitted from an external device to search for peripheral devices. Furthermore, when the printer 3 receives a status command requesting status information, it transmits a signal indicating the status information to the sender of the command. The status information includes at least one of information indicating the state of the printer 3, for example, waiting, printing, or error, and information indicating the remaining amount of consumables.

[0023] Next, an overview of the procedure for causing the printer 3 to execute printing using the print application 22 of this embodiment will be described with reference to the sequence diagram of Fig. 2. Fig. 2 describes the case where the printer 3 is in a sleep state at the start of this procedure. The print application 22 is executed by the CPU 11 of the electronic device 1.

[0024] A user starts the print application 22 on the electronic device 1, displays an editing screen that accepts editing of an image to be printed on the user IF 13, and inputs an editing instruction (A01). The print application 22 of this embodiment accepts an instruction to transition to an editing screen that accepts editing of an image to be printed, for example, on a top screen that is displayed immediately after the print application 22 is started.

[0025] An example of the appearance of the electronic device 1 in a state where the editing screen is displayed is shown in Fig. 3. Fig. 3 is an example of displaying an editing screen for creating a label, and is a diagram showing a surface of the electronic device 1 including a display screen 131 of the user IF 13. As shown in Fig. 3, the display screen 131 is disposed in the center of one surface of the housing of the electronic device 1, and occupies almost the entire surface. The millimeter wave radar 15 of this embodiment is provided on the same surface as the display screen 131, on the surface of the housing outside the display screen 131, and can detect objects near the display screen 131.

[0026] The editing screen displays a plurality of objects for accepting user operations. For example, the editing screen in FIG. 3 includes an editing object 51, a printer selection object 52, a print setting object 53, a print object 54, and a status object 55. The editing object 51 is an object that accepts editing of an image to be printed. The printer selection object 52 is an object that accepts the selection of a printer to execute printing. The print setting object 53 is an object that accepts an instruction to start editing print settings such as the color and length of the tape. The print object 54 is an object that accepts an instruction to move to a print execution screen that is a screen that accepts an instruction to execute printing. The status object 55 is an object that accepts an instruction to display status information of the selected printer.

[0027] When the electronic device 1 detects contact or pressure of a fingertip or the like within a range set for each displayed object, it determines that an operation on that object has been accepted. For example, when an operation on an editing object 51 is accepted, the print application 22 enters a state in which it is ready to accept an editing instruction, edits the image based on the accepted editing instruction (A02), and displays the edited image on the editing screen. When the user has completed editing, he or she inputs a print instruction by operating a print object 54 on the editing screen (A03).

[0028] When the print application 22 receives an operation on the print object 54 on the edit screen of FIG. 3, the print application 22 causes the user IF 13 to display a print execution screen on the display screen 131, as shown in FIG. 4 (A04). The print execution screen shown in FIG. 4 includes a print execution object 61. The print execution object 61 is an object that receives an instruction to send a print command. In other words, when an operation on this print execution object 61 is received, the print application 22 generates a print command and executes processing to send it to the printer 3.

[0029] The print application 22 acquires the position of the print execution object 61 on the displayed print execution screen (A05). The position of the object is information that specifies the area that the object occupies on the display screen 131, and is indicated by, for example, the coordinates of the four corners of the object, or the coordinates of each pixel included in the object. The position of the object is determined in advance according to the shape and size of the user IF 13 of the electronic device 1 and the attitude of the electronic device 1.

[0030] Then, the electronic device 1 drives the millimeter wave radar 15 to obtain a waveform signal output from the millimeter wave radar 15. Furthermore, the electronic device 1 obtains the position of the user's fingertip based on the obtained waveform signal (A11). The fingertip is an example of a pointer. The position of the fingertip may be the position of the tip closest to the display screen 131, or may be a position including the entire range considered to be the fingertip. When the user moves the fingertip (A12), the electronic device 1 obtains the position of the fingertip after the movement. For example, the electronic device 1 repeatedly obtains the position of the fingertip at a predetermined timing (loop) until some operation is accepted.

[0031] The printing application 22 determines whether the acquired fingertip position is near the print execution object 61 (alt). For example, the printing application 22 determines that the fingertip is near the print execution object 61 when the distance between the fingertip position and the print execution object 61 is within 5 cm. Alternatively, the printing application 22 determines that the fingertip is near the print execution object 61 when the fingertip position is directly above the print execution object 61. Then, when the printing application 22 determines that the fingertip is near the print execution object 61, it transmits a return command to the printer 3 via the communication IF 14 (A13).

[0032] When the printer 3 receives the return command, it starts a return operation to return from the sleep state to a non-sleep state (A14). Note that the printer 3 takes, for example, several seconds to a dozen seconds from start to completion of the return operation (A15).

[0033] When the print application 22 receives an operation on the print execution object 61 on the currently displayed print execution screen (A21), it stops acquiring the fingertip position (break). Then, the print application 22 generates a print command and executes a process for sending the generated print command to the printer 3. As a result, the electronic device 1 sends the print command to the printer 3 (A22). After receiving the print command, the printer 3 waits for the return operation to be completed and then executes printing (A23).

[0034] Before accepting an operation on the print execution object 61, the position of the fingertip is close to the print execution object 61. In other words, a return command is sent by A13 before a print command is sent by A22. By receiving the return command sent by A13, the printer 3 starts a return operation before receiving the print command. When the printer 3 receives a print command, it can immediately execute printing if the return operation is complete. In this way, it is highly likely that the time until printing is completed can be shortened compared to when the return command is not sent first. Note that when the printer 3 receives a return operation command in a non-sleep state, it does nothing.

[0035] Next, the procedure of the print process by the print application 22 of this embodiment will be described with reference to the flowchart in Fig. 5. The print process is executed by the CPU 11 of the electronic device 1 when an execution instruction for the print application 22 is received. Note that the following process and each processing step of the flowchart basically indicate the processing of the CPU 11 according to the instructions written in each program. The processing by the CPU 11 also includes hardware control using the API of the OS 21 of the electronic device 1. In this specification, the description of the OS 21 will be omitted and the operation of the program will be described.

[0036] In the printing process, the CPU 11 displays an operation acceptance screen, which is a screen for accepting an operation by the user, on the display screen 131 of the user IF 13 (S101). The operation acceptance screen is, for example, a top screen displayed immediately after the start of the print application 22, an edit screen shown in FIG. 3, and a print execution screen shown in FIG. 4. On the top screen, a plurality of objects are displayed, including, for example, an object that accepts an instruction to display how to use the application and an object that accepts an instruction to start creating a label. The CPU 11 accepts user operations on the objects being displayed on each screen.

[0037] Then, the CPU 11 determines whether or not a specific object is included in the display screen 131 being displayed (S102). A specific object is an object that accepts an instruction for a specific operation. A specific operation is an operation that may hasten the completion of the operation of the object by sending some command in advance to the printer 3 before accepting an instruction for the object and having the printer 3 start the operation of the command. The print application 22 stores a preliminary operation command, which is a command for hastening the completion by sending the command in advance to the printer 3, in association with each specific object.

[0038] For example, the print execution object 61 included in the print execution screen shown in Fig. 4 is one of the specific objects because there is a possibility that the time from accepting an operation on the object to completing the generation of a printed matter can be shortened by sending a return command to the printer 3 to start the return operation before accepting an operation on the object. The specific operation executed by the operation on the print execution object 61 is a series of operations including generating a print command and sending the print command, and the preliminary operation command corresponding to the print execution object 61 is a return command. And the return operation, which is an operation executed by the printer 3 by sending the return command, is an example of a preliminary operation. The print execution screen including the print execution object 61 is one of the screens including the specific objects.

[0039] Also, for example, the printer selection object 52 and the status object 55 in the edit screen shown in FIG. 3 are each a specific object, and the edit screen is one of the screens that includes specific objects.

[0040] The printer selection object 52 is an object that accepts the selection of the printer to be used for printing, and the specific operation that is executed when an operation on this object is accepted is a series of operations including sending a search command, receiving a response command, displaying a list of devices that have responded, and accepting device selection. Therefore, if a search command is sent from the electronic device 1 to a peripheral device before accepting an operation on the printer selection object 52, it is possible that the response command can be received in advance from the printer 3. If the response command has already been received when an operation on the object is accepted, it is possible that the time required to complete printer selection can be shortened by immediately displaying a list of devices. The preliminary operation command corresponding to the printer selection object 52 is a search command. The operation of sending a response command, which is an operation executed by the printer 3 in response to the sending of the search command, is an example of a preliminary operation.

[0041] The status object 55 is an object that receives an instruction to display the status information of the printer 3, and the specific operation executed when an operation on this object is received is a series of operations including sending a status command requesting the status information, receiving the status information, and displaying the received status information. Therefore, if a status command is sent to the printer 3 before an operation on the status object 55 is received, it may be possible to obtain the status information from the printer 3 in advance. If the status information has already been obtained when an operation on the object is received, the status information can be displayed immediately, and therefore the time until the status information is displayed may be shortened. The preliminary operation command corresponding to the status object 55 is a status command. And the status information response operation, which is an operation executed by the printer 3 in response to the sending of the status command, is an example of a preliminary operation.

[0042] The preparatory operation may be a part of the operation accompanying the specific operation, or may be an operation separate from the specific operation. For example, the operation of sending a response command, which is a preparatory operation of the printer selection object 52, is a part of the operation executed by the printer 3 in association with the transmission of a search command, which is a part of the specific operation. Also, the operation of responding to status information, which is a preparatory operation of the status object 55, is a part of the operation executed by the printer 3 in association with the transmission of a status command, which is a part of the specific operation. On the other hand, the return operation, which is a preparatory operation of the print execution object 61, is an operation that is not executed when the printer 3 is in a non-sleep state, and cannot be said to be a part of the operation executed by the printer 3 in association with the transmission of a print command, which is a specific operation.

[0043] In addition, the object included in the top screen that accepts an instruction to display how to use the app is an object that does not have a corresponding preliminary operation command and is not a specific object. In other words, there is almost no possibility that the completion of the command operation can be expedited by sending a command to the printer 3 before accepting an operation on this object.

[0044] When it is determined that the specific object is not included in the display screen 131 being displayed (S102: NO), the CPU 11 determines whether or not an operation for the object being displayed has been accepted (S103). When it is determined that the operation has not been accepted (S103: NO), the CPU 11 waits until it is accepted. When it is determined that the operation has been accepted (S103: YES), the CPU 11 determines whether or not the accepted operation is an instruction to end the print application 22 (S104). When it is determined that the accepted operation is not an instruction to end the print application 22 (S104: NO), the CPU 11 executes an operation based on the accepted instruction (S105), returns to S101, and causes the operation acceptance screen to be displayed. For example, when an instruction to display how to use the application is accepted, the CPU 11 causes the corresponding screen to be displayed on the user IF 13, and waits for the next operation.

[0045] When it is determined that the specific object is included in the display screen 131 being displayed (S102: YES), the CPU 11 acquires the position of the specific object in the display screen 131 (S110). S110 is an example of a first acquisition process. If the operation acceptance screen displayed in S101 is the edit screen shown in FIG. 3 or the print execution screen shown in FIG. 4, the CPU 11 determines in S102 that the specific object is included. Then, when the edit screen is displayed, the CPU 11 acquires the positions of the printer selection object 52 and the status object 55 in S110. When the print execution screen is displayed, the CPU 11 acquires the position of the print execution object 61 in S110.

[0046] Furthermore, the CPU 11 acquires a waveform signal output from the millimeter wave radar 15, and acquires information on the position and movement of an object existing near the electronic device 1 based on the waveform signal, thereby acquiring position information of the object determined to be a fingertip (S111). S111 is an example of a second acquisition process. The print application 22 can acquire the waveform signal output from the millimeter wave radar 15 from, for example, the OS 21.

[0047] Based on the results of S110 and S111, the CPU 11 judges whether the fingertip is within a predetermined range from the specific object. Specifically, the CPU 11 first judges whether the position of the fingertip is above the specific object (S112). Above the specific object is a columnar range extending from the area of ​​the specific object in a direction perpendicular to the display screen 131. If the position of the fingertip is not in the direction perpendicular to the screen from the position of the specific object, an operation other than an operation on the specific object may be performed, and the possibility of an operation on the specific object decreases. In addition, in S112, the CPU 11 may judge NO if the position of the fingertip is too far from the display screen 131. In addition, the above is not limited to a columnar range, and may be, for example, a shape that widens as it moves away from the screen.

[0048] When it is determined that the fingertip is above the specific object (S112: YES), the CPU 11 determines whether the distance between the fingertip position and the specific object is within a predetermined distance (S113). S113 is an example of a determination process. The predetermined distance is a distance stored in advance in the print application 22, and is, for example, 5 cm. The range above the specific object and within the predetermined distance is an example of the predetermined range. Note that the predetermined distance may be fixed or variable. If it is variable, the predetermined distance may be manually changed by a user's setting operation, or may be automatically changed based on the type of preparatory movement, etc.

[0049] When it is determined that the distance between the fingertip position and the specific object is within a predetermined distance (S113: YES), the CPU 11 transmits a preparatory operation command to the printer 3 (S114). S114 is an example of preparatory processing. As a result, the printer 3 starts the preparatory operation. The preparatory operation command transmitted in S114 is a command that is determined corresponding to the specific object to which the fingertip position is determined to be close. By starting the preparatory operation when the fingertip position is in a direction perpendicular to the screen from the position of the specific object and further within a predetermined distance from the specific object, the possibility of avoiding unnecessary preparatory operations is increased.

[0050] Instead of transmitting the preparatory movement command when it is determined that the position of the fingertip has entered within the predetermined distance, the CPU 11 may transmit the preparatory movement command when the state in which the position of the fingertip is within the predetermined distance continues for a certain period of time. The certain period is a predetermined time, for example, 0.1 to 1 second. By not transmitting the preparatory movement command immediately when the position of the fingertip has entered within the predetermined distance, but transmitting the command when the position of the fingertip has remained within the predetermined distance for a certain period of time, it is possible to avoid unnecessary preparatory movements when the finger happens to pass near a specific object.

[0051] Furthermore, the CPU 11 determines whether or not an operation on the specific object has been accepted (S115). When the fingertip approaches the specific object further and it is determined that an operation on the specific object has been accepted (S115: YES), the CPU 11 executes a specific action corresponding to the specific object (S116).

[0052] For example, when an operation on the printer selection object 52 is accepted on the edit screen, the CPU 11 displays a list of devices that have responded with a response command on the display screen 131 of the user IF 13 as a specific operation, and determines the device to be controlled based on the user's selection. If a response command is received in advance by a preliminary operation, the time required to display the device list can be shortened compared to when no preliminary operation is performed, and therefore it is expected that the device to be controlled can be determined more quickly.

[0053] Furthermore, for example, when an operation on the status object 55 is accepted on the editing screen, the CPU 11 displays the status information as a specific action on the display screen 131 of the user IF 13. If the status information is acquired in advance by a preparatory action, it can be expected that the display of the status information can be made earlier than when the preparatory action is not performed.

[0054] Also, for example, when an operation on the print execution object 61 is accepted on the print execution screen, the CPU 11 transmits a print command as a specific operation to the printer 3. If the printer 3 starts a recovery operation in advance due to the preparatory operation, it is expected that the time until printing is completed will be shortened compared to the case where the preparatory operation is not performed.

[0055] If it is determined that an operation on a specific object has not been accepted after the preparatory movement command is transmitted (S115: NO), the CPU 11 returns to S111 and acquires the fingertip position again. Note that if the fingertip is located near the same specific object after the preparatory movement command is transmitted, there is no need to retransmit the preparatory movement command. Since transmitting the preparatory movement command multiple times increases the communication load, the communication load can be reduced by transmitting the preparatory movement command when the fingertip moves within a predetermined range close to the specific object.

[0056] On the other hand, if it is determined that the position of the fingertip is not above the specific object (S112: NO), or if it is determined that the distance between the position of the fingertip and the specific object is not within a predetermined distance (S113: NO), the CPU 11 determines whether or not an operation on an object other than the specific object has been accepted (S120). If it is determined that an operation on another object has not been accepted (S120: NO), the CPU 11 returns to S111 and acquires the position of the fingertip again. That is, the CPU 11 repeatedly acquires the position of the fingertip while the screen including the specific object is being displayed. Note that the CPU 11 may constantly acquire the position of the fingertip in parallel with this printing process and use that information in S111.

[0057] If it is determined that an operation for another object has been accepted (S120: YES), the CPU 11 proceeds to S104 and determines whether the accepted operation is an instruction to end the print application 22. If it is determined that the operation is an instruction to end the print application 22 (S104: YES), the CPU 11 erases the displayed screen and ends the print process.

[0058] If an instruction other than an end instruction is received, the CPU 11 executes an operation based on the received instruction (S105). After S105 or S116, the CPU 11 returns to S101 and causes the user IF 13 to display an operation reception screen based on the user instruction.

[0059] For example, when an operation on the print object 54 is received on the edit screen shown in Fig. 3, the operation on the print object 54 is not an operation on a specific object, nor is it an instruction to end the process, so in S101, the CPU 11 causes the user IF 13 to display the print execution screen shown in Fig. 4. Then, in S102, the CPU 11 determines whether or not there is a specific object on the displayed print execution screen. Note that the print execution screen may contain a specific object other than the print execution object 61, and if there is, the CPU 11 also obtains the position of that object in S110.

[0060] If the printer 3 requires preheating of the print head before starting printing, the printer 3 may be made to start preheating as a preliminary operation of the print execution object 61. That is, when the CPU 11 determines that the fingertip is close to the print execution object 61, the CPU 11 may transmit a preheat command instructing the printer 3 to perform a preheating operation as a preliminary operation command. When the printer 3 receives the preheat command, if the printer 3 is in a sleep state, the printer 3 performs a recovery operation and then a preheating operation. The preheating operation is an example of a preceding operation. If the printer 3 starts preheating by the preliminary operation, it is expected that the completion of printing will be accelerated compared to when the printer 3 receives a print command without preheating. Also, if the printer 3 can start feeding the print medium in advance to accelerate the start of printing, the electronic device 1 may transmit a command to feed the print medium as a preliminary operation command.

[0061] As described above in detail, the print application 22 described in this specification acquires the position of the user's fingertip, and instructs the printer 3 to execute a preparatory operation when the user's fingertip is within a predetermined range from the specific object. The user's fingertip is in a state within a predetermined range from the specific object before performing an operation on the specific object. By starting a preparatory operation for the specific operation before an instruction to execute the specific operation is input, it is expected that the specific operation can be completed sooner. In addition, since it is unlikely that the user will not perform an operation on the specific object after bringing their fingertip within a predetermined range from the specific object, it is unlikely that the preparatory operation will be wasted.

[0062] It should be noted that the present embodiment is merely an example and does not limit the present invention in any way. Therefore, the present invention can be modified and changed in various ways without departing from the spirit and scope of the present invention. For example, the electronic device 1 is not limited to a portable device, but may be a stationary device such as a personal computer.

[0063] Also, for example, the printer connected to the electronic device 1 is not limited to a label printer, but may be an inkjet printer or a laser printer. That is, the printing method of the printer 3 is not limited to a thermal transfer method, but may be, for example, a thermal method, an inkjet method, or an electrophotographic method. Also, the printing medium used in the printer 3 is not limited to label paper, but may be, for example, simple roll paper or cut paper. Also, the printer 3 may not have a user IF 33.

[0064] In this embodiment, the print application 22 acquires the position of the fingertip based on the waveform signal output from the millimeter wave radar 15, but this is not limited to the above. For example, there may be another program that acquires the position of the fingertip based on the waveform signal output from the millimeter wave radar 15. For example, the OS 21 may acquire the position of the fingertip, and the print application 22 may acquire the position information from the OS 21. Furthermore, the configuration for acquiring the information on the position of the fingertip is not limited to the millimeter wave radar 15, and may be a camera, an infrared sensor, an optical sensor, or the like, or may be acquired by a combination of a plurality of configurations.

[0065] Also, for example, the print application 22 may have an option of whether or not to output a preliminary operation command. Furthermore, the print application 22 may accept a manual change to the option by a user operation, or may automatically change the option depending on the state of the printer 3 or the time elapsed since the previous specific operation.

[0066] In addition, in the present embodiment, the position of the fingertip is acquired when a specific object is included in the displayed screen, but the electronic device 1 may acquire the position of the fingertip regardless of the presence or absence of a specific object. For example, while the print application 22 is being executed, the electronic device 1 may periodically acquire the position of the fingertip.

[0067] In addition, although the electronic device 1 including a touch panel has been described in this embodiment, the user IF 13 does not have to be a touch panel. The user IF 13 may be, for example, a combination of a display and an input device such as a keyboard or a mouse. In this case, each button displayed on the display is an example of an object, and the millimeter wave radar 15 may be provided around the display. Even in this case, the user can instruct the execution of a preparatory action by bringing a fingertip close to the object before moving the cursor with the input device to perform an operation on the object.

[0068] In addition, in this embodiment, the position of the user's fingertip is detected, but the detection target is not limited to the fingertip. In other words, it is not necessary to determine whether it is a fingertip or not. For example, even if the pen tip of a touch pen is located near a specific object, a preliminary action command may be transmitted. In this case, the pen tip of the touch pen is an example of an indicator.

[0069] In addition, in any of the flowcharts disclosed in the embodiments, the order of execution of multiple processes in any of the steps can be arbitrarily changed or the processes can be executed in parallel as long as no contradiction occurs in the process content.

[0070] The processes disclosed in the embodiments may be executed by a single CPU, multiple CPUs, hardware such as ASIC, or a combination of these. The processes disclosed in the embodiments may be realized in various forms, such as a recording medium having a program for executing the processes recorded thereon, or a method. [Explanation of symbols]

[0071] 1. Electronic Devices 3. Printer 11 CPU 13 User Interface 15 Millimeter wave radar

Claims

1. A program executable by a computer of an electronic device including an electromagnetic wave sensor and a display, the electromagnetic wave sensor being a sensor that receives electromagnetic waves having wavelengths in units of millimeters or units smaller than millimeters and outputs a waveform signal based on the received electromagnetic waves; The computer includes: displaying an object that accepts input of an instruction to execute a specific operation on a screen of the display device, and executing a process to execute the specific operation when an operation on the object is accepted, the specific operation being an operation of transmitting a print command to a printer to instruct printing; The computer further comprises: a first acquisition process for acquiring a position within a screen of the display device of the object displayed on the screen; a second acquisition process for acquiring a position of a pointer based on the waveform signal output from the electromagnetic wave sensor at least during a period in which the object is displayed on the display device; a determination process for determining whether or not the position of the indicator acquired by the second acquisition process is within a predetermined range based on the position of the object acquired by the first acquisition process; a preliminary process of transmitting to the printer a preliminary operation command for instructing the printer to execute a preliminary operation for hastening completion of printing, the preliminary operation command being executed before the start of printing by the printer when the determination process determines that the position of the indicator is within the predetermined range; Execute the A program characterized by:

2. In the program according to claim 1, the preparatory operation is a return operation for switching the printer from a sleep state to a non-sleep state, the printer consumes less power in the sleep state compared to the non-sleep state and is unable to print; In the preliminary treatment, sending the preparatory operation command to the printer, the preparatory operation command instructing the execution of the recovery operation, and the printer, when receiving the preparatory operation command, switching the printer to the non-sleep state if it is in the sleep state; A program characterized by:

3. In the program according to claim 1, The preliminary operation is a preceding operation that is executed to enable printing before starting printing, and when the printer receives the print command, the printer starts printing after the preceding operation is completed; In the preliminary treatment, transmitting the preparatory operation command to the printer, the preparatory operation command instructing the execution of the preceding operation, the printer starting the execution of the preceding operation when the printer receives the preparatory operation command; A program characterized by:

4. In the program according to any one of claims 1 to 3, The computer includes: executing the determination process and the preliminary process when the position of the indicator acquired by the second acquisition process is in a direction perpendicular to the screen of the display device from the position of the object acquired by the first acquisition process; A program characterized by:

5. The program according to any one of claims 1 to 4, In the preliminary treatment, when the position of the pointer acquired by the second acquisition process moves from outside the predetermined range to within the predetermined range, the preparatory operation command is transmitted to the printer. A program characterized by:

6. The program according to any one of claims 1 to 4, In the preliminary treatment, transmitting the preparatory operation command to the printer when the position of the indicator acquired by the second acquisition process remains within the predetermined range for a certain period of time from a time when the position of the indicator changes from outside the predetermined range to within the predetermined range; A program characterized by:

7. An electromagnetic wave sensor; A display device; A computer, Equipped with the electromagnetic wave sensor is a sensor that receives electromagnetic waves having wavelengths in millimeter units or units smaller than millimeters and outputs a waveform signal based on the received electromagnetic waves; The computer includes: displaying an object that accepts an input of an instruction to execute a specific operation on a screen of the display device, and executing a process to execute the specific operation when an operation on the object is accepted, the specific operation being an operation of transmitting a print command to a printer to instruct printing; The computer further comprises: a first acquisition process for acquiring a position within a screen of the display device of the object displayed on the screen; a second acquisition process for detecting a position of a pointer based on the waveform signal output from the electromagnetic wave sensor at least during a period in which the object is displayed on the display device; a determination process for determining whether or not the position of the indicator acquired by the second acquisition process is within a predetermined range based on the position of the object acquired by the first acquisition process; a preliminary process of transmitting to the printer a preliminary operation command for instructing the printer to execute a preliminary operation for hastening completion of printing, the preliminary operation command being executed before the start of printing by the printer when the determination process determines that the position of the indicator is within the predetermined range; Execute 1. An electronic device comprising:

Images