Display control program
Patent Information
- Application Number
- PCT/JP2026/002606
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-18
- Filing Date
- 2026-01-27
- Publication Date
- 2026-08-27
Smart Images
Figure JP2026002606_27082026_PF_FP_ABST
Abstract
Description
Indicating control program
[0001] The present invention relates to a display control program.
[0002] The information processing apparatus described in Patent Document 1 displays an editing window on a touch screen display. A display area and a plurality of software buttons are displayed in the editing window. A plurality of items (objects) are displayed in the display area. Each item is an aggregate of one or a plurality of strokes, and is identified by being classified according to the distance between strokes and the input time. A stroke is one handwriting input by a user to the display area while the pen button is selected.
[0003] Item information is stored in the flash memory of the information processing apparatus. The item information includes the coordinate information of each item and the information of the strokes. The information of the strokes includes the coordinate information of a plurality of points forming from the start point to the end point of the stroke and the time information when a plurality of points forming the stroke are input. When a stroke is added or deleted in the display area, the CPU of the information processing apparatus updates the stroke data according to the addition or deletion of the stroke and stores it in the flash memory.
[0004] Japanese Patent Application Laid-Open No. 2021-177315
[0005] In a conventional information processing apparatus, since a stroke has a plurality of points forming from the start point to the end point of the stroke, the data amount of the stroke data increases. Since the stroke data is updated every time a stroke is added or deleted in the display area, the more strokes are added, the more the stroke data increases. For example, when a large number of strokes are displayed in the display area, the information processing apparatus needs to process a large amount of data. Therefore, there is a problem that the information processing apparatus is loaded and it becomes difficult to update the display in real time and the operability deteriorates.
[0006] An object of the present invention is to provide a display control program that reduces the load on an information processing apparatus when displaying strokes in a display area.
[0007] A display control program according to one aspect of the present invention is a display control program executed by a computer that controls an information processing apparatus comprising a display unit and an input unit, wherein the computer is caused to execute: a first acquisition process that acquires information of strokes handwritten by the input unit onto a document displayed on the display unit; a display area acquisition process that acquires information indicating the display position and display magnification of a display area, which is a display area on the document; a display change process that changes the display position and / or display magnification of the display area upon detection of input from the input unit; and a first change process that, in the display change process, reduces the amount of dot thinning of the sequence of points constituting the stroke displayed on the display unit when the display magnification of the display area increases, and increases the amount of dot thinning of the sequence of points of the stroke displayed on the display unit when the display magnification of the display area decreases.
[0008] In a display control program, increasing the display magnification narrows the display area, so reducing the amount of dot thinning does not significantly increase the load on the computer controlling the information processing device. However, decreasing the display magnification widens the display area, which tends to increase the load on the computer's display processing. In this case, increasing the amount of dot thinning reduces the load on the computer's display processing.
[0009] This is a block diagram showing the electrical configuration of the information processing device 1. This is a diagram showing the editing window 5. This is a diagram showing stroke 27 and item 25. This is an explanatory diagram showing the difference in distance A due to different page magnifications. This is a flowchart for the handwritten stroke addition process. This is a flowchart for the handwritten stroke display process. This is a diagram showing the editing window 5 with the document 60 magnified. This is a diagram showing the editing window 5 with the handwritten stroke displayed outside the page display area 65. This is a diagram showing the handwritten stroke before the drawing point decimation process. This is a diagram showing the handwritten stroke after the drawing point decimation process. This is a diagram showing the handwritten stroke before the resolution reduction process. This is a diagram showing the handwritten stroke after the resolution reduction process. This is a diagram showing the state in which the document 60 has hierarchical information. This is a flowchart for the same item discrimination process. This is a flowchart for the threshold determination process. This is a flowchart for the first discrimination process. This is a flowchart for the second discrimination process. This is a diagram showing the distance L_m between point B_k' of stroke ST1 and point C_k of stroke ST8. This is a flowchart for the fill judgment process. This is a diagram showing the filled handwritten stroke.
[0010] Referring to Figure 1, the electrical configuration of the information processing device 1 will be explained. The information processing device 1 comprises a CPU 10, a system controller 11, a ROM 12, a RAM 13, a graphics controller 15, a touchscreen display 16, and a communication unit 19. The information processing device 1 is a portable information terminal device, such as a smartphone, tablet terminal, or tablet PC.
[0011] The CPU 10 controls the information processing device 1 and executes a display control program according to one embodiment of the present invention. The system controller 11 connects the local bus of the CPU 10 to each component. The ROM 12 stores the BIOS, OS, setting values, etc. The RAM 13 temporarily stores data generated when the CPU 10 executes processing. The flash memory 14 stores the display control program executed by the CPU 10. The display control program includes a program for adding handwritten strokes and a program for displaying handwritten strokes.
[0012] The touchscreen display 16 includes a flat panel display 17 and a sensor 18. The flat panel display 17 displays a visual output to the user. The sensor 18 detects the contact position of a stylus pen 4 or finger on the screen of the touchscreen display 16. The graphics controller 15 controls the display of the flat panel display 17.
[0013] The communication unit 19 is an interface that can connect to the network 3. The network 3 connects to the server 2 wirelessly or via a wired connection. The CPU 10 obtains page drawing information for the document 60 (see Figure 2) to be displayed on the touchscreen display 16 from the flash memory 14 or the server 2. The information processing device 1 may also obtain the program executed by the CPU 10 via the network 3. The information processing device 1 may also store the obtained page drawing information and program in its internal flash memory 14. The communication unit 19 communicates with the stylus pen 4 via wireless communication such as Bluetooth®.
[0014] Referring to Figure 2, the editing window 5 will be described. When the operation to launch the editing application is detected by the sensor 18, the CPU 10 displays the editing window 5 on the touchscreen display 16. The editing window 5 comprises a display area 50 and a plurality of software buttons 55. The display area 50 is, for example, a horizontally elongated rectangle. The plurality of software buttons 55 include a pen button 51, an eraser button 52, a selection button 53, and a structure button 54.
[0015] The display area 50 shows document 60 and items 21-23. Document 60 is, for example, a document in PDF format. Document 60 may also be document data in other formats, images, etc. In Figure 2, the PDF data of document 60 includes, for example, data of characters 61-63. Character 61 is the letter "Y", character 62 is the letter "O", and character 63 is the letter "S". Items 21-23 are collections of one or more handwritten strokes, distinguished and identified by the distance between strokes and the input time. An item means the content of information that includes at least one of the following: characters, strings of characters, numbers, symbols, figures, etc.
[0016] A stroke is a single handwritten input by the user on the display area 50 when the pen button 51 is selected. Handwritten input can be done with a stylus pen 4 or a fingertip. A stroke is drawn by the user by hand on the screen of the flat panel display 17 from the time the stylus pen 4 or finger touches the screen until it is lifted. Stroke data includes coordinate information of multiple points that form the stroke from the start point to the end point, and time information of when the multiple points forming the stroke were input. When a stroke is added or deleted from the display area 50, the CPU 10 updates the stroke data according to the addition or deletion and saves it to the flash memory 14. Hereinafter, the multiple points that form the stroke from the start point to the end point are also referred to as a sequence of points.
[0017] As shown in Figure 2, the display area 50 is the area where, for example, the document 60 and items 21 to 23 are displayed. The area of the document 60 that is displayed in the display area 50 is called the page display area 65. Hereafter, when referring to items 21 to 23 collectively, they will be called item 20. Item 20 is a collection of one or more strokes, which are distinguished and identified by the distance between strokes and the input time. In the example shown in Figure 2, item 21 is represented by the string "ABC", item 22 is represented by the string "DEFG", and item 23 is represented by the string "HIJK".
[0018] Item information is stored in the flash memory 14. The item information includes coordinate information and stroke data for item 20. The coordinate information for item 21 refers to the coordinate information of the four corners of the minimum rectangular frame M surrounding item 21 in the display area 50. The coordinate information for items 22 and 23 is similar. The stroke data includes coordinate information of multiple points that form the stroke from the start point to the end point, and time information of the multiple points that form the stroke. Note that the frame M is data processed internally by the CPU 10 and is not displayed on the touchscreen display 16. The origin (0,0) that serves as the reference point for the coordinate information is located near the upper left corner of the display area 50. The left-right direction corresponds to the X-axis direction, and the up-down direction corresponds to the Y-axis direction. In this embodiment, the origin is near the upper left corner, and the direction extending to the right from the origin corresponds to the positive X-axis direction. The direction extending downward from the origin corresponds to the positive Y-axis direction.
[0019] Referring to Figure 3, the distance threshold β_L will be explained. The distance threshold β_L is the distance threshold between strokes and is used to determine whether a handwritten stroke 27 is the same item or a different item from another handwritten item 25 that was entered earlier. The default value of the distance threshold β_L is stored in the flash memory 14 in advance according to user settings. The stroke 27 shown in Figure 3 is a handwritten stroke from the upper left to the lower right. The CPU 10 creates an item frame M1 for the stroke 27. The item frame M1 is defined as the smallest rectangle surrounding the stroke 27 based on the smallest X coordinate, the largest X coordinate, the smallest Y coordinate, and the largest Y coordinate of the multiple points A_k (k=1 to n) that constitute the stroke 27. Based on the created item frame M1, the CPU 10 creates a threshold frame N1. The threshold frame N1 is a rectangular frame the size of the item frame M1 with the distance threshold β_L added to the top, bottom, left, and right.
[0020] The adjacent item closest to stroke 27 is item 25. Item 25 is represented by the string "ABC1" arranged horizontally. The lower end of the rightmost digit "1" in item 25 is located within the threshold frame N1 of stroke 27 (see arrow K in Figure 3). In this case, the CPU 10 recognizes stroke 27 as the same item as item 25 in the first discrimination process described later (see Figure 16). On the other hand, if item 25 is not located within the threshold frame N1, the CPU 10 recognizes stroke 27 as a different item from item 25.
[0021] Referring to Figure 4, the relationship between the distance threshold β_L and the page magnification will be explained. The distance threshold β_L is a value set by the page magnification of the display area 50 specified by the user. Here, if the user changes the page magnification, it is necessary to calculate the distance threshold α corrected according to the page magnification. For example, as shown in Figure 4(1), in the display area 50 with a page magnification of 100%, two strokes 28 and 29 that have been handwritten by the user are displayed. Stroke 28 is a vertical straight line, and stroke 29 is a horizontal straight line. The apparent distance between strokes 28 and 29 is A.
[0022] Users generally tend to handwrite characters and other elements at a consistent size, regardless of the page magnification. Therefore, even at a page magnification of 50%, as shown in Figure 4(2), the user handwrites strokes 28 and 29 at the same size as when the page magnification is 100%. Thus, the apparent distance between strokes 28 and 29 at this time is also A.
[0023] In the display area 50 shown in Figure 4(2), even though it is the same size as in Figure 4(1), strokes 28 and 29 are displayed at a page magnification of 50%. Therefore, as shown in Figure 4(3), the CPU 10 calculates the length, positional relationship, etc., of strokes 28 and 29 using a scale converted to a page magnification of 100%. Consequently, the respective lengths of strokes 28 and 29 become twice that of when the page magnification is 50%, and the distance between strokes 28 and 29 also becomes 2A. Here, if the page magnification of the display area 50 when the distance threshold β_L is set is taken as the reference page magnification (100%), and the distance threshold when the page magnification is 50% is taken as α, the distance threshold α is calculated by the following formula. Note that the distance threshold α is stored in the flash memory 14. ・Distance threshold α = Distance threshold β_L × (100 / 50) = 2β_L
[0024] <Handwriting stroke addition process> The handwriting stroke addition process will be explained with reference to Figure 5. When the CPU 10 detects an instruction to start the handwriting stroke addition process from the sensor 18, it reads the handwriting stroke addition process program from the flash memory 14 and executes the process.
[0025] The CPU 10 determines whether the user has selected a page in the document to which they want to add handwritten strokes (S1). An example of a document is a PDF document. If the sensor 18 detects an instruction from the user to select a specific page in the document (S1: YES), the CPU 10 obtains the page drawing information for that page from the flash memory 14 or the server 2 and displays it in the display area 50 (S2). If the sensor 18 does not detect a touch in response to the user's instruction (S1: NO), the CPU 10 continues the determination process in S1.
[0026] Following the processing in S2, the CPU 10 determines whether the user has entered a handwritten stroke (S3). For example, the CPU 10 enters pen input mode when the sensor 18 detects that the pen button 51 has been selected. Then, when input is detected from the sensor 18, the CPU 10 determines that the user has entered a handwritten stroke (S3: YES). If the CPU 10 does not determine that the user has entered a handwritten stroke (S3: NO), it continues the determination in S3. If the CPU 10 determines that the user has entered a handwritten stroke (S3: YES), it obtains the coordinate information of the user's page display area and the entered sequence of points (S4).
[0027] The user's page display area is the area where the document 60 shown in Figure 2 is displayed in the display area 50. The input point sequence coordinate information is the coordinate information of multiple points A_k (k=1 to n) that constitute the stroke 27 shown in Figure 3. The CPU 10 performs point sequence thinning from the page display area and the input point sequence coordinate information (S5). For example, in point sequence thinning, the CPU 10 calculates the physical coordinates on the terminal display from the page display area 65 and the point sequence coordinate information. Then, on the physical coordinates of the touchscreen display 16, the CPU 10 thins out points that are close together, and does not thin out points that constitute lines with high curvature. This process is performed using a commonly used algorithm.
[0028] Next, the CPU 10 saves the coordinate information of the handwritten stroke sequence after thinning (S6). For example, the CPU 10 stores the coordinate information of the thinned-out stroke sequence together with the page drawing information of the document 60 in the flash memory 14 or server 2 (S6). Next, the CPU 10 proceeds to process S3 and executes the processes from S3 to S6 in the same manner as above.
[0029] <Handwriting Stroke Display Processing> The handwriting stroke display processing will be explained with reference to Figure 6. When the CPU 10 detects an instruction to start the handwriting stroke display processing from the sensor 18, it reads the handwriting stroke display processing program from the flash memory 14 and executes the processing.
[0030] The CPU 10 determines whether the user has selected a page of the document to view (S11). An example of a document is a PDF document. When the sensor 18 detects a touch indicating the selection of a specific page of the document, the CPU 10 determines that the user has selected a page to view (S11: YES). Next, the CPU 10 obtains page drawing information from the flash memory 14 or the server 2 and displays it in the display area 50 (S12). The page drawing information is the page drawing information stored in the flash memory 14 or the server 2 by the processing of the handwriting stroke input processing in S6, and includes coordinate information of the point sequence of added strokes.
[0031] If the CPU 10 does not determine that the user has selected a specific page of the document (S11: NO), it continues the determination in S11. After processing in S12, the CPU 10 determines whether the user has operated the page display area (S13). Examples of page display area operations include pinch or pan operations. If the CPU 10 detects input of an operation on the display area, such as a pinch or pan operation, via the sensor 18, it determines that the user has operated the page display area (S13: YES). If the CPU 10 does not determine that the user has operated the page display area (S13: NO), it continues the determination in S13.
[0032] If the CPU 10 determines that the user has manipulated the page display area (S13: YES), it determines the page display area according to the user's operation (S14). In the S14 process, the CPU 10 obtains information indicating the display position and display magnification of the page display area. The display position is the position from the origin (0,0) shown in Figure 2. The page display area is determined, for example, by the number of points in the horizontal width and vertical width of the displayed pages of document 60. Pinch operations include pinch in and pinch out.
[0033] Pinch-in is the action of narrowing the distance between the tips of two fingers on the touchscreen display 16. When the CPU 10 detects pinch-in using the sensor 18, it shrinks the document 60 and enlarges the page display area 65 of the document 60 as shown in Figure 2 (S14). The display magnification is the page magnification.
[0034] When the CPU 10 detects a pinch-out gesture using the sensor 18, it enlarges the document 60 as shown in Figure 7. Consequently, the CPU 10 narrows the page display area 65, which is a portion of the document 60 displayed in the display area 50 (S14). In the example shown in Figure 7, the portion of the enlarged document 60 displayed in the display area 50 is the page display area 65. When the display magnification of the document 60 increases, that is, when the document 60 is enlarged, the amount of items included in the page display area 65 decreases. Conversely, when the display magnification of the document 60 decreases, that is, when the document 60 is reduced, the amount of items included in the page display area 65 increases. When the amount of items included in the page display area 65 increases, the load on the CPU 10's display processing increases.
[0035] Panning is an operation in which the document 60 is moved parallel to the front, back, left, or right while pressing and holding the touchscreen display 16 with the stylus pen 4 or a finger. When the document 60 cannot be fully displayed on one screen, the desired portion of the document 60 can be moved to the display position. When the CPU 10 detects a pan operation using the sensor 18, it moves the document parallel to the display area 50 (S14).
[0036] Next, the CPU 10 determines whether the display area of the handwritten stroke is included within the page display area 65 (S15). The CPU 10 determines that the display area of the handwritten stroke is included within the page display area if even a part of the handwritten stroke is included within the page display area (S15: YES). In Figure 7, the page display area 65 displayed in the display area 50 of the document 60 contains items 21 to 23 composed of handwritten strokes (S15: YES). The process for determining that the handwritten strokes are the same item, which is a single string of characters or a graphic, will be described later with reference to Figures 14 to 17.
[0037] If the CPU 10 determines YES in S15, it determines whether the number of items on the page exceeds a first quantity (S16). An example of the first quantity is 10,000 items. If the number of items exceeds 10,000, the CPU 10 will need to process a large amount of data, increasing the load, making real-time display updates difficult and reducing operability. If the CPU 10 does not determine that the number of items on the page exceeds a first quantity (S16: NO), it does not perform the decimation process of the drawing point sequence of handwritten strokes (S18, S21) and the resolution reduction process (S20, S22) that reduces the drawing resolution, as described later. Therefore, the CPU 10 draws the handwritten strokes according to the decimation and resolution parameters without performing the decimation and resolution reduction processes (S23).
[0038] If the CPU 10 determines YES in the judgment of S16, it determines whether the display ratio of the handwritten stroke within the page display area 65 exceeds a certain amount (S17). If the vertical or horizontal length of the handwritten stroke in the display area 50 is greater than or equal to a predetermined threshold, the CPU 10 determines that the display ratio of the handwritten stroke within the page display area 65 exceeds a certain amount (S17: YES). An example of a predetermined threshold is that the vertical or horizontal length of the handwritten stroke is 5 mm in the display area 50. If the CPU 10 determines that the display ratio of the handwritten stroke within the page display area 65 exceeds a certain amount (S17: YES), it performs a thinning process of the drawing point sequence of the handwritten stroke (S18).
[0039] This section describes the process of thinning out drawing point sequences. Thinning out drawing point sequences is a process of thinning out the data of point sequences that make up a stroke by a certain percentage. An example of the thinning amount is described below. In the following, the minimum value of the thinning amount of drawing point sequences in this embodiment is set to 1, and the maximum value is set to 10. For example, if the thinning amount is 1, every other point in a pixel sequence is thinned out. If the thinning amount is 10, out of 11 points in a pixel sequence, 10 points are thinned out, leaving one at the end. If the thinning amount is 5, out of 6 points in a pixel sequence, 5 points are thinned out, leaving one at the end.
[0040] When the maximum amount of drawing point thinning is 10, this is called "thinning amount: large". When the amount of drawing point thinning is half of the maximum amount of 10, which is 5, this is called "thinning amount: medium". If the CPU 10 determines that the display ratio of handwritten strokes within the page display area exceeds a certain amount (S17: YES), it thins the drawing point sequence of the handwritten strokes with a thinning amount of medium (S18).
[0041] If the display ratio of handwritten strokes within the page display area exceeds a certain amount, the handwritten strokes become conspicuous, and if the thinning amount is set to large, the blurring of the handwritten strokes due to the thinning of the drawing point sequence becomes noticeable. Therefore, if the CPU 10 determines that the display ratio of handwritten strokes within the page display area exceeds a certain amount (S17: YES), it executes a process to thin the drawing point sequence of the handwritten strokes (S18). In this case, the CPU 10 thins the drawing point sequence by a medium amount (S18).
[0042] Next, the CPU 10 determines whether the number of items on the page exceeds a second quantity (S19). An example of the second quantity is 15,000 items. If the number of items exceeds 15,000, the CPU 10 needs to process an even larger amount of data, further increasing the load, making real-time display updates difficult and reducing usability. In the following, the lowest resolution of the drawing quality in this embodiment will be referred to as "lowest quality". Also, if the resolution from the lowest to the highest drawing quality is divided into 10 levels from 1 to 10, the 5th level resolution will be referred to as "medium quality".
[0043] If the CPU 10 determines that the number of items on the page exceeds the second amount (S19: YES), it performs a resolution reduction process to set the rendering resolution to the lowest quality (S22). This minimizes the load on the CPU 10's display processing and reduces the amount of RAM 13 used. If the CPU 10 does not determine that the number of items on the page exceeds the second amount (S19: NO), it performs a resolution reduction process (S20). In this case, the CPU 10 sets the resolution to medium quality (S20). This reduces the load on the CPU 10's display processing and reduces the amount of RAM 13 used.
[0044] Also, when the CPU 10 determines that the display area of the handwritten stroke is not included in the page display area 65 (S15: NO), the CPU 10 executes a process of thinning out the drawing dot sequence of the handwritten stroke (S21). In this case, the CPU 10 thins out the drawing dot sequence with a large thinning amount (S21). Next, the CPU 10 executes a resolution reduction process. In this case, the CPU 10 sets the drawing resolution to the lowest image quality (S22). When the display area of the handwritten stroke is not included in the page display area, even if the drawing dot sequence of the handwritten stroke is thinned out with a large thinning amount or the image quality is set to the lowest, it is difficult for the user to recognize the thinning or the reduction in image quality.
[0045] When the CPU 10 determines that the display ratio of the handwritten stroke within the page display area does not exceed a certain amount (S17: NO), the CPU 10 thins out the drawing dot sequence of the handwritten stroke with a large thinning amount (S21). Next, the CPU 10 sets the drawing resolution to the lowest image quality (S22). When the display ratio of the handwritten stroke within the page display area 65 does not exceed a certain amount, the handwritten stroke is not prominent, and even if the drawing dot sequence of the handwritten stroke is thinned out with a large thinning amount or the image quality is set to the lowest, it is difficult for the user to recognize the thinning or the reduction in image quality.
[0046] After the process of S20 or the process of S22, the CPU 10 draws the handwritten stroke according to the determination parameters of thinning and resolution determined in the processes of S18, S19, S21, and S22 (S23). After that, the CPU 10 proceeds to the determination of S13.
[0047] Incidentally, the CPU 10 may acquire the hardware information of the information processing apparatus 1 and perform a setting process of setting the thinning amount and the resolution reduction amount according to the operation clock number of the CPU 10. For example, when the operation clock number of the CPU 10 is less than 2 GHz, the thinning amount is set to large and the lowest image quality is set. When the operation clock number of the CPU 10 is 2 GHz or more, the thinning amount may be set to medium and the medium image quality may be set. This is because when the operation clock number of the CPU 10 is less than 2 GHz, the processing ability of the CPU 10 is not high.
[0048] Referring to FIGS. 9 and 10, the graphic before and after the decimation process of the drawing dot sequence of the handwritten stroke will be described. The graphic after the decimation of the drawing dot sequence of the handwritten stroke shown in FIG. 10 has fewer points constituting the stroke 27 and a simpler shape compared to the graphic before the decimation process of the drawing dot sequence shown in FIG. 9. Therefore, the size of the data of the handwritten stroke after the decimation process becomes smaller, the load on the display process of the CPU 10 becomes smaller, and the memory consumption also becomes smaller.
[0049] Referring to FIGS. 11 and 12, the character string before and after the resolution reduction process of the handwritten stroke will be described. The graphic after the resolution reduction process of the drawing dot sequence of the handwritten stroke shown in FIG. 12 has a less distinct shape compared to the graphic before the resolution reduction process of the drawing dot sequence shown in FIG. 11. Therefore, the graphic after the resolution reduction process of the drawing dot sequence has a smaller data size, the load on the display process of the CPU 10 becomes smaller, and the memory consumption also becomes smaller.
[0050] In addition, when the document 60 has hierarchical information, the following process may be performed without performing the processes of S16 to S22. Referring to FIG. 13, the case where the document 60 has hierarchical information will be described. In the example shown in FIG. 13, three documents 60A, 60B, and 60C are displayed in the display area 50 overlapping each other in order from top to bottom. The document 60C includes data of characters ~63.
[0051] The uppermost document 60A describes the item 21 of the character string "ABC" added by the handwritten stroke addition process. Also, the middle layer document 60B describes the item 22 of the character string "DEFG" added by the handwritten stroke addition process. Also, the lowermost document 60C describes the item 23 of the character string "HIJK" added by the handwritten stroke addition process. In this case, the CPU 10 may increase the decimation amount of the dot sequence of the stroke constituting the item as the layer goes down as the "fifth change process".
[0052] For example, CPU 10 does not perform decimation on item 21 of the top-level document 60A. CPU 10 also performs decimation on item 22 of the middle-level document 60B with a medium decimation amount. Furthermore, CPU 10 performs decimation on item 23 of the bottom-level document 60C with a large decimation amount. As the document hierarchy goes lower, even if the amount of decimation for the strokes constituting the items is increased, the lower levels are less visible, so the impact on the display on the display unit is minimal, and the load on the CPU 10's display processing can be reduced.
[0053] Next, referring to Figures 14 to 17, we will explain the same-item determination process used to determine the number of items in the determination processes S16 and S19 of Figure 6. The same-item determination process shown in Figure 14 determines whether the strokes stored in the flash memory 14 in the S6 process of the handwritten stroke addition process are of the same item.
[0054] As shown in Figure 14, the CPU 10 performs threshold determination processing (S31). In the threshold determination processing (S31), as shown in Figure 15, the CPU 10 obtains a distance threshold β_L from the flash memory 14 (S41). The CPU 10 calculates a distance threshold α by correcting the obtained distance threshold β_L with the page magnification (S42). The CPU 10 obtains a time threshold β_t from the flash memory 14 (S43). A predetermined value for the time threshold β_t is stored in the flash memory 14 in advance.
[0055] The CPU 10 finishes the threshold determination process and returns to the same item discrimination process shown in Figure 14. The CPU 10 executes the first discrimination process (S32). The first discrimination process will be explained with reference to Figure 16. The first discrimination process is a process that determines whether a stroke is the same item as an adjacent item using a simplified method that uses the threshold frame N1 shown in Figure 3.
[0056] As shown in Figure 16, the CPU 10 creates a threshold frame N1 for, for example, a stroke 27 that has been handwritten into the display area 50 shown in Figure 3 (S51). In the process of S51, for example, the CPU 10 creates an item frame M1 shown in Figure 3, and then creates the threshold frame N1 by adding a distance threshold β_L to the created item frame M1. Alternatively, for example, the CPU 10 may add a distance threshold β_L to a plurality of points A_k (k=1 to n) that constitute the stroke 27, and create a rectangular frame with the largest range as the threshold frame N1. Furthermore, the threshold frame N1 may have a shape other than a rectangle; for example, it may be a roughly elliptical frame with a distance threshold β_L added around the stroke 27.
[0057] The CPU 10 determines whether or not item 25 exists in the display area 50 (S52). If item 25 does not exist in the display area 50 (S52: NO), the CPU 10 recognizes stroke 27 as a new item (S56). On the other hand, if one or more items 25 exist in the display area 50 (S52: YES), the CPU 10 saves the item 25 closest to stroke 27 as an adjacent item (S53). The CPU 10 determines whether or not the adjacent item is located within the threshold frame N (S54). As shown in the example in Figure 3, if the adjacent item 25 is located within the threshold frame N1 (S54: YES), the CPU 10 recognizes stroke 27 as the same item as item 25 (S55).
[0058] If the adjacent item is not within the threshold frame N1 (S54: NO), the CPU 10 recognizes the stroke as a new item separate from the adjacent item (S56). The CPU 10 terminates the first discrimination process and returns to the same item discrimination process shown in Figure 14.
[0059] In the first discrimination process in S32, the CPU 10 determines whether or not it recognized the stroke as the same item as another item (S33). If the stroke is recognized as a different item (S33: NO), the CPU 10 terminates this process. If the stroke is recognized as the same item (S33: YES), the CPU 10 executes the second discrimination process (S34). The second discrimination process determines whether the items are the same more accurately than the first discrimination process.
[0060] The second discrimination process will be explained with reference to Figure 17. As shown in Figure 17, the CPU 10 searches for the closest stroke among adjacent items (S61). For example, the example shown in Figure 18 shows the state after the string "ABC" has been handwritten horizontally, and then the first stroke ST8 of "D" in "DEFG" has been handwritten below "A". "A" is composed of three strokes ST1 to ST3. "B" is composed of three strokes ST4, ST5, and ST6.
[0061] The letter "C" is composed of a single stroke ST7. The string "ABC" is recognized as item 21. The adjacent item to stroke ST8 is item 21. Of the strokes in item 21, the one closest to stroke ST8 is the first stroke ST1 of "A".
[0062] The CPU 10 calculates the distance L_m between multiple points C_k (k=1 to n) that constitute the stroke ST8 and multiple points B_k' (k'=1 to n') that constitute the stroke ST1 in item 21 (S62). In the example in Figure 18, the distance L_m shown is between point B_k', the lowest point of the stroke ST1, and point C_k, the starting point of the stroke ST8. The CPU 10 determines whether there is at least one distance L_m among the calculated distances L_m that is smaller than the distance threshold β_L (S63). If all distances L_m are greater than the distance threshold β_L (S63: NO), the CPU 10 recognizes the stroke ST8 as a new item (S67).
[0063] On the other hand, if there is at least one distance L_m smaller than the distance threshold β_L (S63: YES), the first discrimination process described above recognizes them as the same item. In contrast, the second discrimination process makes a more detailed judgment based on the time information of the input points that form the stroke in order to determine more accurately that they are the same item. If the CPU 10 determines YES in the judgment process of S63, it calculates the time difference t_m between the input time t_z of the input points that form the strokes ST1 to ST7 in item 21 and the input time t of the starting point C_k of stroke ST8 (S64). The CPU 10 stores the input time for each stroke in the flash memory 14.
[0064] The CPU 10 determines whether there is at least one time difference t_m that is smaller than the time threshold β_t (S65). The time threshold β_t is the threshold for the time difference t_m. The default value of the time threshold β_t is stored in the flash memory 14 in advance according to user settings.
[0065] If all time differences t_m are greater than the time threshold β_t (S65: NO), the CPU 10 recognizes the stroke as a new item separate from the adjacent items (S67). In the example in Figure 17, the user recognizes "ABC" and "DEFG" as separate strings, so when they are entered by hand, a certain time difference occurs between "ABC" and "DEFG". Therefore, even if the distance L_m is smaller than the distance threshold β_L, the time difference t_m will be greater than the time threshold β_t (S65: NO). Consequently, the CPU 10 can recognize the stroke ST8 as a new item separate from item 21 (S67). Note that the time difference between each string varies depending on the user, so the time threshold β_t should be set according to the user using the information processing device 1.
[0066] On the other hand, if there is at least one time difference t_m smaller than the time threshold β_t (S65: YES), the CPU 10 recognizes stroke ST8 as the same item as item 21 (S66). Next, the CPU 10 terminates the second decision process and returns to the same item determination process, and terminates the same item determination process. Therefore, the CPU 10 can determine whether the input stroke is a single item. Thus, the CPU 10 can count the number of items written on document 60.
[0067] Next, with reference to Figures 19 and 20, the fill determination process for determining whether multiple strokes belong to the same fill item will be explained. The CPU 10 acquires the coordinate information of the start point P1 and end point P2 of the stroke ST10 shown in Figure 20 (S71). Next, the CPU 10 calculates the distance between the start point P1 and end point P2 of the stroke ST10 (S72). Next, the CPU 10 calculates the length of the sequence of points that make up the stroke (S73). Next, the CPU 10 determines whether the distance L_m between the start point P1 and end point P2 of the stroke is less than the second distance threshold (S74). The second distance threshold is a predetermined value stored in the flash memory 14 in advance for determining whether it is a fill contour line.
[0068] If the CPU 10 determines that the distance between the start and end points of the stroke ST10 is less than the second distance threshold (S74: YES), the CPU 10 determines whether the length of the stroke ST10 is greater than or equal to the length threshold (S75). The length threshold is a predetermined value stored in the flash memory 14 in advance for determining whether it is a filled outline.
[0069] If the CPU 10 determines that the stroke length ST10 is greater than or equal to a length threshold (S75: YES), it determines whether there are other strokes ST11 within the area enclosed by stroke ST10 that occupy a certain percentage or more of the area (S76). The threshold for the certain percentage is a predetermined value stored in the flash memory 14 in advance for determining a filled area. The threshold for the certain percentage is 50%. If the CPU 10 determines YES in S76, it recognizes the strokes ST11 that can be determined to be filled areas as the same item as stroke ST10 (S77). Therefore, the CPU 10 can determine that the strokes ST11 enclosed by stroke ST10 are a single filled item. Consequently, the number of filled items can also be counted in S16 and S19 of Figure 6.
[0070] In the above embodiment, as shown in Figure 7, as the display magnification of the document 60 increases, the page display area 65 displayed in the display area 50 becomes narrower. Consequently, the number of items included in the page display area 65 decreases. Therefore, even if the amount of dot thinning is reduced, the load on the CPU 10 that controls the information processing device 1 for display processing does not increase significantly. However, as shown in Figure 2, as the display magnification decreases, the page display area 65 displayed in the display area 50 expands, so the number of items included in the page display area 65 increases. Consequently, the load on the CPU 10 for display processing tends to increase. In this case, increasing the amount of dot thinning reduces the load on the CPU 10 for display processing.
[0071] Furthermore, in the above embodiment, as shown in Figure 8, the amount of dot thinning that constitutes the stroke of item 21 located outside the display area 50 may be increased compared to the amount of dot thinning that constitutes the stroke of item 23 located inside the display area 50. In this case, the load on the CPU 10 for display processing can be reduced. Also, since item 21 located outside the display area 50 is not displayed in the page display area 65, it does not affect the user.
[0072] Furthermore, in the above embodiment, the resolution of the sequence of points constituting the stroke of item 21 located outside the display area 50 may be lower than the resolution of the sequence of points constituting the stroke of item 23 located within the display area 50. In this case, the load on the CPU 10 for display processing can be reduced. Also, since item 21 located outside the display area 50 is not displayed in the page display area 65, it does not affect the user.
[0073] Furthermore, in the above embodiment, the first modification process (S18, S21) is executed after the completion of the display magnification change process (S13: YES, S14), which places a heavy burden on the terminal's display processing, thus reducing the load on the CPU 10's display processing.
[0074] Furthermore, in the above embodiment, when the total number of items exceeds a certain number, the resolution of the sequence of points in the stroke within the display area is reduced, thereby reducing the load on the CPU 10 for display processing.
[0075] Furthermore, in the above embodiment, when the total number of items exceeds a certain number, the resolution of the sequence of points in the stroke within the display area is reduced, thereby reducing the load on the CPU 10 for display processing.
[0076] Furthermore, in the above embodiment, in the second discrimination process shown in Figure 17, if the distance calculated by the distance calculation process (S62) is less than the distance threshold β_L (S62: YES), the CPU 10 recognizes the two strokes as the same item (S66). Therefore, it is easy to determine whether they are the same item based on the distance between the two strokes.
[0077] Furthermore, in the above embodiment, if the time difference calculated by the time difference calculation process S64 is less than the time threshold (S64: YES), the CPU 10 recognizes the two strokes as the same item. Therefore, the determination of whether they are the same item can be accurately made based on the time difference between the input strokes.
[0078] Furthermore, in the above embodiment, if the distance calculated by the start-end distance calculation process (S72) in Figure 19 is less than the second distance threshold, processes S74, S75, and S76 are performed to recognize whether they are the same item, and a decision is made as to whether or not they are filled in. Since the parts that can be determined to be filled in are recognized as the same item, it is easy to determine whether they are the same item.
[0079] Furthermore, in the above embodiment, if the operating clock speed of the CPU 10 is not high, the amount of dot thinning that constitutes the stroke is increased. Therefore, the load on the display processing of the CPU 10 with a low operating clock speed can be reduced.
[0080] In the above description, the flat panel display 17 of the touchscreen display 16 is an example of the "display unit" of the present invention. The sensor 18 is an example of the "input unit" of the present invention. The process in S4 is an example of the "first acquisition process" of the present invention. The process in S14 is an example of the "display area acquisition process" and "display change process" of the present invention. The processes in the first change process (S18, S21) are an example of the "first change process" of the present invention. The process in S21 is an example of the "second change process" of the present invention, and the process in S22 is an example of the "third change process" and "fourth change process" of the present invention.
[0081] The process in S62 is an example of the "first distance calculation process" of the present invention. The process in S66 is an example of the "first recognition process" of the present invention. The process in S67 is an example of the "individual recognition process" of the present invention. The process in S64 is an example of the "time calculation process" of the present invention. The distance threshold β_L is an example of the "first distance threshold".
[0082] The process in S72 is an example of the "second distance calculation process" of the present invention. The process in S73 is an example of the "length calculation process" of the present invention. The processes in S74, S75, S76, and S77 are examples of the "second recognition process" of the present invention.
[0083] Furthermore, the present invention can be modified in various ways beyond the above-described modifications. For example, in determining identical items, the CPU 10 may perform an "image determination process" to determine that multiple strokes recognized as a single character or figure by image recognition processing are the same item. In this case, since the determination of whether they are the same item is made by image recognition processing, it is easy to determine whether they are the same item. In the above embodiment, a distance threshold β_L and a time threshold β_t are used to determine whether two strokes are the same item or different items, but the determination may also be made using only the distance threshold β_L.
[0084] Next, a modified example of the S4 process in the handwriting stroke addition process shown in Figure 5 will be described. In the S4 process, the CPU 10 may acquire stroke information by reducing the number of dots by a predetermined percentage based on the density of the dot sequence when the stylus pen 4 is input. In this case, since the stroke information is acquired by reducing the number of dots by a predetermined percentage based on the density of the dot sequence when the stylus pen is input, the load on the computer's display processing can be reduced.
[0085] The amount of thinning in the thinning process is not limited to medium and large. The amount of thinning may be set finely. Similarly, the resolution reduction process is not limited to two stages, medium and lowest quality, but may be set in multiple fine stages. Also, an example of the first quantity in the judgment criterion of S16 is not limited to 10,000 items, but may be set according to the performance of the CPU 10. Likewise, an example of the second quantity in the judgment criterion of S19 is not limited to 15,000 items, but may be set according to the performance of the CPU 10.
[0086] An example of a predetermined threshold for the judgment process in S17 is that the vertical or horizontal length of the handwritten stroke is not limited to 5 mm in the display area 50. The predetermined threshold may be set according to the performance of the CPU 10. Also, the judgment based on the number of items may be set in more detail, such as three or five stages, rather than just two stages in S16 and S19. Furthermore, the judgment based on the number of items may be set in only one stage. Also, the resolution reduction process is not limited to two stages, medium quality and lowest quality; it may be one stage, or multiple stages such as three or five stages.
[0087] As shown in Figure 13, if document 60 contains hierarchical information, the CPU 10 may reduce the resolution of the strokes that make up the items as the hierarchy level decreases. For example, the CPU 10 does not perform a resolution reduction process on item 21 of the top-level document 60A. The CPU 10 also performs a resolution reduction process to medium quality on item 22 of the middle-level document 60B. Furthermore, the CPU 10 performs a resolution reduction process to minimum quality on item 23 of the bottom-level document 60C. In this case, the lower levels are difficult to see, so the impact on the display unit is small, and the load on the CPU 10's display processing can be reduced.
[0088] In Figures 2, 7, and 8, the editing window 5 is displayed across the entire surface of the touchscreen display 16. However, the present invention is not limited to this display, and the editing window 5 may be displayed as a partial window rather than covering the entire surface of the touchscreen display 16. Furthermore, instead of the CPU 10, a microcomputer, ASIC (Application Specific Integrated Circuits), FPGA (Field Programmable Gate Array), etc., may be used as the processor.
[0089] 1. Information processing device 4. Stylus pen 5. Editing window 10. CPU 14. Flash memory 16. Touchscreen display 21-23. Items 50. Display area 60. Document 65. Page display area
Claims
1. A display control program executed by a computer controlling an information processing device comprising a display unit and an input unit, characterized in that it causes the computer to execute: a first acquisition process for acquiring information of strokes handwritten by the input unit onto a document displayed on the display unit; a display area acquisition process for acquiring information indicating the display position and display magnification of a display area which is a display area on the document; a display change process for changing the display position and / or display magnification of the display area upon detection of input from the input unit; and a first change process for decreasing the amount of dot thinning of the stroke constituting the stroke displayed on the display unit when the display magnification of the display area increases in the display change process, and increasing the amount of dot thinning of the stroke displayed on the display unit when the display magnification of the display area decreases.
2. The display control program according to claim 1, characterized in that the computer is instructed to perform a second modification process which increases the amount of thinning of the dot sequence in the stroke outside the display area to the amount of thinning of the dot sequence in the stroke inside the display area.
3. The display control program according to claim 1, characterized in that the computer is instructed to perform a third modification process that lowers the resolution of the sequence of points of the stroke located outside the display area to a lower resolution of the sequence of points of the stroke located within the display area.
4. The display control program according to claim 1, characterized in that the computer is made to execute the first change process after the completion of the display change process.
5. The display control program according to claim 1, characterized in that, in the first acquisition process, the computer is instructed to acquire information about the stroke, in which the number of dots in the stylus pen used for inputting the stroke is reduced by a predetermined percentage based on the density of the dot sequence at the time of input.
6. The display control program according to claim 1, characterized in that when the total number of items including the stroke displayed in the display area exceeds a certain number, the computer is instructed to perform a fourth modification process to reduce the resolution of the sequence of points of the stroke in the display area.
7. The display control program according to claim 1, characterized in that the document displayed on the display unit has hierarchical information, and causes the computer to execute a fifth modification process that increases the amount of dot thinning in the stroke as the hierarchy goes down.
8. The display control program according to claim 1, characterized in that it causes the computer to perform an image determination process to determine whether the plurality of strokes acquired by the first acquisition process constitute a single character or figure; a first recognition process to recognize the plurality of strokes as the same item if the image determination process determines that they constitute a single character or figure; and an individual recognition process to recognize the plurality of strokes as separate items if the image determination process does not determine that they constitute a single character or figure.
9. The display control program according to claim 1, characterized in that it causes the computer to execute: a first distance calculation process that calculates the distance between two of the plurality of strokes acquired by the first acquisition process; a first recognition process that recognizes the two strokes as the same item if the distance calculated by the first distance calculation process is less than a first distance threshold; and an individual recognition process that recognizes the two strokes as separate items if the distance calculated by the first distance calculation process is equal to or greater than the first distance threshold.
10. A display control program according to claim 1, characterized by comprising: a time calculation process for calculating the time difference between two stroke inputs input by the input unit; and, if the time difference calculated by the time calculation process is less than a time threshold, recognizing the two strokes as the same item; and, if the time difference calculated by the time calculation process is equal to or greater than the time threshold, recognizing the two strokes as separate items.
11. The display control program according to claim 1, characterized in that it causes the computer to execute: a second distance calculation process that calculates the distance between the start and end points of the stroke acquired by the first acquisition process; a length calculation process that calculates the length of the sequence of points constituting the stroke; and a second recognition process that, if the distance calculated by the second distance calculation process is less than a second distance threshold, and the length calculated by the length calculation process is equal to or greater than the length threshold, and if the area enclosed by the stroke includes other strokes occupying a certain percentage or more of the area, determines the area enclosed by the stroke to be a filled portion and recognizes the portion that can be determined to be a filled portion as the same item.
12. The display control program according to claim 1, characterized in that the computer is made to perform a setting process to set the amount of thinning of the sequence of points constituting the stroke according to the operating clock number of the computer.