Work record extraction device, work record extraction system

Abstract

To provide a work record extracting apparatus which can correctly select drawing elements corresponding to handwriting even if there is any displacement of handwriting, when positional coordinates are collated between handwriting data obtained by overwriting drawing data with handwriting and the drawing elements on the drawing data.SOLUTION: A work record extracting apparatus 100 according to the present invention has a handwritten data acquiring unit 120 for acquiring handwritten data indicating a route obtained by an operator's tracing drawing elements on drawing data on a work terminal by handwriting, and a data collation unit 130 for collating the handwritten data and the drawing data to identify the drawing elements on which the handwritten data is traced. The data collation unit 130 sets a boundary region including at least a part of the drawing elements at the surroundings of the drawing elements, determines whether or not the handwritten data has passed through at least a part of the boundary region, and determines, if the data has passed through it, that the handwritten data has passed through the drawing element to output the determination result.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for extracting a record of work performed by a worker through handwritten input on a work terminal. [Background technology]

[0002] Due to population decline and an aging society with a low birthrate, manufacturing sites are lacking experienced technicians with high on-site skills. In order to maintain work quality, it is a challenge to transfer the advanced skills of experienced technicians to younger technicians. To promote the transfer of skills, it is necessary to create a system in which the work procedures and know-how of experienced technicians can be compiled into manuals so that work can be carried out regardless of skill level. In particular, work that requires the use of blueprints is highly dependent on on-site skills, and there is an urgent need to create manuals for experienced know-how.

[0003] An example of drawing work is the task of checking the continuity and sequence operation of a circuit using an electrical drawing that describes the layout and wiring of circuit components, and then hand-painting the relevant drawing elements. In order to compile the work procedures and know-how of skilled engineers into a manual, it is necessary to extract and analyze the work records on the drawings as digital information. When trying to digitize work records using wearable devices such as electronic paper or tablets, it is necessary to compare the position coordinates of the handwritten data overlaid on the drawing with the drawing elements on the back. Drawing elements here include, for example, wiring, drawing symbols, and characters.

[0004] Patent Document 1 discloses a technique for searching for a route handwritten by a user by collating handwritten data overlaid on map data with the position coordinates of roads on the map. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2001-074481 A Summary of the Invention [Problem to be solved by the invention]

[0006] The method described in Patent Document 1 always selects one road that is closest to the sampling position of the handwritten data. When this method is applied to the above-mentioned drawing work, if the handwritten route is far away from a drawing element, but the drawing element is closest to the handwriting, the drawing element will be selected against the worker's intention. In addition, if the handwritten part deviates significantly locally during the work and approaches another drawing element, the other drawing element will be selected unintended by the worker. Therefore, a system that can correctly extract work records according to the worker's intention even when there is a handwriting deviation is desired.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a work record extraction device that, when comparing position coordinates between handwritten data manually overwritten on drawing data and drawing elements on the drawing data, can correctly select drawing elements corresponding to the handwritten data even if there is handwriting misalignment. [Means for solving the problem]

[0008] The work record extraction device of the present invention sets a boundary area around a drawing element that encompasses at least a portion of the drawing element, and determines whether handwritten data has passed through at least a portion of the boundary area, and if so, determines that the handwritten data has passed through the drawing element. Effect of the Invention

[0009] According to the work record extraction device of the present invention, when comparing position coordinates between handwritten data manually overwritten on drawing data and drawing elements on the drawing data, it is possible to correctly select drawing elements corresponding to the handwritten data even if there is misalignment. Other problems, configurations, and effects will become clear from the description of the embodiments below. [Brief description of the drawings]

[0010] [Figure 1] 1 shows examples of drawing data and handwritten data that has been manually overwritten onto the drawing data. [Diagram 2] 1 is a functional block diagram of a work record extraction device 100 according to a first embodiment. [Figure 3A] An example of handwritten representation 122 on which handwritten data 121 is based is shown. [Figure 3B] An example of handwritten data 121 is shown below. [Figure 4A] 13 is a diagram showing an example of drawing element data 111. FIG. [Figure 4B] 13 is a diagram showing an example of drawing element data 111. FIG. [Figure 4C] 13 is a diagram showing an example of drawing element data 111. FIG. [Diagram 5] 13 is a flowchart illustrating the operation of the data collating unit 130. [Figure 6A] 13A and 13B are diagrams illustrating an allowable width that is set to provide a boundary area. [Figure 6B] 13A and 13B are diagrams illustrating an allowable width that is set to provide a boundary area. [Figure 6C] 13A and 13B are diagrams illustrating an allowable width that is set to provide a boundary area. [Figure 7A] 13A and 13B are diagrams illustrating unit widths that are set for dividing a boundary area. [Figure 7B] 13A and 13B are diagrams illustrating unit widths that are set for dividing a boundary area. [Figure 8A] 11A and 11B are diagrams for explaining advantages obtained by increasing the number of divisions by unit width. [Figure 8B] 11A and 11B are diagrams for explaining advantages obtained by increasing the number of divisions by unit width. [Figure 9A] 13A and 13B are diagrams showing examples of methods for setting an allowable width and a unit width for an obliquely inclined wiring; [Figure 9B] 13A and 13B are diagrams showing examples of methods for setting an allowable width and a unit width for an obliquely inclined wiring; [Figure 9C] 13A and 13B are diagrams showing examples of methods for setting an allowable width and a unit width for an obliquely inclined wiring; [Figure 10A] An example of setting the tolerance width and unit width for a drawing symbol is shown below. [Figure 10B] An example of setting the tolerance width and unit width for a drawing symbol is shown below. [Figure 10C] An example of setting the tolerance width and unit width for a drawing symbol is shown below. [Figure 11] 11A and 11B are diagrams for explaining a method for narrowing down drawing elements to be matched; [Figure 12] 13 shows a data table illustrating the configuration of the check judgment data 131 and an example of the data. [Figure 13] 13 is a flowchart illustrating the operation of a work record generating unit 141. [Figure 14A] 1 shows an example of work record data 112. [Figure 14B] 1 shows an example of work record data 112. [Figure 15A] An example of the display data 113 is shown. [Figure 15B] An example of the display data 113 is shown. [Figure 16] FIG. 11 is a configuration diagram of a work record extraction device 100 in a second embodiment. [Figure 17] 11 is a flowchart illustrating an operation of the data collating unit 130 in the second embodiment. [Figure 18A] FIG. 15 is a diagram for explaining a determination method for extracting a note in S1501. [Figure 18B] FIG. 15 is a diagram for explaining a determination method for extracting a note in S1501. [Figure 18C] FIG. 15 is a diagram for explaining a determination method for extracting a note in S1501. [Figure 19] 13 shows an example of the note determination data 132. [Figure 20] 13 is a flowchart for explaining the operation of a work record generating unit 141 in the second embodiment. [Figure 21] 13 shows an example of work record data 112 in the second embodiment. [Figure 22] 13 is a flowchart illustrating an operation of the data collating unit 130 in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <Embodiment 1> FIG. 1 shows an example of drawing data and handwritten data that has been manually overwritten on the drawing data. In this example, the drawing data describes the arrangement and wiring of circuit components. The circuit components and wiring are described as drawing elements. The worker displays the drawing data on a work terminal and inputs a work record using an input device or the like equipped on the work terminal. For example, a conduction path is input by handwriting tracing the circuit components that have been confirmed to be electrically conductive. In FIG. 1, two conduction paths starting from SIG1 and SIG2, respectively, have been handwritten. The worker may also handwrite a character string described in the drawing data (such as breaker A in FIG. 1).

[0012] As shown in FIG. 1, handwritten input may be locally misaligned. For example, the handwritten conductive path starting from SIG2 is locally misaligned to the right of the character string "CD" so as to approach the wiring one line below. In this case, it is difficult to identify which wiring was traced on the data. The present invention aims to correctly identify the handwritten path (i.e., which drawing elements were traced) even in such a case.

[0013] 2 is a functional block diagram of a work record extraction device 100 according to the first embodiment of the present invention. The work record extraction device 100 is connected to a first terminal 10 used by a worker and a second terminal 200 used by a manager. An example of the first terminal 10 is a wearable device (hereinafter referred to as a WD). A WD is a device, such as a tablet or electronic paper, that can record handwritten time-series data when handwriting is written on a screen using an electronic pen or the like.

[0014] The work record extraction device 100 includes: a data communication unit 101 that transmits and receives data to the first terminal 10 and the second terminal 200; a display control unit 102 that controls the data communication unit 101 to output data to at least one of the first terminal 10 and the second terminal 200; a data storage unit 110; a handwritten data acquisition unit 120 that acquires data (handwritten data 121) describing a route handwritten by a worker on the first terminal 10; a data comparison unit 130 that determines whether the worker selected or checked the drawing element data by handwriting by comparing the position coordinates between the handwritten data 121 and drawing element data 111 corresponding to the work drawing; and a data generation unit 140 that generates work record data 112 based on the determination result by the data comparison unit 130, and generates display data 113 in which the work record data 112 is visualized on the drawing.

[0015] The handwritten data acquisition unit 120 acquires handwritten data 121 describing the results of handwritten input by the worker on the first terminal 10, for example, by communicating with the first terminal 10. The handwritten data 121 is a collection of data including the generation time, attributes, coordinates, etc. of ink data generated by handwriting. The ink data is a collection of strokes that are generated when the user touches the screen of the first terminal 10 with an electronic pen to write by hand and then moves away from the screen. If the user moves away from the screen and touches the screen again within a predetermined time to write by hand, one piece of ink data will have multiple strokes. The details of a stroke will be described later, but it refers to the path of a single stroke while touching the screen.

[0016] The data matching unit 130 has check judgment data 131. The check judgment data 131 describes the result of matching the position coordinates between the handwritten data 121 and the drawing element data 111, and is a collection of data including time information added by the matching and the judgment result.

[0017] The data generation unit 140 includes: a work record generation unit 141 that generates work record data 112 based on the determination result of the data comparison unit 130; and a display data generation unit 142 that generates display data 113 based on the work record data 112.

[0018] The data storage unit 110 stores drawing element data 111, work record data 112, and display data 113. The drawing element data 111 is a collection of data including the coordinates and attributes of each element included in a drawing to be displayed on the first terminal 10. The work record data 112 is a collection of work record data including check presence / absence information, work sequence, and required work time obtained based on the check judgment data 131. The display data 113 is a display drawing in an image format, a portable document format (PDF), or the like, in which the work record data 112 is visualized on a drawing so that the work record can be easily understood by a third party.

[0019] Examples of the file formats of the drawing element data 111, the work record data 112, the handwritten data 121, and the check judgment data 131 include text files and CSV (Comma Separated Value) files. Examples of the file format of the display data 113 include image files such as PDF files and PNG (Portable Network Graphics) files.

[0020] <First embodiment: Outline of operation of work record extraction device 100> The handwritten data acquisition unit 120 acquires time-series handwritten data 121 handwritten by an operator on the first terminal 10 from the first terminal 10 via the data communication unit 101 and the network at any timing, and outputs the data to the data collation unit 130. The handwritten data 121 may be stored in advance in the data storage unit 110 and acquired from the data storage unit 110.

[0021] The data comparison unit 130 compares the position coordinates between the input handwritten data 121 and the drawing element data 111 obtained from the data storage unit 110, and records the comparison result as check judgment data 131. The comparison here refers to determining whether or not the area of ​​the corresponding drawing element has been hand-painted. If the area of ​​the drawing element has been painted, the drawing element is considered to be checked, and if not, the drawing element is considered to be unchecked. The generated check judgment data 131 is sent to the work record generation unit 141 in the data generation unit 140.

[0022] The work record generating unit 141 rearranges the check judgment results in chronological order based on the input check judgment data 131. Furthermore, it calculates the time required for each check task. The work record generating unit 141 records these results as work record data 112 and outputs them to the data storage unit 110.

[0023] The display data generation unit 142 acquires the work record data 112 output to the data storage unit 110. The display data generation unit 142 extracts work records required by the manager based on the acquired work record data 112, and generates display data 113 by visualizing the extracted work records on a drawing so that the manager can easily view it.

[0024] When there is a data display request from the first terminal 10 or the second terminal 200, the work record data 112 and the display data 113 are transmitted to the requesting terminal via the display control unit 102 and the data communication unit 101. Each terminal displays the data. At this time, the display control unit 102 selects a terminal to output the data, and commands the data communication unit 101 to transmit the data to the selected terminal.

[0025] <First embodiment: handwritten data 121> FIG. 3A shows an example of handwritten display 122 that is the source of handwritten data 121. In the following description, it is assumed that the first terminal 10 is a WD. As shown in FIG. 3A, a stroke indicates a path drawn in one stroke by an operator from the time the operator touches the WD screen with an electronic pen until the operator releases it. As shown in handwritten display 122, a stroke is made up of a collection of points that spatially sample the path based on the position coordinates of the pen tip touching the WD. For example, one stroke shown in ink No. 1 is made up of six sample points, (X1, Y1) to (X6, Y6). Ink data is a data set of strokes, and a stroke is a data set of sample points (hereinafter, points).

[0026] Since the sampling frequency of the WD touch panel is finite, the density of the point cloud coordinates depends on the speed at which the person writes; as shown in Ink No. 2, the slower the handwriting, the higher the density, and the faster the handwriting, the lower the density. Multiple strokes written within a specified time after the electronic pen leaves the WD screen may be considered as the same ink data. In the example shown in Ink No. 3, three strokes are treated as one ink data, and each stroke can be distinguished by a unique stroke number assigned to each stroke. A unique point number may be assigned to each point in each stroke to make them distinguishable.

[0027] FIG. 3B shows an example of the handwritten data 121. The handwritten data 121 can be configured as a data table. The handwritten display 122 shown in FIG. 3A is an example including three ink data, each of which has time information t=t1, t=t2, and t=t3, and is numbered No. 1 to No. 3 in chronological order. Each row of the handwritten data 121 is ink data corresponding to each of the handwritings No. 1 to No. 3. Each row includes a unique ink number, the handwriting generation time of the handwritten start point, the user ID of the handwritten handwriting, the terminal ID of the handwritten handwriting, the pen attributes (including color, width, transparency, pen type such as marker / pen), sample point cloud coordinates included in the stroke (hereinafter, point cloud coordinates), and WD resolution.

[0028] <First embodiment: drawing element data 111> 4A to 4C are diagrams showing examples of the drawing element data 111. The drawing element data 111 indicates a collection of data including the names, coordinate information, and grouping information of each element of wiring, drawing symbols, and characters in a drawing. Wiring indicates a conductor that connects drawing symbols included in a circuit in a drawing. Drawing symbols indicate symbols representing components included in a circuit (e.g., electrical symbols defined by JIS standards, etc.). Characters indicate attribute information of wiring or drawing symbols by being written around the wiring or drawing symbols in a circuit.

[0029] FIG. 4A shows an example of data describing wiring among the drawing element data 111. Wiring data 111-1 is an example of data in a data table describing wiring. Wiring display 111-A shows the drawing element that is the source of wiring data 111-1. Each row of wiring data 111-1 is data corresponding to wiring No. 1 to No. 4 in wiring display 111-A. Each row includes a unique wiring number, name, coordinates (x, y coordinates of the start point and end point), and line information. When the line information includes multiple coordinates (for example, wiring No. 4 is composed of three lines), the line information includes the x, y coordinates of the start point and end point of each line, and the coordinates of the endpoints of the three lines are entered as the start point and end point in the coordinate column. In the example of Figure 4A, the -X direction is the start point and the +X direction is the end point (when the x coordinates are the same like the wire No. 2, the -Y direction is the start point and the +Y direction is the end point), but the start point and end point may be changed as appropriate depending on the origin position of the drawing. The connected wires like No. 4 may be managed as one group, or each may be managed as a separate wire.

[0030] FIG. 4B shows an example of data describing a drawing symbol among the drawing element data 111. Drawing symbol data 111-2 is an example of data in a data table describing a drawing symbol. Drawing symbol display 111-B shows drawing elements that are the source of drawing symbol data 111-2. Each row of drawing symbol data 111-2 is data corresponding to each of the drawing symbols No. 1 to No. 3 in drawing symbol display 111-B. A drawing symbol is composed of multiple lines, but in order to handle it as a meaningful symbol unit, its area is defined by the coordinates of the diagonal corners of a rectangular area. FIG. 4B shows an example in which the coordinates are defined as the lower left (start point) and upper right (end point) of a rectangular area. The positions of the start point and end point may be changed as appropriate depending on the position of the origin of the drawing. Each row includes a unique drawing symbol number, name, coordinates (x, y coordinates of the start point and end point of the rectangular area), and the connection direction with the wiring. If the connection direction with the wiring is one direction, such as No. 1 and No. 2, it is described as the "x direction" and "y direction", respectively. When there is a connection relationship in both the x and y directions, as in No. 3, it is described as "xy direction." The information on the connection direction with the wiring may include the relationship between input and output. For example, when describing "xy direction," x may be input and y may be output.

[0031] FIG. 4C shows an example of data describing a character among the drawing element data 111. Character data 111-3 is an example of data in a data table describing a character. Character display 111-C shows a character that is the source of character data 111-3. Each row of character data 111-3 is data generated for each of characters No. 1 to No. 3 in character display 111-C. In order to handle a character as a unit of meaningful character information, its area is defined by the coordinates of the diagonal corners of a rectangular area. In FIG. 4C, the coordinates are defined by the x and y coordinates of the lower left (start point) and upper right (end point) of the rectangular area. The positions of the start point and end point may be changed as appropriate depending on the origin position of the drawing. Each row includes a unique character number, name, coordinates (x and y coordinates of the rectangular area), and the direction of the character string. The character string direction is the x direction for No. 1 and No. 2, and the y direction for No. 3.

[0032] By indicating the direction for each drawing element, the direction of the unit width, which will be described later, can be set based on the direction of the drawing element. For example, in the case of a drawing symbol, an operator hand-paints an area along the connection direction between the wiring and the drawing symbol when checking the drawing symbol, so the division direction of the unit width can be set horizontal to the connection direction. In the case of characters, an operator hand-paints an area along the direction in which the characters are read, so the division direction can be set horizontal to the direction of the character string.

[0033] As will be described later, even if the wiring extends in an oblique direction, the wiring and the drawing symbol (or the wiring and the character string) are almost always connected in either the x or y direction. This tendency is particularly noticeable in circuit diagrams. Therefore, it should be noted that the present invention is based on the premise that the connection direction is either the x or y direction.

[0034] <First embodiment: Operation of data collating unit 130> FIG. 5 is a flow chart for explaining the operation of the data matching unit 130. First, the overall flow will be explained, and then each step will be explained in detail. The data matching unit 130 acquires the handwritten data 121 from the handwritten data acquisition unit 120, and acquires the corresponding drawing element data 111 from the data storage unit 110 (S501). The data matching unit 130 confirms that the coordinate space of the handwritten data 121 (WD resolution) and the coordinate space of the drawing element data 111 (drawing file resolution) match (S502). If they do not match, one coordinate space is converted to match the other coordinate space so that both resolutions match (S503). The data matching unit 130 checks the point cloud coordinates of each stroke of the handwritten data 121 and the position coordinates of the drawing element data 111 to determine whether the drawing element has been checked (S504). After the judgment, the data collating unit 130 generates check judgment data 131 as the judgment result (S505), and outputs the check judgment data 131 to the data generating unit 140 (S506).

[0035] The method of coordinate conversion in S503 will be described. If the resolution of the WD and the resolution of the drawing file do not match, the coordinate spaces of the two are different from each other, so that the position coordinates cannot be correctly collated in S504. In this case, the data collation unit 130 needs to convert one coordinate space to match the other coordinate space so that the resolutions of both match. The conversion method will be specifically described below.

[0036] The number of pixels in the X direction of the resolution of the handwritten WD (for example, the number of pixels for a tablet, the number of dots for electronic paper) is Xwd, and the number of pixels in the Y direction is Ywd, and the number of pixels in the X direction of the resolution of the drawing file is Xdd, and the number of pixels in the Y direction is Ydd. For example, when matching the coordinate space of the handwritten data 121 with the coordinate space of the drawing element data 111, assuming that the origins of each are the same, and assuming that the X and Y coordinates of the handwritten data 121 before the coordinate conversion are Xhw and Yhw, and the X coordinate Xc and the Y coordinate Yc after the coordinate conversion, the calculations can be made as follows: Xc = (Xdd / Xwd) x Xhw, Yc = (Ydd / Ywd) x Yhw. If the origins are different, for example, when the coordinate space of the handwritten data 121 is the upper left origin of the screen and the coordinate space of the drawing element data 111 is the lower left origin of the screen, a coordinate conversion that inverts in the Y direction is required, and in this case, Yc can be calculated as follows: Yc = (Ydd / Ywd) x (Ywd-Yhw).

[0037] A method of matching in S504 between the point cloud coordinates of each stroke in the handwritten data 121 and the element coordinates of the drawing element data 111 will be described. By matching the handwritten data 121 with the drawing element data 111 based on the position coordinates, it is possible to extract information on whether the corresponding drawing element has been checked, and check time information based on the time information in the handwritten data 121. Below, a method of matching wiring, drawing symbols, and characters will be described.

[0038] In the present invention, a boundary area is provided for matching with the point cloud coordinates included in the stroke of the handwritten data 121 based on the coordinates of the start point and end point of the drawing element data 111. The boundary area refers to a rectangular area provided around each drawing element. The boundary area is divided in either the horizontal or vertical direction. The length set in the division direction is defined as the unit width, and the length set in the direction other than the division direction is defined as the allowable width. The method of setting the allowable width and unit width will be described in detail later. The allowable width is set in the direction perpendicular to the direction of the drawing element, and the unit width is set in the direction horizontal to the direction of the drawing element. For example, in the case of wiring, the allowable width is set in the direction perpendicular to the extension direction, and the unit width is set in the horizontal direction. In the case of drawing symbols, the allowable width is set in the direction perpendicular to the connection direction with the wiring, and the unit width is set in the horizontal direction. In the case of characters, the allowable width is set in the direction perpendicular to the direction of the character string, and the unit width is set in the horizontal direction.

[0039] 6A to 6C are diagrams for explaining the allowable width set to provide a boundary region. Here, an explanation is given using a wiring as an example of a drawing element. Fig. 6A shows an example in which the wiring is considered not to have been checked based on the positional relationship between the stroke and the wiring, Fig. 6B shows an example in which the wiring is considered to have been checked based on the positional relationship between the stroke and the wiring, and Fig. 6C shows an example of setting the allowable width to provide a boundary region.

[0040] The worker leaves a record of the work by hand-painting a stroke on the area of ​​the drawing element corresponding to the part in the drawing where the check work was performed. As shown in Fig. 6A, when the stroke and the wiring are far apart, the stroke does not overlap the area of ​​the drawing element (the wiring in this example), so the data collation unit 130 considers that the drawing element has not been checked. As shown in Fig. 6B, when the stroke overlaps the area of ​​the drawing element, the data collation unit 130 considers that the drawing element has been checked.

[0041] The allowable width is set to determine whether or not the drawing element is checked when there is a vertical handwriting deviation that occurs substantially with respect to the direction of the drawing element. The size of the rectangle of the bounding area is the allowable width in either the horizontal or vertical direction, and the other is the length of the drawing element. The setting direction of the allowable width is set based on the direction of the drawing element as described above. For example, in the case of horizontal wiring as shown in Figure 6C, the length in the direction perpendicular to the bounding area is the allowable width, and the horizontal direction is the wiring length. In other words, if the coordinates of the start point and the end point are (xs, ys) and (xe, ye), respectively, the horizontal size of the bounding area is |xe-xs|. The length of the allowable width is set appropriately within the pen width that can be set on the terminal according to the operator. Different values ​​may be used for each drawing element (wiring, drawing symbol, or character).

[0042] The pen width here refers to the spot size of a virtual pen device virtually generated on the work terminal. For example, when a worker touches the screen of the work terminal, a virtual pen device is generated on the application, and the tip of the pen device moves along the position touched by the worker. This tip size corresponds to the pen width (or spot size) of the virtual pen device used to input handwritten data.

[0043] The data collation unit 130 regards the stroke point as checked if it is included in the boundary area, and regards it as unchecked if it is not included. By setting the length of the allowable width to the pen width, it is possible to allow vertical deviation to the lower or upper end of the pen width for the drawing element. For example, FIG. 6C shows a state in which the pen width is set to w and a line is drawn so that the upper end of the stroke line width overlaps the drawing element. By setting the length of the allowable width δtole to the pen width w, the stroke point is included in the boundary area set for the drawing element. Therefore, the drawing element can be determined to be checked.

[0044] For drawing symbols and characters, the size of the allowable width is a value that includes the length of the drawing element (x direction is Lx, y direction is Ly) in addition to the pen width components mentioned above. The length of the drawing element can be calculated from the coordinates of the start and end points of the rectangular area. For example, if the pen width is w and the coordinates of the start and end points are (xs, ys) and (xe, ye), respectively, then the allowable width in the y direction is δtole=w+Ly=w+|ye-ys|. For the allowable width in the x direction, it is δtole=w+Lx=w+|xe-xs|.

[0045] By setting a tolerance as described above, when determining whether a drawing element has been checked based on the worker's handwriting, if the drawing element and the handwriting are significantly different, they are deemed not to have been checked, and deviations in the worker's handwriting in the vertical direction relative to the direction of the drawing element can be tolerated.

[0046] When the drawing element is a wire (or a straight line), the allowable width is set in the direction perpendicular to the extension direction (the vertical direction in Figs. 6A to 6C), and a unit width (described later) is set along the extension direction. When the drawing element is a drawing symbol or a character string, the connection direction with the wire is used, as described later. The data collating unit 130 can acquire these directions as "direction data".

[0047] Next, the unit width will be described in detail. When observing actual human handwriting, there are not only handwriting deviations in the direction perpendicular to the direction of the drawing element, but also handwriting deviations that are unique to humans and are largely localized. Strokes that are largely localized and deviate may be included in a boundary area provided for a drawing element other than the drawing element to be checked. Therefore, in the present invention, in addition to the allowable width, a unit width is set in the direction horizontal to the direction of the drawing element, and the boundary area is divided into the unit width, so that it is possible to correctly determine whether an element is checked or not even when handwriting deviations that are unique to humans and are largely localized and deviate occur.

[0048] 7A and 7B are diagrams for explaining the unit width set to divide the boundary area. Fig. 7A shows an example in which a check judgment is erroneous due to a large localized deviation in handwriting. Fig. 7B shows a method for performing a correct check judgment by setting a unit width and dividing the boundary area. Here, an example is shown in which a unit width is set for wiring.

[0049] In Figure 7A, traces 1 and 2 are close to each other, and the stroke used by the worker to check trace 1 is locally significantly off due to misalignment of the handwriting, and part of it is unintentionally included from the outside in the boundary area of ​​trace 2. In this case, it is correct to determine that trace 1 has been checked, and that trace 2 has not been checked because the handwriting has only been applied locally on the trace area. However, if a boundary area with only the allowable width is used for trace 2, the stroke points will be included in the boundary area, and it will be erroneously determined that it has been checked.

[0050] In order to avoid such erroneous judgment, in this application, in addition to the allowable width, a unit width is set in the horizontal direction relative to the direction of the drawing element, and the boundary area is divided into the unit widths. By judging whether or not a point in a stroke is included for each unit width in the entire boundary area, it becomes possible to correctly check and judge even if there is a stroke that is locally included in the boundary area of ​​another drawing element due to handwriting misalignment. Specifically, the data collating unit 130 judges whether or not a check is performed based on a judgment formula for the judgment threshold (if N / Nunit≧Th_check is satisfied, it is considered that the check is performed), where Nunit is the number of unit widths in the entire boundary area, N is the number of unit widths that are included in the divided area by the point in the stroke, and Th_check is the judgment threshold.

[0051] The strictness of the check judgment for handwriting misalignment can be dynamically determined by the judgment threshold. In other words, the larger the judgment threshold, the less overlap between the stroke and the drawing element will be judged as checked. Conversely, the smaller the judgment threshold, the less overlap between the stroke and the drawing element will be considered as checked.

[0052] The determination threshold can be arbitrarily set between 0 and 1. In an actual usage environment, when an operator handwrites and paints more than half of the area of a drawing element, it is highly likely that the drawing element has been checked. Therefore, if the minimum value of Th_check is set to 0.5 and it is determined that there is a check when more than half of the unit widths within the boundary region contain points within the stroke, it is effective.

[0053] Figure 7B shows an example of the determination procedure using unit widths. It shows the case where the boundary regions of wiring 1 and wiring 2 are each divided into six unit widths in the horizontal direction. For example, if the determination threshold Th_check is 0.5. For wiring 1, N / Nunit = 5 / 6 ≈ 0.83 ≥ Th_check = 0.5, which satisfies the determination formula, so it can be determined that the drawing element has been checked. For wiring 2, N / Nunit = 1 / 6 ≈ 0.17 < Th_check = 0.5, which does not satisfy the determination formula, so it can be determined that the drawing element has not been checked.

[0054] When the determination formula is satisfied, information on the check time can be attached based on the time information when the stroke was generated. For example, as shown in Figure 7B, if the stroke used for checking wiring 1 was generated at time t = t1, it can be regarded as checking wiring 1 at time t1, so information on the check time t = t1 can be attached.

[0055] Regarding the check time, when multiple strokes (for example, two handwritings generated at t = t2 and t = t5) are painted on the area of one drawing element, it is better to set the stroke generation time with more points within the boundary region as the check time. This is because the stroke with more points within the boundary region is more likely to be handwritten for checking the drawing element.

[0056] Regarding the judgment method, in the case of wiring including group information consisting of multiple horizontal and vertical lines such as L-shaped wiring or U-shaped wiring, the ratio of the sums of N and Nunit of the constituent lines (N_total / Nunit_total) may be taken, and a check judgment may be performed on the entire wiring including the group information (i.e., the L-shaped wiring or U-shaped wiring as it is).

[0057] 8A and 8B are diagrams for explaining the advantages obtained by increasing the number of divisions by unit width. The greater the number of divisions by unit width, the more accurate the judgment can be. FIG. 8A is a diagram showing an example in which an erroneous judgment is prevented by increasing the number of divisions when a stroke is locally included as noise from the outside due to handwriting deviation in a boundary area divided by unit width. FIG. 8B is a diagram showing an example in which N / Nunit used for check judgment can be expressed more accurately when a stroke is locally significantly deviated from the divided boundary area.

[0058] As shown in Fig. 8A, when a stroke is unintentionally included from outside the boundary area (for example, a single dot made by simply placing the tip of an electronic pen, or handwritten notes that overlap a portion of a drawing element), if the judgment threshold is set to 0.5, the judgment formula is satisfied when the number of divisions in the upper part of Fig. 8A is small, and the stroke is deemed to be checked. In contrast, when the number of divisions is increased as shown in the lower part of Fig. 8A, the proportion of the division boundary area containing noise relative to the number of divisions becomes small, and the stroke is correctly determined to be unchecked. Increasing the number of divisions in this way enables a check judgment that is robust against noise.

[0059] As shown in Fig. 8B, when a stroke deviates significantly from the inside of the boundary region in only a part, if the number of divisions in the upper part of Fig. 8B is small, the handwritten points are included in the unit width divided into three, and N / Nunit=3 / 3=100%. However, N / Nunit=100% does not express a state in which the handwriting is locally significantly off, and it cannot be said to accurately express the result of matching with the actual handwriting. On the other hand, if the number of divisions in the lower part of Fig. 8B is increased, N / Nunit=5 / 6=83%, making it possible to express a state in which a part is significantly off.

[0060] By increasing the number of divisions in this way and accurately representing the state of the strokes used for checking, it becomes possible to extract the characteristics of the handwriting person and optimize the judgment threshold for each worker. By optimizing for each worker, it becomes possible to more accurately reflect each worker's intention in checking drawing elements in the final work record.

[0061] Next, how to determine the length of the unit width will be explained. As mentioned above, the reason for setting the unit width is to allow a check to be made when the boundary area set for the drawing element is divided into unit widths, and to make it possible to determine that a check is not made when the stroke is not hand-painted so as to overlap the area of ​​the drawing element or when there is unintentional noise. Therefore, it is effective to set the unit width to a length that can divide the boundary area into two or more.

[0062] The unit width for dividing a drawing element into two or more regions may be determined as half the minimum length of all drawing elements contained in the drawing. This is because if the unit width is set to half the minimum width of all drawing elements contained in the drawing, all drawing elements can be divided into at least two regions. The minimum width is detected from the coordinates of the drawing element data contained in the drawing element data 111. The minimum width may be different in the x and y directions, and may be detected separately for each wiring, drawing symbol, or character and used as the unit width for each.

[0063] When the unit width is set to the minimum width, in order for each divided area to contain at least one stroke point cloud, it is necessary to follow a conditional formula calculated based on the worker's stroke speed and the sampling frequency of the WD. If the worker's maximum stroke speed is Vmax and the sampling frequency of the WD is fs, the conditional formula is unit width δunit ≧ Vmax / fs. As a numerical example, the JIS standard for working drawings (JIS Z 8313-5) specifies that the minimum character size is 1.8 mm (height direction). Based on this value, the unit width δunit required to divide a drawing element into at least two or more parts can be calculated to be 0.9 mm. The sampling frequency of the WD that satisfies δunit = 0.9 mm is fs ≧ 344.1, using the measured Vmax of 0.31 mm / msec. In other words, it can be calculated that a WD with a sampling performance of approximately 344 Hz or more is required.

[0064] FIG. 7 shows an example of setting the tolerance width and unit width of the boundary area for wiring parallel to the horizontal direction (x direction). Next, a method of setting the tolerance width and unit width for wiring that is inclined at an angle will be described.

[0065] 9A to 9C are diagrams showing examples of methods for setting the allowable width and unit width for an obliquely inclined wiring. Fig. 9A shows an example of setting the allowable width and unit width after rotating an obliquely inclined wiring so that it is horizontal with respect to the x-axis. Fig. 9B shows an example of setting the allowable width and unit width without rotating the wiring in order to reduce the amount of calculation. Fig. 9C shows an example in which the pen width component included in the allowable width changes depending on the inclination.

[0066] When the wiring is not horizontal or vertical but is inclined diagonally as in Figure 9A, setting the tolerance width and unit width in the diagonal direction in the Cartesian coordinate system becomes complicated. Therefore, after rotating the coordinate system so that it is horizontal with the x-axis, the unit width and tolerance width are set in the same way as for the horizontal wiring described above. The angle φ between the diagonal wiring and the x-axis is calculated by φ=tan -1The diagonal wiring is rotated so that it is horizontal to the x-axis by the rotation matrix shown in Fig. 9A. The point cloud coordinates in the stroke included in the handwritten data 121 are also rotated by the same angle by the same rotation matrix, and the position coordinates between the stroke and the wiring are compared.

[0067] In Fig. 9A, by rotating the coordinate system, it is possible to make the same judgment as the tolerance width and unit width described above, but the coordinate system needs to be rotated each time each drawing element is compared, which may result in a large amount of calculation. Therefore, as shown in Fig. 9B, a method is used in which the tolerance width and unit width are appropriately set between each coordinate without rotating the coordinate system, which has the advantage of reducing the amount of calculation.

[0068] Fig. 9B shows a method for setting the boundary area in a stepped manner by appropriately setting the tolerance width and unit width between each coordinate without rotating the coordinate system. For diagonal wiring that is long in the x direction, the tolerance width is set in the y direction and the unit width in the x direction, and for diagonal wiring that is long in the y direction, the tolerance width is set in the x direction and the unit width in the y direction.

[0069] Figure 9B shows an example of dividing the boundary area into three by unit width for a diagonal wiring long in the x direction, and providing the boundary area in a stepped shape. The x and y coordinates of the start point and end point are (xs, ys) and (xe, ye), respectively, and the x and y coordinates of the position that divides the boundary area included between the start point and end point are (xe1, ye1) and (xe2, ye2), respectively. The allowable width of each area is calculated using the pen width w and the y coordinates of the start point and end point of each area. For example, for a diagonal wiring long in the x direction, δtole=w+|ye1-ys|. For a diagonal wiring long in the y direction, δtole=w+|xe1-xs|.

[0070] In the case of diagonal wiring, the worker draws a line diagonally along the extension direction, so the closer the angle between the x-axis and the wiring is to 45°, the more accurate the allowable width will be if the pen width component of the allowable width is made smaller. Therefore, as shown in FIG. 9C, instead of w, w_tilt, whose magnitude changes depending on the tilt, may be used as the pen width component included in the allowable width. w_tilt is expressed as w_tilt=w×(45°-tan -1It can be calculated as (|ye-ys| / |xe-xs|)) / 45°. w_tilt becomes smaller as the angle between the wiring and the x-axis approaches 45°, and becomes larger as the angle moves away from the wiring.

[0071] As shown in the upper right of FIG. 9C, when the angle between the wiring and the x-axis is close to 45°, the portion (|xe-xs| or |ye-ys|) of the size of the boundary area formed by the difference between the coordinates is relatively large. Therefore, since there is no need to further secure a large pen width component w_tilt, w_tilt can be small. This is represented by the fact that the thick dotted rectangle and the thin dotted rectangle in the upper right of FIG. 9C are almost the same size. As shown in the lower right of FIG. 9C, when the angle between the wiring and the x-axis is away from 45°, the portion (|xe-xs| or |ye-ys|) of the size of the boundary area formed by the difference between the coordinates becomes relatively small. Therefore, in order to secure the same level of judgment accuracy as in the upper right of FIG. 9C, it is necessary to secure a large pen width component w_tilt. This is represented by the fact that the size of the thick dotted rectangle and the size of the thin dotted rectangle in the lower right of FIG. 9C are significantly different.

[0072] 10A to 10C show examples of setting an allowable width and a unit width for a drawing symbol. Fig. 10A shows an example where the connection direction with the wiring is the x direction. Fig. 10B shows an example where the connection direction with the wiring is the y direction. Fig. 10C shows an example where the connection direction with the wiring exists in both the x direction and the y direction.

[0073] When checking the symbol area by hand on the WD, a line is often drawn on the relevant area along the direction in which the symbol and the wiring are connected, so the allowable width and unit width are set using the information on the connection direction with the wiring in the symbol data 111-2. The connection direction described in the symbol data 111-2 corresponds to the "direction data" of the symbol. The same applies to the character string direction described later.

[0074] Figure 10A shows how to set a drawing symbol representing a circuit breaker. In this case, the wiring is connected in the x direction, and workers tend to draw lines on the circuit breaker area along the x direction. Therefore, the allowable width is set in the y direction and the unit width in the x direction. The allowable width is calculated as δtole=w+Ly, where w is the pen width used for handwriting and Ly is the length of the drawing element in the y direction.

[0075] Figure 10B shows how to set the drawing symbol for a twist switch. In this case, the wiring is connected in the y direction, and the worker tends to draw a line parallel to the y direction on the twist switch area. Therefore, the allowable width is set in the x direction and the unit width in the y direction. The allowable width is set as δtole=w+Lx, where w is the pen width used for handwriting and Lx is the length of the drawing element in the x direction.

[0076] Figure 10C shows an example where a symbol and wiring are connected in the x and y directions. Depending on the type of symbol, it may be connected in both the x and y directions as shown in Figure 10C. In this case, it is appropriate to first set the allowable width in the y direction and the unit width in the x direction and use a judgment formula, and if the judgment formula is not satisfied, set the allowable width in the x direction and the unit width in the y direction and use a judgment formula.

[0077] The check and judgment method according to the present invention uses a boundary area divided into an allowable width and a unit width, and therefore can handle other check methods in addition to drawing a line on the area of ​​the drawing symbol. For example, even when using a check method such as surrounding the rectangular area of ​​the drawing symbol with a square, filling in the rectangular area, or writing a check mark in the rectangular area, it is possible to judge whether the operator has checked the drawing symbol.

[0078] Next, a method for setting the allowable width and unit width for characters will be explained. In actual work, there are cases where the area of ​​the characters near the checked wiring and drawing symbol is filled with a stroke. In this case, a line is often drawn on the corresponding area along the character string direction. Therefore, by utilizing the character string direction in the character data 111-3, the allowable width and unit width are set in the same way as for the drawing symbol.

[0079] Regarding the matching method, the matching can be speeded up by narrowing down the drawing elements to be matched based on the position coordinates of strokes beforehand and then performing matching. When matching all the point groups included in all strokes with all the drawing elements included in the drawing element data 111 in a round-robin manner, it may take time if there are a large number of strokes and drawing elements. Therefore, when matching the position coordinates of the handwritten data 121 and the drawing element data 111, the drawing elements to be matched may be narrowed down as a preprocessing.

[0080] Fig. 11 is a diagram for explaining a method of narrowing down the drawing elements to be matched. The drawing elements to be matched are shown for the strokes in Fig. 11. As shown in Fig. 11, the area of ​​a circumscribing rectangle indicated by a dashed line is detected from the position coordinates of the stroke, and drawing elements that include at least a part of the area of ​​the circumscribing rectangle are narrowed down as the drawing elements to be matched. This narrows down the drawing elements to be checked, making it possible to speed up the matching process.

[0081] <First embodiment: Check judgment data 131> Next, the check judgment data 131 in the generation of the check judgment data (S505) after the step of comparing the handwritten data 121 with the drawing element data 111 (S504) will be described.

[0082] 12 shows a data table showing a configuration and an example of the check judgment data 131. The check judgment data 131 describes the check judgment result by comparing the handwritten data on the WD with the position coordinates of the drawing elements.

[0083] Each line of the check judgment data 131 includes the type, name, and coordinates (x, y coordinates of the start and end points) of the checked drawing element, the number N of unit widths that contain points in the stroke, the number Nunit of unit widths in the entire boundary area, the ratio (N / Nunit), the judgment result based on the judgment formula (in the example of FIG. 12, the judgment threshold value Th_check=0.5 is set, and if there is a check, it is written as "check" and if there is no check, it is written as "pass"), the check time, the ink number, the stroke number, and the first point number used to check the drawing element in the stroke. The items of the check judgment data 131 may be changed as necessary, and may not include all the items shown in the figure. In addition, other items not shown, such as the name of the checked drawing and the page number, may be included.

[0084] One stroke may pass through multiple drawing elements. For example, assume that one stroke passes through a wiring, a switch, and an IC in that order. In this case, among the sample points that make up the stroke, the one that passes through the wiring first is a sample point that is relatively forward in the stroke, and the one that passes through the IC first is a sample point that is relatively backward. In this way, the sample point that passes through a drawing element first is different for each drawing element. The point number indicates the number of this first sample point. The work order can be identified by sorting each record of the check judgment data 131 using the point number.

[0085] The check judgment data 131 manages the handwriting number, stroke number, and point number for each collated drawing element. As a result, when generating chronologically ordered work record data 112 from the check judgment data 131, the work record generating unit 141 can rearrange the drawing elements in chronological order by assigning a unique order even when multiple drawing elements are checked with one stroke.

[0086] If no points in the stroke are included in the boundary area, no check time information is added from the stroke creation time, so the check time, ink number, stroke number, and point number are left blank or contain "-".

[0087] <First embodiment: Operation of the work record generating unit 141> 13 is a flowchart illustrating the operation of the work record generation unit 141. First, the overall flow will be described, and then each step will be described. When the check judgment data 131 is input by the data collating unit 130, the work record generation unit 141 rearranges the check judgment data 131 in chronological order (S1301), calculates the time required for each task from the check time information (S1302), and generates work record data 112 (S1303). The work record generation unit 141 outputs the generated work record data 112 to the data storage unit 110 (S1304).

[0088] A method for rearranging the judgment results based on the check judgment data 131 in chronological order in S1301 will be described. The check judgment data 131 manages the ink numbers, stroke numbers, and point numbers for each collated drawing element based on the handwritten information superimposed on the drawing element. The judgment results of the checking work can be converted into time series data by sorting the rows of each drawing element in the check judgment data 131 in ascending order of the ink numbers, and further rearranging the stroke numbers within the ink numbers and the point numbers within the stroke numbers in ascending order. Since the time series data of the work indicates the order of work, the check order information of each drawing element can be added to the work record data 112.

[0089] The method of calculating the time required for the check work of the drawing elements in S1302 will be described below. The time required for the check work can be calculated from the difference in check times included in two previous and subsequent check data in the check judgment data 131 rearranged in chronological order. For example, if the check time of drawing element 1 is t1 and the check time of drawing element 2 is t2, then the time required to check drawing element 2 is t2-t1. It is not necessary to calculate the check times of two consecutive check data, and calculations may be made by narrowing down the drawing elements (for example, only wiring and drawing symbols) depending on the purpose of the person viewing the work record.

[0090] 14A and 14B show examples of work record data 112. As shown in Fig. 14A, work record data 112 can be configured as a data table. Fig. 14B shows an example of worker information data 112-1 in which information about workers is managed. Worker information data 112-1 may be managed by a system separate from the work record extraction device 100, or may be stored in the data storage unit 110.

[0091] The work record data 112 stores, as a work record, the results of extracting necessary information based on the check judgment data 131. The work record data 112 centrally manages, as work evidence, the presence or absence of checks on drawing elements, the check order obtained by rearranging in chronological order, and the work time obtained by calculating the time required for the check work.

[0092] Each line of the work record data 112 includes the check sequence, work date and time, type of drawing element, name, work time, coordinates (x and y coordinates of the start and end points), name of the work drawing, whether the drawing element was checked, and worker ID. The items in the work record data 112 may be changed as necessary, and do not have to include all items shown in the figure. Other items not shown may include the page number of the drawing, or items included in the check judgment data 131. For example, by entering the ratio N / Nunit used for the check judgment, it is possible to check to what extent the drawing element has been hand-painted.

[0093] When multiple drawing elements are checked together with one stroke, the work date and time will be the same, as shown by #3 to #5 in the work record data 112. In this case, the work time can be considered to be equivalent, so if the work time is added to the first work (#3) and the work times of the subsequent works (#4, #5) are indicated in parentheses or the like to show that they were checked together with one stroke, it will be easy to tell at a glance.

[0094] The worker ID field in work record data 112 is used when linking records in separately managed worker information data 112-1 by worker ID. Each row of worker information data 112-1 includes the worker ID, the worker's name, years of experience, the equipment ID, and the site where the work is performed. Linking work record data 112 and worker information data 112-1 makes it possible to manage "when," "where," "who," "what," and "how" the work was performed as a work record.

[0095] <First embodiment: Operation of display data generating unit 142> 15A and 15B show examples of display data 113. The display data generation unit 142 acquires work record data 112 generated by the work record generation unit 141 from the data storage unit 110, and generates display data 113 based on the work record data 112. The display data 113 does not necessarily need to visualize all the data in the work record data 112, and information to be visualized is extracted and visualized as necessary.

[0096] FIG. 15A shows an example of a display drawing in which the drawing elements checked by the worker are visualized. FIG. 15B shows an example of a display drawing in which the length of time required for checking is visualized by using shades of color. As shown in FIG. 15A, the drawing elements checked by the worker can be visualized by referring to the judgment result items in the work record data 112. The coordinates used for highlighting when visualizing can be obtained by referring to the coordinates of the start point and the end point in the work record data 112. Examples of highlighting include overlaying a thick straight line on a wiring, surrounding a drawing symbol with a square (the inside may or may not be filled in), highlighting characters, etc. By dividing the colors used for highlighting by type of drawing element, it is possible to easily see whether each drawing element has been checked or not.

[0097] In this way, by visualizing the drawing elements that the worker has checked by changing the color, line width, and transparency to appropriate settings, it becomes clear which parts have been checked, which is useful for managing the progress of work.

[0098] Other items in the work record data 112 (for example, the type, name, and work time of the drawing element) are preferably written in the annotations so that they can be confirmed when the display data 113 is in PDF format, for example.

[0099] If you want to clarify the incomplete parts, you can highlight the unchecked symbols, preferably with a different color, line width, and transparency to the checked ones to avoid information overload.

[0100] As shown in FIG. 15B, by referencing the work record items in work record data 112 and visualizing the work time using color gradations, the drawing elements that took a long time to check can be seen at a glance, and time-consuming or important tasks can be extracted, which can be used to improve work and extract know-how.

[0101] 15A and 15B, the strokes used to check the drawing elements are not visualized so as to avoid excessive information in the drawing for display, but if necessary, the strokes may also be visualized according to the point cloud coordinates in the handwritten data 121. In this case, it is advisable to visualize the strokes with appropriate settings for line thickness, color, and transparency to avoid excessive information.

[0102] <Embodiment 2> In the second embodiment of the present invention, an example configuration will be described in which notes are output to work record data 112 and display data 113 including the notes is generated. A note refers to handwritten notes (ink data) for a purpose other than checking drawing elements. Examples of notes include the following: handwritten notes when writing measured voltage values ​​or resistance values ​​in a drawing after checking continuity or circuit operation; handwritten notes to indicate corrections when there is an error in the drawing. In order to more faithfully digitize the actual work results as a work record, it is necessary to extract notes as data and manage them as work records.

[0103] 16 is a configuration diagram of a work record extraction device 100 in the present embodiment 2. Compared to the embodiment 1, the data matching unit 130 has note judgment data 132 in addition to check judgment data 131, and the operations of the data matching unit 130 and the work record generation unit 141 are different. Explanations of blocks that share processing with the embodiment 1 will be omitted, and only the internal processing of the data matching unit 130 and the work record generation unit 141 will be explained.

[0104] The note determination data 132 refers to a collection of data including the time information when the note was created and the note determination result, which is used to add and store the result of determining whether or not a note is a note as an item in the handwritten data 121.

[0105] 17 is a flowchart for explaining the operation of the data collating unit 130 in the second embodiment. The difference from the first embodiment is that after the handwriting-drawing element collation step (S504), a note extraction step (S1501) is added, the check judgment data generation step (S505) is replaced with a check judgment data and note judgment data generation step (S1502), and the check judgment data output (S506) is replaced with a check judgment data and note judgment data output (S1503). The same reference numerals are used to designate parts common to the first embodiment, and explanations are omitted, and only parts where the operation is different will be explained below.

[0106] The note extraction step (S1501) is executed after the handwritten data and the position coordinates of the drawing element are collated (S504). Details will be described later, but in the note extraction (S1501), whether or not it is a note is determined from the ratio of the number of points contained in the stroke to the number of points not used to check the drawing element. Check determination data 131 is generated as a result of whether or not the drawing element is checked, and at the same time, note determination data 132 is generated as a result of the note determination (S1502), and the check determination data and note determination data are output to the work record generation unit 141 of the data generation unit 140 (S1503).

[0107] 18A to 18C are diagrams for explaining the determination method for extracting notes in S1501. Fig. 18A shows an example of the note determination method. Fig. 18B shows a specific example of note determination. Fig. 18C shows the advantage of performing note determination for each ink data.

[0108] In Fig. 18A, points in strokes that are included in a boundary area set for a drawing element are represented by white circles, and points in strokes that are not included in the boundary area are represented by black circles. Since notes are strokes that are not used in checking drawing elements, whether or not a stroke is a note is determined based on the ratio of black circles (points outside the boundary area) to the total points included in the stroke. The note determination formula is expressed as n_out / n_stroke≧Th_note, where n_stroke is the number of points in one stroke, n_out is the number of points outside the divided areas, and Th_note is the determination threshold. If the formula is satisfied, the stroke is determined to be a note, and if not, it is determined to not be a note.

[0109] The larger the determination threshold Th_note, the less likely the stroke is to be determined as a note, and the smaller the determination threshold Th_note, the more likely it is to be determined as a note. The determination threshold Th_note can be set anywhere between 0 and 1, but in an actual usage environment, if more than half of the points in a stroke are outside the boundary area, there is a high possibility that the stroke has been left as a note, so it is effective to set the minimum value of Th_note to 0.5.

[0110] Figure 18B is an example of a working drawing in which a circuit breaker of a drawing symbol is connected between two wirings, and the character "CBE2" is described on the circuit breaker. The area surrounded by the dashed line represents the boundary area corresponding to each drawing element. In this example, the drawing contains two types of strokes: diagonal strokes and horizontal strokes. In the case of the diagonal strokes in Figure 18B, for example, when the determination threshold Th_note is set to 0.5, the number of points n_stroke included in the stroke is 9, and the number of points n_out outside the boundary area is 6. Therefore, the determination formula n_out / n_stroke = 6 / 9 ≈ 0.67 ≥ Th_note = 0.5 is satisfied, and it can be determined that it is a note. In the case of the horizontal strokes in Figure 18B, the number of points n_stroke included in the stroke is 10, and the number of points n_out outside the boundary area is 0. Therefore, the determination formula n_out / n_stroke = 0 / 10 = 0.0 < Th_note = 0.5, and it is determined that it is not a note but a handwritten check of a drawing element.

[0111] When multiple strokes are managed as one ink data, it is advisable to perform a note determination by taking the ratio (n_out_total / n_stroke_total) of the sum of n_out and n_stroke for all the strokes included in the ink data. This makes it easier to determine that it is a note even if the stroke is close to a drawing element.

[0112] As shown in Figure 18C, assume that an operator handwrites six strokes as a note and they are managed as one ink. When performing a note determination for individual strokes, for the stroke of the circle at the bottom of Figure 18C, since it is close to a drawing element, the determination formula becomes n_out / n_stroke = 2 / 12 ≈ 0.17, and it is determined that it is not a note. On the other hand, when performing a determination by grouping them as one ink data, n_out_total / n_stroke_total = 29 / 42 ≈ 0.69, and it is possible to determine that the stroke close to the drawing element is also a note.

[0113] 19 shows an example of the note determination data 132. The note determination data 132 can be configured as a data table describing the note determination results. In addition to each item included in the handwritten data 121, the note determination data 132 includes the number of points outside the boundary area n_out, the number of points included in the stroke n_stroke, the ratio (n_out / n_stroke), and the determination result (in the example of FIG. 17, the determination threshold value Th_note=0.5 is set, and if the note determination is satisfied, it is described as "note", and if the note determination is not satisfied, it is described as "for checking").

[0114] When multiple strokes are managed as one ink data, as shown in No. 3 of the annotation determination data 132, the items n_out, n_stroke, and ratio may store numerical values ​​for each stroke, or may store the total value of the ink data. In Figure 19, the total value for each ink data is shown in parentheses.

[0115] 20 is a flowchart for explaining the operation of the work record generating unit 141 in the second embodiment. The difference from the first embodiment is that a data synthesis step (S2001) is added before the chronological sorting step (S1301). Steps in common with the first embodiment are given the same reference numerals and explanations are omitted, and only the parts of the operation that are different will be explained below.

[0116] In a data synthesis step (S2001), before proceeding to a chronological sorting step (S1301), the check judgment data 131 and the note judgment data 132 are synthesized. In the chronological sorting step (S1301), the records are sorted in chronological order based on the synthesized data of the check judgment data 131 and the note judgment data 132. By synthesizing the check judgment data 131 and the note judgment data 132, it becomes possible to manage the check results and notes of the drawing elements as a single chronological work record data 112.

[0117] As an example of a synthesis method, necessary information is extracted from the note judgment data 132 and synthesized to match the items in the check judgment data 131. As an example of matching items in the note judgment data 132 with items included in the check judgment data 131, "note" is stored in the drawing element column, a "unique note name (for example, a distinguishable name such as note 1 and note 2)" is stored in the name column, "point cloud coordinates included in the ink data" is stored in the coordinate column, "n_out value" is stored in the N column, "n_total value" is stored in the Nunit column, "n_out / n_total ratio" is stored in the ratio column, "note judgment result" is stored in the judgment result column, and "generation time" is stored in the check time column. Other items that do not correspond (for example, ink No., stroke No., point No.) are stored as blanks or "-".

[0118] FIG. 21 shows an example of work record data 112 in the second embodiment. The difference from the first embodiment is that note information has been newly added. In FIG. 21, #6 and #9 correspond to note data. According to the work record data 112 in FIG. 21, in addition to checking the drawing elements, the timing when the note was generated can be grasped as a work record. In addition, when the display data generating unit 142 generates the display data 113, the notes written in the work record data 112 in addition to the drawing elements are visualized, and the strokes on the drawing elements are not visualized, so that only the information necessary for the user can be displayed, and the display data 113 can be displayed with high visibility.

[0119] According to the second embodiment, by adding a note extraction step (S1501) to the data collating unit 130, note information entered by an operator can be digitized as a work record. For example, the timing and contents of when an operator entered measured voltage values ​​and resistance values, correction contents in the case of an error in a drawing, etc. can be grasped, and visualization that reflects the actual work in detail can be realized.

[0120] <Embodiment 3> In the third embodiment of the present invention, a configuration will be described in which the determination accuracy of note extraction in the second embodiment is improved to convert notes into data with higher accuracy. In the third embodiment, the pen attributes (color, width, transparency, pen type) in the handwritten data 121 are utilized, and the pen attributes when writing notes are determined in advance as work rules, making it easy to separate notes. The pen attributes referred to here are virtual input devices on an application used to input handwritten data on a work terminal. For example, when a worker touches the screen of the work terminal, the pen on the application moves on the screen along the touch position. This pen can be used in the same way as a pen on an application, so it is referred to as a virtual pen device here.

[0121] The third embodiment is different from the second embodiment in the operation of the data matching unit 130. Description of blocks that share the same processing as the second embodiment will be omitted, and only the internal processing of the data matching unit 130 will be described.

[0122] 22 is a flowchart for explaining the operation of the data matching unit 130 in the third embodiment. The difference from the second embodiment is that a pen attribute extraction step (S2201) and a branch based on the annotation determination rule of the pen attribute (S2202) are added. Steps in common with the second embodiment are given the same reference numerals and their explanations are omitted, and only the parts of the operation that are different will be explained below.

[0123] The data matching unit 130 extracts pen attributes from the handwritten data 121 acquired in the data collection step (S501) (S2201). For strokes that do not fit the predetermined annotation rules, the data matching unit 130 proceeds to a handwritten-drawing element matching step (S504). For strokes that fit the annotation determination rules, they can be regarded as annotations, so the process proceeds to S1502, where annotation determination data 132 for the strokes is generated.

[0124] The annotation rules specify the pen attributes used when inputting annotations, so if any of these rules are met, the annotation is deemed to be present without implementing steps S502 to S1501. Handwriting that does not meet the annotation rules may have been used to trace the drawing, so it is judged in the same way as in the first and second embodiments by implementing steps S502 to S1501.

[0125] As an example of the annotation rule, drawing attributes of the image drawn by the pen can be used, such as the pen color when handwriting is black, the thickness is 2pt, the transparency is 0%, the pen type is a normal pen, or a combination thereof. Also, a rule of the pen attributes corresponding to the handwriting-drawing matching (S504) can be determined. In this case, as a rule, for example, the pen color when handwriting is red, the thickness is 6pt, the transparency is 60%, the pen type is a marker, or a combination thereof can be used.

[0126] According to the third embodiment, by adding pen attribute extraction (S2201) to the data collation unit 130 and separating notes according to predetermined pen attribute determination rules, even handwritten notes close to drawing elements can be easily separated as notes. In addition, by combining this with note extraction (S1501) that determines based on the number of points in a stroke, it becomes possible to determine whether a note is a note even if the worker writes a note with a pen attribute that does not fit the note rules, so that more accurate note extraction is possible.

[0127] <Modifications of the present invention> The present invention is not limited to the above-described embodiment, and includes various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0128] 2, only one first terminal 10 is shown, but data may be transmitted and received between several hundred first terminals 10 and one data communication unit 101. In FIG. 2, the data storage unit 110 is shown as being included in the work record extraction device 100, but the data storage unit 110 may be located in a different spatial location and communicate with the work record extraction device 100 via a network.

[0129] The information included in handwritten data 121 may be changed as necessary, and may not include all of the items shown in Fig. 3B. Other items not shown may also be included, such as the time handwriting ended, the name of the handwritten drawing, the page number, the stroke number included in each ink data, or the point number included in each stroke. Fig. 3B shows an example in which handwritten data 121 is configured as a data table, but the format does not have to be a table, and the data format may be text data or the like as long as the above-mentioned information is included. The same applies to other data tables.

[0130] The name and page number of the handwritten drawing may be written in the name of the file in which the handwritten data 121 is stored, so that the handwritten data can be distinguished by drawing and page number. The handwritten data 121 may be managed by page included in the drawing, or multiple pages may be managed together by drawing name.

[0131] The information included in the wiring data 111-1, the drawing symbol data 111-2, and the character data 111-3 described in Figures 4A to 4C may be changed as necessary, and may not include all the items shown in the figures. Also, other items not shown in the figures, such as the name of the drawing including each element, and the page number, may be included. [Explanation of symbols]

[0132] 10: Terminal 1 100: Work record extraction device 101: Data Communications Department 102: Display control unit 110: Data storage unit 111: Drawing element data 111-1: Wiring data 111-2: Drawing symbol data 111-3: Character data 111-A: Wiring display 111-B: Drawing symbol display 111-C: Character display 112: Work record data 112-1: Worker information data 113: Display data 120: Handwritten data acquisition unit 121: Handwritten data 122: Handwritten display 130: Data matching unit 131: Check judgment data 132: Note judgment data 140: Data generation unit 141: Work record generation unit 142: Display data generation unit 200: Second terminal

Claims

1. a data acquisition unit for acquiring handwritten data describing a path taken by a worker through drawing elements on the drawing data by manually inputting the path on the work terminal; a data collation unit that identifies the drawing element through which the handwritten data has passed by collating the handwritten data with the drawing data; Equipped with the data collation unit sets a boundary area around the drawing element, the boundary area including at least a part of the drawing element; the data collation unit determines whether the handwritten data has passed through at least a portion of the boundary area, and if so, determines that the handwritten data has passed through the drawing element; The data collating unit outputs a result of the determination, the data collation unit acquires direction data representing a direction in which the drawing element extends or a direction in which the drawing element connects to another drawing element; The data matching unit divides the boundary area into unit widths along a first direction designated by the direction data, the data collating unit determines, for each divided unit width, whether or not the handwritten data passes through an area divided into the unit width; The data collation unit determines that the handwritten data passes through the drawing element when the number of the unit widths in the boundary area through which the handwritten data passes satisfies a threshold value with respect to a total number of the unit widths in the boundary area. A work record extraction device comprising:

2. When the drawing element is a straight line, the data matching unit obtains a direction in which the straight line extends as the first direction; When the drawing element is a character string, the data collating unit acquires a direction in which the character string is arranged as the first direction.

2. The work record extraction device according to claim 1.

3. The size of the unit width is half the minimum width of all the drawing elements included in the drawing data.

2. The work record extraction device according to claim 1.

4. The boundary region has an allowable width in a second direction perpendicular to the first direction, the tolerance has a size of the drawing element in the second direction plus a spot size of an input device used by the worker to input the handwritten data on the work terminal, The data comparison unit determines that the handwritten data passes through the boundary area when a path through which the handwritten data has passed is included within the allowable width.

2. The work record extraction device according to claim 1.

5. If the drawing element extends in a diagonal direction with respect to the first direction in the drawing data, The data collating unit sets the boundary area around the drawing element by continuously arranging one or more rectangles having the unit width and the allowable width along the drawing element.

5. The work record extraction device according to claim 4.

6. When the drawing element extends in a diagonal direction with respect to the first direction in the drawing data, the data collation unit the larger the angle between the extension direction of the drawing element and the 45° direction on the coordinate axis in the drawing data, the larger the allowable width becomes; the minimum allowable width is the size of the drawing element in the second direction; The maximum allowable width is the size of the drawing element in the second direction plus the spot size. Set the tolerance so that 5. The work record extraction device according to claim 4.

7. If the drawing element is connected to other drawing elements both horizontally and vertically within the drawing data, When the threshold value is not satisfied when the determination is performed with either the horizontal direction or the vertical direction set as the first direction, the data collating unit further sets the other of the horizontal direction or the vertical direction as the first direction and performs the determination.

2. The work record extraction device according to claim 1.

8. The data matching unit calculates a circumscribing rectangle for the handwritten data; The data collation unit determines whether the handwritten data passes through the boundary area only for those drawing elements at least partially included within the circumscribing rectangle.

2. The work record extraction device according to claim 1.

9. the data collation unit performs a note identification process to classify a path traversed by the handwritten data into a path passing through the drawing element and a path on which a note is handwritten; The data collating unit classifies the handwritten data as the note when the number of sample points in the handwritten data that are included within the boundary area set around the drawing element does not satisfy a threshold value with respect to the total number of sample points.

2. The work record extraction device according to claim 1.

10. When a plurality of pieces of handwritten data are generated within a predetermined time period, The data collating unit classifies the handwritten data as the note when the number of sample points of each of the handwritten data generated within the predetermined time that are included within the boundary area set around the drawing element does not satisfy a threshold value with respect to the total number of sample points of each of the handwritten data generated within the predetermined time.

10. The work record extraction device according to claim 9.

11. The data collation unit acquires drawing attributes of a handwritten image drawn on the work terminal according to the handwritten data, the data collation unit performs a note identification process to classify a path through which the handwritten data has passed into a path through the drawing element and a path on which a note has been handwritten according to whether the drawing attribute satisfies a predetermined rule; The drawing attribute is: a pen type of a virtual pen device used to input the handwritten data; the line color of the handwritten image; the line thickness of the handwritten image; the transmittance of the handwritten image; At least one of the following is true:

2. The work record extraction device according to claim 1.

12. The work record extraction device further includes a work record generation unit that generates a record of the work performed by the worker on the work terminal, The work record generation unit arranges the results of the determination by the data collating unit in chronological order, the work record generation unit calculates a required time for the work represented by the handwritten data according to the arranged time series, and generates work record data describing the result of the calculation; The work record generating unit stores the work record data in a storage device.

2. The work record extraction device according to claim 1.

13. The work record extraction device further includes a display data generation unit that generates display data for displaying the work record described in the work record data on a screen, the display data generation unit generates the display data by visualizing whether the handwritten data has passed through the drawing element using at least one of a color, a transparency, and a line thickness corresponding to a required time for each task described in the work record data; The display data generating unit stores the generated display data in a storage device.

13. The work record extraction device according to claim 12.

14. The work record extraction device according to claim 1, The work terminal, having The data acquisition unit acquires the handwritten data from the work terminal. A work record extraction system comprising:

Images