Work support device, work support system, analysis program

Abstract

The present invention provides a technique for determining a conduction path by matching coordinates of hand-written data overlaid on a drawing with coordinates of circuit components even when the drawing is converted into a data form suitable for display by a work terminal. A work assisting apparatus of the present invention detects circuit symbols and conductive lines from drawing data that does not have information inherent to the circuit components, and matches the detection results with results obtained by a worker along a conduction path by hand-writing, thereby determining the circuit components and conductive lines through which the conduction path passes.

Description

Technical Field

[0001] This invention relates to a technology for assisting workers in their on-site operations. Background Technology

[0002] The declining population and aging population are causing increasingly serious labor shortages. In particular, there is a shortage of highly skilled workers on the manufacturing floor, leading to a situation where only specific individuals can perform certain tasks – a problem known as personnel management. When transferring advanced technologies, successors also need corresponding skills, but there is a shortage of such mid-level technical personnel, resulting in many companies struggling to advance technology transfer. To promote technology transfer to younger technical personnel, a structure must be established that makes explicit knowledge such as work sequences and know-how readily available, regardless of skill level.

[0003] Given this situation, work assistance systems that fully utilize wearable devices such as e-paper have gained attention. For example, in tasks like checking the continuity of a circuit by highlighting corresponding areas on a drawing (the so-called "red-marking task"), the operator displays the circuit diagram on a terminal screen and traces the confirmed conductive components along the diagram to record the continuity path. Thus, the continuity path on the circuit diagram is visually displayed as the highlighted path, making the continuity path visible. In this way, by fully utilizing wearable devices, explicit knowledge that was previously impossible to extract from paper-based work can be obtained.

[0004] The handwritten continuity path, recorded by highlighting in red, represents the operator's handwritten data. This handwritten data is merely a cluster of points and therefore not directly linked to the circuit components and wires on the circuit diagram. To achieve the original purpose of highlighting in red—visualizing the continuity path—it is necessary to compare the coordinates of the handwritten data with the coordinates of the circuit components on the circuit diagram to identify which circuit component is conductive.

[0005] Data described in circuit diagrams, such as CAD drawings, sometimes utilizes information inherent in the circuit components (e.g., the type of circuit symbol, identifier, coordinates of the graphic representing the circuit symbol). For data created in this format, identifying the circuit components along the handwritten path is relatively easy when recording the conduction path by hand. This is because it is only necessary to determine the coordinates of the handwritten path and compare them with the coordinates of the graphic representing the circuit component.

[0006] On the other hand, wearable devices used by workers at the work site sometimes lack the processing capability to display circuit diagrams created in this data format. In such cases, the circuit diagram data must be converted into a data format that can be displayed by the terminal. For example, a data format that uses pixels to represent graphics, such as raster data, is equivalent to this. When recording conduction paths by hand on circuit diagrams in this data format, it is not easy to identify the circuit components along the handwritten paths. This is because the coordinate information of the circuit components is lost during the data format conversion, and only pixel information is generated.

[0007] Patent Document 1 described below discloses a technique for structuring (determining the connection relationships between constituent elements) the components on a raster-format drawing. The document, with the subject matter "automatic structuring of raster-format drawings," describes the following technique: "A structuring system 10 for raster-format drawings, which converts raster-format drawings 1 into vector-format drawings and structures them, comprising: a vector conversion function 12 that converts the raster-format drawing into a vector-format drawing; a structuring rule definition DB 15 that specifies rules for structuring structural elements in the vector-format drawing; and a structuring function 18 that, with reference to the structuring rule definition DB, assigns attribute information and connection information to the structural elements in the vector-format drawing, thereby structuring the structural elements" (see abstract).

[0008] Patent Document 2, titled "Simple and accurate creation of a map showing the route to a destination," describes the following technology: "A map containing the desired route is input from a scanner using a map reading unit 1, and the map information is stored using a map image storage unit 2. Coordinates and a trajectory are input using a coordinate input unit 3, a pen, and a tablet, and the input trajectory is extracted using a trajectory extraction unit 4. A consistency detection unit 5 compares the coordinates of the road information with the input trajectory information to detect consistent information or information within the error range. A path extraction unit 6 extracts map information around the detected points, and the extracted path information is stored using an output image storage unit 7. A text input unit 8 appends text information such as place names to the extracted path information, and an output unit 10 outputs the information to a file or printer" (Reference Abstract).

[0009] Existing technical documents

[0010] Patent documents

[0011] Patent Document 1: Japanese Patent Application Publication No. 2018-206250

[0012] Patent Document 2: Japanese Patent Application Publication No. 09-305106 Summary of the Invention

[0013] The problem the invention aims to solve

[0014] In Patent Document 1, the structural elements on the vector drawing are structured after the raster-format drawing is converted into a vector-format drawing. However, this document does not address the issue of determining the correspondence between the conductive paths handwritten by the operator on the work terminal and the circuit components on the drawing. The same applies to Patent Document 2. Furthermore, Patent Document 2 lacks a process for converting the data into a format suitable for display on the work terminal. Therefore, a technique is needed that can accurately determine the correspondence between the conductive paths handwritten in a data format suitable for display on the work terminal and the circuit components on the circuit drawing.

[0015] The present invention was made in view of the above-mentioned problems, and its object is to provide a technique for determining the conduction path by comparing the coordinates of handwritten data overlaid on the drawing with the coordinates of circuit components, even when the circuit drawing is converted into a data format suitable for display on a work terminal.

[0016] Technical means to solve the problem

[0017] The work assistance device of the present invention detects circuit symbols and wires from circuit drawing data that does not have inherent information about circuit components, and matches the detection results with the results obtained by the operator by tracing the conduction path by hand, thereby determining the circuit components and wires through which the conduction path passes.

[0018] The effects of the invention

[0019] According to the work assistance device of the present invention, even when the circuit diagram is converted into a data format suitable for the work terminal, the coordinates of the handwritten data overlaid on the diagram can be compared with the coordinates of the circuit components to determine the conduction path. Other issues, configurations, and effects will be clarified through the following description of the embodiments. Attached Figure Description

[0020] Figure 1A Examples of circuit diagrams described by circuit diagram data.

[0021] Figure 1B This indicates the result obtained by the operator manually inputting the conduction path from the circuit diagram data.

[0022] Figure 2 This is a configuration diagram of the work assistance device 200 according to Embodiment 1.

[0023] Figure 3A The diagram shows the result of the line detection unit 208 detecting the line 300 on the circuit diagram data.

[0024] Figure 3B This diagram shows the results of the circuit symbol detection unit 209 detecting circuit symbols on the circuit drawing data.

[0025] Figure 3C This diagram shows the results of the wire detection unit 210 detecting the wires on the circuit diagram data.

[0026] Figure 4 This is a schematic diagram illustrating the method by which the circuit symbol detection unit 209 detects circuit symbols.

[0027] Figure 5 A flowchart illustrating the operation of the work assistance device 200.

[0028] Figure 6A Examples of circuit diagram data that include information beyond the circuit diagram itself.

[0029] Figure 6B Indicates to Figure 6A An example is provided with a detection area 604 and a non-detection area 605.

[0030] Figure 7 A circuit diagram that represents an example of assigning attribute information to circuit symbols.

[0031] Figure 8 This diagram illustrates the order in which the lines detected by the line detection unit 208 are classified according to each type.

[0032] Figure 9 Examples of circuit symbols enclosed by dashed lines.

[0033] Figure 10 This is a configuration diagram of the work assistance device 200 according to Embodiment 2.

[0034] Figure 11 A flowchart illustrating the operation of the work assistance device 200 in Embodiment 2.

[0035] Figure 12 This is an example of a circuit diagram consisting of multiple lines, with one conductor.

[0036] Figure 13 This is a schematic diagram illustrating an example of the structuring performed by the structuring unit 213.

[0037] Figure 14 Here is an example illustrating the order of structuring implemented by the structuring unit 213.

[0038] Figure 15 This is an example of how wires connected to a single circuit symbol can form multiple input-output relationships.

[0039] Figure 16This represents an example of the structuring process performed by the structuring unit 213, which resulted in the formation of a loop path.

[0040] Figure 17 This illustrates an example of the data stored in DB 211 in the work assistance device 200 of Embodiment 3.

[0041] Figure 18 An example of a control screen provided for the work assistance device 200. Detailed Implementation

[0042] <Implementation Method 1>

[0043] Figure 1A Examples of circuit diagrams described by circuit diagram data. Circuit diagram data is constructed by using geometric information to describe the circuit components and wires. For example, CAD drawing data in vector form (first data form) is equivalent to this. Figure 1A The circuit diagram shown describes the circuit components as follows: wire 101, twisted pair 102, power supply 103, capacitor 104, connection point 105, resistor 106, and IC 107. Attribute information 100 uses strings to express the attributes of the circuit components.

[0044] Figure 1B This demonstrates the results obtained by an operator manually inputting conductive paths from circuit diagram data. The operator displays the circuit diagram data on a work terminal (e.g., a wearable device) and manually inputs the paths of confirmed conductive components. The handwritten data 108 is obtained by tracing the coordinates of this input, representing the conductive path entered by the operator. The handwritten data 108 includes the time of writing, so the time-series data of the operation indicates the order of operations.

[0045] The purpose of handwritten input of the conductive path is to visualize the connection relationship between conductive circuit components on a circuit diagram. Therefore, it is necessary to determine the circuit components tracked by the handwritten data 108. However, when converting the circuit diagram data into a data format suitable for display on a work terminal (e.g., image data, PDF data), the coordinate information of the circuit components sometimes disappears. As a result, it is not easy to compare the coordinates of the handwritten data 108 with the coordinates of the circuit components on the converted circuit diagram data. The work assistance device of Embodiment 1 of the present invention aims to automatically determine the circuit components on the conductive path tracked by the handwritten data 108 by providing this comparison process.

[0046] Figure 2This is a configuration diagram of the work assistance device 200 according to Embodiment 1. The work assistance device 200 is a device that converts circuit drawing data 205 created in a first data form (e.g., vector form) into circuit drawing data 206 in a second data form (e.g., raster form) and provides it to the work terminal 204. The work assistance device 200 and the work terminal 204 form a system to assist the operator.

[0047] The work assistance device 200 includes a drawing parsing unit 201, a handwritten data parsing unit 202, a communication unit 203, and a database (DB) 211. The drawing parsing unit 201 also includes a conversion unit 207, a line detection unit 208, a circuit symbol detection unit 209, and a wire detection unit 210.

[0048] The conversion unit 207 converts the circuit diagram data 205 (first data form) into circuit diagram data 206 (second data form). The line detection unit 208 detects at least one line from either the circuit diagram data 205 or the circuit diagram data 206. The circuit symbol detection unit 209 detects the circuit symbols described in the circuit diagram data 206. The wire detection unit 210 detects the wires described in the circuit diagram data 206. These detection sequences will be described later.

[0049] The communication unit 203 sends the circuit diagram data 206 to the work terminal 204. The work terminal 204 sends handwritten data (e.g., handwritten data 108) recording the conduction path input by the operator in the circuit diagram data 206 to the work assistance device 200, and the communication unit 203 receives the handwritten data. The handwritten data analysis unit 202 compares the coordinates of the handwritten data with the coordinates of the circuit components and wires detected by the diagram analysis unit 201, thereby determining the circuit components and wires on the conduction path. The database 211 stores the processing results of the work assistance device 200. The database 211 can be composed of a data storage device.

[0050] Figure 3A The diagram illustrates the result of the line detection unit 208 detecting a line 300 in circuit diagram data. The detected line can be represented by a start point (x0, y0) and an end point (x1, y1). The line detection unit 208 detects the line 300 from circuit diagram data 205 or 206. If the start and end points of the line 300 are defined in the data, the line 300 can be detected according to those definitions. Alternatively, the line 300 can be detected by applying an appropriate line detection algorithm, such as line detection filtering, to the pixel data.

[0051] Figure 3BThis diagram illustrates the results of the circuit symbol detection unit 209 detecting circuit symbols on circuit drawing data. The circuit symbol detection unit 209 detects the regions constituting circuit symbols based on lines detected by the line detection unit 208. For example, template matching or deep learning can be used to detect regions that match the shape of the circuit symbol, as described later. The circuit symbols detected by the circuit symbol detection unit 209 do not include wires. The coordinates of the detected circuit symbols can be represented, for example, by the lower left (starting point) and upper right (ending point) of the region. Figure 3B The example shown illustrates the detection of power supply region 301, capacitor region 302, and resistor region 303. Depending on the detection method, sometimes a portion of a wire connected to a circuit symbol may also be identified as part of the circuit symbol. In this case, the wire can also be considered part of the circuit symbol.

[0052] Figure 3C This diagram illustrates the results of the wire detection unit 210 detecting wires on the circuit diagram data. The wire detection unit 210 detects the portion of the wires detected by the wire detection unit 208 after removing the circuit symbols detected by the circuit symbol detection unit 209 as wires. Therefore, a portion of the wires 300 detected by the wire detection unit 208 is redefined as wires 304 with new start and end points.

[0053] Figure 4 This is a schematic diagram illustrating the method by which the circuit symbol detection unit 209 detects circuit symbols. The circuit symbol detection unit 209 receives a circuit drawing (input image 400) and applies one or more of the following methods to obtain a circuit symbol detection result 406.

[0054] Figure 4 Template matching 401: The circuit symbol detection unit 209 matches the template image 402 with the input image 400 to detect circuit symbols. The template image 402 is an image of a circuit symbol that may be included in the circuit drawing data 205 or 206. Template matching is easy to implement; however, if the image is magnified, reduced, or rotated, the detection accuracy may decrease.

[0055] Figure 4 Object detection model 403: Circuit symbol detection unit 209 detects circuit symbols using a detector learned through deep learning, employing object detection models such as R-CNN (Region with CNN features), YOLO, and SSD (Single Shot MultiBox Detector). This method uses a single detection model; therefore, detection failure may occur if the model is not sufficiently learned.

[0056] Figure 4The circuit symbol detection unit 209 uses a detector learned through deep learning, which includes object detection model 403 and object recognition models 405 such as ResNet, DenseNet, AmoebaNet, and EfficientNet, in addition to object detection model 403, to detect circuit symbols. Specifically, the circuit symbol image 404 detected by object detection model 403 is fed into object recognition model 405. Object recognition model 405 identifies the category of the circuit symbol detected by object detection model 403. By using object recognition model 405, detection failures can be eliminated, thus improving detection accuracy.

[0057] Figure 5 This is a flowchart illustrating the operation of the work assistance device 200. The work assistance device 200 begins this flowchart after the conversion unit 207 converts the circuit diagram data 205 into circuit diagram data 206. The wire detection unit 208 detects wires from the circuit diagram data (S01). The circuit symbol detection unit 209 detects circuit symbols other than wires from the circuit diagram data (S02). The wire detection unit 210 detects wires (S03). The communication unit 203 sends the circuit diagram data 206 to the work terminal 204 (S04). The operator inputs the conduction path on the work terminal 204, and the work terminal 204 sends handwritten data (S05). The communication unit 203 receives the handwritten data (S06). The handwritten data parsing unit 202 parses the handwritten data to determine the circuit components on the conduction path (S07) and stores the result in the database 211 (S08). The handwritten data parsing unit 202 notifies the operator of the parsing result (S09).

[0058] Figure 6A This is an example of circuit drawing data that includes information beyond the circuit diagram. Actual circuit drawing data sometimes includes various other information besides the circuit diagram. In this example, in addition to the circuit diagram, it also includes grid 600, grid reference 601, outline 602, and title block 603. If this information is also detected by the line detection unit 208, non-wire parts may be detected as wires.

[0059] Figure 6B Showing Figure 6A An example is provided with a detection area 604 and a non-detection area 605. By excluding information outside the circuit diagram from the detection target before detecting circuit components and wires from the circuit diagram data in the drawing parsing unit 201, it is possible to avoid misdetecting such unwanted information as part of the circuit diagram. Therefore, the user can specify at least one of the areas where the circuit diagram is to be detected (detection area 604) and the areas where the circuit diagram is not detected (non-detection area 605) via an appropriate interface. For example, each area can be specified using coordinates such as the lower left corner and the upper right corner. Figure 6BThe example shown illustrates designating title block 603 as non-detection area 605 and circuit diagram portion as detection area 604. Drawing analysis unit 201 detects circuit components only from detection area 604, or only from areas other than non-detection area 605.

[0060] For example, if there is redundant information (such as a table describing the characteristics of the circuit symbol) inside the circuit symbol, this redundant information should be excluded from the detection targets of the line detection unit 208. In this case, the non-detection area 605 is useful. Furthermore, if there are redundant scribe lines or the like around the circuit diagram, it is useful to use the detection area 604.

[0061] Figure 7 This is a circuit diagram illustrating an example of assigning attribute information to circuit symbols. Circuit diagrams sometimes include information about circuit components and their attributes, such as part number 700. When detecting circuit symbols, the circuit symbol detection unit 209 can detect this attribute information simultaneously and assign it to the detected circuit symbols. The handwritten data parsing unit 202 can output the parsing result and its attribute information simultaneously. Thus, it is possible to identify the conduction path and its attribute information together.

[0062] For example, if the distance between the center coordinates of the detected circuit symbol and the center coordinates of the text area is within a threshold, the circuit symbol detection unit 209 considers the text as an attribute of the circuit symbol. As a method for extracting the text area, for example, if it is circuit diagram data containing text information, then that text information can be extracted; alternatively, the text can be extracted by applying the same method to pixels as for circuit symbols.

[0063] Figure 8This diagram illustrates the order in which the lines detected by the line detection unit 208 are classified according to each category. The line detection unit 208 reclassifies a line as a point if its length is less than a threshold, and reclassifies it as a line if its length is greater than or equal to the threshold. An example of the threshold for distinguishing between points and lines is, for instance, the level at which dotted lines created by drawing software can be recognized as points. The line detection unit 208 further reclassifies two adjacent lines on the same straight line as dashed lines if the distance between them is less than a threshold, and reclassifies them as solid lines if the distance is greater than or equal to the threshold. The line detection unit 208 then reclassifies the line category according to the number of points present between two lines classified as dashed lines. If there are 0 points, it is considered a dashed line; if there is 1 point, it is considered a single-dash line; if there are 2 points, it is considered a double-dash line. The line detection unit 208 further reclassifies two points on the same straight line as dotted lines if the distance between them is less than a threshold, and reclassifies them as points if the distance is greater than or equal to the threshold. For each of the above scenarios, an appropriate value can be set for each step in each step; there is no need to use the same threshold.

[0064] Figure 9 This shows an example of a circuit symbol enclosed in dashed lines. Circuit diagrams sometimes use dotted or dashed lines to enclose reference information to make it easier for the reader to understand. Figure 9 In the circuit, wire 900 is connected to galvanometer 901, which is surrounded by a dashed line 902. Within the dashed line 902, the model number 903 and the corresponding number 904 are arranged. Text enclosed by lines of a specific type can be used as attribute information for circuit symbols.

[0065] When a circuit symbol and text exist within an area enclosed by dotted or dashed lines, and the text is positioned within a predetermined distance from the circuit symbol, the circuit symbol detection unit 209 considers the text as an attribute of the circuit symbol. In this case, the predetermined distance between the circuit symbol and the text is not necessarily... Figure 7 The distance threshold is the same under the conditions described. The reason is that the text is initially inferred to be an attribute of the circuit symbol by surrounding the circuit symbol and the text.

[0066] <Implementation Method 2>

[0067] Figure 10 This is a structural diagram of the work assistance device 200 according to Embodiment 2 of the present invention. In addition to the configuration described in Embodiment 1, the work assistance device 200 of Embodiment 2 also includes a structuring unit 213. Other configurations are the same as in Embodiment 1. The structuring unit 213 generates data recording the connection relationships between circuit symbols and wires as shown in the circuit diagram data 206. This process is called structuring. The specific order of structuring will be described later.

[0068] Figure 11 This is a flowchart illustrating the operation of the work assistance device 200 in Embodiment 2. Between S03 and S04, the structuring unit 213 performs structuring according to the order described below (S10). Other steps are the same as... Figure 5 The same applies. In S07, the handwritten data parsing unit 202 can use the structured results given by the structuring unit 213 to determine the conduction path of the handwritten data. For example, the connection relationships between circuit components and wires described in the structured results can be compared with the connection relationships shown in the matching results, thereby outputting a list of circuit components and wires through which the conduction path passes.

[0069] Figure 12 This is an example of a circuit diagram consisting of multiple lines, with one conductor. Figure 12 In (1), the wire 1200 connecting the power supply 1201 and the IC 1203 consists of four wires, at least two of which face different directions and are in contact at their ends, thereby forming a conductive path. Furthermore, a wire 1204 is arranged between the connection point 1202 and the IC 1203 in a manner close to the fourth wire of the wire 1200.

[0070] If like Figure 12 If, as in (2), a conductive path is marked by handwriting where other nearby wires are located, the handwritten path may sometimes overlap with other wires. Figure 12 In example (2), the handwritten path is simultaneously traced along the fourth section of wire 1200 and wire 1204. Therefore, it may be impossible to correctly determine the conductive path.

[0071] Therefore, the conductor detection unit 210 integrates the four partial conductors constituting the conductor 1200 into one conductor 1205. Figure 12 (3)). Thus, even if a portion of wire 1205 overlaps with wire 1204 on the handwritten path ( Figure 12 (4)), the handwritten path will only trace along a portion of the wire 1205, so the handwritten data parsing unit 202 can determine that the handwritten path does not trace the wire 1205. For example, the wire detection unit 210 in Figure 11 Such wire integration can be implemented in advance in S03.

[0072] Figure 13 This is a schematic diagram illustrating an example of the structuring performed by the structuring unit 213. Figure 13 In the circuit diagram on the left, the path between power supply 1201 and IC 1203 can be like... Figure 13As described on the right. That is, the path can be described as a path in which circuit symbol 1300 and wire 1301 are connected alternately. The structuring unit 213 uses a data structure in which circuit symbols and wires are connected alternately to express the connection relationship described in the circuit diagram data according to this principle.

[0073] Figure 14 Here is an example illustrating the order of structuring implemented by the structuring unit 213. Specifically, regarding... Figure 14 As shown in (1), when power supply 1400 ⇒ wire 1403 ⇒ capacitor 1401 ⇒ wire 1404 ⇒ resistor 1402 ⇒ wire 1405 are connected in series in this order, data representing this connection relationship is created (that is, data representing the connection relationship is created by connecting the power supply 1400 ⇒ wire 1403 ⇒ capacitor 1401 ⇒ wire 1404 ⇒ resistor 1402 ⇒ wire 1405 in series). Figure 14 (1) The structured order is explained.

[0074] The structuring section 213 lists the circuit elements within the circuit drawing data, consisting of circuit symbols and the wires connected to those circuit symbols. For example, power supply 1400 is connected to wire 1403, so this group is listed as one circuit element. Similarly, capacitor 1401 is connected to wires 1403 and 1404, so this group is also listed as one circuit element. The circuit is constructed in this order. Figure 14 The circuit element list shown in (2) is as follows.

[0075] The structuring section 213 lists circuit elements within the circuit element list that could potentially become the starting point of a connection path. Specifically, it extracts and lists the connection relationships where circuit symbols and wires are connected in a 1:1 ratio from the circuit element list. For example... Figure 14 As shown in (3), at the point in time when structuring begins, only the power supply 1400 and the wire 1403 are connected together in a 1:1 ratio. Therefore, at that point in time, they become the starting elements of the connection relationship.

[0076] The structuring unit 213 compares the starting element with each circuit element in the circuit element list. The structuring unit 213 then identifies circuit elements from the circuit element list that have the same wires as the starting element but do not have the same circuit symbols. At this point in time, the group 1401 / wire 1403 / wire 1404 corresponds to this.

[0077] The structuring unit 213 connects the determined circuit elements and the starting element using common wires. At this point in time, wire 1403 is common, so wire 1403 is used to connect the starting element and capacitor 1401. The connection result is updated to a new starting element. Thus, the starting element becomes... Figure 14The structured part 213 repeats the same process using a new starting element. The structured part 213 stores the content of the starting element at the point in time when the starting element can no longer be updated as the result of the structured process in DB 211.

[0078] Figure 15 This is an example of forming multiple input-output relationships from wires connected to a single circuit symbol. The structuring section 213 is created in this case. Figure 14 When listing the circuit elements as described above, the circuit elements are listed according to each of these input-output relationships. Therefore, even when multiple input-output relationships exist for a single circuit symbol, all connection paths determined by these input-output relationships can be included. For example, the circuit symbol detection unit 209 can pre-store the input-output relationships and the category of each circuit symbol.

[0079] Figure 15 In (1), the twisted pair 1500 has the following input-output relationships: (a) receiving input from wire 1501 and outputting from wire 1503; (b) receiving input from wire 1502 and outputting from wire 1504. The structuring section 213 lists these two input-output relationships as separate circuit elements. Therefore, the twisted pair 1500 is like... Figure 15 As shown in (2), they are listed as two circuit elements.

[0080] Figure 15 In (3), IC 1505 has two input wires 1506 and 1507 and one output wire 1508. In the absence of an input-output relationship between them, the structuring section 213 lists each wire as a separate circuit element for IC 1505. Therefore, as... Figure 15 As shown in (4), IC 1505 is listed as three circuit elements. Alternatively, for example, in the case where wire 1506 is the input, wire 1508 is its output, but wire 1507 does not have a corresponding output, such as... Figure 15 As shown in (5), IC 1505 is listed as two circuit elements.

[0081] Figure 16 An example is shown where a loop path is formed as a result of the structuring performed by structuring unit 213. Figure 16 In the example shown, the path starting from point 1600 loops through loop path 1601. When such a loop path is formed, the same circuit symbols will appear multiple times in the connection relationships obtained through structuring. The structuring unit 213 removes connection paths containing such loop paths from the structuring result. This is because such paths cannot be considered conductive paths.

[0082] <Implementation Method 3>

[0083] In Embodiment 3 of the present invention, a specific example of the data stored in DB 211 and the user interface provided by the work assistance device 200 will be described. Other configurations are the same as in Embodiments 1 to 2.

[0084] Figure 17 An example of the data stored in DB 211 in the work assistance device 200 of this embodiment 3 is shown. In addition to storing the parsing results given by the handwritten data parsing unit 202 and other functional units, DB 211 can also store, for example, (a) a worker table that records information related to the workers, (b) a task table that records information related to the work performed by the workers, etc.

[0085] The operator table can store, for example, the operator's personal ID, name, affiliation, the ID of the work terminal used, and information identifying the work performed (task information). The task information is referenced in the task table. The task table can store, for example, task No., project name, delivery date, manager, person in charge, and progress rate. The progress rate represents the progress rate of the handwritten input task. The handwritten data parsing unit 202 can calculate the progress rate based on, for example, (a) the proportion of completed handwritten input paths relative to all completed paths recorded in the circuit diagram data, and (b) the proportion of completed handwritten input circuit components relative to all circuit components recorded in the circuit diagram data.

[0086] Figure 18 An example of a control screen provided for the work assistance device 200. The control screen can be used, for example, by a manager overseeing the work progress of each worker. The control screen can be provided, for example, by a handwritten data parsing unit 202. The control screen includes a parsing file designation unit 1800, a parsing area designation unit 1801, a terminal operation unit 1805, and a progress display unit 1810.

[0087] In the file selection unit 1800, when a user (e.g., an administrator) clicks the ▲ button, a file selection screen is displayed. The user selects circuit diagram data or a PDF file of the circuit diagram created by drawing software. The conversion unit 207 converts the selected data into a data format suitable for display on the work terminal 204 (circuit diagram data 206).

[0088] The user uses the page designation unit 1802 to specify the page number of the circuit diagram data. The parsing area designation unit 1801 displays the circuit diagram described on the page selected by the user. The user can specify the detection area 1803 and the non-detection area 1804, for example, by dragging the mouse. The storage device (e.g., DB 211) stores the detection area data describing the result.

[0089] In the terminal operation unit 1805, the file selection area 1806 displays a list of circuit diagram data 206. The user specifies any one of them. The terminal selection area 1807 displays a list of work terminals 204. The user specifies the receiving terminal to send the circuit diagram data 206 or the sending source terminal to receive the handwritten data, and clicks the send button 1808 or the receive button 1809. The communication unit 203 sends the circuit diagram data 205 or the circuit diagram data 206 to the specified recipient, or receives the handwritten data from the specified sending source.

[0090] In the progress display unit 1810, the work progress rate is displayed as a numerical value 1811 or a pie chart 1812 according to the content of the handwritten data. The display content change unit 1813 can switch the display content, for example, by each project / each worker.

[0091] In addition to the above, the control screen can also display the processing results given by the drawing analysis unit 201 or the handwritten data analysis unit 202. For example, it can display structured results, matching results between structured results and handwritten data, attribute information of circuit symbols, etc. Furthermore, the same content can be presented to the user in an appropriate data format in a way that replaces the display of this information and the control screen, or is used in conjunction with it.

[0092] <Regarding variations of the present invention>

[0093] This invention includes various modifications and is not limited to the embodiments described above. For example, the above embodiments are detailed descriptions provided to illustrate the invention in an easily understandable manner and are not necessarily limited to all the described configurations. Furthermore, a portion of the configuration of one embodiment may be replaced with the configuration of another embodiment, and the configuration of one embodiment may be added to the configuration of another embodiment. In addition, other configurations may be added, deleted, or replaced to a portion of the configuration of each embodiment.

[0094] In the above embodiments, it is envisioned that the line detection unit 208 detects the straight lines described in the circuit diagram data, but appropriate techniques for detecting curves can also be used to detect curves. The circuit symbol detection unit 209 uses methods such as pattern matching that are not solely based on line detection, so even if the line detection unit 208 only detects straight lines, curves can still be included in the circuit symbol.

[0095] In the above embodiments, the work assistance device 200 sends data to the work terminal 204 in a format suitable for display by the work terminal 204. Either circuit diagram data 205 or circuit diagram data 206 can be sent, as long as it is suitable for display by the work terminal 204. In either case, it is envisioned that the circuit diagram data received by the work terminal 204 will only represent the circuit components as graphics (geometric information of lines or pixels), thus losing the inherent information of the circuit components.

[0096] Depending on the method of notation in circuit diagrams, wires are sometimes also considered a type of circuit symbol. However, it should be noted that in the above implementation, a distinction must be made between circuit symbols and wires; a circuit symbol is the remaining part of a line after removing the wire.

[0097] In the above embodiments, the drawing parsing unit 201 (and the various functional units provided by the drawing parsing unit 201) and the handwritten data parsing unit 202 can be constructed by hardware such as circuit devices that have these functions deployed, or they can be constructed by a computing device such as a CPU (Central Processing Unit) executing software that has these functions deployed.

[0098] In the above embodiments, the drawing parsing unit 201 includes a conversion unit 207, a line detection unit 208, a circuit symbol detection unit 209, and a wire detection unit 210. However, these functional units can also be deployed as separate components.

[0099] In the above embodiments, the drawing parsing unit 201, the handwritten data parsing unit 202, and each functional unit of the drawing parsing unit 201 can all be equipped on the same device, or a portion of them can be implemented on another device. For example, the drawing parsing unit 201 (parsing program) can be configured on another device and implemented separately from the handwritten data parsing unit 202 to detect circuit components and wires from circuit drawing data (which may further include structuring processing).

[0100] Symbol Explanation

[0101] 200…work assistance devices

[0102] 201…Drawing Analysis Department

[0103] 202…Handwritten Data Analysis Department

[0104] 203…Ministry of Communications

[0105] 204…Work Terminal

[0106] 205…Circuit drawing data

[0107] 206…Circuit drawing data.

Claims

1. A work assistance device that provides a work terminal used by a worker with circuit diagram data showing the connection relationships of circuit components, characterized in that, The work assistance device includes a conversion unit that converts first circuit diagram data created in a first data format into second circuit diagram data created in a second data format. The first data form is configured to represent lines using geometric information, and these lines constitute a graphic depicting the circuit components. The second data format is configured to represent a graphic depicting the circuit components in pixels. The work assistance device also includes: The line detection unit detects straight lines as described in the first circuit diagram data or from the image area of ​​the second circuit diagram data; The circuit symbol detection unit detects circuit symbols other than wires from the image area of ​​the second circuit drawing data; The wire detection unit detects the remaining portion of the straight line detected by the wire detection unit after removing the circuit symbol detected by the circuit symbol detection unit as a wire. The communication unit sends the first circuit diagram data or the second circuit diagram data to the operating terminal; as well as The analysis unit analyzes the conduction path data, which records the results of the operator tracing the conduction path on the first or second circuit drawing data by handwriting input on the work terminal. The analysis unit matches the conduction path described in the conduction path data with the circuit symbols detected by the circuit symbol detection unit and the wires detected by the wire detection unit, thereby determining the circuit components and wires through which the conduction path passes. The analysis unit notifies the operator of the results of determining the circuit components and wires through which the conduction path passes.

2. The work assistance device according to claim 1, characterized in that, The line detection unit detects the straight line based on the geometric information described in the first circuit diagram data, or by applying a line detection filter to the second circuit diagram data. The circuit symbol detection unit detects the circuit symbols by using template matching, deep learning of an object detection model, or deep learning of both an object detection model and an object recognition model on the second circuit drawing data.

3. The work assistance device according to claim 1, characterized in that, The work assistance device also includes a storage unit for storing detection area data, which specifies at least one of the detection area and other non-detection areas as the object of processing the detection of the straight line in the second circuit drawing data. The line detection unit detects the straight line from the detection area specified by the detection area data, or does not detect the straight line from the non-detection area specified by the detection area data.

4. The work assistance device according to claim 1, characterized in that, The circuit symbol detection unit detects text positioned within a predetermined distance of the circuit symbol on the second circuit drawing data, and associates this text as attribute information of the circuit symbol. The parsing unit also presents the circuit components through which the conduction path passes and the attribute information associated with each of the circuit symbols.

5. The work assistance device according to claim 1, characterized in that, The line detection unit classifies lines whose length is less than a first threshold as points, and lines whose length is greater than or equal to the first threshold as lines. The line detection unit classifies lines whose interval between two lines is greater than or equal to a second threshold as solid lines, and lines whose interval is less than the second threshold as dashed lines. The line detection unit classifies the dashed lines obtained from the classification into any one of dashed lines, single-dot dashed lines, or double-dot dashed lines according to the number of points present in the interval. When the points obtained from the classification are arranged on the same straight line at intervals within a third threshold, the line detection unit classifies the points arranged on the straight line as dotted lines.

6. The work assistance device according to claim 5, characterized in that, The circuit symbol detection unit detects text on the second circuit drawing data that is positioned within a predetermined distance from the circuit symbol and surrounded by the dotted line, and associates this text as attribute information of the circuit symbol. The parsing unit also presents the circuit components through which the conduction path passes and the attribute information associated with each of the circuit symbols.

7. The work assistance device according to claim 1, characterized in that, The operation assistance device also includes a structuring unit, which structures the circuit symbols and wires on the second circuit drawing data according to their connection relationships. The analysis unit uses the structured results provided by the structuring unit to determine the circuit components and wires through which the conduction path passes.

8. The work assistance device according to claim 1, characterized in that, When the conductor detection unit detects two or more conductors that form a path by contacting each other at their ends, it treats these two or more conductors as a single conductor.

9. The work assistance device according to claim 7, characterized in that, The structuring is implemented by alternating the connection of the circuit symbols with the wires removed with the wires.

10. The work assistance device according to claim 7, characterized in that, The structured section creates a list of circuit elements, each circuit element consisting of a circuit symbol and a wire connected to that circuit symbol. The structuring section creates a list of starting points for circuit elements in which the circuit symbols and wires are connected one-to-one in a list of more than one of the circuit elements. The structuring unit identifies circuit elements listed in the circuit element list that have the same conductors as the circuit elements listed in the starting point list but do not have the same circuit symbols. The structuring unit connects the circuit elements listed in the starting point list with the determined circuit elements using the same wires, thereby updating the circuit elements listed in the starting point list. The structuring unit uses the updated list of starting points to implement the structuring.

11. The work assistance device according to claim 10, characterized in that, In the case where there are multiple input-output relationships in one of the circuit components, formed by at least one of the wires that input signals to the circuit component and the wires that output signals from the circuit component, the structuring section lists the circuit elements in the circuit element list according to each input-output relationship.

12. The work assistance device according to claim 7, characterized in that, If the same circuit component exists in the connection path between the circuit component and the wire formed by the structuring, the structuring unit deletes that connection path from the result of the structuring.

13. The work assistance device according to claim 3, characterized in that, The work assistance device also provides a control screen, on which user instructions to the work assistance device are input. The control screen has the following features: The data designation unit receives the instruction that selects the first circuit drawing data; The detection area designation unit inputs the instruction specifying the detection area and the non-detection area; The terminal designation unit inputs the instruction specifying the operating terminal; The sending instruction unit receives an instruction to send the first circuit diagram data or the second circuit diagram data to the operation terminal; A receiving instruction unit receives an instruction from the work terminal to receive the conduction path data. as well as The progress display unit displays the work progress status of the operator based on the analysis results provided by the analysis unit. The progress display unit displays the work progress status based on the ratio of the completed handwritten input conduction path to all conduction paths recorded in the first circuit drawing data or the second circuit drawing data, or the ratio of the completed handwritten input circuit component to all circuit components recorded in the first circuit drawing data or the second circuit drawing data.

14. A job assistance system, characterized in that, have: The work assistance device according to claim 1; and The operation terminal receives the first circuit diagram data or the second circuit diagram data from the operation assistance device.

15. A parsing program that causes a computer to perform parsing of circuit drawing data representing the connection relationships of circuit components, characterized in that, The circuit diagram data is composed of pixels representing the graphics that depict the circuit components. The parsing program causes the computer to perform the following steps: Detect straight lines from the image region of the circuit diagram data; Detect circuit symbols other than wires from the image area of ​​the circuit drawing data; The remaining portion after removing the circuit symbol detected in the step of detecting the circuit symbol from the straight line detected in the step of detecting the straight line is detected as a wire; The circuit symbols and wires on the circuit drawing data are structured according to their connection relationships. Using the structured results, the circuit components and wires through which the conduction path passes are determined by parsing the conduction path data, which records the results obtained by tracing the conduction path on the circuit drawing data with a line input by handwriting on a work terminal displaying the circuit drawing data; as well as The operators were informed of the results confirming the circuit components and wires through which the conductive path passed. In the structuring step, the computer performs the structuring by alternately connecting the circuit symbols with the wires removed.

Images