Line extraction method

Abstract

To improve extraction accuracy in extracting a line from an image.SOLUTION: A filter at a first time point with a pixel representing a point as a center pixel is set; at a filter at an i-th time point, a filter at the next time point with a neighborhood pixel being a pixel other than the center pixels of filters before the i-th time point and representing a point as a center pixel is set; and a line obtained by connecting the center pixels of filters after the first time point is converted into numbers. When two or more neighborhood pixels are present in the filter at a j-th time point, a filter at the next time point with the two or more neighborhood pixels as the center pixels is set; a first line obtained by connecting the center pixels of filters before the j-th time point and a second line obtained by connecting the center pixels of filters after the next time point of the j-th time point are compared with each other in terms of a pattern, thickness, light and shade, or color, and the second line to be connected to the first line is determined.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a line extraction method.

Background Art

[0002] Patent Document 1 discloses a data reading device that moves a window sensor having a grid on a graph input as an image, extracts intersections between the grid of the window sensor and the graph, and recognizes the graph from the extracted intersections.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, due to the influence of line overlap or noise such as comments, the recognition accuracy when recognizing a graph may decrease.

[0005] An object of the present disclosure is to improve the extraction accuracy when extracting a line from an image.

Means for Solving the Problems

[0006] The line extraction method according to the present disclosure is a line extraction method for extracting at least one line from an image including one or more lines, selecting a pixel representing a point on any of the one or more lines from among a plurality of pixels included in the image; setting, in the image, a certain range centered on the selected pixel as a filter at a first time point; When each time point after the first time point is defined as the i-th time point, in the filter at the i-th time point, if there is a neighboring pixel that is a pixel representing any point on the one or more lines, other than the central pixel of the filter before the i-th time point, a certain range centered on the neighboring pixel is set as the filter for the next time point. Digitizing the line obtained by connecting the central pixels of the filters after the first time point. including The setting includes, in the filter at the j-th time point, which is any time point after the first time point, when there are two or more neighboring pixels, setting a certain range centered on each of the two or more neighboring pixels as the filter for the next time point, and comparing the pattern, thickness, shading, or color of the first line obtained by connecting the central pixels of the filters before the j-th time point with the second line obtained by connecting the central pixels of the filters after the next time point for each of the two or more neighboring pixels, and determining the second line to be connected to the first line.

Advantages of the Invention

[0007] According to the present disclosure, the extraction accuracy when extracting lines from an image is improved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0009] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0010] In each figure, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0011] The outline of this embodiment will be described with reference to FIGS. 1 and 2.

[0012] In step S1, a document such as a PDF file is input into the image processing apparatus 20. "PDF" is an abbreviation for Portable Document Format. The image processing apparatus 20 is, for example, a mobile device such as a mobile phone, smartphone, or tablet, a PC, or a dedicated device. "PC" is an abbreviation for personal computer.

[0013] In step S2, the area in the document input in step S1 is specified.

[0014] In step S3, a line that the user wants to extract from graph 11 included in the area specified in step S2 is selected. In the example shown in FIG. 1, graph 11 is a waveform graph including two waveforms. In this example, the waveform that the user wants to read is specified.

[0015] In step S4, the image processing apparatus 20 digitizes the line selected in step S3. In the example shown in FIG. 1, the image processing apparatus 20 digitizes the waveform specified in step S3.

[0016] Referring to FIG. 2, the configuration of the image processing apparatus 20 will be described.

[0017] The image processing apparatus 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, and an output unit 25.

[0018] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or a GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 21 executes processes related to the operation of the image processing apparatus 20 while controlling each unit of the image processing apparatus 20.

[0019] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 22 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores data used for the operation of the image processing apparatus 20 and data obtained by the operation of the image processing apparatus 20.

[0020] The communication unit 23 includes at least one communication interface. The communication interface is, for example, a LAN interface, an interface corresponding to a mobile communication standard such as LTE, 4G standard, or 5G standard, or an interface corresponding to a short-range wireless communication standard such as Bluetooth (registered trademark). "LAN" is an abbreviation for local area network. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. The communication unit 23 receives data used for the operation of the image processing apparatus 20 and transmits data obtained by the operation of the image processing apparatus 20.

[0021] The input unit 24 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with a display, a camera, LiDAR, or a microphone. "LiDAR" is an abbreviation for light detection and ranging. The input unit 24 receives an operation for inputting data used for the operation of the image processing apparatus 20. Instead of being provided in the image processing apparatus 20, the input unit 24 may be connected to the image processing apparatus 20 as an external input device. As the connection interface, for example, an interface corresponding to a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0022] The output unit 25 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. The output unit 25 outputs data obtained by the operation of the image processing apparatus 20. Instead of being provided in the image processing apparatus 20, the output unit 25 may be connected to the image processing apparatus 20 as an external output device. As the connection interface, for example, an interface corresponding to a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0023] The functions of the image processing apparatus 20 are realized by executing the program according to this embodiment on a processor as the control unit 21. That is, the functions of the image processing apparatus 20 are realized by software. The program causes a computer to execute the operations of the image processing apparatus 20, thereby enabling the computer to function as the image processing apparatus 20. That is, the computer functions as the image processing apparatus 20 by executing the operations of the image processing apparatus 20 according to the program.

[0024] The program can be stored in a non-transitory computer-readable medium. Examples of the non-transitory computer-readable medium include a flash memory, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a ROM. The program can be distributed, for example, by selling, transferring, or lending a portable medium such as an SD card storing the program, a DVD, or a CD-ROM. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program may be stored in the storage of a server and transferred from the server to other computers to distribute the program. The program may be provided as a program product.

[0025] A computer stores, for example, a program stored in a portable medium or a program transferred from a server, once, in a main storage device. Then, the computer reads the program stored in the main storage device with a processor and executes processing according to the read program with the processor. The computer may directly read a program from a portable medium and execute processing according to the program. The computer may sequentially execute processing according to the received program each time a program is transferred from a server to the computer. Processing may be executed by a so-called ASP-type service that realizes functions only by execution instructions and result acquisition without transferring a program from a server to a computer. "ASP" is an abbreviation of application service provider. A program includes information for use in processing by an electronic computer and those conforming to the program. For example, data that is not a direct instruction to a computer but has a property of defining processing of the computer corresponds to "those conforming to the program".

[0026] Some or all functions of the image processing apparatus 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. That is, some or all functions of the image processing apparatus 20 may be realized by hardware.

[0027] Hereinafter, the operation of the image processing apparatus 20 will be described. This operation corresponds to the line extraction method according to the present embodiment.

[0028] In step S1, the control unit 21 of the image processing apparatus 20 receives an input of a document from a user via the communication unit 23 or the input unit 24.

[0029] In step S2, the control unit 21 receives a designation of an area in the document input in step S1 from the user via the communication unit 23 or the input unit 24.

[0030] In step S3, the control unit 21 receives from the user, via the communication unit 23 or the input unit 24, the selection of at least one line in the graph 11 included in the area specified in step S2.

[0031] In step S4, the control unit 21 generates an image 12 as shown in FIG. 3 by pixelating the graph 11. The image 12 includes one or more lines corresponding to the lines in the graph 11. Although the graph 11 is a two-dimensional graph in the present embodiment, it may be replaced with other types of two-dimensional data such as a two-dimensional map, or three-dimensional data such as a three-dimensional CAD or a three-dimensional map. "CAD" is an abbreviation for computer-aided design. The control unit 21 extracts, from the generated image 12, at least one line selected in step S3 as the original line.

[0032] Before step S4, the control unit 21 of the image processing apparatus 20 may receive from the user, via the communication unit 23 or the input unit 24, the specification of the position and direction of the vertical axis and the horizontal axis of the graph 11. Alternatively, the control unit 21 may detect the vertical axis and the horizontal axis from the area specified in step S2 and specify the position and direction of the vertical axis and the horizontal axis. When the directions of the vertical axis and the horizontal axis are inclined, the control unit 21 may rotate the graph 11 so that the directions of the vertical axis and the horizontal axis are in the vertical direction and the horizontal direction, respectively.

[0033] A specific example of the process in step S4 will be described. In the example shown in FIG. 3, waveforms W1 and W2 of the same system color are included in the graph 11, and it is assumed that the waveform W1 is selected in step S3.

[0034] The control unit 21 of the image processing apparatus 20 selects pixels representing any point on one or more lines included in the image 12 from among a plurality of pixels included in the image 12. Specifically, the control unit 21 receives, via the communication unit 23 or the input unit 24, the designation of pixels representing points from the user, and selects the designated pixels. Alternatively, the control unit 21 searches the image 12 downward from the upper-left pixel to detect pixels representing points, and selects the detected pixels. The control unit 21 expands the search range by one pixel to the right until it detects pixels representing points. It may be possible to select between a mode in which the user designates pixels representing points and a mode in which pixels representing points are automatically detected. In the example shown in FIG. 3, it is assumed that the third pixel from the top and the second pixel from the left are selected.

[0035] The control unit 21 of the image processing apparatus 20 sets, in the image 12, a certain range with the selected pixel as the central pixel C1 as the filter F1 at the first time point. When there is a neighboring pixel Ni at each time point after the first time point, which is defined as the i-th time point, the control unit 21 sets a certain range with the neighboring pixel Ni as the central pixel as the filter for the next time point. The neighboring pixel Ni is a pixel representing any point on one or more lines included in the image 12 other than the central pixel of the filter at a time point prior to the i-th time point in the filter Fi at the i-th time point. The filter shape may be any shape, but in the example shown in FIG. 4, it is a square. The filter size may be any size, but in the example shown in FIG. 4, it is a total of 25 pixels with 5 pixels in the vertical direction and 5 pixels in the horizontal direction. In the filter Fi at the i-th time point, pixels other than the central pixel Ci, such as the pixel Pi[2,4] which is the second pixel from the top and the fourth pixel from the left, are pixels to be evaluated. The central pixel Ci may be any number of pixels, but in the example shown in FIG. 4, it is 1 pixel.

[0036] Applying the example shown in FIG. 4 to the example shown in FIG. 3, the control unit 21 of the image processing apparatus 20 executes a process of reading a waveform from the continuity between the central pixel and the neighboring pixels of the filter at each time point. That is, as shown in FIG. 5, in the filter F1 at the first time point, the pixel P1[4,5] which is the fourth from the top and the fifth from the left corresponds to the neighboring pixel N1. Therefore, the control unit 21 sets the neighboring pixel N1 as the central pixel C2 of the filter F2 at the second time point. As shown in FIG. 6, in the filter F2 at the second time point, the pixel P2[3,5] which is the third from the top and the fifth from the left corresponds to the neighboring pixel N2. Therefore, the control unit 21 sets the neighboring pixel N2 as the central pixel C3 of the filter F3 at the third time point. As shown in FIG. 7, in the filter F3 at the third time point, the pixel P3[2,5] which is the second from the top and the fifth from the left corresponds to the neighboring pixel N3. Therefore, the control unit 21 sets the neighboring pixel N3 as the central pixel C4 of the filter F4 at the fourth time point. Similarly, in the filter F4 at the fourth time point, the pixel P4[1,5] which is the first from the top and the fifth from the left corresponds to the neighboring pixel N4. Therefore, the control unit 21 sets the neighboring pixel N4 as the central pixel C5 of the filter F5 at the fifth time point. In the filter F5 at the fifth time point, there is no neighboring pixel N5. Therefore, the process of reading the waveform ends.

[0037] The control unit 21 of the image processing apparatus 20 extracts the original line by digitizing the line obtained by connecting the central pixels of the filters after the first time point. In the example shown in FIG. 3, the control unit 21 extracts the waveform W1 as the original line by digitizing the line obtained by connecting the central pixels C1, C2, C3, C4, C5.

[0038] Unlike the example shown in FIG. 3, in the example shown in FIG. 8, since there are intersections 13 between the waveform that the user wants to read and other waveforms, it is difficult to determine continuity. Therefore, as shown in FIG. 9, the control unit 21 of the image processing apparatus 20 analyzes the waveform pattern and determines continuity based on the obtained analysis result. In FIG. 9, "B" represents black and "W" represents white. For example, a method of sampling two cycles from a continuous pattern and detecting a waveform pattern can be used. FIGS. 10, 11, and 12 show examples of detecting a broken line, a one-dot chain line, and a two-dot chain line, respectively. By evaluating the waveform pattern, it is possible to determine whether the line types are the same.

[0039] Specifically, when there are two or more neighboring pixels in the filter Fj at the j-th time point, which is any time point after the first time point, the control unit 21 of the image processing apparatus 20 sets a certain range centered on each of the two or more neighboring pixels as the filter for the next time point. The control unit 21 compares the pattern between the first line obtained by connecting the central pixels of the filters before the j-th time point and the second line obtained by connecting the central pixels of the filters after the next time point for each of the two or more neighboring pixels, and determines the second line to be connected to the first line.

[0040] In the example shown in FIG. 8, assume that a combination of the solid line L1a and the solid line L2a is selected as the waveform that the user wants to read in step S3. In this case, the control unit 21 uses the filters from the first time point to the j-th time point to detect the solid line L1a as the first line. In the filter Fj at the j-th time point, each of the pixels representing the points of the solid line L2a and the pixels representing the points of the broken line L2b corresponds to a neighboring pixel. Therefore, the control unit 21 sets each of these two neighboring pixels as the central pixel of the filter for the next time point after the j-th time point, and separately executes the process of reading the remaining waveforms. As a result, the control unit 21 detects the solid line L2a and the broken line L2b as the second lines, respectively. Since the pattern of the solid line L1a and the broken line L2b is different, while the pattern of the solid line L1a and the solid line L2a is the same, the control unit 21 determines the solid line L2a as the second line to be connected to the first line.

[0041] As described above, in the present embodiment, the control unit 21 of the image processing apparatus 20 acquires the graph 11 as the image 12. The control unit 21 recognizes the difference in the plot pattern according to the line type and reads the lines in the graph 11. Therefore, according to the present embodiment, the lines in the graph 11 can be read with high precision. That is, the extraction accuracy when extracting lines from the image 12 is improved. For example, considering the continuity of an arbitrary waveform, the detection rate can be improved.

[0042] In the present embodiment, when only an arbitrary line in the space is to be extracted, it is less affected by noises included in the space, such as other lines overlapping the line or comments, and the extraction rate is improved. In the example shown in FIG. 8, assume that the continuity is detected only from the change in the slope of the continuous value. In that case, when looking at the slopes of the solid line L2a and the broken line L2b with respect to the solid line L1a after the intersection point 13, since the slope of the broken line L2b is smaller, it is erroneously determined that the solid line L1a is connected to the broken line L2b. However, according to the present embodiment, by looking at the line type, it can be accurately determined that the solid line L1a is connected to the solid line L2a.

[0043] As a modification of the present embodiment, the continuity may be determined by the thickness, shade, or color of the line. That is, when there are two or more neighboring pixels in the filter Fj at the j-th time point, the control unit 21 of the image processing apparatus 20 may compare the thickness, shade, color, or any combination thereof of the first line and the second line instead of, or together with, the pattern of the first line and the second line, and determine the second line to be connected to the first line.

[0044] As a modification example of this embodiment, continuity may be determined by the thickness of a line. That is, when there are two or more neighboring pixels in the filter Fj at the j-th time point, the control unit 21 of the image processing apparatus 20 may compare the thicknesses of the first line and the second line instead of, or together with, the pattern of the first line and the second line, and determine the second line to be connected to the first line. For example, when it is desired to extract only an arbitrary waveform among waveforms W3, W4, and W5 having different thicknesses as shown in FIG. 13, it becomes less susceptible to the influence of noise and the extraction rate is improved.

[0045] As a modification example of this embodiment, continuity may be determined by the shading of a line. That is, when there are two or more neighboring pixels in the filter Fj at the j-th time point, the control unit 21 of the image processing apparatus 20 may compare the shading of the first line and the second line instead of, or together with, the pattern, thickness, or both of the first line and the second line, and determine the second line to be connected to the first line. For example, when it is desired to extract only an arbitrary waveform among waveforms W6, W7, and W8 having different shading as shown in FIG. 14, it becomes less susceptible to the influence of noise and the extraction rate is improved.

[0046] As a modification example of this embodiment, continuity may be determined by the color of a line. That is, when there are two or more neighboring pixels in the filter Fj at the j-th time point, the control unit 21 of the image processing apparatus 20 may compare the color of the first line and the second line instead of, or together with, the pattern, thickness, shading, or any combination thereof of the first line and the second line, and determine the second line to be connected to the first line. Also in that case, it becomes less susceptible to the influence of noise and the extraction rate is improved.

[0047] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps described in the flowchart in time series according to the description, depending on the processing capabilities of the device that executes each step, or as necessary, they may be executed in parallel or in a different order. In addition, changes can be made without departing from the spirit of the present disclosure.

Explanation of Signs

[0048] 11 Graph 12 Image 13 Intersection point 20 Image processing apparatus 21 Control unit 22 Storage unit 23 Communication unit 24 Input unit 25 Output unit

Claims

【Claim 1】 A line extraction method for extracting at least one line from an image including one or more lines, comprising: receiving from a user a designation of a pixel representing a point on any one of the one or more lines, and selecting the designated pixel, or searching the image from the pixels at the corners of the image until a pixel representing a point on any one of the one or more lines is detected, expanding the search range one pixel at a time, and selecting the detected pixel, thereby selecting, from among a plurality of pixels included in the image, a pixel representing a point on any one of the one or more lines; setting, in the image, a certain range centered on the selected pixel as a filter at a first time point; when each time point after the first time point is defined as the i-th time point, in the filter at the i-th time point, if there is a neighboring pixel that is a pixel representing a point on any one of the one or more lines other than the central pixel of the filter at a time point before the i-th time point, setting a certain range centered on the neighboring pixel as a filter at the next time point; digitizing the line obtained by connecting the central pixels of the filters after the first time point; and the setting includes, in the filter at the j-th time point, which is any time point after the first time point, when there are two or more neighboring pixels, setting a certain range centered on each of the two or more neighboring pixels as a filter at the next time point, and comparing the pattern, thickness, shading, or color of a first line obtained by connecting the central pixels of the filters before the j-th time point with a second line obtained by connecting the central pixels of the filters after the next time point of the j-th time point for each of the two or more neighboring pixels, and determining the second line to be connected to the first line. A line extraction method.

Images