METHOD FOR DETECTING DISCONTINUITIES ON THE SURFACE OF A BODY

Description

[0001] The invention discussed below relates to a method for detecting geometric discontinuities on the surface of the lateral surface of a body, which body is illuminated with a light source emitting light rays and which body is recorded by means of a camera outputting image data, which image data comprise color property values ​​at a pixel.

[0002] By definition, a geometric discontinuity represents a deviation in the geometric properties of the surface of a body. A geometric discontinuity results in a different amount of reflected and / or transmitted light rays compared to the rest of the body. In a body made of a weldable material, a discontinuity can represent a weld seam visible on the surface of the body. A geometric discontinuity can also be a deformation or fracture visible on the surface.

[0003] The body can be made of an opaque workpiece. The body can be, for example, a metallic workpiece, a plastic, or a material containing a proportion of natural raw materials, such as wood or a wood-based material.

[0004] The color property values ​​of the image data comprising a pixel thus comprise a color property value and a coordinate indication or other suitable location information of the pixel having the color property value.

[0005] Since the method according to the invention is based on the application of image acquisition, it is limited to the detection of discontinuities visible on the surface.

[0006] The current state of the art uses image recordings to detect discontinuities. One possible approach is to compare an image of a body exhibiting no discontinuities with another image of a body exhibiting at least one discontinuity. However, it has been shown that methods based solely on comparison are neither sufficiently accurate nor sufficiently robust against errors in the image recordings.

[0007] A state-of-the-art method, which is limited to a comparison of image data, does not provide usable results, especially when used to detect discontinuities in natural workpieces such as wood, since the possible discontinuities are too different.

[0008] DE102007014475 describes a method in which, in addition to a physical property, another value is determined. In contrast to the method described below, the method disclosed in DE102007014475 is based on the determination of two values.

[0009] US20020088952 describes a method based on the overlay of measurement images with filters and reference functions. There is no reference to the design of the filters or the design of the reference function, which makes discontinuities visible by exploiting the coincidence effect.

[0010] DE102016114465 is based on the comparison of measured values ​​and reference values, which does not lead to a satisfactory result.

[0011] EP1304560 is based on the use of electromagnetic diagnostic beams and not imaging technology.

[0012] US2017046615, US2013073221, and DE10102387 do not disclose the determination of a deviation of a color property value from an average color measure or a dominant color property value. These documents also do not disclose the superposition of a function describing the color property values ​​with a reference function.

[0013] DE 101 02 387 describes an optical image recognition method for detecting and evaluating defects on a sample surface, such as a paint surface. The surface to be examined is first irradiated with collimated light, and the reflected radiation is fed to a spatially resolving image processing device. Damage in the paint appears as dark areas when an image is captured with collimated light. A threshold value is then used to generate a mask, in which the masked areas are formed by the bright, defect-free areas of the image, and the unmasked areas are formed by the dark defect areas of the image.

[0014] The method according to the invention aims to replicate the human detection of discontinuities in a manner as similar as possible using a computer-implemented method, which is explained in detail below. This objective is linked to the inventive method being resistant to interference. This is achieved according to the invention by claim 1.

[0015] A person perceives a discontinuity through their eyes primarily through color differences and attempts to create areas of similar colors. Furthermore, the person compares an area of ​​a similar color with their experience in order to potentially evaluate such an area as a discontinuity.

[0016] The colors perceivable by the human eye are incorporated into computer-implemented processes as color values. Color values ​​can be defined according to defined codes.

[0017] The inventive solution, like human perception, is based on a two-stage detection approach. The inventive method is characterized by the fact that very precise image position information of geometric discontinuities is determined. From the image position information, position information can be calculated, which indicates the location of the discontinuity on the surface of the body.

[0018] The method according to the invention is characterized by the fact that only the color properties of the surface of the body are determined. No other physical parameters of the body are determined.

[0019] In a first method step, the value subranges with deviating color property values ​​are detected. The first method step can include the condition that a minimum number of pixels exhibit deviating color properties. This can prevent a color deviation caused by an interference signal or similar from being interpreted as a discontinuity. The first method step includes considering a cluster of pixels and thus a minimum number of pixels with deviating color properties as an initial indication of the occurrence of a discontinuity.

[0020] The minimum number of pixels can also be considered as a threshold for a minimum size of a discontinuity to be detected.

[0021] In a second step, an overlay image data set is examined for a minimum or maximum. The first step provides a first indication of the image position of a discontinuity located on the surface of the hollow cross-section. The second step provides a second indication of the image position of a discontinuity located on the surface of the hollow cross-section.

[0022] A color property value can, for example, be a state-of-the-art color code such as RGB or a grayscale value. The person skilled in the art selects the color property value underlying the method according to the invention depending on the detectable colors of the surface of the body. If the body essentially exhibits only variants of a specific color value, such as the gray tones of a metal cross-section or the surface, the use of gray tones as color property values ​​is sufficient.

[0023] Carrying out the method according to the invention may require that the body to be examined be illuminated. For this purpose, a person skilled in the art can use light sources with directed, preferably parallel light beams. According to current teachings, LED light sources can be used. For example, an LED ring extending annularly around the image sensor or lens can be used to illuminate the body.

[0024] Conventional digital cameras can be used to perform the procedure.

[0025] In the second step of the process, the color property values ​​of the pixels of a selected image sub-area and / or a color property function describing the color property values ​​can be mathematically superimposed by means of a mathematical function.

[0026] The mathematical superposition can be carried out by multiplication (mathematical convolution) and / or addition and / or subtraction and / or division et cetera of the color property values ​​of the pixels and / or the progression of the color property values ​​of the pixels with a function and / or a number of functions.

[0027] In the second process step, the optically perceivable color property values ​​can be overlaid with an optical filter to generate an optically perceivable overlay image data set.

[0028] The second step of the method can be based on the application of the method of generating neural networks, which is known according to current teaching, and the training of the neural networks from image data, which is also known according to current teaching.

[0029] The first process step and the second process step can be carried out in parallel or consecutively in any order.

[0030] The mathematical functions to be superimposed or defined by the optical filter can be reference functions that describe the progression of color property values ​​of image data of a surface exhibiting a discontinuity. The method according to the invention can provide that a reference function describing a group of discontinuities is stored in a database or is present as a group of filters. The group of discontinuities includes the discontinuity that is to be detected using the method according to the invention.

[0031] Alternatively, the mathematical functions or functions defined by the optical filter to be superimposed can also be reference functions that describe the course of color property values ​​of image data of a surface without any discontinuity.

[0032] The function can also describe environmental influences and thus minimize the influence of environmental influences on the detection of geometric discontinuities, since this smooths out changes in the color property values ​​caused by environmental influences. However, this is not the subject of the claimed invention.

[0033] Overlaying the color property values ​​with a reference function describing the reference color property values ​​has the effect of clearly revealing deviations in the image data due to a coincidence with the function. Essentially, the skilled person expects the occurrence of certain discontinuities and can select the reference function based on this expectation. Preferably, the reference functions are scalable, so that the aforementioned coincidence effect becomes more pronounced when the reference function is scaled.

[0034] For example, the expert may expect a weld seam to be a discontinuity.

[0035] The specialist can mathematically or optically overlay the color property values ​​with a reference function, which reference function represents the usual course of the color property values ​​of a weld seam.

[0036] The method according to the invention is by no means limited to the requirement that the first clue and the second clue must be identical. The method according to the invention may include the step of comparing the first clue and the second clue to determine the position of a discontinuity.

[0037] The method according to the invention can comprise that the course of the color property values ​​at pixels of an image sub-area is superimposed with a reference interference function, which reference interference function describes the course of interference signals or the inverse course of interference signals, with generation of a further overlay image data set comprising further overlay color property values ​​at the pixels.

[0038] In this sense, filters can also be applied which have a similar effect.

[0039] The environmental influences listed above are generally considered a possible type of interference. According to current theory, those skilled in the art are familiar with other interference signals that can distort the result. The method according to the invention is characterized by the fact that potential interference signals can be highlighted or attenuated by superimposing them with reference interference functions, utilizing coincidence.

[0040] The method according to the invention can be characterized in that multiple images are taken of a body, with the method according to the invention being applied to each image. The latter can, in turn, include the detection of a discontinuity with sufficient accuracy for a defined number of matching first and second indicators.

[0041] The first method step may provide a first indication, which first indication comprises a first probability measure of the occurrence of a discontinuity.

[0042] The second method step may provide a second indication, which second indication comprises a second probability measure of the occurrence of a discontinuity.

[0043] The above-mentioned comparison of the first evidence and the second evidence can be carried out in the form that the first probability measure and the second probability measure must exceed a threshold value so that the first evidence and the second evidence can be combined into a statement about the existence of a discontinuity.

[0044] The method according to the invention can be advantageously applied to both stationary and moving bodies. In particular, the above-mentioned superposition of the color property values ​​with reference functions, which describe environmental influences, allows the method according to the invention to be applied to moving bodies, since the influence of environmental influences such as dust dispersal can be prevented.

[0045] The comparison of clues may require that a certain number of images be captured within a certain time period, to which the method according to the invention is applied to generate the clues. The person skilled in the art calculates the required number of images per second from the required number of images within a certain time period.

[0046] Image data can be created and / or used that represent light rays with a defined waveform and / or with a defined wavelength and / or with a defined light frequency. The method according to the invention can include the use of filters according to the prior art, such as H-wavelet filters.

[0047] Image data may also be created and / or used in the form of a negative image. The image data may be processed using state-of-the-art processing techniques.

[0048] The method according to the invention can be characterized with respect to the use and / or creation of image data by means of filters and / or negative image technology in that only certain light rays are examined with respect to the expected discontinuity. This allows the discontinuities to be displayed with greater contrast, either visually or in the form of image data. The method according to the invention can be characterized by high effectiveness with respect to the use of filters or negative image technology.

[0049] The method according to the invention can be characterized in that a network area comprising a geometric discontinuity is stored in the database as a reference discontinuity. As a result, the database, in which the shapes of the discontinuities are stored in the broadest sense, is continuously expanded, essentially with information describing the shape of a network area exhibiting a discontinuity.

[0050] The ongoing expansion of the database by network areas classified as network areas containing a discontinuity implies that this process can be system-controlled and / or user-controlled.

[0051] This continuous expansion of the database by adding network areas classified as discontinuities is particularly advantageous when the method according to the invention is applied to natural materials. In a similar manner, the color property values ​​of the pixels of a value sub-range comprising a geometric discontinuity can be stored in the database. This method step also allows for the storage of gradients or patterns of the color property values ​​across the pixels of the geometric discontinuity if the geometric discontinuity extends across multiple pixels.

[0052] The method according to the invention can thus comprise routines by means of which a database comprising the color property values ​​of pixels is continuously expanded, which pixels comprise a geometric discontinuity. Information about the color property values ​​of individual pixels and / or about the progression of color property values ​​across multiple pixels can be stored.

[0053] The progression of color property values ​​across multiple pixels can be described using mathematical functions.

[0054] Those skilled in the art will recognize that the above method is essentially applicable to any body shape that includes a surface that can be captured using imaging techniques. However, this may require knowing when a body is depicted in the image.

[0055] The second method step, in which the overlay image data set is generated, can be characterized in that the image sub-area is selected depending on the occurrence of a maximum value and / or a minimum value at pixels.

[0056] Preferably, the image sub-area is selected by a person or by independent process routines such that the image sub-area includes the pixels comprising the maximum or minimum value as well as other pixels adjacent to these pixels. The maximum or minimum value is determined by applying the method according to the invention using a previous image sub-area. The method according to the invention can be characterized in that the amount of image data is reduced because the image sub-area is smaller than the previous image sub-area.

[0057] The method according to the invention may also comprise detecting and verifying the presence of a body in the image, wherein the edges of the body are detected by methods according to the prior art and the verification is carried out via the virtual construction of vanishing points.

[0058] The method according to the invention is further explained with reference to the following description and the following figures. In the figures, the following elements are identified by the preceding reference numerals. 1 body 2 cross section 3 edges 4 weld seam 5 right wall 6 free 7 direction of movement 8 reflection surface light 9 partial value range 10 rectangular area 11 further value range

[0059] Figure 1 and Figure 2 show black and white images of the body to be examined.

[0060] Figure 3 shows an excerpt of the Figure 2 depicted view of the body, where Figure 3created using a filter.

[0061] Figure 4 shows a diagram which diagram from the Figure 3 derived from the color property values ​​shown.

[0062] Figure 1 shows a hollow body as body 1 with a cross section 2. The viewing direction of the Figure 1 The view of body 1 shown is parallel to the longitudinal axis of body 1.

[0063] In the Figure 1 In the embodiment shown, a pipe having a length of 12.0 m and a cross-section of approximately 10.0 cm by 10.0 cm is moved as body 1 at a high speed - up to 6 m / sec - past cameras.

[0064] Figure 1shows a direction of movement 7 of the pipes in a system according to the state of the art. In the system, the pipes are moved past the cameras in a continuous process. The cameras required to carry out the method according to the invention are integrated into the system.

[0065] The object of the method according to the invention is to detect the body in the image recorded by the camera in a first step and to detect discontinuities in the form of a weld seam on the surface of the body 1 in a second step. The step of detecting the discontinuities includes also determining the image position of the discontinuity, so that the pipe can subsequently be positioned depending on the position and / or depending on a target position of the weld seam.

[0066] The first and second steps of the method according to the invention can be carried out essentially independently of one another. The second step may require that the presence of the tube as a body is known, which the person skilled in the art can also determine using methods other than the first step of the method according to the invention.

[0067] To carry out the method according to the invention, the body is illuminated with a light source emitting light rays and recorded by cameras outputting image data.

[0068] The cameras are essentially aligned along the pipe axes. The camera for recording a Figure 1 The image shown as an example is aligned parallel to the longitudinal axis of body 1. The focus of the camera for the image shown in Figure 1 The image is adjusted so that a working area of ​​20.0-200.0cm is sharply displayed.

[0069] The recording direction of the additional camera for recording a Figure 2 The image shown as an example is essentially aligned at right angles to the surface to be examined.

[0070] In the Figure 1 and Figure 2 In the images shown, the image data contains a gray value per pixel. Reducing the image data from color property values ​​to gray data results in a reduction in the amount of data. The method according to the invention is characterized by providing sufficiently precise information to enable this reduction in the amount of data.

[0071] In a first step, the edges 3 of the body 1 are detected. However, the person skilled in the art need not detect all edges when implementing the method according to the invention. Prior art edge detection methods are used. In the method illustrated in the figures, the edges are determined using the gradients of the gray values ​​of neighboring pixels.

[0072] Based on the detected edges, the vanishing points of these edges can be determined and thus the presence of body 1 in the image can be verified.

[0073] In Figure 1 The approximate positions of the determined edges 3 are shown by dashed lines. The vanishing points determined from the edges 3 include Figure 1 illustrated embodiment a vanishing point located at "infinity".

[0074] Determining at least one vanishing point of the body's edges at "infinity" has the advantage that the body can be differentiated from its surroundings without significant computational effort. In the embodiment discussed here, the surroundings are formed by a hall, in which the system for processing and / or transporting the pipes is located as body 1.

[0075] In a second step, the discontinuities and their image position are detected.

[0076] The expert recognizes that the Figure 1 The hollow body 1 shown in a cross-sectional view comprises a weld seam 4 in the right-hand wall 5. The weld seam 4, which represents a geometric discontinuity, is recognizable to the human eye by the gradient of the gray tones in the right-hand wall 5 and by identifying partial surfaces of the right-hand wall.

[0077] Figure 2shows a frontal view of the right wall 5. The camera for recording the Figure 2 The image shown is essentially aligned parallel to a cross-sectional axis of the body. Here, too, weld seam 4, which represents a discontinuity, is recognizable by the gradient of the gray tones and by identifying partial areas.

[0078] Figure 2 shows next to the pipe as body 1 together with the weld 4 a bright surface, which bright surface is a reflection surface 8 of light, with which light the body 1 for making the in Figure 1 and Figure 2 images shown. It is also an object of the method according to the invention to recognize such reflection surfaces 8 as such.

[0079] The subjective identification of partial areas by a person includes that the person compares the recognized partial areas with his or her wealth of experience about discontinuities and also selects the partial areas accordingly depending on the color property values.

[0080] The method according to the invention attempts to recreate this briefly described human perception, which is known according to current teaching, by means of a computer-implemented method.

[0081] For this purpose, the color property values ​​at the individual pixels are compared, and sub-areas with deviating color property values ​​are determined. These deviating color property values ​​differ by a threshold value from an average color value measured across the surface or from a dominant color property value. Sub-areas with color values ​​deviating from the average or with the dominant color values ​​within a value range defined by threshold values ​​are then searched for.

[0082] Figure 3 shows an excerpt of the Figure 2 shown weld seam, where the Figure 3created using an edge detector based on the established Sobel or Lenny theory. With the help of the edge detector, only those pixels with the properties are displayed at which the gradient of a function describing the color values ​​has a high gradient in the horizontal or vertical direction. Figure 4 shows one from the Figure 3 derived diagram. The Figure 4 The diagram shown represents a color value at a pixel on the z-axis as a function of the pixel's position shown in the xy area. Figure 4 also includes the Figure 3 contained image in a three-dimensional representation.

[0083] The detection of discontinuities using the method according to the invention is essentially based on one of the Figure 3 and Figure 4The implementation of the method according to the invention does not require the creation of such graphics, but can also be based exclusively on the processing of data. To illustrate the method according to the invention, it is described using the Figures 3 and 4 explained by example.

[0084] Figure 3 comprises a rectangular area 10. In this area 10, information has been covered, which information is in a format for display in Figure 3 are written in too small a font and have no influence on the subject matter of the patent application.

[0085] Figure 3 shows a value subrange 9, which value subrange 9 includes color property values ​​deviating from an average measure of the color property values.

[0086] The value subrange 9 therefore includes color property values ​​which deviate from the dominant color property value by at least a defined limit value.

[0087] A dominant color property value is the range of color property values ​​that dominates over the surface. Figure 3 shows the special case that the average color value is also the dominant color property value.

[0088] The expert recognizes that in Figure 3 and Figure 4 two sub-areas can be identified as such value ranges 9 with deviating color property values, while three further value sub-areas with average color properties equal to the dominant color property values ​​can be identified.

[0089] The value subranges 4 in Figure 3 In addition to the reference symbols, they are marked with a dashed ellipse; the other sub-areas are not further marked.

[0090] In Figure 4 the value ranges 4 are only marked by reference symbols.

[0091] The presence of the two sub-areas as value sub-areas 9 with differing color property values ​​is considered as the first indication of a discontinuity on the mantle surface.

[0092] The method according to the invention is characterized in particular by the fact that the first indication of the presence of a discontinuity is obtained by a further analysis of the Figure 1 and Figure 2 included images is verified.

[0093] Figure 5 shows the basic course of the pixel graph 12 in Figure 4 shown area curve along a section plane parallel to the x-axis. The exact position of the section plane is shown in the Figure 4 and the exact course of the Figure 5 The graph shown is for the following discussion of the Figure 5 illustrated effects of the application of the method according to the invention are irrelevant.

[0094] The pixel graph 12 thus represents the course of the color property values. The pixel graph 12 is an approximate, sufficiently accurate representation of the course of the Figure 4 represented surface curve along a cutting plane parallel to the x-axis.

[0095] Figure 5 further comprises a reference curve 13. The skilled person selects the reference curve 13 and, if applicable, the position of the reference curve 13 based on his specialist knowledge.

[0096] The reference curve 13 can be chosen with regard to an expected discontinuity.

[0097] The person skilled in the art will recognize that the application of the method according to the invention also allows the use of several reference curves, in which Figure 5 For the sake of clarity, only the use of a reference curve is shown.

[0098] By applying the second method step of the method according to the invention, the pixel graph 12 and the reference graph 13 are convolved to form an overlay graph 14.

[0099] Figure 6 shows an optical reference model comprising reference curves for superimposing with a sub-area of ​​the Figure 4 shown area curve.

[0100] Figure 7 shows the result of an overlay of a part of the Figure 4 shown area curve with the one in Figure 6 shown reference model. The discontinuities are clearly visible due to the bright areas.

[0101] This verification can be done by forming neural networks according to the current teaching, which can be done with reference to the Figure 8 is explained.

[0102] Figure 8shows an image 12, which image 12 has a size of 224x224. The image 12 comprises image data with color property values ​​per pixel. The image data of image 12 are overlaid with two reference functions, forming two so-called levels. The levels each comprise an overlay image data set, which is generated by multiplying the color property values ​​by a function. Thus, in the Figure 8 The embodiment shown generates two overlay image data sets, each comprising maximum values ​​and minimum values.

[0103] The occurrence of these maximum and minimum values ​​is interpreted as a second indication of the presence of geometric discontinuities.

[0104] Accordingly, the image sub-area is selected when overlaying the image data with three additional functions. While the image sub-area in the first overlay of the image data with two functions corresponded to the size of image 12, the image sub-area can be significantly reduced in the second overlay.

[0105] The skilled person can perform the overlay of the image data n times and accordingly reduce the image sub-area and limit the local presence of the discontinuity.

Claims

1. A computer-implemented method for determining indications of geometrical discontinuities (4) on the surface of the surface of a body (1), which body (1) is illuminated using a light source emitting light beams, and which body (1) is recorded using a camera issuing image data, which image data comprises colour characteristics values on an image point, wherein - the colour characteristics values are compared at the individual image points and an accumulation of deviant colour characteristics values determined in a value subarea (9) of the surface, which deviant colour characteristics values deviate from a colour average of the colour characteristics values of the surface or from a dominant colour characteristics value by a predefined limit value, is interpreted as a first indication of an image position of a geometrical discontinuity (4) at image points neighbouring the value subarea or in the value subarea, and - the course of the colour characteristics values on image points of an image subarea is superimposed with a reference function (13), which reference function (13) describes the course of colour characteristics values of image data of a surface having either a discontinuity or no discontinuity, to generate a superimposition image data quantity (14) comprising superimposition colour characteristics values on the image points, and a maximum value and / or a minimum value in the superimposition image data quantity on an image point is interpreted as second indication of a geometrical discontinuity (4).

2. The method of claim 1, characterised in that the first indication and the second indication are matched to determine a position of a discontinuity (4).

3. The method of any one of claims 1 and 2, characterised in that the course of the colour characteristics values on image points of an image subarea is superimposed with a reference interference function, which reference interference function describes the course of interference signals or the inverse course of interference functions, thereby generating a further superimposition image data quantity comprising further superimposition colour characteristics values on the image points.

4. The method of any one of claims 1 to 3, characterised in that the image data is created and / or used, which image data render light beams with a predefined waveform and / or wavelength and / or light frequency.

5. The method of any one of claims 1 to 4, characterised in that the image data is created and / or used in the form of a negative image.

6. The method of any one of claims 1 to 5, characterised in that a network subarea comprising a geometrical discontinuity (4) is saved in a database as a reference discontinuity.

7. The method of any one of claims 1 to 6, characterised in that the colour characteristics values of the image points of a value subarea comprising a geometrical discontinuity (4) are saved in a database.

8. The method of any one of claims 1 to 7, characterised in that the image subarea is chosen depending on the occurrence of a maximum value and / or a minimum value on image points.

9. The method of any one of claims 1 to 8, characterised in that the edges (3) of the body (1), which body can be entirely depicted in the image, are detected and the presence of a body (1) is verified by determining the vanishing points defined by the edges.