Crack analysis apparatus and method

Abstract

A crack analysis device and method is disclosed that is a method and device that can analyze cracks in a vehicle glazing panel without a technician being present at the site. The method and device capture and process images of the cracks to enable a determination of whether the glazing panel is suitable for repair or replacement.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to an apparatus and a method. In particular, but not exclusively, the present invention relates to an apparatus and a method for analysing cracks in a glazing panel of a vehicle. Further in particular, but not exclusively, the present invention relates to a method and apparatus for analysing cracks in a glazing, in particular a glazing panel of a vehicle. BACKGROUND

[0002] During driving, if there are debris and other matter on the road, this can result in the matter being transferred into the carriageway, and when this matter hits the windscreen, this can result in cracks, fissures or other damage to the windscreen, which can require repair or replacement of the glazing panel of the vehicle.

[0003] For safety reasons and for economic reasons, the repair or replacement work must be carried out as soon as possible, as the cracks can further propagate into the windscreen due to the effects of cold weather, for example, which can cause the cracks to increase in size. This can result in the cracks changing from requiring a small scale repair to requiring a complete replacement of the windscreen.

[0004] There is a need to assess whether the damage to the glazing panel of the vehicle can be remedied by repair. If the assessment indicates that repair is not feasible, then replacement of the glazing panel is required.

[0005] Based on the foregoing, aspects and embodiments are conceived. SUMMARY

[0006] Viewed from a first aspect, the invention provides a crack analysis method for analysing cracks in a glazing panel of a vehicle, the method comprising: capturing an image of the cracks in the glazing panel of the vehicle; processing the image of the cracks. Viewed from a second aspect, the invention provides a crack analysis apparatus for analysing cracks in a glazing panel of a vehicle, the apparatus comprising: a camera for capturing an image of the cracks in the glazing panel of the vehicle; a processing module for processing the image of the cracks.

[0007] Optionally, the apparatus can comprise a mobile computing device comprising the camera. The mobile computing device is an electronic device for capturing the image, which can comprise a mobile phone (e.g. a smartphone), a notebook computer, a tablet computer, a phablet or a camera. The mobile computing device comprises the camera for capturing the image.

[0008] The mobile computing device can further comprise the processing module.

[0009] The method or apparatus according to the first and second aspects enables the cracks in a surface to be analysed using an image of the cracks. This enables a technician not to have to attend the scene to carry out any analysis of the cracks.

[0010] The need for replacement of the glazing panel can be determined based on processing of the image. Thus, the method can comprise the step of determining, based on processing of the image, whether the glazing panel needs to be replaced and / or whether the glazing panel is suitable for repair.

[0011] The image of the crack can be captured at an angle that is inclined relative to the vehicle glazing panel.

[0012] The image can be captured by a mobile computing device that is in contact with the surface of the glazing panel, wherein the mobile computing device comprises a camera. The mobile computing device can for example be a mobile phone, such as a smartphone provided with a camera. The application can be implemented by a software component for processing image data from the camera in order to determine whether the crack can be repaired or whether the glazing panel needs to be replaced. The software component can be downloaded to the device, for example as a stand-alone application (app) or the like. The software component can comprise an algorithm for making the determination, and preferably also display user instructions indicating how to implement the determination method.

[0013] Thus, according to another aspect, the application comprises a computer-implemented method for determining technical information of a crack present in a vehicle glazing panel, wherein the method comprises the step of downloading a software component to a mobile computing device having a camera, the software component being configured to process image information captured by the camera in order to determine whether the crack is preferably repaired or replaced.

[0014] The mobile computing device can be tilted in order to view the crack at a predetermined position in the field of view of the camera. The reference position can be achieved by ensuring that the computing device is tilted towards the glazing panel and that the edge is in contact with the glazing panel.

[0015] The predetermined position at which the mobile computing device can be tilted in order to view the crack can be indicated by a marker displayed on the camera or on the mobile computing device comprising the camera, for example on a screen.

[0016] Additionally or alternatively, the marker can indicate the center of the field of view, for example on the screen.

[0017] Optionally, the camera or the mobile computing device can initially be laid flat on the surface of the glazing panel. Then, the mobile computing device can be pivoted away from the glazing panel or tilted with respect to the glazing panel, wherein at least part of the mobile computing device remains in contact with the glazing panel.

[0018] Optionally, the camera is positioned at a predetermined position relative to the crack before the mobile computing device is pivoted. For example, the method can comprise aligning a feature of the mobile computing device at a predetermined position relative to the crack.

[0019] In some embodiments, an edge of the image capture module, e.g., the top edge, can be aligned proximate (or immediately below) the lowest point of the crack. This means that the geometry of the mobile computing device can be used to determine the initial distance between the camera and the lowest point of the crack.

[0020] The image capture module or mobile computing device can then be pivoted away from the glass window pane, with the bottom edge of the mobile computing device remaining in contact with the glass window pane. An image of the crack is then captured.

[0021] The method can include pivoting or tilting the mobile computing device until the crack is visible at a predetermined location in the field of view of the camera.

[0022] Optionally, the method includes pivoting the mobile computing device until the crack is centered in the field of view of the camera.

[0023] The method can include estimating parameters of the crack using the geometric parameters of the mobile computing device and the lens parameters of the camera. The parameters of the crack include one or more spatial dimensions indicative of the size of the crack.

[0024] For example, using the above-described method, the geometric parameters of the mobile computing device and the lens parameters of the camera can be used to determine the length of one or more branches of the crack and / or the size (e.g., width / diameter) of the center of the crack.

[0025] The geometric parameters of the mobile computing device can be used to determine the angle of rotation (or angle of pivot) between the mobile computing device and the glass window pane.

[0026] If the estimated size of the crack (e.g., the length of one branch of the crack) exceeds a given threshold, the method can determine that the glass window pane needs to be replaced. If the estimated size of the crack (e.g., the length of one branch of the crack) is less than a given threshold, the method can determine that the glass window pane is suitable for repair.

[0027] The processing of the image can capture the image of the crack based on the geometric parameters of the mobile computing device.

[0028] The processing of the image can also be based on the chip parameters of the camera and / or mobile computing device.

[0029] The processing of the image can generate a set of data points that are used to generate a diameter of the crack, which can be used to determine the requirement to replace the vehicle glass window pane.

[0030] The method can include outputting a signal or indication indicating whether the glass window pane needs to be repaired or replaced.

[0031] A crack in a glazing panel can comprise a centre and one or more branches. This form of crack is common when a small stone or other object impacts the glazing panel. The one or more branches (or cracks) typically radiate from the break centre to the periphery.

[0032] The shape of the break centre can be substantially circular.

[0033] Determining the need to replace or repair the glazing panel can comprise generating data indicative of the break centre and the one or more branches of the crack.

[0034] The method can comprise identifying the break centre and the one or more branches of the crack.

[0035] The method can comprise generating a scale factor indicative of a relative length of the one or more branches of the crack compared to a size of the break centre. The size of the break centre can be a diameter, a width and / or a length of the break centre.

[0036] The method can comprise estimating a length (i.e. an absolute length in cm or mm etc.) of the one or more branches of the crack using the scale factor. For example, the length of the one or more branches can be estimated by multiplying the scale factor by a predetermined value.

[0037] The predetermined value can be an estimate of an actual (i.e. absolute) size of the break centre, with the advantage that no calibration object is required, which is at least more convenient for a user.

[0038] The method of the present application can therefore comprise determining a scale value of the image to estimate a length of the one or more branches of the crack.

[0039] The method can determine that the glazing panel needs replacing if the estimated length of the one or more branches exceeds a given threshold. The method can determine that the crack in the glazing panel is suitable for repair if the estimated length of the one or more branches is less than a given threshold.

[0040] The method can comprise outputting a signal indicative of a need to repair the glazing panel if the estimated length of the branch or branches is less than a given threshold.

[0041] The method can comprise outputting a signal indicative of a need to replace the glazing panel if the estimated length of the branch or branches exceeds a given threshold.

[0042] It has been found that the size of the break centre is typically different and smaller than the length of the one or more branches of the crack when comparing different cracks in a glazing panel. As such, the predetermined value can be an average or a mode of measured sizes of break centres in glazing panels.

[0043] Optionally, the estimated value (i.e. predetermined value) of the actual width (or diameter) of the crack centre can be between 1 mm and 3 mm. A particularly preferred predetermined width (or diameter) of the crack centre can be 2 mm. These ranges / values have been determined by crack studies carried out by the applicant.

[0044] The effect of this is that the estimated size of the crack centre, for example the estimated diameter of the crack centre, can be used to estimate the length of the crack branch, as the predetermined value can be used to vary between the relative length of the branch or branches relative to the size of the crack centre, and the estimated actual length of the branch or branches of the crack.

[0045] For example, if we know that the crack centre is always about 2 mm wide (or 2 mm in diameter), and the generated image data indicates that the branch length is twice the diameter of the crack centre, then the method can comprise multiplying the 2 mm by a scaling factor of 2. This estimates the length of the branch to be 4 mm, which helps to build a picture of the crack dimensions in the data.

[0046] The generated (or estimated) length of the branch or branches can be used to indicate the estimated size of the crack. The size of the crack can be compared to a threshold parameter to determine whether the glazing panel needs to be replaced or repaired.

[0047] If the estimated crack size exceeds a given threshold, then the method can determine that the glazing panel needs to be replaced. If the estimated crack size is less than a given threshold, then the method can determine that the glazing panel is suitable for repair.

[0048] The crack threshold parameter can be compared to the maximum span distance of the crack.

[0049] Optionally, the predetermined estimate of the size of the crack centre can depend on one or more parameters. The parameters can be input by a user and / or pre-set into the device or processing module. For example, the parameters can include: one or more properties of the glazing panel (e.g. type, size, etc.), and / or the speed at which the vehicle was travelling when the breakage occurred.

[0050] The processing of the image can comprise filtering the image to remove background portions to identify the crack.

[0051] Morphological thinning can be applied to the image to remove any impurities from the image and improve the quality of the image data which is used as the basis for determining whether the glazing panel needs to be replaced.

[0052] The method can comprise cleaning the glazing panel prior to capturing the image of the crack. This can help to remove any dirt which can affect the image processing. For example, there is a risk that the dirt can be built into the image processing software as a crack.

[0053] The method can comprise disabling the flash function of the image capture module or device prior to capturing the image of the crack. If flash photography is used, the light can adversely affect the accuracy of the image processing software. For example, the flash can reflect in the glass panel which can affect the identification or analysis of the crack.

[0054] The method can be implemented using computer implemented instructions which, when installed into a memory, instruct a processor to implement the method as defined above, preferably a downloadable software build (e.g. app).

[0055] It will be appreciated that any of the features of the method can be performed using the apparatus of the application.

[0056] These and other aspects of the application will become apparent in light of the embodiments described herein. BRIEF DESCRIPTION OF DRAWINGS

[0057] A first and second embodiment of the application will now be described by way of example only and with reference to the accompanying drawings. In which:

[0058] Figure 1 A windscreen with a crack is shown;

[0059] Figure 2 An image of the crack is shown being captured using a camera;

[0060] Figure 3 A processing module which can be used to analyse cracks in a windscreen is shown; Figure 1

[0061] Figure 4 A flow chart showing the steps involved in assessing a crack in a windscreen using the system is shown; Figure 3

[0062] Figure 5 a, 5b shows an image of the Fourier transform of the crack; and

[0063] Figure 6 a, 6b shows a profile image of the crack;

[0064] Figure 7 An arrangement which can be used to simulate a crack 102 is shown;

[0065] Figure 8 The steps involved in generating crack parameters are shown;

[0066] Figure 9 a shows schematically the top view field of view of the camera to the windscreen;

[0067] Figure 9 b shows schematically the tilted view field of view of the camera to the windscreen; ​​

[0068] Figure 10 It is shown how the camera is modelled in order to analyse the crack using the parameters of the camera; and

[0069] Figure 11 It is shown an image output from the system to determine the size of the crack in the windscreen. DETAILED DESCRIPTION

[0070] In a first embodiment, Figure 1 It is shown a windscreen 100 having a crack 102 caused by a stone bouncing onto the windscreen. The driver of the car in which the windscreen 100 is enclosed then uses a mobile phone 104 to capture an image of the crack 102, the mobile phone 104 containing a camera 106 for capturing the image of the crack 102. This arrangement is shown in Figure 2 in side view. The focal length of the camera 106 is fixed below 100mm so that the focus of the camera is tight at close distances.

[0071] The image of the crack 102 is then captured in response to a user input, the mobile phone 104 being used to provide a prompt to the user to request that they indicate that they wish to transfer the image from the camera 106 to an image processing module 108. We will now describe the image processing module 108. This step enables the user to self-evaluate the quality of the image so that, if they consider that the image is not clear - for example, in bad weather conditions, there can be water condensation on the lens of the camera 106 - they can choose to capture another image. Figure 3

[0072] The camera 106 converts the captured image into an array of image data using any suitable method. The camera 106 can save the image data as an exchangeable image file (EXIF) in which the lens parameters of the camera are also stored.

[0073] That is, the camera 106 is an example of an image capture module which can be run to capture an image of the crack 102 and transfer the captured image in the form of captured image data to an image processing module 108 in which the image can be further processed to extract the details of the crack.

[0074] The image processing module 108 can be formed as part of the mobile phone 104 or it can be geographically relatively remote from the mobile phone 104. The image data is transferred to the image processing module 108 by any suitable means - for example, a data bus or the internet.

[0075] ​In a preferred embodiment, the image processing module 108 is implemented as a software build downloaded to the mobile phone. This can be achieved by downloading the software build as an app. The software build is able to process image data from the camera to determine whether a crack is amenable to repair, or whether replacement of the glazing panel is required as a preferred solution.

[0076] The crack analysis module 112 can be a software build downloaded to the mobile phone, preferably as a single download in conjunction with the image processing module 108. Preferably, the single downloaded software build I is used to process image data from the phone camera and analyse the crack using one or more algorithms implemented in the software.

[0077] In one embodiment, the captured image data is received by the image processing module 108 at the data input interface 110. The image data is then transmitted to the crack analysis module 112 which is used to access the library of routines 114 in which routines can be stored to perform operations on the captured data during analysis of the captured image data.

[0078] The crack analysis module 112 is also used to access a device parameters database 116 in which parameters relating to the mobile phone 104 are stored.

[0079] The parameters relating to the mobile phone 104 include chip parameters defining the image capture performance of the camera 106, for example, focal length, lens, sensor size, and dimensional parameters of the mobile phone 104, for example, the length of the mobile phone 104, and the distance between the top edge of the mobile phone 104 and the image centre of the camera 106.

[0080] The crack analysis module 112 can also be used to interface with a display module 118 which can be operable to display image data transmitted from the crack analysis module 112 on a display, and to display parameter data transmitted from the crack analysis module 112 on the display.

[0081] We will now refer to Figure 4 the analysis of the crack 102 using the crack analysis module 112.

[0082] In step S400, the crack analysis module 112 receives the image data. Then, in step S402, the crack analysis module 112 calls a Fourier transform routine from the library of routines 114 and applies a discrete two-dimensional Fourier transform to the image data using the Fourier transform routine to produce a transformed image as shown in Figure 4a. Figure 5 a.

[0083] In Figure 5In a, we can see the transformed image. Figure 5 In a, the spatial frequencies have been plotted according to the magnitude of the corresponding Fourier components. It can be seen that the low spatial frequencies occupy the centre of the transformed image, and as we move away from the centre of the transformed image, we can see higher spatial frequencies.

[0084] The Fourier transform of the image enables the crack analysis module 112 to perform image analysis in terms of the component spatial frequencies and the phase of the image. As will now be described, this removes spatial frequencies that we are not interested in and enables us to reconstruct the image that we are interested in by retaining the spatial frequencies that we are interested in.

[0085] In step S404, a Butterworth bandpass filter is then applied to the transformed image by the crack analysis module 112. The mask achieved by the Butterworth bandpass filter is shown in Figure 5 b. The Butterworth bandpass filter achieves a mask on the transformed image shown in Figure 5 a, and removes the low spatial frequencies (shown by the dark spots in the centre of Figure 5 b) and very high spatial frequencies (shown by the dark edges in the image in Figure 5 b), which represent dust and dirt spots on the image.

[0086] In step S406, the Fourier transform of the image data is then inverted by calling a Fourier transform inversion routine from the routine library 114 to perform an inverse discrete two-dimensional Fourier transform on the transformed image data.

[0087] The inverse Fourier transform process on the transformed image data transforms the transformed image data from the Fourier domain to the real domain to generate real domain image data. The resulting real domain image data is shown in Figure 6 a and 6b.

[0088] Using the Fourier transform produces Figure 6 the image shown in b, with the effect of isolating the crack from the background.

[0089]

[0090] Figure 6 a shows the real domain image data without the use of the Butterworth bandpass filter. Figure 6b shows the application of the Butterworth bandpass filter. Figure 5 The real-domain image data shown in example a is transformed data, and thresholding is applied to generate a binary image with a threshold intensity of 4. The Butterworth bandpass filter in this example has a rolloff value of 3.

[0091] The upper and lower cutoff frequencies of the Butterworth bandpass filter can be modeled because they are linearly related to the number of pixels on the longest side of the image (denoted as m), and can be expressed as follows:

[0092]

[0093]

[0094] This relationship can be altered using standard experiments and numerical experiments.

[0095] Figure 6 The image shown in b is not limited to the image of crack 102. The image may also include image data that has been processed through steps S400 to S408, but caused by dirt spots on the windshield and other artifacts from the processing performed by the crack analysis module 112.

[0096] Generate using threshold strength 4 Figure 6 The binary image shown in b helps to display the region of interest more clearly. Figure 6 The image data shown in b, representing the real-domain image, highlights the crack—including the central crack region—which is a low spatial frequency area.

[0097] As can be seen, the Fourier method does a very clever job, namely isolating the crack region from the cluttered background, thus assuming that it is in focus while the background is not.

[0098] Then, the crack analysis module 112 can call the morphological routine from the routine library 114 to perform the operation in step S410. Figure 6 Remove any impurities from the image shown in b.

[0099] Morphological routines Figure 6 The image shown in b undergoes several operations. This image is a binary image. Black areas have zero values, and white areas have non-zero values. The pixel values ​​are stored in the memory of the crack analysis module 112 and are the result of the processing in steps S400 to S408.

[0100] The first of these operations is a fill operation, which uses shape reconstruction to fill a pixel-sized black area surrounded by a white area with white, by replacing zero values ​​with non-zero values ​​according to the process listed in [1].

[0101] The second of these operations is a clean-up operation which discards very small non-zero value regions. A very small non-zero value region is defined as a non-zero value region occupying less than (maximum dimension of the image / 500) squared. The maximum dimension of the image can be determined by the crack analysis module simply by comparing the width of the image with the length of the image.

[0102] The first morphological operation is then repeated to fill any pixel-sized black regions surrounded by white regions resulting from the second morphological operation. This is a third morphological operation.

[0103] A fourth morphological operation is then implemented to connect any branches of the image of the crack 102 which have gaps. This is achieved using the morphological closing operation described in [2] by eroding with a disc shaped structuring element with a radius of (maximum dimension of the image\5312) multiplied by 20 and then dilating. The value 20 is determined empirically and can be varied. The value can be determined without any undue burden on different image resolutions.

[0104] The first morphological operation is then repeated to fill any pixel-sized black regions surrounded by white regions resulting from the fourth morphological operation. This is a fifth morphological operation.

[0105] A sixth morphological operation is then performed to discard any non-zero small regions. A small region is defined as a region with an area equal to (maximum dimension of the image / 100) squared.

[0106] A seventh morphological operation is then performed to remove any disconnected objects in the image. The disconnected objects of interest are objects which are further than 3 / 4 of the radius of the largest object closest to the centre of the image. This means that branches of the crack which are still not coherent are included, but spurious artefacts are included. The seventh morphological operation is achieved by finding the centre of mass (i.e. the centre of mass of the image) and the length of the major axis of each remaining region in the image. Each region is assigned an additional weighting based on how close the centre of mass is to the centre of the image.

[0107] The weighting w = 1 / d 2 where d is the Euclidean distance between the centre of mass and the centre of the image. The largest region closest to the centre of the image is selected and its major axis length is used to set the radius (or 3 / 4 of the major axis length from its centre of mass) and any regions outside this radius are discarded. That is, the morphological routine and the centre of mass to boundary calculation are used to keep all "blobs" within a distance which is the distance from the centre of the crack to the radius of the largest object in the image plus half of that radius to ensure that no discontinuities in the crack 102 are lost.

[0108] After applying morphology to refine the image data, the image data can be used to determine the dimensions of the crack 102.

[0109] The crack analysis module 112 applies further edge detection, morphology, blur processing and thresholding to determine the crack 102 centre.

[0110] It has been observed by experiment that the crack centre diameter is typically around 2mm. The crack analysis module 112 is used to estimate the length of the branch of the crack 102 using the refined image data and the data resulting from the determination of the crack 102 centre, and to determine a scale value representing the length of the branch compared to the diameter of the crack 102 centre, i.e. a scale factor for the branch compared to the crack 102 centre. Using the observed crack centre of typically 2mm, the scale factor can be used to determine the length of the branch. This provides an uncalibrated analysis of the size of the crack 102.

[0111] The determined length of the branch can then be used to approximate the size of the crack 102 and enable the crack analysis module 112 to output whether the windscreen needs to be replaced or whether it is sufficient to repair the windscreen, as the size of the crack is an important factor in deciding which decision to make. By comparing the size of the crack 102 to repair / replace thresholds, the crack analysis module 112 can automate this decision. The crack analysis module 112 outputs this decision to the display module 118.

[0112] In step S412, the output, i.e. whether the windscreen needs to be replaced, is then displayed using the display module 118.

[0113] Using the observed estimate of the crack centre to estimate the size of the crack branch depends on the assumption of the sharpness of the crack radially, which means that an image of the crack can be taken and used to analyse the crack to provide the size of the crack 102 without the aid of any scale on site.

[0114] The method enables crack analysis under a variety of conditions without the need for the involvement of a technician.

[0115] In a second embodiment, we now describe how to use the parameters of the mobile phone 104 and the lens of the camera 106 to obtain the parameters of the crack 102. This can help to correct any effects of the angle on the image.

[0116] The second embodiment can be combined with the first embodiment without departing from the invention.

[0117] Figure 2 The arrangement shown in Figure 4 enables the use of the chip parameters of the camera 106 and the geometric parameters of the mobile phone 104 to estimate the dimensions of the crack.

[0118] To calculate the angle of rotation (or angle of pivot or angle of inclination) of the mobile phone 104 with respect to the windshield, we can use the geometric parameters of the mobile phone 104.

[0119] In positioning the crack 102 at the center of the field of view of the camera 106 lens, a right triangle can be drawn. This is referred to as Figure 7 and is described.

[0120] After finding the crack 102, the mobile phone 104 is laid flat on the windshield with the top edge at the bottom of the crack 102. This means that the distance between the bottom edge of the mobile phone 104 and the bottom of the crack is equal to the length of the mobile phone 104. The mobile phone 104 is tilted from the bottom edge of the mobile phone 104 until the crack 102 is at the center of the field of view of the camera 106. A mark can be made on the display area of the mobile phone 104 to indicate the center of the field of view.

[0121] The distance between the bottom edge of the mobile phone 104 and the lens of the camera 106 can be obtained from the device parameters database 116. Thus, a right triangle is formed by the angle of rotation between the bottom edge of the mobile phone 104 and the windshield 100, the z-axis of the camera lens, and the distance formed between the bottom edge and the bottom of the crack.

[0122] We now describe how to estimate the parameters of the crack using the geometric parameters of the mobile phone 104 and the lens parameters.

[0123] The image of the crack is captured in accordance with the above procedure, in which the mobile phone 104 is rotated until the crack 102 is at the center of the field of view of the camera 106.

[0124] This enables a right triangle to be formed by the z-axis of the camera lens, the distance formed between the bottom edge and the bottom of the crack, and the length between the bottom edge and the camera lens.

[0125] We refer to Figure 8 to describe how to estimate the parameters of the crack using the geometric shape of the mobile phone 104 and the lens parameters.

[0126] In step S800, the crack analysis module 112 obtains from the device parameters database 116 the distance between the bottom edge of the mobile phone 104 and the bottom of the crack (i.e. the length of the mobile phone 104), and the length between the bottom edge of the mobile phone 104 and the camera lens. The angle of rotation of the mobile phone 104 can then be calculated in step S802 using the cosine relationship between the distance between the bottom edge and the bottom of the crack and the length between the bottom edge and the camera lens.

[0127] We then need to use the camera parameters to derive a plane-to-plane homographic mapping between the pixels of the camera and the real-world spatial dimensions of the image. The plane-to-plane homographic mapping routine is then called from the library of routines 114 in step S804 to derive the real-world spatial dimensions of the image.

[0128] The derivation of the homographic mapping to provide the real-world spatial dimensions of the image is based on a "pinhole camera model", in which the camera is considered to be a rectangular-based cone whose view region is unrolled with respect to the lens of the camera 106, as shown in Figure 9 a and 9b.

[0129] Figure 9 a is for illustration only and shows the mobile phone 104 positioned directly above the windscreen 100. That is, the camera 106 provides an overhead view of the windscreen 100. In this case, the field of view region Al is rectangular and each pixel occupies the same amount of real-world space (in millimetres).

[0130] In this case, as shown in Figure 9 b, the mobile phone 104 is at an angle with respect to the windscreen 100. This angle is calculated in step S802. The field of view region A2 then becomes a trapezium, which means that the pixels close to the camera represent less millimetres than the pixels further away.

[0131] We have described the theoretical basis for how the plane-to-plane homographic mapping is derived, but it will be appreciated that this will be implemented using routines in numerical form using the library of routines 114, which will be made available to the crack analysis module 112.

[0132] Consider a rectangular image sensor forming part of the camera 106 and the sensor tilted by a rotation angle Θ from the plane, the region observed by the sensor maps to an isosceles trapezium. The base width of this trapezium depends directly on Θ. Using the plane-to-plane homographic mapping routine, we can use this principle to make numerical estimates of the parameters of the crack 102 using the knowledge of the pixels on the camera 106.

[0133] We define a 3D rotation matrix about the x-axis as a function of Θ as follows:

[0134]

[0135] It will be appreciated that Θ is the angle of the mobile phone 104 relative to the windshield. We can define the origin of the Cartesian x, y and z dimensions at (0, 0, 0), i.e. the world origin. This is the point in the middle of the bottom edge of the mobile phone 104 aligned with the x-axis. The y-axis of this coordinate system then points vertically from the bottom to the top of the phone. For simplicity without loss of generality, if we assume that the camera is located on the y-axis, a distance d from the bottom of the phone c then the camera center is defined as:

[0136]

[0137] Then, in step S806, the focal length of the lens of the camera 106 and the dimensions of the vertical and horizontal sensors can be obtained from the device parameters database 116. These parameters can be referred to as the chip parameters. This enables us to compute the field of view region from the camera. The field of view region is defined by two quantities, referred to as the horizontal and vertical view angles (denoted αΗ and αν respectively), defined by the following equations:

[0138]

[0139]

[0140] where S x and S y are the dimensions of the horizontal and vertical sensors, and f is the focal length.

[0141] Step S808, after the horizontal and vertical view angles have been computed, the crack analysis module 112 uses a plane-to-plane homography mapping routine to compute the edges of the field of view pyramid to provide us with the field of view on the windshield 100. This provides Figure 9 the trapezoid shown in b, i.e. the trapezoid that we need to correct to compensate for the different amount of space occupied by pixels further from the lens relative to pixels closer to the lens. That is, we need to scale the trapezoid to ensure that the ongoing calculations assign equal amounts of real world space to each pixel.

[0142] This is modeled by a line, i.e. a ray, in the plane-to-plane homography mapping routine used by the crack analysis module 112, which extends from the lens along the line of sight between the lens and the crack 102. This line will intersect the plane represented by the windshield, i.e. the plane-to-plane homography mapping routine models a plane.

[0143] In step S810, the plane-to-plane homography routine calls a numerical solver routine from the routine library 114 to solve the simultaneous equations defining the plane of the windshield and the line extending from the lens along the line of sight between the lens and the crack 102. The plane-to-plane homography routine is programmed assuming that the plane defining the windshield 100 is flat and that the camera 106 is tilted with respect to that plane. This provides the intersection between the line extending from the lens along the line of sight and the plane of the windshield 100.

[0144] In theory, the above can be expressed as a calculation of rays radiating from the center point of the camera, through the various angles of the sensor / image plane onto the windshield, forming the above trapezoid.

[0145] We first obtain the intersection of the rays with the plane that is parallel to the image plane at a unit distance, given the horizontal and vertical angles of view aH and aV as described above.

[0146] There are four rays, one for each corner of the rectangular sensor. The minimum and maximum x values can be defined as:

[0147]

[0148]

[0149] Similarly, we can define the minimum and maximum y values as:

[0150]

[0151]

[0152] We can then define the corners of the rectangular sensor as:

[0153]

[0154] Normalizing these coordinates by their magnitude gives us the direction of the four rays. We define the direction of the rays for each coordinate as:

[0155]

[0156] If we assume that the phone is rotated by an angle of 0 in the x-axis, we can calculate the position of the camera center as:

[0157]

[0158] This allows us to define the direction of the rays as:

[0159]

[0160] This gives us a ray in Cartesian coordinates with a known point of intersection with a plane parallel to the image plane, and we know that this intersection happens only once. This gives us a trapezoid that indicates the field of view in the real world.

[0161] We define the corners of the trapezoid as:

[0162] V i ,i∈(tl,tr,br,bl)

[0163] We calculate the vertices of the trapezoid using the line plane intersection formula described in [3].

[0164] We know that the normal to the windshield plane is the vector n = (0, 0, -1) at the world origin, which means that the intersection formula simplifies to:

[0165]

[0166]

[0167] where the points V i ,i∈(tl,tr,br,bl) are the vertices of the trapezoid, we need to define the homography H from the image plane to the plane in the real world using the four-point correspondence technique between the trapezoid vertices and the image coordinates:

[0168] u tl =(0,0) T

[0169] u tr =(h,0) T

[0170] u br =(h,ω) T

[0171] u bl =(0,ω) T

[0172] where w is the width of the image and h is the height of the image. The algorithm involved in obtaining this homography is discussed in [4].

[0173] The height of the camera above the windshield can be calculated by the crack analysis module 112 using Pythagoras theorem, since the distance between the bottom edge of the mobile phone 104 and the base of the crack (i.e. the length of the mobile phone 104) and the length between the bottom edge of the mobile phone 104 and the camera lens have already been obtained from the device parameters database 116 and are still in the memory of the crack analysis module 112 in step S800.

[0174] The output of step S810 is a trapezoidal view in the real world (X1, X2, X3, X4). The comparison between the parameters (X1, X2, X3, X4) and the respective corners of the image captured on the windshield (performed by the crack analysis module 112 in step S812) provides the scaling needed to map the position of the pixels of the camera 106 to the position in millimeters on the field of view on the windshield 100. This gives us a plane-to-plane homography mapping. The scaling is in the form of a 3x3 matrix representing the scale, rotation, skew and translation values between the field of view of the camera and the windshield 100.

[0175] The plane-to-plane homography mapping corrects for the effect of the angle of view on the captured image, and the conversion from pixel dimensions to millimeters enables the crack analysis module 112 to derive the dimensional parameters of the crack 102.

[0176] The plane-to-plane homography mapping is a matrix that maps the two-dimensional image plane of the camera 106 to a plane representing the windshield.

[0177] The output of the plane-to-plane homography mapping provides an orthorectified mask in millimeters indicating the position and shape of the crack.

[0178] In response to this output of the plane-to-plane homography mapping, as understood to be the output of the plane-to-plane homography routine, the crack analysis module 112 calls a convex hull computation routine from the routine library 114. The positions in millimeters on the field of view of the windshield are provided to the convex hull computation routine from the routine library 114.

[0179] In summary, the convex hull is a space that covers every position in millimeters on the field of view. The output of the convex hull computation routine is data that can be simply represented as a “binary large object” (blob) that has the same dimensions as the detected crack 102. This enables the use of the blob to perform analysis on the detected crack 102.

[0180] The crack analysis module 112 then calls a minimum circle routine from the routine library 114, which implements a numerical solution to the minimum circle problem for the convex hull output by the convex hull computation routine. This module outputs the minimum circle that encloses every point in the convex hull, thus providing the minimum radius for the crack 102.

[0181] The data representing the convex hull, the data representing the solution to the minimum circle problem for the convex hull and the computed crack radius are respectively stored in memory by the crack analysis module 112, either in the local memory of the processing module 108, or remote from the processing module 108.

[0182] That is, the crack analysis module 112 has used the geometric parameters of the mobile phone 104 and the parameters of the camera 106 to generate the radius of the crack 102.

[0183] Then, in step S814, the parameters and the circle output by the minimum circle routine can be displayed using the display module 118.

[0184] An example image that can be provided by the display module 118 is shown in Figure 11 In this case, the diameter of the minimum circle is indicated as 16 mm, i.e. a radius of 8 mm. The maximum crack diameter estimated in this case is 16 mm. The effect here is that the minimum size of the crack is estimated and that the minimum size of the crack can be used to determine the necessity of replacing the windscreen.

[0185] The estimated radius can be compared by the crack analysis module 112 to a replacement / repair threshold to determine whether the crack 102 needs to be replaced or whether a repair only is sufficient.

[0186] The housing present on the mobile phone 104 can introduce an error in the measured parameters as this will increase the length of the mobile phone 104, but the error is typically about 3%. A 3% error margin is built into the calculations of the crack analysis module 112 and provided on the display by the display module 118.

[0187] The distance between the bottom of the mobile phone 104 and the camera 106 can also not be available from the device parameters database. In this case, we can estimate the parameter to improve the robustness of the described method.

[0188] When capturing the image of the crack 102, we can use the inclinometer built into the mobile phone 104 to obtain the angle of the mobile phone. This can be used to calculate the height h, based on the following equation:

[0189] h = l * sin(0)

[0190] where l is the length of the mobile phone 104 and the angle 0 is the angle obtained from the inclinometer.

[0191] Similarly, the phone angle can be estimated using the angle of view, the image resolution and the sensor size.

[0192] As described for the first technique, in a preferred embodiment, the image processing module 108 is implemented as a software build downloaded to the mobile phone. This can be achieved by downloading the software build as an app. The software build is able to process the image data from the camera to determine whether the crack is amenable to repair or whether replacing the glass window pane is the preferred solution.

[0193] The crack analysis module 112 can be a software build downloaded to the mobile phone, preferably as a single download in combination with the image processing module 108. The single download software build I is preferably used to process image data from the phone camera and analyze the crack using one or more algorithms performed in the software.

[0194] The crack analysis module 112 can be run to provide an alert on the display as to whether a complete windshield replacement is required based on the radius of the smallest circle. If the crack 102 is above a specified threshold, the crack analysis module 112 will indicate that a windshield replacement is required, otherwise it is not. The alert can be displayed on the display of the mobile phone 104.

[0195] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the application as defined by the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim or the whole specification. The word 'comprises' means 'includes or consists of' and 'comprising' means 'including or consisting of'. A singular reference of an element does not exclude the presence of plural elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain methods are recited in mutually different dependent claims does not indicate that a combination of these methods cannot be used to advantage.

[0196] Reference is made to:

[0197] [1] http: / / www.mathworks.com / tagteam / 64199 91822v00eddins final.pdf

[0198] [2] http: / / en.wikpedia.org / wiki / closing(morphology)

[0199] [3] Joseph O'Rourke "Computational Geometry in C", Cambridge University Press, 2012

[0200] [4] Richard Hartley and Andrew Zisserman“Multiple View Geometry in Computer Vision”, Cambridge University Press, 2011.

Claims

1. A method for analyzing cracks in a window panel of a vehicle, the method comprising: Capture images of cracks in the vehicle's window sill; Processing the image of the crack includes: Identify the center of the crack and one or more branches of the crack; Generate a scaling factor, the scaling factor indicating the relative length of one or more branches of the crack compared to the size of the crack's center; and Using the scaling factor, the length of the one or more branches is estimated, wherein, Estimating the length of one or more branches of the crack includes multiplying the scaling factor by a predetermined value. The predetermined value is a predetermined estimate of the actual size of the center of the crack, wherein the size is the diameter, width, and / or length of the center of the crack.

2. The method of claim 1, wherein the estimated length of one or more branches is used to indicate the size of the crack, and the size of the crack is compared with a threshold parameter to determine whether the window panel needs to be replaced or repaired.

3. The method according to claim 2, wherein the step of determining that the glass window panel needs to be replaced or repaired includes determining whether the estimated length of the one or more branches exceeds a given threshold.

4. The method of claim 1, wherein the image processing comprises filtering the image and removing background portions to identify the crack.

5. The method of claim 4, wherein filtering the image comprises applying morphological refinement to the image.

6. The method of claim 3, further comprising an output signal indicating whether the windshield needs repair or replacement.

Images