Windshield damage detection system

Abstract

Image 25 of windshield 80 comprises damage sites, each having a position in the image. In an aspect, a software application operates a camera to capture the image and provides user interface 64 for a

Description

Field The technology relates to the field of automotive maintenance and repair, specifically focusing on the assessment and decision-making process for vehicle windshield damage, utilizing computer vision and machine learning algorithms for accurate detection and classification of damages. Background Vehicle damage detection systems play a crucial role in identifying and assessing damages to various parts of a vehicle, including the windshield. Windshields are particularly susceptible to damages such as chips, cracks, and combinations thereof. There are different type ofchips, such as surface chip, bullseye, star, straight line, half moon, angel wings, star burst, extended ray on bullseye, etc. Accurate and efficient detection of windshield damages is essential for determining whether a repair or replacement is necessary, which has significant implications for the vehicle owner, insurance companies, and the environment. In recent years, there have been several attempts to develop methods and systems for detecting and assessing windshield damages. Some of these attempts have employed computer vision-based artificial intelligence (Al) damage detection services that analyze images of windshields to identify and classify damages. However, these systems often struggle to accurately locate damages and their types due to the presence of reflections, dirt, and imperfections on the windshield, as well as the transparent nature of the glass. Additionally, the response rate for repairability decisions may not be instant, because accuracy of such general ML models are fairly low and to increase the accuracy for acceptable level for their customers, they have large amount of annotators, who will review each prediction the ML model made, making it actually manual labor (not Al) thus being slow and causing delays &human errors. Another approach to windshield damage detection involves the use of pattern matching algorithms, where the size of the damage is compared to a reference object, such as a 2-euro coin. This method, however, is limited by the availability of a suitable reference object during the inspection process and may not provide an accurate assessment of the damage size. These shortcomings in the prior art have led to several problems in the field of vehicle damage detection systems. Firstly, the low detection rate of glass damages results in missed opportunities for repair and unnecessary windshield replacements. This contributes to higher economic costs for vehicle owners and insurance companies, as well as increased CO2 emissions due to the manufacturing and disposal of replacement windshields. Secondly, the underutilization of repairable glass damages is a significant issue, as a very small percentage of glass damages are actually repaired. Improved detection and assessment methods could potentially increase the rate of repairs, reducing the need for costly replacements and the associated environmental impact. In light of these problems, there is a need for an improved method and system for detecting and assessing windshield damages that can accurately identify and classify damages, determine their repairability, and guide the decision-making process between repair and replacement. Such a system would address the shortcomings of the prior art and provide significant benefits to vehicle owners, insurance companies, and the environment. Summary According to a first aspect of the disclosure, a method of determining damage to a vehicle windshield comprises providing an image of a windshield comprising one or more damage sites, providing a position of each damage site in the image, processing the image to determine a repairable area of the windshield, determining damage criteria for the one or more damage sites on the windshield, the damage criteria comprising a type of damage of the damage site and a location of the damage site relative to the repairable area, and determining a repair or replacement decision for the windshield in dependence on the damage criteria. This method allows for an objective and standardized assessment of windshield damage, reducing human error and inconsistencies in repair decisions. Optionally in some examples, the method further comprises detecting a windshield segment in the image. This step helps in accurately identifying the windshield area within the image, improving the overall accuracy of the damage assessment process. Optionally in some examples, determining the repairable area comprises subtracting a non-repairable area from the windshield segment. This approach ensures that critical areas of the windshield are not mistakenly considered for repair, enhancing safety and compliance with regulations. Optionally in some examples, the non-repairable area comprises at least one of an area within a distance from the edges of the windshield, a field of view of a driver, a visibility zone for Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield, and a zone surrounding any installed heads-up display. By defining these specific non-repairable areas, the method ensures that repairs do not compromise the structural integrity or functionality of the windshield. Optionally in some examples, the method further comprises calculating a midpoint of a top windshield X-axis to determine a position of ADAS sensors. This calculation helps in accurately identifying and protecting the ADAS sensor zones, ensuring the continued proper functioning of these critical safety systems. Optionally in some examples, the method further comprises compensating for image distortion by calculating a top and bottom width of the windshield to determine a change in X-axis based on a Y-axis distance of the top and bottom windshield. This compensation improves the accuracy of damage site localization, especially in images taken at an angle. Optionally in some examples, the method further comprises providing a close-up image of each damage site using the close-up image. This additional step allows for a more detailed assessment of each damage site, improving the accuracy of damage type classification. Optionally in some examples, the method further comprises matching objects in the close-up image to the image to determine a size and location of the damage site on the image. This matching process enhances the precision of damage site localization and size estimation. Optionally in some examples, the method further comprises determining a safe zone around each damage site, the safe zone being an area at least 3 cm from the edges of the damage site. This safe zone helps prevent unintended damage during repair processes and ensures the structural integrity of the windshield around the damage site. Optionally in some examples, the safe zone is included in the non-repairable area. This inclusion further refines the repairable area, reducing the risk of attempting repairs in potentially compromised areas of the windshield. Optionally in some examples, the method further comprises counting a number of damage sites with a specific type of damage. This counting feature allows for a more comprehensive assessment of overall windshield condition and aids in making more informed repair or replacement decisions. Optionally in some examples, the repair decision for the windshield is "repair" if the type of damage is a chip, the damage site is located in the repairable area, the damage site is not located in the non-repairable area, and the size of the chip is smaller than a specific size. These specific criteria provide a clear and objective basis for repair decisions, ensuring consistency across assessments. Optionally in some examples, the repair decision for the windshield is "replace" if the number of damage sites with the specific type of damage is greater than a predetermined threshold. This threshold-based decision helps in identifying windshields with extensive damage that would be more cost-effective to replace than repair. Optionally in some examples, the repair decision for the windshield is "replace" if there are any types of damage other than a chip. This rule simplifies decision-making for more complex types of damage, ensuring that potentially unsafe windshields are replaced rather than repaired. Optionally in some examples, the method further comprisesallowing a user to indicate the position of each damage site in the image using a user interface. This user input feature allows for human verification and correction of automatically detected damage sites, improving the overall accuracy of the assessment. Optionally in some examples, the method further comprises comparing the detected type of damage to the type of damage indicated by the user for the damage site using the user interface. This comparison helps in validating the automatic damage type classification and can be used to improve the system's accuracy over time. Optionally in some examples, the method further comprises allowing a user to book an appointment at a glass workshop based on the repair or replacement decision for the windshield. This integrated booking feature streamlines the process for users, increasing the likelihood of timely repairs or replacements. According to a second aspect of the disclosure, a software application for a mobile phone comprises operating a camera to capture the image of a windshield comprising one or more damage sites, providing a user interface for a user to indicate the position of each damage site in the image, processing the image to determine a repairable area of the windshield, determining damage criteria for the one or more damage sites on the windshield, the damage criteria comprising a type of damage of the damage site and a location of the damage site relative to the repairable area, and determining a repair or replacement decision for the windshield in dependence on the damage criteria. This mobile application makes the windshield damage assessment process more accessible and convenient for users, potentially leading to more frequent and timely assessments and repairs. Brief Description of the Drawings Examples are described in more detail below with reference to the appended drawings. Figure 1 is a schematic representation of a user device capturing an image of a vehicle windshield. Figures 2a and 2b are views of the user interface of the software application, showing the damage indicator. Figure 3 is a representation of the image processing steps, including windshield segmentation and damage determination. Figures 4a-4c show the windshield, highlighting the repairable and non-repairable areas. Figure 5 is a flowchart illustrating the method of determining windshield damage. Figure 6 is a schematic representation of the decision-making process for windshield repair or replacement. Detailed Description The detailed description set forth below provides information and examples of the disclosed technology with sufficient detail to enable those skilled in the art to practice the disclosure. Figure 1 shows a schematic representation of a user device 10 capturing an image of a vehicle windshield 80. The user device 10, which can be a mobile phone, is equipped with a camera 20. The camera 20 is used to produce an image 25 of the windshield 80. The image 25 captures a detailed view of the entire windshield 80. The camera 20 can also produce a close-up image 26 of the windshield 80, which captures a detailed view of a specific damage site 90 on the windshield 80. The user device 10 also includes a processor 30, which processes the image 25 from the camera 20 to analyze the windshield 80. The user device 10 also includes a touchscreen 40, which allows users to select areas of the image 25 using a software application 60 to indicate the position of the damage site 90. The user device 10 also includes a wireless communication module 50, which enables data transfer between the user device 10 and external systems via wireless networks. Figure 2a shows a view of the user interface 64 of the software application 60, showing the damage indicator. The user interface 64 includes instructions to the user for capturing high-quality photos of the windshield 80, image quality validation, and a damage indicator. Figure 2b shows close-up image 26. The damage indicator 27 allows users to mark the general location of the damage site 90 of the windshield 80 in the image 25. Figure 3 shows a representation of the image processing steps, including windshield segmentation and damage determination. The windshield segmentation model 66 detects the windshield segment 67, repairable area 68, and non-repairable area 69 on the image 25. The windshield damage determination model determines a set of damage criteria for one or more damage site 90 on the windshield 80 using the image 25. Figure 4a shows a depiction of image 25, showing the windshield segment 67 of windshield 80. Figure 4b shows image 25 with detected damage site 90, and nonrepairable area(s) 69. Figure 4c shows the repairable area 68. The repairable area 68 identifies areas of the windshield 80 where damage site 90 can be repaired. The nonrepairable area 69 identifies areas of the windshield 80 where damage site 90 cannot be repaired. Figure 5 shows a flowchart illustrating the method of determining windshield damage. The method includes steps such as receiving the user's initiation of inspection, providing the image 25 of the windshield 80, validating the quality of the captured image 25, providing the position of each damage site 90 in the image 25, processing the image 25 to detect the windshield segment 67, determining a repairable area 68 for the image 25, determining non-repairable area 69 for the image 25, detecting the damage criteria for the one or more damage site 90 on the windshield 80, and determining a repair or replacement decision for the windshield 80 in dependence on the damage criteria. Figure 6 shows a schematic representation of the decision-making process for windshield repair or replacement. The repair decision calculator determines whether the windshield 80 should be repaired or replaced based on the detected damage site 90 and their locations. The decision-making process includes counting the number of damage site 90 with the type “chip”, determining if the type of damage site 90 is a chip and is located in the repairable area 68, and not located in the non-repairable area 69, and if the chips are smaller than a certain size, then the repair decision for the windshield 80 is "repair". If the damage count is greater than a certain number then the repair decision for the windshield 80 is "replace". If there are any types of damage other than “chip”, the repair decision for the windshield 80 is "replace". 1. Vehicle Details The vehicle 70 can be a passenger car, a van, a motorcycle, a bus, or any other type of vehicle that includes a windshield 80. The vehicle 70 can be a new or used vehicle, and it can be a vehicle that is currently in use or a vehicle that is being inspected for potential purchase or sale. The vehicle 70 can be a vehicle that is being inspected as part of a routine maintenance check, a vehicle that has been involved in an accident, or a vehicle that is being inspected for any other reason. The vehicle 70 can be located in any suitable location, such as a garage, a parking lot, a driveway, a street, etc. 1.1. Windshield The windshield 80 is a component of the vehicle 70 that provides a barrier between the interior of the vehicle 70 and the exterior environment. The windshield 80 can be made of any suitable material, such as glass or a glass-like material, and it can be transparent or semi-transparent to allow the driver of the vehicle 70 to see through it. The windshield 80 can be flat or curved, and it can have any suitable shape or size. The windshield 80 can include one or more damage sites 90, which are areas of the windshield 80 that have been damaged in some way. 1.1.1. Damage Site The damage site 90 is an area of the windshield 80 that has been damaged. The damage site 90 can be any area of the windshield 80 that has been damaged in any way, such as by a chip, a crack, ora combination of these, or any other type of damage. There may be multiple damage sites 90 on the windshield 80, and each damage site 90 can be separate from the other damage sites 90, or they can overlap or intersect with each other. The damage site 90 can be located anywhere on the windshield 80, such as in the center of the windshield 80, near the edges of the windshield 80, or anywhere in between. 1.1.1.1. Size Of Damage In one implementation, the size of the damage at the damage site 90 is a factor that can be considered when determining whether the windshield 80 should be repaired or replaced. The size of the damage can be determined by measuring the diameter of the damage site 90, or by using any other suitable method for determining the size of the damage. The size of the damage can be a factor that is considered when determining whether the damage site 90 is repairable or not. For example, if the size of the damage is larger than a certain threshold, the damage site 90 may not be repairable, and the windshield 80 may need to be replaced. Conversely, if the size of the damage is smaller than a certain threshold, the damage site 90 may be repairable, and the windshield 80 may not need to be replaced. The threshold for determining whether the size of the damage is repairable or not can be determined based on various factors, such as the type of damage, the location of the damage site 90 on the windshield 80, and other factors. 1.1.1.2. Type Of Damage In some examples, the type of damage at the damage site 90 is another factor that can be considered when determining whether the windshield 80 should be repaired or replaced. The type of damage can include a chips, cracks, and combinations thereof. There are different type ofchips, such as surface chip, bullseye, star, straight line, half moon, angel wings, star burst, extended ray on bullseye, etc. The type of damage can be a factor that is considered when determining whether the damage site 90 is repairable or not. For example, certain types of damage, such as chips or small cracks, may be repairable, while other types of damage, such as large cracks, may not be repairable and may require the windshield 80 to be replaced. 2. User Device Details The present disclosure describes the use of a user device 10, which can be any type of device that is capable of capturing an image 25 of the windshield 80 and processing the image 25 to analyze the windshield 80. The user device 10 can be a mobile phone, a tablet, a laptop, a desktop computer, a digital camera, or any other type of device that includes a camera 20 and a processor 30. The user device 10 can also include a touchscreen 40, which allows users to interact with the user device 10 and to select areas of the image 25 using a software application 60. The user device 10 can also include a wireless communication module 50, which enables data transfer between the user device 10 and external systems via wireless networks, including 4G &5G networks, etc. 2.1. Software Application In one example, the software application 60 is a program or set of programs that is installed on or accessed via a webbrowser via the user device 10 and that facilitates user-reported assessment and automated decision-making for windshield damage repair or replacement. The software application 60 can include various components, such as a user interface 64, a windshield segmentation model 66, a windshield damage determination model, a repair decision calculator, and other components. The software application 60 can be a mobile application that is installed and used on e.g. a mobile phone or a tablet, or it can be a web-based application that is designed to be used on e.g. a mobile phone, tablet, laptop ora desktop computer. Where it is installed, the software application 60 can be downloaded and installed on the user device 10 from an application store or a website, or it can be pre-installed on the user device 10. 2.1.1. User Interface In some implementations, the user interface 64 is a component of the software application 60 that allows users to interact with the software application 60 and to provide input to the software application 60. The user interface 64 can provide instructions to the user for capturing high-quality photos of the windshield 80, and it can validate the quality of the captured images using Al models. The user interface 64 can also include a damage indicator, which allows users to mark the general location of the damage site 90 on the windshield 80 in the image 25. The user interface 64 can also provide instructions to the user for capturing a close-up photo of the damage site 90, and it can include a booking tool, which enables users to book a time at a glass workshop after receiving a repair decision. 2.1.1.1. Instructions To User For Capturing High Quality Photos Of The Windshield In one implementation, the user interface 64 provides instructions to the user for capturing high-quality photos of the windshield 80. These instructions can be provided in various forms, such as text, images, videos, or any other suitable form. The instructions can guide the user on how to position the camera 20, how to focus the camera 20, how to adjust the lighting conditions, how to clean the windshield 80 before taking the photo, and other aspects of capturing a high-quality photo of the windshield 80. The instructions can also guide the user on how to capture a photo of the entire windshield 80, including all edges and corners of the windshield 80. The instructions can be provided in a step-by-step manner, and they can be interactive, allowing the user to proceed to the next step only after successfully completing the current step. 2.1.1.2. Image Quality Validation In some configurations, the user interface 64 includes an image quality validation feature, which validates the quality of the captured image 25. The image quality validation feature can check various aspects of the image 25, such as whether the windshield 80 is centered in the frame, whether the car is in the expected pose, whether the lighting conditions are adequate for clear visibility of the damage site 90, whether the image 25 contains reflections or glare that could obscure the damage site 90, whether the resolution is high enough to detect small damage site 90 such as chips, whether the windshield 80 is clean and free from obstructions like dirt or stickers, whether the angle of the shot provides an unobstructed view of the windshield 80, whether the entire windshield 80 is visible and not partially cropped out of the frame, and whether the captured image 25 is not blurry, ensuring sharpness and clarity necessary for damage site 90 detection. 2.1.1.3. Damage Indicator In one example, the user interface 64 includes a damage indicator, which allows users to mark the general location of the damage site 90 on the windshield 80 in the image 25. The damage indicator can be a tool or feature that allows users to draw, mark, or otherwise indicate the location of the damage site 90 on the image 25. The damage indicator can allow users to mark multiple damage sites 90 on the same image 25, and it can allow users to adjust the size, shape, and position of the marked damage site 90. The damage indicator can also allow users to provide additional information about the damage site 90, such as the type of damage, the size of the damage, and other information. 2.1.1.4. Instructions To User For Capturing Close-Up Photo In some implementations, the user interface 64 provides instructions to the user for capturing a close-up photo of the damage site 90. These instructions can guide the user on how to position the camera 20, how to focus the camera 20, how to adjust the lighting conditions, how to clean the area around the damage site 90 before taking the photo, and other aspects of capturing a high-quality close-up photo of the damage site 90. The instructions can also guide the user on how to capture a photo that includes the entire damage site 90 and a sufficient area around the damage site 90. The instructions can be provided in a step-by-step manner, and they can be interactive, allowing the user to proceed to the next step only after successfully completing the current step. 2.1.1.5. Booking Tool In one configuration, the user interface 64 includes a booking tool, which enables users to book a time at a glass workshop after receiving a repair decision. The booking tool can allow users to select a date and time for the appointment, and it can allow users to select a location for the appointment. The booking tool can also provide information about the availability of appointments at the selected date, time, and location. The booking tool can also allow users to provide additional information, such as their contact information, the make and model of their vehicle 70, and other information. The booking tool can also provide a confirmation of the booking, and it can provide reminders or notifications about the upcoming appointment. 2.1.2. Windshield Segmentation Model In some examples, the software application 60 includes a windshield segmentation model 66. The windshield segmentation model 66 is a computer vision machine learning model that is used to detect the windshield segment 67, the repairable area 68, and the non-repairable area 69 on the image 25. The windshield segmentation model 66 can use various types of machine learning algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), support vector machines (SVMs), decision trees, random forests, gradient boosting machines, k-nearest neighbors (KNN), Bayesian networks, principal component analysis (PCA), feature engineering techniques, hyperparameter tuning methods, and other types of machine learning algorithms. The windshield segmentation model 66 can also use various types of mathematical algorithms to smoothen the detected area and increase the accuracy of the segmentation, such as linear regression and other types of mathematical algorithms. 2.1.2.1. Windshield Segment In one implementation, the windshield segment 67 is a part of the image 25 that is identified by the windshield segmentation model 66 as representing the windshield 80. The windshield segment 67 can include the entire windshield 80, or it can include a portion of the windshield 80. The windshield segment 67 can be identified based on various features of the image 25, such as the color, texture, shape, size, location, and other features of the windshield 80 in the image 25. The windshield segment 67 can be represented in the image 25 as a region of interest (ROI), a mask, a bounding box, a contour, or any other suitable representation. 2.1.2.2. Non-Repairable Area In some configurations, the windshield segmentation model 66 also determines the non-repairable area 69 of the windshield 80. The non-repairable area 69 represents the areas of the windshield 80 where a damage site 90 cannot be repaired due to various factors such as proximity to the edges of the windshield 80, location within the driver's field of view, or presence of Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield 80. The non-repairable area 69 is subtracted from the windshield segment 67 to accurately define the repairable area 68. The model 66 uses a combination of rules and algorithms to identify the non-repairable area 69. For example, any area within a certain distance from the edges of the windshield 80, such as 6cm, may be marked as non-repairable. The model 66 may also calculate the midpoint of the top windshield 80 x-axis to determine the position of ADAS sensors and mark this area as a non-repairable zone. Other non-repairable zones may include the driver's field of view and zones surrounding any installed heads-up display. All these non-repairable zones are combined into a unified non-repairable area 69 mask for the image 25. The model 66 may also compensate for image distortion by calculating the top and bottom width of the windshield 80 to get the change in x-axis based on the y-axis distance of the top and bottom windshield. 2.1.2.3. Repairable Area In one example, the repairable area 68 is a part of the windshield segment 67 that is identified by the windshield segmentation model 66 as representing an area of the windshield 80 where a damage site 90 can be repaired. The repairable area 68 can be determined by subtracting the non-repairable area 69 from the windshield segment 67. The repairable area 68 can include various zones on the windshield 80, such as an area that is greater than a certain distance from the edges of the windshield 80, an area that is not within the field of view of the driver, an area that is not within the visibility zone for Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield 80, an area that is not surrounding any installed heads-up display, and other zones. The repairable area 68 can be identified based on various factors, such as the location of the damage site 90 on the windshield 80, the type of damage at the damage site 90, the size of the damage at the damage site 90, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other factors. The repairable area 68 can be represented in the image 25 as a region of interest (ROI), a mask, a bounding box, a contour, or any other suitable representation. 2.1.3. Windshield Damage Determination Model In some implementations, the software application 60 includes a windshield damage determination model. The windshield damage determination model is a machine learning model that is used to determine a set of damage criteria for one or more damage sites 90 on the windshield 80 using the image 25. The windshield damage determination model can use various types of machine learning algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), support vector machines (SVMs), decision trees, random forests, gradient boosting machines, k-nearest neighbors (KNN), Bayesian networks, principal component analysis (PCA), feature engineering techniques, hyperparameter tuning methods, and other types of machine learning algorithms. 2.1.3.1. Damage Criteria In one configuration, the damage criteria are a set of criteria that are used to assess the damage at the damage site 90. The damage criteria can include various factors, such as the type of damage at the damage site 90, the size of the damage at the damage site 90, the location of the damage site 90 on the windshield 80, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other factors. The damage criteria can be determined based on the image 25, the close-up image 26, the user's input, and other sources of information. The damage criteria can be used to determine whether the windshield 80 should be repaired or replaced. 2.1.3.2. Windshield Damage Segmentation Model In some examples, the windshield damage determination model includes a windshield damage segmentation model. The windshield damage segmentation model is a machine learning model that is used to detect the type of damage at the damage site 90, to determine if the damage site 90 is fully within the repairable area 68, and to compare the detected type of damage to the type of damage indicated by the user for the damage site 90 using the damage indicator. The windshield damage segmentation model can use various types of machine learning algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), support vector machines (SVMs), decision trees, random forests, gradient boosting machines, k-nearest neighbors (KNN), Bayesian networks, principal component analysis (PCA), feature engineering techniques, hyperparameter tuning methods, and other types of machine learning algorithms. The windshield damage segmentation model can improve the accuracy of damage detection and assessment by ensuring that the detected type of damage matches the user's input. 2.1.3.3. Windshield Damage Characteriser In some configurations, the windshield damage determination model includes a windshield damage characteriser. The windshield damage characteriser is a feature that is used to estimate the size of the damage at each damage site 90 using the closeup image 26. The windshield damage characteriser can improve the accuracy of damage detection and assessment by identifying whether the damage is within the repairable size range, which can guide the decision between repair and replacement. 2.1.3.4. Distance Calculation In one implementation, the windshield damage determination model includes a distance calculation feature. The distance calculation feature is a feature that is used to calculate the distance between multiple damage sites 90. The distance calculation feature can use various methods to calculate the distance, such as by measuring the distance between the centers of the damage sites 90 in the image 25, by using a scale or ruler that is included in the image 25, or by using any other suitable method for calculating the distance. The distance calculation feature can improve the accuracy of damage detection and assessment by identifying whether the damage sites 90 are too close to each other for individual repairs. 2.1.3.4.1. Safe Zone In some examples, the distance calculation feature includes a safe zone calculation feature. The safe zone is an area around each damage site 90 that is considered to be safe for repair. The safe zone can be defined as an area that is a certain distance from the edges of the damage site 90, such as 3 cm from the edges of the damage site 90. The safe zone can be included in the non-repairable area 69, which means that if a damage site 90 is located within the safe zone of another damage site 90, the windshield 80 may not be repairable and may need to be replaced. The safe zone calculation feature can improve the accuracy of damage detection and assessment by identifying groups of damage sites 90 that might make the windshield 80 nonrepairable. 2.1.4. Repair Decision Calculator In some implementations, the software application 60 includes a repair decision calculator. The repair decision calculator is a feature that is used to determine whether the windshield 80 should be repaired or replaced based on the detected damage sites 90 and their locations. The repair decision calculator can use various factors to make the decision, such as the type of damage at the damage sites 90, the size of the damage at the damage sites 90, the location of the damage sites 90 on the windshield 80, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other factors. The repair decision calculator can automate the decision-making process, providing an instant decision and reducing economic costs and CO2 emissions associated with unnecessary replacements. 2.1.4.1. Damage Count In one example, the repair decision calculator includes a damage count feature. The damage count feature is a feature that is used to count the number of damage sites 90 with a specific type of damage, such as a chip. The damage count feature can use various methods to count the damage sites 90, such as by identifying each damage site 90 in the image 25 that has the specific type of damage, by using the user's input, or by using any other suitable method for counting the damage sites 90. The damage count feature can improve the accuracy of damage detection and assessment by helping to determine whether the windshield 80 should be repaired or replaced based on the number of damage sites 90. 2.2. Camera Describing now the required hardware components of the user device 10, the camera 20 is a component of the user device 10 that is used to produce an image 25 of the windshield 80. The camera 20 can be any type of camera that is capable of capturing an image 25, such as a digital camera, a smartphone camera, a tablet camera, a laptop camera, a webcam, or any other type of camera. The camera 20 can have various features, such as a lens, a sensor, a shutter, a flash, a zoom function, a focus function, a resolution setting, a color setting, a brightness setting, a contrast setting, and other features. The camera 20 can also produce a close-up image 26 of the windshield 80, which captures a detailed view of a specific damage site 90 on the windshield 80. 2.2.1. Image In one example, the image 25 is a digital representation of the windshield 80 that is captured by the camera 20. The image 25 can be a still image, a video frame, or any other type of image. The image 25 can be captured in any suitable format, such as a JPEG format, a PNG format, a RAW format, or any other suitable format. The image 25 can be stored in the memory of the user device 10, or it can be transmitted to an external system for processing. The image 25 can capture a detailed view of the entire windshield 80, including all edges and corners of the windshield 80. The image 25 can also capture a view of the surrounding area of the windshield 80, such as the hood of the vehicle 70, the roof of the vehicle 70, the dashboard of the vehicle 70, and other areas. 2.2.2. Close-Up Image In some implementations, the camera 20 can also produce a close-up image 26 of the windshield 80. The close-up image 26 is a digital representation of a specific damage site 90 on the windshield 80 that is captured by the camera 20. The close-up image 26 can be a still image, a video frame, or any other type of image. The close-up image 26 can be captured in any suitable format, such as a JPEG format, a PNG format, a RAW format, or any other suitable format. The close-up image 26 can be stored in the memory of the user device 10, or it can be transmitted to an external system for processing. The close-up image 26 can capture a detailed view of the specific damage site 90, including the edges of the damage site 90 and the surrounding area of the damage site 90. 2.3. Processor In some configurations, the user device 10 includes a processor 30. The processor 30 is a component of the user device 10 that processes the image 25 from the camera 20 to analyze the windshield 80. The processor 30 can be any type of processor that is capable of processing the image 25, such as a microprocessor, a microcontroller, a digital signal processor (DSP), a graphics processing unit (GPU), a neural processing unit (NPU). The processor 30 can execute various types of software, including the software application 60. The processor 30 can also execute various types of algorithms, such as image processing algorithms, machine learning algorithms, artificial intelligence algorithms, or any other type of algorithms. 2.4. Touchscreen In one example, the user device 10 includes a touchscreen 40. The touchscreen 40 is a component of the user device 10 that allows users to interact with the user device 10 and to select areas of the image 25 using the software application 60. The touchscreen 40 can be any type of touchscreen that is capable of detecting the touch of a user's finger or a stylus, such as a capacitive touchscreen, a resistive touchscreen, an infrared touchscreen, or any other type of touchscreen. The touchscreen 40 can display various types of content, such as the image 25, the user interface 64 of the software application 60, a keyboard, a menu, a dialog box, a notification, etc. 2.5. Wireless Communication Module (Wi-Fi, Bluetooth, Cellular) In some configurations, the user device 10 includes a wireless communication module 50. The wireless communication module 50 is a component of the user device 10 that enables data transfer between the user device 10 and external systems via wireless networks. The wireless communication module 50 can be used to transmit the image 25 and the close-up image 26 from the user device 10 to an external system for processing, and it can be used to receive data from the external system, such as the results of the processing, notifications, updates, or any other type of data. 3. Method Of Determining Windshield Damage Method Details An example method of determining windshield damage is now described. 3.1. User Initiation and Image Capture The method begins with the user's initiation of the inspection. The user can initiate the inspection by launching the software application 60 on the user device 10, selecting an option to start the inspection, or performing any other suitable action. Once the inspection is initiated, the user can capture the image 25 of the windshield 80 using the camera 20 of the user device 10. The user can capture the image 25 by pointing the camera 20 at the windshield 80, adjusting the focus and other settings of the camera 20, and triggering the shutter of the camera 20. The user can also follow the instructions provided by the user interface 64 to capture a high-quality image 25 of the windshield 80. 3.1.1. Quality Validation of Captured Image In some configurations, after the image 25 is captured, the quality of the captured image 25 is validated. The quality validation can be performed by the software application 60 using the image quality validation feature of the user interface 64. The image quality validation feature can validate the quality of the image 25 by checking various aspects of the image 25, such as whether the windshield 80 is centered in the frame, whether the car is in the expected pose, whether the lighting conditions are adequate for clear visibility of the damage site 90, whether the image 25 contains reflections or glare that could obscure the damage site 90, whether the resolution is high enough to detect small damage site 90 such as chips, whether the windshield 80 is clean and free from obstructions like dirt or stickers, whether the angle of the shot provides an unobstructed view of the windshield 80, whether the entire windshield 80 is visible and not partially cropped out of the frame, and whether the captured image 25 is not blurry, ensuring sharpness and clarity necessary for damage site 90 detection. 3.2. Damage Site Identification and Analysis In one implementation, after the quality of the image 25 is validated, the damage site 90 on the windshield 80 is identified and analyzed. The damage site 90 can be identified by the user using the damage indicator of the user interface 64, or it can be automatically detected by the software application 60 using the windshield damage determination model. The damage site 90 can be analyzed to determine various damage criteria, such as the type of damage, the size of the damage, the location of the damage site 90 on the windshield 80, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other damage criteria. The analysis of the damage site 90 can be performed by the software application 60 using various features, such as the windshield damage segmentation model, the windshield damage characteriser, the distance calculation feature, and other features. The analysis of the damage site 90 can result in a set of damage criteria that are used to determine whether the windshield 80 should be repaired or replaced. 3.2.1. Close-Up Image Analysis In some examples, the method includes providing a close-up image 26 of each damage site 90. The close-up image 26 can be captured by the user using the camera 20 of the user device 10, following the instructions provided by the user interface 64. The close-up image 26 can capture a detailed view of the specific damage site 90, including the edges of the damage site 90 and the surrounding area of the damage site 90. The close-up image 26 can be used to provide more detailed information about the damage site 90, such as the exact size of the damage, the exact shape of the damage, the exact color of the damage, and other detailed information. The close-up image 26 can also be used to validate the quality of the captured image 25, to match the objects in the close-up image 26 to the objects in the image 25, and to estimate the size of the damage at the damage site 90. 3.3. Windshield Segmentation and Repairable Area Determination Next, the image 25 is processed to detect the windshield segment 67. The windshield segment 67 can be detected by the software application 60 using the windshield segmentation model 66. The windshield segmentation model 66 can use various types of machine learning algorithms and mathematical algorithms to detect the windshield segment 67. The windshield segment 67 can be used to identify the exact outline of the windshield 80 in the image 25. In some implementations, the method includes determining a repairable area 68 for the image 25. The repairable area 68 can be determined by the software application 60 using the windshield segmentation model 66. The repairable area 68 can be defined as the part of the windshield segment 67 where a damage site 90 can be repaired. The repairable area 68 can be determined by subtracting the non-repairable area 69 from the windshield segment 67. 3.3.1. Non-Repairable Area Determination The non-repairable area 69 can be determined by the software application 60 using the windshield segmentation model 66. The non-repairable area 69 can be defined as the part of the windshield segment 67 where a damage site 90 cannot be repaired. The non-repairable area 69 can include various zones on the windshield 80, such as an area within a certain distance from the edges of the windshield 80, a field of view of the driver, a visibility zone for Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield 80, a zone surrounding any installed heads-up display, and other zones. 3.4. Damage Criteria Detection and Analysis Next, damage criteria for the one or more damage site 90 on the windshield 80 is detected. The damage criteria can be detected by the software application 60 using the windshield damage determination model. The damage criteria can include various factors, such as the type of damage at the damage site 90, the size of the damage at the damage site 90, the location of the damage site 90 on the windshield 80, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other factors. 3.4.1. Repair or Replacement Decision Making A repair or replacement decision for the windshield 80 is then made in dependence on the damage criteria. The repair or replacement decision can be determined by the software application 60 using the repair decision calculator. The repair decision calculator uses the type of damage at the damage sites 90, the size of the damage at the damage sites 90, the location of the damage sites 90 on the windshield 80, the number of damage sites 90 on the windshield 80, the distance between the damage sites 90, and other factors. The repair decision calculator can automate the decisionmaking process, reducing economic costs and CO2 emissions associated with unnecessary replacements. Example 1: Single Chip within the Repairable Area Damage Criteria - Type of Damage: Chip - Size of Damage: 2 cm in diameter, within the repairable size threshold of 25.75mm. - Location of Damage: Center of the windshield, away from edges and critical zones - Number of Damage Sites: One Decision-Making Process - Type of Damage Check: The damage is identified as a chip, which is a repairable type of damage. - Size Check: The size of the chip is 2 cm, which is smaller than the specific repairable size threshold (e.g., 25.75mm). - Location Check: The chip is located in the repairable area and is not within any nonrepairable zones such as the field of view of the driver, ADAS sensors, or close to the edges. - Count Check: There is only one chip, meeting the criteria for repairability. Repair Decision - Decision: Repair - Rationale: The chip is small, located in a repairable area, and is the only damage present, making it suitable for repair. Example 2: Multiple Chips with One Larger Than Repairable Size Damage Criteria - Type of Damage: Multiple chips - Sizes of Damage: - Chip 1: 1.5 cm - Chip 2: 4 cm - Chip 3: 2 cm - Location of Damage: All chips are within the repairable area but not within any nonrepairable zones - Number of Damage Sites: Three Decision-Making Process - Type of Damage Check: The damages are identified as chips, which are potentially repairable. - Size Check: - Chip 1 is 1.5 cm, which is repairable. - Chip 2 is 4 cm, which exceeds the repairable size threshold of 3 cm. - Chip 3 is 2 cm, which is repairable. - Location Check: All chips are within the repairable area and not within any nonrepairable zones. - Count Check: The total number of chips is three. Repair Decision - Decision: Replace - Rationale: Although most chips are of repairable size and in repairable areas, Chip 2 exceeds the repairable size threshold, making the overall decision for replacement. Example 3: Crack and Chip in Non-Repairable Area Damage Criteria - Type of Damage: Crack and chip - Sizes of Damage: - Crack: 5 cm in length - Chip: 2 cm, within the repairable size threshold of 25.75mm. - Location of Damage: - Crack: Close to the driver's field of view - Chip: Near the edge of the windshield (within the non-repairable zone) - Number of Damage Sites: Two Decision-Making Process - Type of Damage Check: The damages include both a crack and a chip. - Size Check: - The crack is 5 cm, which is critical and generally non-repairable. - The chip is 2 cm, within the repairable size threshold of 25.75mm. - Location Check: - The crack is in the non-repairable area (driver's field of view). - The chip is also in the non-repairable area (near the edge). - Count Check: There are two damage sites. Repair Decision - Decision: Replace - Rationale: The crack is located within a critical non-repairable area (driver's field of view), and the chip is located near the edge, making the windshield non-repairable and requiring a replacement for safety and compliance. Example 1. A method of determining damage to a vehicle windshield, comprising: providing an image (25) of a windshield (80) comprising one or more damage sites (90); providing a position of each damage site (90) in the image (25); processing the image (25) to determine a repairable area (68) of the windshield (80); determining damage criteria for the one or more damage sites (90) on the windshield (80), the damage criteria comprising: a type of damage of the damage site (90); and a location of the damage site (90) relative to the repairable area (68); and determining a repair or replacement decision for the windshield (80) in dependence on the damage criteria. Example 2. The method according to example 1, further comprising detecting a windshield segment (67) in the image (25). Example 3. The method according to example 2, wherein determining the repairable area (68) comprises subtracting a non-repairable area (69) from the windshield segment (67). Example 4. The method according to example 3, wherein the non-repairable area (69) comprises at least one of: an area within a distance from the edges of the windshield (80); a field of view of a driver; a visibility zone for Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield (80); and a zone surrounding any installed heads-up display. Example 5. The method according to example 4, further comprising calculating a midpoint of a top windshield (80) X-axis to determine a position of ADAS sensors. Example 6. The method according to any one of examples 2 to 5, further comprising compensating for image distortion by calculating a top and bottom width of the windshield (80) to determine a change in X-axis based on a Y-axis distance of the top and bottom windshield. Example 7. The method according to any one of examples 1 to 6, further comprising providing a close-up image (26) of each damage site (90) using the close-up image (26). Example 8. The method according to example 7, further comprising matching objects in the close-up image (26) to the image (25) to determine a size and location of the damage site (90) on the image (25). Example 9. The method according to any one of examples 1 to 8, further comprising determining a safe zone around each damage site (90), the safe zone being an area at least 3 cm from the edges of the damage site (90). Example 10. The method according to example 9, wherein the safe zone is included in the non-repairable area (69). Example 11. The method according to any one of examples 1 to 10, further comprising counting a number of damage sites (90) with a specific type of damage. Example 12. The method according to example 11, wherein the repair decision for the windshield (80) is "repair" if the type of damage is a chip, the damage site (90) is located in the repairable area (68), the damage site (90) is not located in the nonrepairable area (69), and the size of the chip is smaller than a specific size. Example 13. The method according to examples 11 or 12, wherein the repair decision for the windshield (80) is "replace" if the number of damage sites (90) with the specific type of damage is greater than a predetermined threshold. Example 14. The method according to any preceding example, wherein the repair decision for the windshield (80) is "replace" if there are any types of damage other than a chip. Example 15. The method according to any preceding example, further comprising allowing a user to indicate the position of each damage site (90) in the image (25) using a user interface (64). Example 16. The method according to any preceding example, further comprising comparing the detected type of damage to the type of damage indicated by the user for the damage site (90) using the user interface (64). Example 17. The method according to any preceding example, further comprising allowing a user to book an appointment at a glass workshop based on the repair or replacement decision for the windshield (80). Example 18. A software application for a mobile phone, comprising: operating a camera (20) to capture the image (25) of a windshield (80) comprising one or more damage sites (90); providing a user interface (64) for a user to indicate the position of each damage site (90) in the image (25); processing the image (25) to determine a repairable area (68) of the windshield (80); determining damage criteria for the one or more damage sites (90) on the windshield (80), the damage criteria comprising: a type of damage of the damage site (90); and a location of the damage site (90) relative to the repairable area (68); and determining a repair or replacement decision for the windshield (80) in dependence on the damage criteria. The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes," and / or "including" when used herein specify the presence of stated features, integers, actions, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, actions, steps, operations, elements, components, and / or groups thereof. It will be understood that, although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the scope of the present disclosure. Relative terms such as "below" or "above" or "upper" or "lower" or "horizontal" or "vertical" may be used herein to describe a relationship of one element to another element as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. It is to be understood that the present disclosure is not limited to the aspects described above and illustrated in the drawings; rather, the skilled person will recognize that many changes and modifications may be made within the scope of the present disclosure and appended claims. In the drawings and specification, there have been disclosed aspects for purposes of illustration only and not for purposes of limitation, the scope of the disclosure being set forth in the following claims.

Claims

1. A method of determining damage to a vehicle windshield, comprising: providing an image (25) of a windshield (80) comprising one or more damage sites (90);providing a position of each damage site (90) in the image (25);processing the image (25) to determine a repairable area (68) of the windshield (80); determining damage criteria for the one or more damage sites (90) on the windshield (80), the damage criteria comprising:a type of damage of the damage site (90); anda location of the damage site (90) relative to the repairable area (68); and determining a repair or replacement decision for the windshield (80) in dependence on the damage criteria.

2. The method according to claim 1, further comprising detecting a windshield segment (67) in the image (25).

3. The method according to claim 2, wherein determining the repairable area (68) comprises subtracting a non-repairable area (69) from the windshield segment (67).

4. The method according to claim 3, wherein the non-repairable area (69) comprises at least one of:an area within a distance from the edges of the windshield (80);a field of view of a driver;a visibility zone for Advanced Driver Assistance Systems (ADAS) sensors embedded within the windshield (80); anda zone surrounding any installed heads-up display.

5. The method according to claim 4, further comprising calculating a midpoint of a top windshield (80) X-axis to determine a position of ADAS sensors.

6. The method according to any one of claims 2 to 5, further comprising compensating for image distortion by calculating a top and bottom width of the windshield (80) todetermine a change in X-axis based on a Y-axis distance of the top and bottom windshield.

7. The method according to any one of claims 1 to 6, further comprising providing a close-up image (26) of each damage site (90) using the close-up image (26).

8. The method according to claim 7, further comprising matching objects in the closeup image (26) to the image (25) to determine a size and location of the damage site (90) on the image (25).

9. The method according to any one of claims 1 to 8, further comprising determining a safe zone around each damage site (90), the safe zone being an area at least 3 cm from the edges of the damage site (90).

10. The method according to claim 9, wherein the safe zone is included in the nonrepairable area (69).

11. The method according to any one of claims 1 to 10, further comprising counting a number of damage sites (90) with a specific type of damage.

12. The method according to claim 11, wherein the repair decision for the windshield (80) is "repair" if the type of damage is a chip, the damage site (90) is located in the repairable area (68), the damage site (90) is not located in the non-repairable area (69), and the size of the chip is smaller than a specific size.

13. The method according to claims 11 or 12, wherein the repair decision for the windshield (80) is "replace" if the number of damage sites (90) with the specific type of damage is greater than a predetermined threshold.

14. The method according to any preceding claim, wherein the repair decision for the windshield (80) is "replace" if there are any types of damage other than a chip.

15. The method according to any preceding claim, further comprising allowing a user to indicate the position of each damage site (90) in the image (25) using a user interface (64).

16. The method according to any preceding claim, further comprising comparing the detected type of damage to the type of damage indicated by the user for the damage site (90) using the user interface (64).

17. The method according to any preceding claim, further comprising allowing a user to book an appointment at a glass workshop based on the repair or replacement decision for the windshield (80).

18. A software application for a mobile phone, comprising:operating a camera (20) to capture the image (25) of a windshield (80) comprising one or more damage sites (90);providing a user interface (64) for a user to indicate the position of each damage site (90) in the image (25);processing the image (25) to determine a repairable area (68) of the windshield (80); determining damage criteria for the one or more damage sites (90) on the windshield (80), the damage criteria comprising:a type of damage of the damage site (90); anda location of the damage site (90) relative to the repairable area (68); and determining a repair or replacement decision for the windshield (80) in dependence on the damage criteria.

Images