AUTOMATIC FAULT SOURCE DIAGNOSIS SYSTEM AND METHOD BASED ON THE CORRELATION OF IMAGE DISTORTION CHARACTERISTICS IN VEHICLE-BASED IMAGING SYSTEMS AND VEHICLE SUB-SYSTEM DATA.
Patent Information
- Application Number
- TR202614203
- Authority / Receiving Office
- TR · TR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-08-21
- Publication Date
- 2026-09-21
Smart Images
Figure 00000036_0000 
Figure 00000037_0000
Abstract
Description
1 TARIFF IMAGE DISTORTION IN VEHICLE-BOARD DISPLAY SYSTEMS CORRELATION BETWEEN VEHICLE SUBSYSTEM DATA AND THEIR CHARACTERISTICS AUTOMATIC FAULT SOURCE DIAGNOSIS SYSTEM AND METHOD BASED ON 5 Subject of the Invention The invention is a display system used on moving platforms. Detecting image distortions occurring in systems and the possible sources of these malfunctions 10 It is related to a system and method for determining this. More specifically, the invention is created from camera images. image distortion profile and moving platform bottom 15 from the timestamped operational data of the systems temporal and logical aspects of the generated vehicle status profile As a result of associating them, possible sources of failure can be identified. with automatic identification and prioritization It is related. State of the Art Today, environmental concerns are particularly important in military and service vehicles. vision, front view, thermal imaging, near infrared various imaging such as viewing and driver monitoring These systems use cameras. The images obtained are a record and / or image. It is transferred to the processing system. During the operation of imaging systems, in the images vertical line, horizontal band, tingling, mosaic effect or 30 Blocking, image freezing, black screen, and color distortion. Disorders such as these can occur. The aforementioned The malfunctions affect the camera as well as electromagnetic interference and power. power surges, chassis or grounding problems, 2 wiring faults, connector contact problems, DVR malfunctions, task computer malfunctions, Ethernet communication problems or other electronic components on the vehicle This can be caused by the systems. In the current state of the technology, the source of image distortion is... This determination largely depends on the experience of the maintenance personnel. external measurement tools such as oscilloscopes and spectrum analyzers the use of devices or individual system components 10 by checking and eliminating through trial and error It is based on. In some applications, however, only CAN communication is used. data is being evaluated or specific electronic modules By disabling them one by one, the image distortion is eliminated. It is being checked whether it has gotten up or not. In technology, occurrences in imaging systems to detect defects and to identify those defects improvements related to identifying sources of failure Documents on the subject are available. As an example of the known state of the art, consider CN113038122A. A patent document may be provided. This document may include details about the camera. by examining the quality of the images, the imaging malfunction determining the type and the resulting image diagnosis 25 alarm information related to hardware and software components A system that locates the cause of a failure by correlating it. It is explained. However, the document includes images. one of the characteristics of the different types of deterioration that occur Creating an image distortion profile for vehicle subsystems. a vehicle status profile from timestamped operational data 30 the creation and the beginning of the deterioration of these profiles and end times and state changes of subsystems using temporal correlation that takes into account The relationship is not explained. Also, a possible malfunction. 3 logical correlation and dynamic scoring of resources a regulation regarding prioritization using It is not available. Another example of the state of the art is CN202282837U 5 Patent document number [number] may be provided. In the said document cloudiness, blurred vision, tingling, streaks, or band formation, color aberration, image freezing, image loss, and Detects similar video quality issues; also DVR 10 that examine the operating status and network and video transmission conditions A video quality diagnostic system is described. together with the technical aspects of image distortion in the document characteristics of the electronic and / or Time-stamped operation of electrical subsystems. The combined evaluation of the data is not explained. 15 The document also covers image distortions and vehicle subsystems. to establish a temporal and logical correlation between them and Dynamics based on correlation results of possible failure sources It does not include scoring as such. The documents in question show a deterioration in image quality. determination or imaging diagnostic results with infrastructure status and alarm information It offers solutions for establishing the relationship. With this... together, the type and severity of image distortion in these solutions, 25 with start time, duration and affected camera information Time-stamped operational data of vehicle subsystems They are not considered together. Furthermore, image distortion... between the time of occurrence and the operating states of the subsystems temporal correlation, characteristic of image distortion 30 failure characteristics that subsystems may exhibit a possible fault by establishing a logical correlation between them dynamic scoring of resources and Prioritization is not ensured. 4 As a result, the source of the image distortion can be identified. assessment by maintenance personnel and additional checks It may require the following procedures; fault diagnosis time It can take longer, unnecessary module changes can be made, and Additional testing equipment may be required. 5 Military or mission vehicles used especially in field conditions. inability to quickly identify the source of malfunctions in vehicles, maintenance period and vehicle return to duty period It increases. In order to overcome the disadvantages mentioned above, characteristics of image distortions and vehicle undercarriage Time-stamped operational data of the systems in temporal and logically relating and this relation As a result, it dynamically scores potential sources of failure, resulting in 15 a prioritizing fault source diagnostic system and method It has been developed. Detailed Description of the Invention The invention relates to imaging systems on mobile platforms. technical characteristics of the image distortions that occur with timestamped operational data of subsystems as a result of its temporal and logical correlation Identifying potential sources of failure and 25 with a system and method for prioritization It is related. The primary purpose of the invention is to correct problems occurring in imaging systems. image distortions using technical image analysis methods detecting and identifying possible malfunctions caused by these malfunctions 30 a system that automatically identifies and prioritizes resources The goal is to develop systems and methods. Another aim of the invention is to correct the detected image distortion. type, intensity, start time, duration, and the camera it affected or an image containing data relating to a group of cameras The goal is to create a degradation profile. Another purpose of the invention is to create a mobile platform. Timestamped electronic and electrical subsystems Creating a vehicle status profile using operational data. to create. Another purpose of the invention is to improve the image distortion profile of the vehicle. state profile using temporal and logical correlation to associate and with the characteristic of image distortion failure characteristics that subsystems may exhibit The aim is to evaluate the technical compatibility between them. 15 Another purpose of the invention is to identify potential sources of malfunction. type, severity of disruption, number of affected cameras, relevant sub-cameras temporal and logical correlation with the system's operating status 20 with a dynamic scoring mechanism that takes the results into account It is to evaluate. Another aim of the invention is to provide a final solution for every possible source of failure. to create a failure score, by normalizing those scores The goal is to calculate the source of failure confidence scores. 25 Another purpose of the invention is for maintenance personnel to check. by prioritizing potential sources of failure and eliminating unnecessary ones. Module removal or replacement procedures are not based on trial and error. fault finding procedures based on and additional test equipment 30 The goal is to reduce the need for it. The invention applies to display systems on mobile platforms. technical characteristics of the image distortions that occur 6 with timestamped operational data of subsystems as a result of its temporal and logical correlation Identifying potential sources of failure and with a system and method for prioritization The system covered by the invention is shown in Figure 1. The subject of this system is a camera system (1), a digital video recording device (2), a task computer (3), a vehicle data collection module (4), an image analysis module (5), a correlation engine (6), a fault source diagnostic module (7) and 10 a user interface (8) and possible image distortion types matching that includes relationships between fault sources It includes data (9). The camera system (1) has one or more 15 movable platforms It includes at least one camera that captures images of the area. In one application, the camera system (1); left of the movable platform and the 360° cameras located on their right sides, on the back a 360° camera located on the front, and a VNIR the camera, a DOME camera located inside the cabin, and 20 optional on the sides of the movable platform It includes IR cameras that are used. Camera system (1) limited to the camera types in question. not 360° surround view cameras, front view cameras, 25 near infrared cameras, infrared cameras, thermal cameras, night vision cameras, driver assistance cameras, environmental sensing cameras and image generating or similar sensor systems that provide images It may include. 30 Images obtained by the camera system (1) are digital The digital video recording is transmitted to the video recording device (2). 7 The device (2) collects camera images and performs the task transfers it to his computer (3). Mission computer (3), images and vehicle subsystem It ensures the processing and management of data. Task 5 computer (3) with data processing and management module Application and data management modules are available. Image analysis module (5), correlation engine (6) and fault source Functions of the diagnostic module (7) Task computer (3) It can be carried out within the framework of [the organization / institution]. 10 Task computer (3); reference image characteristics creation, performing image analysis, Image distortion profile creation, vehicle status Profile creation, image distortion profile with vehicle 15 Temporal and logical correlation of the state profile By using this method, potential sources of failure can be identified. determination of final failure scores and failure confidence It performs the processes of calculating the scores. Reference image characteristics show whether the system is normal or not. This is created for each camera assuming trouble-free operation. Within this scope, a valid certificate will be issued for each camera found to be in working order. A reference image representing the image is being recorded, and from the reference image in question reference image 25 characteristics are obtained. Reference image characteristics for the initial setup of the system. maintenance procedure, camera replacement or by the user 30 during the initiated calibration process Camera replacement, maintenance, and calibration can be performed. or any system that affects the image characteristics Reference image characteristics after modification It is being recreated. 8 Reference image characteristics; edge density, noise level, color distribution, brightness level, and blocking It consists of image parameters such as ratio. Live images from cameras in operation of the moving platform. is taken. Image analysis module (5), live images Analysis by comparing with reference image characteristics. During comparison, brightness and line quality are examined. density, bandwidth, noise, blocking or 10 mosaic effect, color balance and color distribution in squares Criteria such as the changes between them are evaluated. Between live image and reference image characteristics Image distortion 15 if a significant difference is found. the type of deterioration in question is being identified is determined. The system uses a single image analysis method. not related, different analyses for different types of degradation It can use various methods. Multiple images can be displayed on the same image. multiple analysis methods simultaneously 20 It can be operated. Vertical lines, horizontal lines are analyzed by the image analysis module (5). bands, tingling, mosaic-like or blocking, image 25 issues such as freezing, black screen, color distortion and choppy image Image distortions can be detected. Edge analysis is used to detect vertical lines. The vertical edge density in the live image is calculated as a reference. 30 with vertical edge density found in image characteristics They are being compared. The vertical edge density is determined. Exceeding the threshold value, line density increases, and vertical lines that continue throughout the image frame its arrival is an indicator of vertical line distortion. 9 This analysis is being evaluated. As a result of this analysis, line density and The severity of the deterioration is being determined. Frequency analysis and edge analysis in the detection of horizontal bands. It is used. The image shows a horizontal 5 The density of periodic structures is determined; horizontal band increasing density, band thickness and repetition frequency This is considered a horizontal band distortion. This analysis... As a result, the bandwidth is determined. Noise analysis is used to detect tingling sounds. The pixel noise level of the image is calculated and... the subject level reference image characteristics The deviation is being evaluated. It generates random noise. Increasing pixel ratio and noise level 15 It is classified as a tingling sensation, and as a result of the analysis... The noise level is determined. Block analysis for detecting mosaicism or blocking. is used. Block artifacts within the image and 20 The block artifact rate is determined. The block artifact rate... exceeding the defined level causes a block in the image. the formation of structures and the number or size of blocks increase as mosaic-like or blocking is being evaluated. As a result of the analysis, the blocking level is 25. It is determined. Sequential frame analysis in detecting image freezing. It is used. The change between consecutive image frames. is determined and the difference between the squares is set within a specified time of 30. The image remains below the threshold value determined throughout. It is classified as freezing. Consecutive squares highly similar to each other and movement in the image Its absence also helps in determining the image freeze. is used. The freezing time was determined as a result of the analysis. It is determined. Histogram analysis and brightness in detecting black screen Analysis is used. The average brightness of the image is 5. The value is calculated and the average brightness is below the determined threshold. Falling below the threshold value results in a black screen. is being evaluated. The brightness values in the image are very high. also low and pixel values close to zero It is used in determining the black screen. Analysis 10 As a result, the level of darkening is determined. Color dispersion analysis in the detection of color distortion. It is used. The color distribution of the live image is used as a reference. 15 from the color distribution found in the image characteristics Deviation above the specified tolerance results in discoloration. It is classified. The reference of RGB color distribution. deviation according to the image and color balance in the image deterioration is also used in determining color discoloration. It is used. As a result of the analysis, the color change rate is 20. It is determined. Wavy image, wave-shaped shift in the image, or with periodic brightness changes occurring is determined. 25 Image analysis module when image distortion is detected An image distortion profile is created by (5). Image distortion profile; distortion type, distortion severity, the onset time of the deterioration, the duration of the deterioration, and the 30 affected It includes information about the camera or group of cameras. The presence of more than one type of distortion in an image simultaneously. In this case, each distortion is displayed separately or together. It can be included in the degradation profile. 11 The type of malfunction detected is shown as system output. It is being created, recorded, and reported. Image Deterioration profile, for fault source analysis. It is sent to the correlation engine (6). 5 In an example of a malfunction detection output, the malfunction type is vertical. lines, detection confidence score 85%, detection time 12:45:03 and The affected camera can be shown as the front camera. (The word...) The values in question are only an example of a detected output. 10 It shows. Vehicle data collection module (4), on the mobile platform the operation of the imaging system directly or indirectly electronic, electrical and communication 15 that can affect Time-stamped operational data of subsystems The vehicle data collection module (4) collects the said data. It transfers the data to the task computer (3). Vehicle data acquisition module (4) CAN or J1939 interface, Ethernet 20 interface, power and supply data, digital or analog It can retrieve data through inputs. In different applications. LIN, FlexRay, MIL-STD-1553, ARINC, RS-485, CAN FD, automotive Obtained from Ethernet or similar communication infrastructures Study data can also be used. 25 DC / DC converter by vehicle data collection module (4), alternator or engine, radio system, battery or power supply system, chassis or grounding system, digital video recording device (2), Ethernet or data line, task computer (3), 30 camera system (1), camera cable harness, connector, projector or high current loads, CTIS compressor or valve, CBRN or positive pressure fan, fuel heating system and blackout 12 operation of subsystems or hardware such as the system Data can be collected. On or off operating status in terms of DC / DC converter. It can be received via CAN interface or digital input and 5 It can be used in electromagnetic interference correlation. Alternator maintenance, engine speed or charge status via CAN interface. can be obtained through and in fluctuation analysis It can be used. In terms of radio systems, TX or RX. The status can be obtained via CAN or Ethernet and radio 10 Frequency can be used in interference analysis. In terms of the power supply system, the battery voltage is transmitted via the CAN interface. can be obtained and used in the evaluation of voltage fluctuation. It can be used. Digital video recorder (2) 15 The operating status can be obtained via Ethernet and video. It can be used in the evaluation of transfer. Ethernet connection and error status are received via Ethernet. It can be used in communication analysis. Task The computer's (3) processing load is taken via Ethernet 20 It can be used in evaluating system load. The camera's operating status is obtained via Ethernet. It can be used to verify the status. 25 timestamped data collected by the vehicle data collection module (4) A vehicle status profile using operational data. The vehicle status profile is created. The image distortion is assessed. the relevant subsystems during the time interval in which it occurred timestamped to enable assessment of their status It includes data. 30 Vehicle status profile is only available from electronic control units. not limited to the data received, but also the power distribution system data, communication network performance data, sensor data 13 health information, power quality data, and images. other systems that may affect the operation of the system It may include data. Correlation engine (6), image distortion profile with vehicle status 5 They are jointly evaluating the profile. This evaluation It includes temporal correlation and logical correlation. Onset time of image distortion in temporal correlation By comparing the runtime of the related subsystem, the statement 10 the question is whether the events occurred simultaneously is being evaluated based on the end time of the image distortion. The downtime of the relevant subsystem is also compared. the issue is between the subsystem and image distortion. The causal relationship is strengthened. 15 Immediately after an electronic subsystem is activated The onset of image distortion is due to the subsystem in question. It can increase the correlation between image distortion. Image 20 after the subsystem is disabled. The elimination of the malfunction is related to the subsystem. causal assessment between deterioration It can strengthen it. Type of image distortion in logical correlation or 25 the characteristics that the subsystem can form electromagnetic interference or electronic malfunction characteristic Technical compatibility between them is being evaluated. Image The number of cameras affected by the malfunction is also logically correlated. Within this scope, local and systemic failures are evaluated. They are separated from each other. In a single camera. the malfunction that occurred simultaneously on multiple cameras The resulting deterioration points to different sources of failure. is able to. 14 As a result of temporal and logical correlation operations, a It is generated as a result of correlation. Fault source diagnostic module (7), correlation engine (6) 5 Using the correlation result obtained, the image potential sources of failure that could cause it to malfunction It determines the types of image distortion in the system for this purpose. and possible technical issues that could cause these malfunctions 10 including relationships and scores between sources of failure Matching data (9) is used. Possible sources of failure; originating from the DC / DC converter. electromagnetic interference, radio or radio frequency sources electromagnetic interference, camera cable shielding problem, 15 Chassis or grounding problem, camera power line problem, antenna proximity, alternator-induced ripple, digital video recorder (2) malfunction, Ethernet communication problem, camera malfunction, task computer (3) problem, power supply The problem is a cable harness malfunction, connector contact issue, and 20 There may be a communication infrastructure failure. In one example of the matching data (9), the vertical Line distortion caused by DC / DC converter. electromagnetic interference 40 points, camera power line 25 25 points for parasites, 20 points for chassis or grounding problems. And camera cable shielding problems are given 15 points. Alternator fluctuation is scored at 40 points for horizontal band distortion. 30 points for power line surge, 30 points for chassis or grounding. 20 points for the problem and electromagnetic field related to motor speed. The initiative is given 10 points. Tingling sensation can be caused by radio or radio frequency signals. 40 points for electromagnetic interference, camera cable shielding. 25 points for the problem, 20 points for antenna proximity, and 25 points for chassis or A grounding problem is given 15 points. Ethernet data error 35 for mosaicking or blocking. 30 points for malfunction of digital video recorder (2), Ethernet switch problem 25 points and task computer (3) Image processing errors are given 10 points. Digital video recorder (2) for image freezing 35 points for deadlock, 30 points for Ethernet packet loss, task 25 points for the computer (3) problem and camera data transfer A deduction of 10 points is given. 35 points for camera feed interruption due to black screen, cable 30 points for disconnection or connector contact failure, numerical video recorder (2) channel malfunction 25 points and task 10 points to the computer (3) or image transmission line is given. 20 35 points for camera sensor malfunction due to color distortion, video. Signal line problem: 25 points, digital video recorder. (2) 20 points for image processing error and cable shielding The problem is worth 20 points. 25 A chassis or grounding problem causes a choppy display, scoring 40 points. 30 points for supply line fluctuation, 20 points for alternator effect. 10 points are awarded for both standard and common mode noise. Examples relating to vehicle subsystems or vehicle data. In the scoring system, the 24 VDC-24 VDC converter received 25 points, radio. 25 points for system or TX information, alternator or engine. 25 points for the circuit, 25 points for the battery or supply voltage, chassis. 16 or 20 points to grounding reference, digital video recording 25 points to the device (2) status, 25 to the Ethernet or data line points, task computer (3) status 25 points and camera cable The bundle or connector is awarded 25 points. Projector or 20 points for high current loads, CTIS compressor or 5 20 points for the valve, 20 points for the CBRN or positive pressure fan, 15 points for fuel heating system and 20 points for blackout system These points are given. A deterioration has occurred in these point values. when it comes to the evaluation of the relevant subsystem It is used. 10 When an image distortion is detected, it is classified according to the type of distortion. from matching data (9) to relevant possible sources of failure Basic or starting scores are assigned. Then the vehicle Depending on the status of the subsystems, an additional 15 points are added to the starting scores. applied or points from initial scores is being removed. In an example of applying the point effect rules, the relevant When this type of malfunction occurs, the associated subsystem becomes active. 20 and the malfunction begins when the subsystem becomes active. 10 additional points if there is a timing relationship between them Timing points are applied. The subsystem is closed. If the deterioration continues, a penalty of 10 points will be applied. is being implemented. The subsystem is active, but the malfunction is 25. No points are awarded if it is not technically compatible with the type. These values relate to a sample scoring system. and the scores used in the system are based on the type of image distortion, severity, number of affected cameras, subsystem operational status According to the temporal and logical correlation results, 30 It can be determined. During dynamic scoring, the type of image distortion, the image the severity of the disruption, the number of affected cameras, the relevant sub-cameras 17 the system's operating status, the result of temporal correlation, and They are evaluated together as a result of logical correlation. In determining the initial impairment type score; impairment The severity and the number of cameras affected can be used to increase the score or in reducing; subsystem activity, temporal correlation and 5 In increasing the logical correlation score; the subsystem Being passive is used to reduce the score. Multiple image distortions linked to the same possible source of failure. If associated with it, the image in question is 10 points obtained from their malfunctions together by evaluating each possible source of failure and determining the final fault. A score is being generated. During the calculation, the first step is matching according to the type of malfunction. 15 Basic scores are obtained from the data (9), vehicle subsystem Depending on the situation, additional points are applied or points are deducted. The obtained points are added together to arrive at the final failure score. The calculations are made and the final failure scores are in the range of 0-100. It is being normalized. 20 As a result of normalization, a percentage is given for each possible source of failure. A fault source confidence score is created in this way. The fault source confidence score is a measure of the identities of candidate fault sources. They express the relative level of trust between them. 25 The sum of these scores does not need to be 100%. Potential failure sources have the highest failure source confidence score. the source with the lowest fault source confidence score They are aligned towards the source. Detection of image distortion. The confidence value related to the "detection confidence score" indicates a potential failure of 30. Normalized confidence value for source identification. This is expressed as the "source of failure confidence score". 18 In a sample confidence score output, the DC / DC converter was 91%. 58% for alternator, 27% for Ethernet and digital video recording. 19% failure source confidence score for device (2) is being created. In a normalization example, originating from a DC / DC converter. 65 final failure points and 100% for electromagnetic interference. fault source confidence score, radio or radio frequency origin. 65 final failure points and 100% for electromagnetic interference. Fault source confidence score, camera cable shielding problem 10 40 ultimate failure points and 62% failure source confidence score for, 35 final failure points for chassis or grounding problems and 54% fault source confidence score, camera power line interference. for 25 ultimate failure points and 38% failure source confidence score and Antenna proximity resulted in a final failure score of 20 and a 31% failure source of 15. A confidence score is obtained. In the normalization example in question, vertical lines appear in the image. and tingling are observed together. Vehicle subsystem In this case, the 24 VDC-24 VDC converter is active, radio system TX 20 is doing, alternator or engine speed 2100 rpm, battery or Normal digital video recorder with a supply voltage of 27.1 V. (2) status is normal, Ethernet or data line status is normal and camera cable or connector status is normal is shown. As a result of the evaluation, 25 possible malfunctions were identified. Their sources are, respectively, DC / DC converter sources. electromagnetic interference and wireless or radio frequency It is identified as electromagnetic interference originating from a specific source. In another example of data-source mismatching, the frequency is 30. Horizontal band distortion determined by analysis of alternator load It is associated with fluctuation and 65% for the alternator. A fault source confidence score is generated. This is done through edge analysis. The defined vertical lines represent the DC / DC output voltage. 19 It is associated with fluctuation and is for DC / DC converters. A 70% fault source confidence score is established. Noise tingling determined by analysis of radio TX status It is associated with and accounts for 60% of the failures for the radio system. A confidence score is generated. Determined by block analysis, 5 Mosaic distortion is associated with Ethernet error rate, and 55% source of failure confidence score for Ethernet communication. It is created. It is made with chassis connection resistance. 40% failure rate assurance for chassis connection in matching. The score is being generated. 10 The user interface (8) displays the type of image distortion detected, potential sources of failure, and the characteristics of those sources of failure fault source confidence scores, prioritization of fault sources list and maintenance or repair recommendations to the user 15 It offers. As a maintenance recommendation, the relevant subsystem should be checked. Verification of connections and supply lines and necessary If observed, it is recommended to take measurements. Relevant 20 The image distortion ends when the subsystem is shut down. The suggestion to check that it has not been received is also given to the user. can be presented. The workflow within the scope of the invention is shown in Figure 2. 25 This flow is primarily within the scope of image acquisition step (101) Images are being captured from the cameras. Image distortion occurs. Detection and image distortion profile creation step (102) live images within the scope of reference images Image distortion 30 when compared with its characteristics They are being identified and classified. Possible malfunctions. the step of identifying resources and assigning starting scores (103) possible sources of failure according to the type of failure Initial scores are given. Time-stamped work. step of collecting data and creating vehicle status profile (104) status and operational data from vehicle subsystems within this scope Vehicle status profiles are created by collecting data. Temporal and Image distortion within the scope of the logical correlation step (105) Vehicle status profile with temporal and logical profile 5 It is related using correlation. Initial updating scores and creating final failure scores Correlation results obtained within the scope of step (106) Scores are updated using this data, and final failure scores are determined. is calculated. The final failure scores are normalized to 10. Percentage failure sources are converted into confidence scores. Step to prioritize potential sources of failure (107) The possible sources of failure are listed within this scope. Results possible listed within the scope of the step of presenting to the user (108) Fault sources are presented to the user. Diagnosis 15 within the scope of the step of recording and reporting the results (109) The results are recorded and reported. Python will be used to demonstrate the technical working principle of the invention. Streamlit-based and native 20 using a programming language A prototype demo software was prepared to run in the environment. In the prototype demonstration in question, the reference image was shown live. image, number of affected cameras and vehicle subsystem status. In the demo environment, these were manually selected and the system responded to these situations. The reaction given has been observed. The elections in question are prototype 25 Sample inputs used in conducting the demonstration It represents. In the prototype demonstration, real vehicle subsystems were used. Time data was not collected directly; 30 to vehicle subsystems Related status data was manually entered in the demo environment. This has been ensured. This situation only applies to the prototype demonstration. It relates to the way it is implemented. The system on the vehicle if implemented, in order to collect real-time data 21 internal software and communication of the units in the vehicle adapting the protocols to the system configuration It is anticipated that, within this scope, the subsystems will be integrated into the vehicle. communication protocols during system integration 5. Structuring according to its design and appropriate data communication The environment needs to be prepared. The hardware used in the demo presentation included left, right, front, and four 360° cameras located in the rear areas, one VNIR camera, two IR cameras, a digital video recorder 10 (2), a task computer (3) and Ethernet with CAN / J1939 The interfaces are specified. Within the scope of the software infrastructure. Image analysis with OpenCV, data processing with Python, CAN Logger data collection, correlation with correlation engine (6), troubleshooting Scoring and evaluation with source diagnostic module (7), 15 In addition, the user interface (8) application is specified. At the start of the demo scenario, at 14:35:10, the left 360° A clear image is being obtained from the camera. In this case, the engine speed... 1850 rpm, battery voltage 27.4 V, DC / DC converter active, radio 20 The system is down, the CBRN system is down, the Ethernet line is normal. digital video recorder (2) normal and camera power supply It is shown as normal. Initially, the vertical edge density is 3%, and the horizontal band density is 25%. density 2%, noise level 4%, mosaic effect level 1% and The brightness change is shown as 2%. System As a result, no image distortion was detected by the system. It was working normally, the result time was 14:35:10 and it was detected. It is stated that a confidence score has not been established. 30 In the continuation of the demo scenario, at 14:35:18, the radio PTT... The radio system is activated in TX mode by pressing the button. is brought in. In this case, in the left 360° camera view 22 Vertical line distortion is occurring. Engine speed 1850 RPM, battery voltage 27.4 V, DC / DC converter active, radio System active, CBRN system deactivated, Ethernet cable normal. digital video recorder (2) normal and camera power supply It is shown as normal. 5 In case of failure, the vertical edge density is 47%, horizontal band. density 3%, noise level 6%, mosaic effect level 2% and The brightness change is shown as 3%. System Deterioration was detected by; the type of deterioration is vertical 10 The lines show the beginning time at 14:35:18, the affected camera... The left 360° camera detected a distortion level of 47% or higher. The confidence score is stated to be 96%. The radio system switches from off to active mode, 15 From a situation where vertical line distortion is absent to a situation where it is present. The time difference between the transition to this state is 0.08 seconds. This relationship is shown as a very strong correlation. is being evaluated. In this example, radio frequency electromagnetic waves originating from radio transmission. 40 initial points and 25 correlation points for the initiative. with the addition of 65 final failure points and a 100% normalized score. This is obtained from electromagnetic fields originating from DC / DC converters. For the venture, a starting score of 40, a correlation score of zero, 40 25 final failure score and 62% normalized score obtained It is being done. 25 starting points for the camera power supply line, zero correlation score, 25 final failure score, and 38% normalized A calculated score is obtained. Chassis or grounding problem. 20 starting points, zero correlation score, 20 final 30 A failure score and a normalized score of 31% are obtained. The diagnosis revealed that possible sources of the malfunction were radio-related. frequency electromagnetic interference, originating from DC / DC converter. 23 electromagnetic interference, camera power line and chassis or The problem is listed as a grounding issue. To the user, The image distortion ends when the radio system is switched off. It is recommended to check that it has not been reached. In the demo scenario, the radio system was activated at 14:35:29. It is being turned off and the image distortion is eliminated. This In this case, engine speed is 1850 rpm, battery voltage is 27.4 V, DC / DC. Converter active, radio system off, CBRN system Closed, Ethernet cable normal, digital video recorder (2) 10 Normal and camera feed are shown as normal. After the degradation ends, the vertical edge density is 3%. horizontal bandwidth 2%, noise level 4%, mosaic effect. The level is shown as 1% and the brightness change as 2%. 15 The image distortion ended when the radio system was switched off. reached, the correlation is very strong, the end time 14:35:29 and the source of the failure confidence score is 96% It is stated. In the demo scenario, pressing the radio's PTT button activates the radio. the system has switched to TX state and a vertical line appears in the image The malfunction occurred; the image was lost when the radio system was shut down. It is shown that the disruption has been eliminated. The system, word using temporal and logical correlation, the subject matter of the event is 25 evaluating and identifying the most likely source of the malfunction as radio-related. radio frequency electromagnetic interference It determines. The test environment shown on the demo screen was a vehicle, an open field, sunny weather, 30 As a test user with a trial date of 25.05.2024. This information is stated on the prototype demo screen. It relates to the scenario shown. 24 The invention, in its current form, is used in military tactical wheeled vehicles. vehicle-mounted imaging systems and vehicle undercarriage used working data obtained from the systems together by evaluating the possible source of the malfunction causing the image distortion. It is designed for the purpose of determining this. With this, 5 In different vehicles, while maintaining the same operating principle, on platforms and display infrastructures It is applicable. The invention applies to military tactical wheeled and tracked vehicles, armored 10 in personnel carriers, armored logistics vehicles, tactical in mission and support vehicles, fuel carriers, commercial in vehicles, construction equipment, autonomous land vehicles, rail in system vehicles, marine platforms, unmanned ground in vehicles, unmanned aerial vehicles and similar mobile 15 It can be used on various platforms. The invention only addresses images caused by electromagnetic interference. not in diagnosing malfunctions; but in diagnosing power supply problems, chassis connection problems, wiring harness failures, 20 connector contact issues, camera malfunctions, digital video recorder (2) malfunctions, task computer (3) malfunctions, communication infrastructure failures and image similar electronic devices that could cause malfunctions It can also be applied in the diagnosis of electrical faults. 25 Image distortion profile, vehicle status profile, temporal and logical correlation, matching data (9), dynamic Scoring and fault source confidence score creation methods different vehicle architectures, different communication infrastructures and 30 It can be adapted to different imaging systems. As a result of this study, image distortions are defined according to technical specifications. They are classified according to image distortions and vehicle undercarriage. There is a temporal difference between the operating states of the systems and Logical correlations are being established, potential sources of failure. It is evaluated using dynamic scoring and every possible final failure score for the source of failure and source of failure confidence The score is being calculated. Possible sources of failure: confidence level 5 Maintenance personnel are prioritized according to their scores. It is presented. The invention relates to a display system on a moving platform. The source of the image distortion that occurred is automatically identified as 10. a fault source diagnosis aimed at identifying it It is a system, in its most general form; - at least those who obtain images from the mobile platform in question a camera system including a camera (1), - images obtained from the camera system (1) without malfunction 15 Reference image representing the working condition. image by comparing its characteristics detects distortion and creates an image distortion profile. an image analysis module (5), - lower 20 that could affect the operation of the imaging system timestamped operational data of the systems by collecting data, creating a vehicle status profile. a tool data collection module (4), - the study in question with the image distortion profile a vehicle status profile created from data 25 using temporal and logical correlation by relating and forming a correlation result correlation engine (6) and - Correlation result indicates image distortion type and possible malfunction. matching data between sources (9) 30 dynamically identify potential sources of failure using A fault source diagnostic tool that scores and prioritizes faults. It includes module (7). 26 In one configuration of the invention, the image analysis module (5), correlation engine (6) and fault source diagnostic module (7) it contains and collects data through a vehicle data collection module. It includes a task computer (3) with which it is in communication. In a configuration of the invention, the camera system (1) the task of collecting images obtained by a digital video recorder (2) that transmits to its computer (3) It includes. In a configuration of the invention, the camera system (1) 360° Surround view camera, front view camera, near infrared camera, infrared camera, dome camera, thermal camera, night rearview camera, driver assistance camera, and perimeter sensing. It includes at least one of its cameras. 15 The invention also includes display on a moving platform. image distortion occurring in the system malfunction a system for automatically identifying the source It is a fault source diagnosis method, and in its most general form; 20 - mobile platform within the scope of image acquisition step (101) Obtaining images from at least one camera on it, - Detecting image distortion and image degradation taken as part of profile creation step (102) 25 represents the image's fault-free operating condition. by comparing with reference image characteristics Detecting image distortion and an image Creating a degradation profile, - identifying potential sources of failure and initiating Image distortion 30 within the scope of assigning points step (103) Matching between type and possible sources of failure using the data, possible sources of failure can be identified. identification and possible sources of failure assignment of initial scores, 27 - collecting timestamped operational data and vehicle status Viewing within the scope of profile creation step (104) subsystems that could affect the operation of the system collection of time-stamped work data and word The subject is a vehicle status profile from study data 5 creation, - within the scope of the temporal and logical correlation step (105) image distortion profile and vehicle status profile using temporal and logical correlation the correlation and the correlation results 10 creation, - updating initial scores and final failure Initial step within the scope of creating points (106) their scores and the operational status of the relevant subsystem According to the temporal and logical correlation results, 15 by updating and generating final failure scores and - Prioritizing potential sources of failure (107) within the scope of possible sources of failure, the final point in question steps for prioritizing based on failure scores It includes. 20 In a conceptualization of the invention, the edge of image distortion analysis, frequency analysis, noise analysis, block analysis, sequential squares analysis, histogram analysis, brightness analysis and Detection using at least one of the color distribution analyses 25 is being done. In a configuration of the invention, the image distortion profile type of degradation, severity of degradation, time of onset of degradation, Duration of the malfunction and 30 cameras or camera group affected It includes related information. In a structuring of the invention, a time-stamped study data from DC / DC converter, alternator, radio system, battery 28 or power supply system, chassis or grounding system, digital video recorder (2), Ethernet or data line, task It belongs to at least one of the computer (3) and the camera system (1). In a structuring of the invention, temporal correlation is shown in image 5. sub-regarding the start and / or end time of the deterioration the system becoming active, passive and / or working The time it takes to change the situation is being compared. In a structuring of an invention, logical correlation 10 within this scope, sub-characteristics of image distortion. electromagnetic interference that the system may generate and / or technical compatibility between electronic failure characteristics and Number of cameras affected by image distortion is being evaluated. 15 In a configuration of the invention, the initial scores are displayed. type of distortion, severity of image distortion, affected number of cameras, operational status of the relevant subsystem, temporal The correlation result and the logical correlation result together form 20 It is updated using [method / service]. In a configuration of the invention, with the same potential source of failure. related to multiple image distortions The scores are evaluated together and the possible malfunction in question is 25. A final failure score is generated for the source. In a conceptualization of the invention, potential sources of failure are identified. Confidence scores are obtained by normalizing the final failure scores. is being created and possible sources of failure are mentioned in the failure 30. The sources are ranked according to their trust scores. In a structuring of an invention, presenting the results to the user. The type of image distortion detected within the scope of step (108), 29 potential sources of failure, belonging to those sources of failure fault source confidence scores and maintenance and / or inspection The suggestions are presented to the user. In a structuring of the invention, recording diagnostic results is 5. and diagnostic results within the scope of reporting step (109) It is being recorded and reported. One advantage of the invention is that image distortions are eliminated from technical images. detection by analysis methods and 10 regarding deterioration It enables the creation of an image distortion profile. Another advantage of the invention is its image distortion profile. Timestamped subsystems of the mobile platform 15 vehicle status profiles created from operational data The goal is to ensure they are evaluated together. Another advantage of the invention is the sub-discretion regarding image distortion. the temporal relationship between the operating states of the system and by evaluating technical compatibility, 20 potential sources of failure Its purpose is to enable its identification. Another advantage of the invention is the dynamic nature of potential sources of failure. by enabling them to be scored and prioritized It provides systematic decision support to maintenance personnel. 25 Another advantage of the invention is that it is troubleshooting based on trial and error. search operations, unnecessary module changes, and additional This contributes to reducing the use of test equipment. Another advantage of the invention is the time required for fault diagnosis. to reduce and make the maintenance process more systematic its contribution to its implementation. Explanations of the Figures Figure 1: Example of the fault source diagnostic system covered by the invention. It shows the system architecture. Figure 2: Operation of the fault source diagnostic method described in the invention. It shows the flow. Explanations of Reference Numbers in Figures 1. Camera system 10 2. Digital video recorder 3. Task computer 4. Vehicle data collection module 5. Image analysis module 6. Correlation engine 15 7. Fault source diagnosis module 8. User interface 9. Matching data 101. Image acquisition step 102. Detecting image distortion and image distortion 20 profile creation step 103. Identifying potential sources of failure and initiating troubleshooting. the step of assigning scores 104. Collecting timestamped work data and vehicle status. Profile creation step 25 105. Temporal and logical correlation step 106. Updating initial scores and final failure. the step of creating scores 107. Prioritizing potential sources of failure. 108. Step 30: Presenting the results to the user. 109. Step: Recording and reporting diagnosis results.
Claims
31 REQUESTS 1. On a display system on a moving platform The source of the image distortion that occurred is 5. a source of failure that can be automatically diagnosed It is a diagnostic system; its characteristic feature is: - the best at obtaining images from the moving platform in question a camera system containing a small number of cameras (1), - images obtained from the camera system (1), 10 Reference image representing fault-free operation. image by comparing its characteristics detects distortion and creates an image distortion profile. an image analysis module (5), - 15 that could affect the operation of the imaging system timestamped operational data of subsystems by collecting data, creating a vehicle status profile. a tool data collection module (4), - the study in question with the image distortion profile vehicle status profile created from data 20 using temporal and logical correlation by relating and forming a correlation result correlation engine (6) and - Correlation result and image distortion type and possible Matching data between fault sources (9) 25 dynamically identify potential sources of failure using A fault source diagnostic tool that scores and prioritizes faults. It includes module (7).
2. According to claim 1, a fault source diagnostic system is 30 Its features include the image analysis module (5), and the correlation engine. (6) and fault source diagnostic module (7) within it containing and communicating with the vehicle data collection module It includes a task computer (3). 32 3. According to claim 2, a source of failure is the diagnostic system, and feature obtained by camera system (1) a task computer (3) that collects images and transfers them to the task computer It includes a digital video recorder (2). 5 4. A source of failure according to any of the previous requirements. It is a diagnostic system, and its feature is; camera system (1) 360° Surround view camera, front view camera, close-up infrared camera, infrared camera, DOME camera, thermal 10 camera, night vision camera, driver assistance camera and It must include at least one environmental sensing camera.
5. On a display system on a moving platform The source of the image distortion that occurred is 15 a source of failure that can be automatically diagnosed It is a diagnostic method, and its characteristic feature is; - mobile platform within the scope of image acquisition step (101) Obtaining images from at least one camera on it, - Detecting image distortion and image distortion 20 taken as part of profile creation step (102) the image represents the fault-free working condition with reference image characteristics Detection of image distortion by comparison and an image distortion profile 25 creation, - identifying potential sources of failure and initiating image within the scope of assigning points step (103) between the type of failure and possible sources of failure Possible fault 30 using matching data. Identifying the sources and potential malfunctions in question assigning starting scores to resources, - collecting timestamped operational data and vehicle status Viewing within the scope of profile creation step (104) 33 subsystems that could affect the operation of the system collection of time-stamped work data and a tool case from the data of the study in question profile creation, - Temporal and logical correlation step (105) 5 Vehicle status within the scope of image distortion profile. temporal and logical correlation of the profile correlation by using generating results, - updating starting points and final failure 10 Initial step within the scope of creating points (106) their scores and the operational status of the relevant subsystem According to temporal and logical correlation results updated and final failure scores are generated. and 15 - Prioritizing potential sources of failure (107) within the scope of possible sources of failure, the final point in question steps for prioritizing based on failure scores It includes.
6. According to claim 5, it is a fault source diagnostic method, Feature; edge analysis of image distortion, frequency analysis, noise analysis, block analysis, sequential squares analysis, histogram analysis, brightness analysis and color Determined using at least one of the distribution analyses 25 It is done.
7. A fault source diagnostic method according to claim 5 or 6. Its characteristic is that the generated image distortion profile type of degradation, severity of degradation, time of onset of degradation, 30 duration of the malfunction and the camera or group of cameras affected It contains information related to the subject. 34 8. A source of failure according to any of claims 5 through 7. It is a diagnostic method; its characteristic feature is time-stamped analysis. data from DC / DC converter, alternator, radio system, battery or power supply system, chassis or grounding system, digital video recorder (2), Ethernet or data cable, 5 task computer (3) and camera system (1) at least It means it belongs to someone.
9. A source of failure according to any of claims 5 through 8. It is a diagnostic method, and its characteristic feature is that the temporal correlation is 10. with the start and / or end time of image distortion Activation or deactivation of the relevant subsystem and / or time of changing employment status It involves comparison.
10. A failure according to any of claims 5 through 9. Its source is a diagnostic method, and its characteristic is logical. correlation with the characteristic of image distortion electromagnetic fields that the relevant subsystem may generate between the intervention and / or electronic failure characteristics 20 technical compatibility and camera affected by image distortion It involves evaluating the number.
11. A failure according to any of claims 5 through 10. Its source is a diagnostic method, and its characteristic is; initial 25 the type of image distortion, the scores the severity of the disruption, the number of affected cameras, the relevant sub-cameras the system's operating status, the result of temporal correlation, and by using them together as a result of logical correlation It is an update. 30 12. According to claim 11, it is a fault source diagnostic method, characteristic; associated with the same potential source of failure scores for multiple image distortions combined 35 by evaluating the possible source of the failure, the final decision will be made. It is the creation of a failure score.
13. Diagnose a source of failure according to claim 11 or 12. This method is characterized by its ability to provide the final 5 information on possible sources of failure. by normalizing failure scores, confidence scores creation and potential sources of failure The sources are ranked according to their trust scores.
14. According to claim 13, it is a fault source diagnostic method, 10 feature; step of presenting the results to the user (108) the type of image distortion detected within this scope, possible fault sources, belonging to those fault sources fault source confidence scores and maintenance and / or inspection The recommendations are presented to the user. 15 15. Diagnose a source of failure according to claim 13 or 14. It is a method whose characteristic is to record diagnostic results and Diagnostic results within the scope of reporting step (109) recording and reporting. 20 30