TEST AUTOMATION OF VEHICLE INSTRUMENT PANEL WARNINGS USING OBJECT RECOGNITION WITH IMAGE PROCESSING

Abstract

The invention relates to systems for testing the indicator warnings of vehicles, buses, and trucks, and its characteristic feature is:It is characterized by the following steps: creating files containing positive photos (1) and negative photos (2) representing the image, training the classifier (6), teaching each vehicle warning to the classifier, monitoring the instrument cluster by the camera, comparing the image detected by the camera via the control unit or computer with the classified image samples, and completing the test automation by evaluating it as suitable or not; and creating object recognition using image processing to enable the automatic testing of the functionality of the indicator warnings on the vehicles without extended hardware equipment, teaching each vehicle warning warning to the classifier, and completing the test automation by creating files containing positive photos (1) and negative photos (2) representing the image, training the classifier (6) and teaching each vehicle warning warning to the classifier, monitoring the instrument cluster by the camera via the control unit or computer, and completing the test automation by evaluating it as suitable or not.

Description

-1 DEFINITION TEST AUTOMATION OF VEHICLE INSTRUMENT PANEL WARNINGS USING OBJECT RECOGNITION WITH IMAGE PROCESSING Technical Area The present invention relates to the automation of test warning signals on vehicle indicators, buses, and trucks. The invention specifically relates to the automation of vehicle display warning signals through object recognition using image processing with the OpenCV library. Each vehicle warning signal is trained, and test automation is performed without human intervention. This allows for software-based testing of whether the triggered warning signal is correct. Meanwhile, the recognition process works independently of the image's resolution, pixels, brightness, and flatness. State of the Art Before vehicles are manufactured, a number of software implementations take place. After the vehicle leaves the production line, if the warnings that should be displayed to the user while driving cannot be shown, problems that can lead to life-threatening situations may arise. In such cases, test supervisors behind the vehicle production run a scenario as if the specified signal were present, and check whether the warning appears on the screen, that is, whether the relationship between the signal and the visual warning is correct. However, since the vehicle has many warning systems, the tester cannot test all of them. Therefore, there is considerable research being conducted on triggering these warning signals and identifying them after they are triggered. High investment costs are required for automating the testing of vehicle display warnings. Using software and various hardware, the system can detect the warning displayed on the screen. -2 systems exist that attempt this. However, this process requires program support, software and hardware resources, and high investment costs. New technical developments are needed that can perform this testing process at more affordable costs. In matching operations, even after the target object is specified as a reference in the Python code, the object can still be searched for in new photos. However, in this type of method, object recognition only occurs when the object is at its reference dimensions. That is, the dimensions of the object in the new photos must be the same as in the reference. Otherwise, it will not be recognized. Size independence is necessary. Recognition must be possible for every dimension of every object in the target photo. In many applications, the resolution quality of photographs is highly valued for the accuracy rate of training. However, in this classifier, resolution quality is not a major factor; with the correct parameters, training results in an accuracy rate of 95%. The object to be recognized must be attempted to be recognized each time. In some applications, the stability of the trained data varies during each test. A stable and high accuracy rate is expected. Problems arise in systems where stable behavior is desired. While it's possible to do this using artificial neural networks, it requires a very large amount of data, making it unsuitable for systems where a large amount of data is not available. High accuracy rates cannot be achieved with limited data. In some examples, operations are performed based on a threshold value. During each recognition process, it is checked whether the object has exceeded the threshold value. Performing operations based on the threshold value every time reduces system stability and increases processing times. The German patent application numbered DE10110038 describes a training set that can be specially expanded to recognize new objects and classifications by classifying them. -3 A solution is provided for the automatic recognition and tracking of objects in retrained image data. It relates to a training set. However, new and different technical solutions are needed for testing high-cost vehicle display warnings. Explanation of the Purposes of the Invention Based on the current state of the art, the aim of the invention is to improve upon and eliminate the shortcomings of existing structures. Another objective of the invention is to enable the automated testing of the functionality of indicator warnings on vehicles without the need for extended hardware equipment. Another aim of the invention is to achieve a solution that does not require high costs. Another aim of the invention is to eliminate data for the testing process, regardless of the object's size. Another aim of the invention is that if a trained data set has an accuracy rate of 95%, it should remain at 95% even after being tested 1000 times, thus ensuring a stable and very high accuracy rate. Another aim of the invention is to save time by making the testing process faster and with a higher accuracy rate, thereby improving the quality of the work done. To achieve these objectives, each vehicle warning signal has been trained, and object recognition using image processing, enabling automated testing without human intervention, has been developed for vehicle display warning signals. This allows for software-based testing of whether the triggered warning signal is accurate. -4Meanwhile, the recognition process does not depend on the resolution, pixels, brightness, or flatness of the example photograph. Explaining the Figures Figure 1 shows a schematic drawing of a representative application of the invention. Reference Numbers 1 Positive photos 6 Training the classifier 2 Negative photos 7 Creating a file for target object recognition 3 Negative.txt file 4 Positive.txt file 5 Vector file Detailed Description of the Invention The invention is the automation of vehicle display warnings by object recognition using image processing. As shown in the block diagram in Figure-1, first, positive and negative photographs (1,2) are collected in quantities of at least 500 each. Then, the negative.txt file (3) and the positive.txt file (4) are created. The file paths of the negative photographs (2) are written one by one into the negative.txt file (3). In the positive.txt file (4), the file paths of the positive photos (1) are written line by line, and in addition, opposite the file path of each positive photo (1), with a space in between, the coordinates of the area occupied by the target object in that positive photo (1) are written, namely the negative.txt file (3) and the positive.txt file (4). Then, in order to convert the photos to grayscale, reduce the pixel values, and train them faster, a vector file (5) is created by referencing the positive.txt file (4). The positive photo (1) of the target object is placed inside the positive.txt file (4). This is why the coordinates of the area it occupies are written in -5. In the images to be trained, only the target object is taken as positive photos (1) for training the classifier (6). The target object is specified as coordinates in the vector file (5). After the vector file (5) is created, the necessary parameters for training the classifier (6) are entered using the trainingcascade in the OpenCV library. These parameters include: the file path of the trainingcascade that will conduct the training, the file path of the .xml file that will be created, the file path of the vector file (5) that will be used as a reference, the file path of the negative photographs (2), the number of stages in the training; the more stages there are, the better the classifier, but the training time will also be longer, how many bad examples will be eliminated in each stage, for example, if there are 1000 negative images, how many of them will be eliminated in each stage. Training of the classifier as bad examples are eliminated (6); specifying how much memory the computer will use during training without wasting time on useless examples, pixel size of the target object; taken as the reduced pixel size when the vector file (5) is created. After these data are written to a text file, saved as a .bat file and run, the training of the classifier (6) will begin. As an example of how the training parameters are written to a text file; “C:\ZAFER\opencv\build\x64\vc14\bin\opencv_traincascade.exe-data This can be done using the following command: `C:\ZAFER\Desktop\data -vec el.vec -bg nesli.txt -numPos 900 -numNeg 1800 numStages 17 -minHitRate 0.995 -maxFalseAlarmRate 0.25 -mem 1024 -w 24 -h 24` After execution, when training is finished, an XML file will be created to introduce the target object to the specified directory (7). The location where the file will be created will be in the directory C:\ZAFER\Desktop\data, for example, during the parameters. -6After the file is created, Python code is written to use the trained XML file of the object that is to be introduced as the reference .xml file. For test automation, very powerful computers are needed to train the object; training the classifier (6). Since computer performance and the parameters to be given in training directly affect the training speed, if training can be done on a better computer, the training time becomes very reasonable. For training, approximately 500 negative and 500 positive photographs (2,1) should be selected, but if the object to be introduced is very complex, the training data is increased. The more data, the better the results. Thanks to the training of the classifier (6) and the creation of a file for the identification of the target object (7), the .xml file finds its target object in the example photograph and encloses it in a rectangle. In this way, each vehicle warning signal is trained, and test automation without human intervention is realized. In this way, it is software-tested whether the triggered warning signal is correct or not. In addition, the recognition process is tested by detecting with high accuracy, regardless of the resolution, pixels, brightness, and flatness of the photograph. With the developed method, the vehicle warning alert is trained and test automation is carried out without humans. It consists of image samples and vector file (5) after object detection and evaluation of appropriate or inappropriate after collecting several image samples of a master image different from the master image in at least one parameter. The file representing the image is trained on a classifier representing an algorithm, after which the set of indicators is monitored by a camera; the image detected by the camera is evaluated by the classifier when compared to classified image samples. It is classified as suitable or not. Ideally, to simplify the image samples, each image is converted to a black-and-white image. Ideally, the pixel count is reduced to simplify the image. -7 (like 24x24). Preferably, image samples can be artificially generated or collected from several empirical trials. The invention relates to systems for testing the indicator warnings of vehicles, buses, and trucks, and its characteristic feature is:It is characterized by the following steps: creating files containing positive photos (1) and negative photos (2) representing the image, training the classifier (6), teaching each vehicle warning to the classifier, monitoring the instrument cluster by the camera, comparing the image detected by the camera via the control unit or computer with the classified image samples, and completing the test automation by evaluating it as suitable or not; and creating object recognition using image processing to enable the automatic testing of the functionality of the indicator warnings on the vehicles without extended hardware equipment, teaching each vehicle warning warning to the classifier, and completing the test automation by creating files containing positive photos (1) and negative photos (2) representing the image, training the classifier (6) and teaching each vehicle warning warning to the classifier, monitoring the instrument cluster by the camera via the control unit or computer, and completing the test automation by evaluating it as suitable or not. The electronic control unit or computer contains a classifier which evaluates the vehicle warning test result by comparing the data obtained by creating a file for target object recognition from the photos taken by the camera and from the previously introduced photos, and by processing the image from the camera and testing the vehicle display warnings with object recognition (7). The system includes the creation of positive photographs (1) representing the image, a negative txt file (3) containing negative photographs (2), a positive.txt file (4), a vector file (5) in which each target test object is paired and taught to introduce and teach, training the classifier that recognizes the target test objects with the data taken from the vector file (5) (6), creating a file for target object recognition (7), and a control unit or computer that evaluates the images taken from the camera and gives the test result. -8. To simplify the image samples, it includes a control unit or computer that converts each image into a black-and-white image. The control unit or computer includes image samples that can be artificially generated or images collected from several empirical trials. The image file represents a classifier that is trained on an algorithm, which is then monitored by a camera set; the image detected by the camera is compared with classified image samples, evaluated by the classifier, and classified as suitable or not, thus completing the testing process in the control unit. Thanks to the training of the classifier (6) and the creation of a file for the introduction of the target object (7), the .xml file contains a control unit or computer that finds its target object in the example photo and puts it in a rectangle, so that each vehicle warning signal is trained and the test automation is carried out without human intervention, and the software tests whether the triggered warning signal is correct or not, the recognition process is detected with high accuracy regardless of the resolution, pixels, brightness, flatness of the photo.

Claims

-9 CLASSIFICATIONS 1. The invention relates to systems for testing the indicator warnings of vehicles, buses and trucks, and its feature is;The process is characterized by the following steps: creating object recognition files containing positive photos (1) and negative photos (2) representing the image and training the classifier (6), teaching each vehicle warning signal to the classifier, and completing the test automation by monitoring the instrument cluster by the camera, comparing the image detected by the camera via the control unit or computer with the classified image samples, and evaluating it as suitable or not; - creating object recognition files containing positive photos (1) and negative photos (2) representing the image, teaching each vehicle warning signal to the classifier, and completing the test automation by teaching each vehicle warning signal to the classifier; 2. It is the automation of vehicle display warnings by object recognition using image processing in accordance with Claim 1, and its feature is: - creating a file for target object recognition from the photos taken by the camera and from the previously introduced photos, which enables testing of vehicle display warnings by object recognition by processing the image from the camera, - including an electronic control unit or computer with a classifier that compares the data obtained by creating a file for target object recognition (7) and evaluates the vehicle warning test result.

3. The automation of vehicle display warnings by object recognition using image processing in accordance with Claim 1, and its features include: - creating a negative txt file (3) containing positive photos (1) representing the image, negative photos (2), positive.txt file (4), creating a vector file (5) in which each target test object is introduced and trained by matching them with each other, - training the classifier that recognizes the target test objects with the data taken from the vector file (5) (6), - creating a file for target object recognition (7), - including a control unit or computer that evaluates the images taken from the camera and gives the test result.

4. Automation of vehicle display warnings via object recognition using image processing compliant with Claim 1, characterized by: - ​​a control unit or computer that simplifies image samples by converting each one into a black-and-white image.

5. Automation of vehicle display warnings via object recognition using image processing compliant with Claim 1, characterized by: - ​​the inclusion of a control unit or computer where image samples can be artificially generated or images collected from several empirical trials can be used.

6. This is a test automation of vehicle display warnings using object recognition with image processing in accordance with Claim 1, and its feature is: - a file representing the image is trained in a classifier representing an algorithm, after which the instrument cluster is monitored by a camera; where the image detected by the camera is compared with classified image samples, evaluated by the classifier, and classified as appropriate or not, thus completing the test process in the control unit.

7. The automation of vehicle display warnings by object recognition using image processing in accordance with Claim 1, its feature is that; - thanks to the training of the classifier (6) and the creation of a file for the identification of the target object (7), the .xml file finds its target object in the photograph shown as example 5 and encloses it in a rectangle, thus training each vehicle warning signal and performing test automation without human intervention, the recognition process is performed by a control unit or computer that tests whether the triggered warning signal is correct or not, with high accuracy, regardless of the resolution, pixels, brightness, flatness of the photograph.

Images