Vehicle Evaluation System

The vehicle evaluation system addresses the challenge of evaluating vehicle conditions by using supervised learning to analyze sound data during a standardized operation, ensuring accurate and consistent assessment of vehicle performance.

JP7586135B2Active Publication Date: 2024-11-19TOYOTA JIDOSHA KK
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022080738
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-05-17
Publication Date
2024-11-19
Estimated Expiration
2042-05-17

AI Technical Summary

Technical Problem

Existing vehicle evaluation systems face challenges in accurately determining vehicle conditions from sound data due to variations in driving conditions and the rarity of abnormal sound data, making it difficult to effectively evaluate vehicle performance.

Method used

A vehicle evaluation system that uses supervised learning to generate operation data from sound data, comparing it with generated data to determine deviations and evaluate vehicle conditions by analyzing sound data recorded during a standardized operation pattern, incorporating a trained model optimized for a reference vehicle state.

Benefits of technology

The system effectively evaluates vehicle conditions by minimizing the influence of driving condition variations and accurately determining differences in vehicle states through sound analysis, providing a standardized evaluation method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586135000001
    Figure 0007586135000001
  • Figure 0007586135000002
    Figure 0007586135000002
  • Figure 0007586135000003
    Figure 0007586135000003
Patent Text Reader

Abstract

To provide a vehicle evaluation system with which it is possible to discriminately evaluate a difference in vehicle degree from sound data.SOLUTION: The vehicle evaluation system comprises a processing circuit 110 and a storage device 120. In the storage device 120, operation data and data of a trained model having been trained so as to generate the operation data from sound data for training are stored. The processing circuit 110 executes generation processing for outputting generation data that is the operation data generated by inputting, to the trained model, data for evaluation including sound data for evaluation having been collected while operating an object vehicle 10 in an operation pattern for measurement. The processing circuit 110 also executes evaluation processing for comparing the generation data with the operation data, and evaluating the object vehicle 10 in accordance with the magnitude of deviation.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a vehicle evaluation system for evaluating vehicles. [Background technology]

[0002] There is a known inspection device that detects anomalies by analyzing sound data recorded from the sound produced by the inspection target. It is also being considered to detect anomalies using such sound data using a trained model trained by machine learning. However, anomalies rarely occur. Therefore, it is difficult to obtain sound data of abnormal sounds to be used for machine learning.

[0003] Patent Document 1 discloses generating pseudo abnormal sound data, which is sound data of pseudo abnormal sound, using normal sound data, which is sound data of normal sound. Patent Document 1 also discloses performing machine learning using normal sound data and pseudo abnormal sound data. According to the method disclosed in Patent Document 1, machine learning can be performed using the pseudo abnormal sound data even in a situation where it is difficult to obtain abnormal sound. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 026829 Summary of the Invention [Problem to be solved by the invention]

[0005] As a vehicle evaluation system for evaluating the condition of a vehicle, there is a system that evaluates the vehicle by analyzing sound data recorded from the vehicle. However, the sound emitted by the vehicle changes depending on the driving condition of the vehicle. In addition, in order to evaluate the condition of a vehicle, it is necessary not only to distinguish from the sound data whether the vehicle is obviously broken and making abnormal noise, but also to distinguish the difference in the level of the vehicle from the sound data and perform the evaluation. Therefore, a vehicle evaluation system suitable for evaluating vehicles is required. [Means for solving the problem]

[0006] The means for solving the above problems and their effects will be described below. A vehicle evaluation system for solving the above problem evaluates a target vehicle using sound data recorded from a target vehicle that is a vehicle to be evaluated. The vehicle evaluation system includes a processing circuit and a storage device. The storage device stores data of a learned model trained by supervised learning to generate operation data from training sound data including training sound data recorded while a reference vehicle in a state serving as a reference for evaluation is operated with a measurement operation pattern over a predetermined time period and operation data showing a transition of operation information in the measurement operation pattern. The processing circuit also performs a generation process of inputting evaluation data including evaluation sound data recorded while the target vehicle is operated with the measurement operation pattern into the learned model, and outputting generated data that is data of the operation information; The training data included in the supervised learning An evaluation process is executed in which the operation data is compared with the generated data, and the target vehicle is evaluated according to the degree of deviation between the operation data and the generated data.

[0007] The storage device of the vehicle evaluation system stores data of a trained model that generates data of operation information from sound data. Therefore, the vehicle evaluation system can generate operation data from sound data using the trained model. However, the trained model is optimized to generate operation data from sounds emitted from a reference vehicle. Therefore, when sound data emitted from a vehicle in a state different from that of the reference vehicle is input, the operation data cannot be generated correctly. In other words, when the state of the target vehicle deviates from the state of the reference vehicle, a discrepancy occurs between the operation data and the generated data.

[0008] Therefore, the above vehicle evaluation system executes an evaluation process to evaluate the target vehicle according to the magnitude of deviation between the operation data and generated data generated based on the sound emitted from the target vehicle when the target vehicle is operated according to the measurement operation pattern. The magnitude of deviation between the generated data and the operation data reflects the difference in the condition between the target vehicle and the reference vehicle. Therefore, the vehicle evaluation system can determine the difference in the level of the vehicles from the sound data and perform the evaluation. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing an embodiment of a vehicle evaluation system. [Diagram 2] FIG. 2 is a schematic diagram showing the configuration of the vehicle control unit. [Diagram 3] FIG. 3 is a time chart showing the measurement operation pattern. [Figure 4] FIG. 4 is a mel spectrogram. [Diagram 5] FIG. 5 is an explanatory diagram illustrating the structure of a trained model. [Figure 6] FIG. 6 is a flowchart of the data acquisition process. [Figure 7] FIG. 7 is a flowchart of the data formatting process. [Figure 8] FIG. 8 is a flowchart of the training process. [Figure 9]FIG. 9 is a flowchart relating to the generation process and the evaluation process. [Figure 10] FIG. 10 is a schematic diagram showing a modified example of the vehicle evaluation system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, one embodiment of the vehicle evaluation will be described with reference to FIGS. <Vehicle evaluation system configuration> Fig. 1 shows a configuration of a vehicle evaluation system according to an embodiment. As shown in Fig. 1, the vehicle evaluation system includes a data center 100 and a vehicle controller 300. The data center 100 is communicatively connected to the vehicle controller 300 via a communication network 200.

[0011] <Configuration of Data Center 100> 1, the data center 100 includes a storage device 120 in which a program is stored, and a processing circuit 110 that executes various processes by executing the program stored in the storage device 120. The data center 100 also includes a communication device 130. The communication device 130 is implemented as hardware such as a network adapter, various communication software, or a combination of these. The communication device 130 is configured to realize wired or wireless communication via a communication network 200.

[0012] The data center 100 may be configured using a plurality of computers. For example, the data center 100 may be configured by a plurality of server devices. <Configuration of vehicle controller 300> The vehicle controller 300 is, for example, a personal computer. The vehicle controller 300 includes a storage device 320 in which a program is stored, and a processing circuit 310 that executes various processes by executing the program stored in the storage device 320. The vehicle controller 300 also includes a communication device 330. The communication device 330 is implemented as hardware such as a network adapter, various communication software, or a combination of these. The communication device 330 is configured to realize wired or wireless communication via the communication network 200.

[0013] In this embodiment, the vehicle controller 300 is connected to the data center 100 by wireless communication via the communication network 200. The vehicle controller 300 also includes a display device 340 that displays information. The vehicle controller 300 may be a smartphone or a tablet terminal. The vehicle controller 300 also includes a microphone 350.

[0014] The microphone 350 is for recording sounds emitted from the target vehicle 10, which is a vehicle to be evaluated. When evaluating the target vehicle 10 using this vehicle evaluation system, the microphone 350 is installed at a predetermined position relative to the target vehicle 10. In addition, the vehicle controller 300 is connected to the vehicle control unit 20 of the target vehicle 10. Then, a command is output from the vehicle controller 300 to the vehicle control unit 20 to operate the target vehicle 10 in a predetermined measurement operation pattern, while recording sounds with the microphone 350.

[0015] <Configuration of vehicle control unit 20> 1, the vehicle control unit 20 includes a storage device 22 in which a program is stored, and a processing circuit 21 that executes various controls by executing the program stored in the storage device 22. The vehicle control unit 20 controls each part of the target vehicle 10.

[0016] The vehicle control unit 20 is connected to various sensors that detect the state of the target vehicle 10. For example, the vehicle control unit 20 is connected to a vehicle speed sensor 35 that detects the vehicle speed SPD, which is the speed of the target vehicle 10. The vehicle control unit 20 acquires information on the vehicle speed SPD detected by the vehicle speed sensor 35. For example, the vehicle control unit 20 is connected to a crank position sensor 34. The crank position sensor 34 outputs a crank angle signal according to a change in the rotation phase of a crankshaft, which is the output shaft of an internal combustion engine mounted on the target vehicle 10. The vehicle control unit 20 calculates the engine rotation speed NE, which is the rotation speed of the crankshaft, based on a detection signal of the rotation angle of the crankshaft input from the crank position sensor 34. For example, the vehicle control unit 20 is connected to an air flow meter 33. The air flow meter 33 detects an intake air temperature THA, which is the temperature of air drawn into a cylinder through an intake passage of the internal combustion engine mounted on the target vehicle 10, and an intake air amount Ga, which is the mass of the drawn air. The vehicle control unit 20 acquires information on the intake air temperature THA and the intake air amount Ga detected by the air flow meter 33. For example, a water temperature sensor 36 is connected to the vehicle control unit 20. The water temperature sensor 36 detects a water temperature THW, which is the temperature of the cooling water. The vehicle control unit 20 acquires the water temperature THW detected by the water temperature sensor 36. For example, an air-fuel ratio sensor 37 is connected to the vehicle control unit 20. The air-fuel ratio sensor 37 is installed in an exhaust pipe of the internal combustion engine. The air-fuel ratio sensor 37 detects the air-fuel ratio of the exhaust. The vehicle control unit 20 acquires the air-fuel ratio detected by the air-fuel ratio sensor 37. For example, a throttle opening sensor 31 is connected to the vehicle control unit 20. The throttle opening sensor 31 detects a throttle opening, which is the opening of a throttle valve provided in an intake passage of the internal combustion engine. The vehicle control unit 20 acquires the throttle opening detected by the throttle opening sensor 31. For example, a brake hydraulic pressure sensor 32 is connected to the vehicle control unit 20. The brake hydraulic pressure sensor 32 detects the brake hydraulic pressure. The vehicle control unit 20 obtains the brake hydraulic pressure detected by the brake hydraulic pressure sensor 32.

[0017] The vehicle control unit 20 is also connected to a transmission control unit 30 that controls the transmission mounted on the target vehicle 10. The vehicle control unit 20 acquires information from the transmission control unit 30, such as the gear ratio and the oil temperature of the transmission.

[0018] When the vehicle controller 300 is connected to the vehicle control unit 20 of the target vehicle 10, it becomes possible to operate the target vehicle 10 according to commands from the vehicle controller 300. In addition, when the vehicle controller 300 is connected to the vehicle control unit 20 of the target vehicle 10, it becomes possible for the vehicle controller 300 to obtain information of the target vehicle 10 through the vehicle control unit 20.

[0019] <Evaluation process using the vehicle evaluation system> As described above, in this vehicle evaluation system, when evaluating the target vehicle 10, the vehicle controller 300 is connected to the vehicle control unit 20 of the target vehicle 10. Then, the target vehicle 10 is operated in a predetermined measurement operation pattern in response to a command from the vehicle controller 300, while recording sounds with the microphone 350. The vehicle controller 300 transmits data including data on the recorded sounds to the data center 100. Then, the data center 100 executes an evaluation process for evaluating the target vehicle 10 using the received data.

[0020] <About the measurement operation pattern> Fig. 3 is a time chart showing a measurement operation pattern in this vehicle evaluation system. As shown in Fig. 3, the measurement operation pattern of this vehicle evaluation system is to operate the throttle opening and brake hydraulic pressure over a predetermined time from time t0 to time t10. Note that the operation in the measurement operation pattern is not feedback control but feedforward control in which the operation is performed directly according to a command.

[0021] Specifically, at time t0 when this measurement operation pattern is started, the throttle opening is 0, as shown in FIG. 3(a). Then, as shown in FIG. 3(b), the brake oil pressure is bp2. Note that bp2 is a value greater than 0. Then, at time t1, the vehicle controller 300 operates the target vehicle 10 via the vehicle control unit 20 so as to set the brake oil pressure to 0. Next, at time t2, the vehicle controller 300 operates the target vehicle 10 via the vehicle control unit 20 so as to set the throttle opening to th2. Note that th2 is a value greater than 0.

[0022] As shown in FIG. 3, at time t3, the vehicle controller 300 operates the target vehicle 10 through the vehicle control unit 20 to set the throttle opening to th1. Here, th1 is a value smaller than th2 and larger than 0. At time t4, the vehicle controller 300 operates the target vehicle 10 through the vehicle control unit 20 to set the throttle opening to 0. Then, the vehicle controller 300 operates the brake hydraulic pressure through the vehicle control unit 20 to increase the brake hydraulic pressure at a constant rate between time t5 and time t6. The vehicle controller 300 increases the brake hydraulic pressure to bp1. Here, bp1 is a value larger than 0 and smaller than bp2. At time t6, the increase in the brake hydraulic pressure is stopped, and the brake hydraulic pressure is maintained at bp1 after time t6. Then, the measurement operation pattern ends at time t10.

[0023] <About the evaluation data> The vehicle controller 300 records the sound data collected by the microphone 350 while operating the target vehicle 10 with the measurement operation pattern in the storage device 320 as evaluation sound data. In addition, the intake air temperature THA detected by the air flow meter 33 is stored in the storage device 320 as information on the outside air temperature when the target vehicle 10 is operated with the measurement operation pattern. The vehicle controller 300 then stores the collected evaluation sound data and outside air temperature data in the storage device 320 as one data set for a predetermined time period.

[0024] The vehicle controller 300 cuts out each data range of a window Tw having a time width shorter than the predetermined time from the data set for the predetermined time stored in the storage device 320, and shapes the data into evaluation data. Note that, in the data shaping process for shaping the evaluation data, the vehicle controller 300 converts the evaluation sound data into a mel spectrogram that has been subjected to frequency analysis, and handles the converted data as image data.

[0025] As shown in FIG. 4, in a mel spectrogram, the vertical axis is frequency, and the frequency on the vertical axis is shown in the mel scale. The horizontal axis is the time axis. In a mel spectrogram, intensity is represented by color. The lower the intensity, the darker the blue color, and the higher the intensity, the brighter the red color. For example, from low to high intensity, the colors are blue, green, yellow, orange, and red. Sound data contained in one data set for a predetermined time period becomes one mel spectrogram for the predetermined time period from time t0 to time t10, as shown in FIG. 4.

[0026] As shown in FIG. 3 and FIG. 4, data extraction is to extract data within a window Tw having a time width shorter than a predetermined time. The data shaping process starts from a point corresponding to time t0, and each time cutting is completed, the window Tw is shifted toward time t10 by a certain stride t_st, and data is cut again. Then, data cutting is repeated until data including the time t10 is cut. Then, data cut at the same time is collected into one list. Note that the stride t_st is a length corresponding to a time width shorter than that of the window Tw. Details of the data shaping process will be described later with reference to FIG. 7.

[0027] The vehicle controller 300 transmits the shaped evaluation data to the data center 100. Then, the data center 100 inputs the evaluation data shaped into a plurality of lists into a trained model trained by supervised learning, and performs evaluation processing.

[0028] <About trained models> The storage device 120 of the data center 100 stores data of the trained model used to evaluate the target vehicle 10.

[0029] In the data center 100, the evaluation sound data is handled as image data, so a model that partially uses ResNet-18, which is an image classification model, is used. Note that ResNet-18 is a pre-trained image classification model trained on the ImageNet dataset. ResNet-18 has been trained on more than one million pieces of image data and can classify input images into 1,000 categories. The trained model stored in the storage device 120 of the data center 100 is a model that has been transferred from the pre-trained ResNet-18.

[0030] Fig. 5 is an explanatory diagram illustrating the configuration of a trained model stored in the storage device 120. As shown in Fig. 5, this trained model is a model in which the classification output layer of ResNet-18 is replaced with a neural network MLP, and this neural network MLP is trained by supervised learning.

[0031] The input layer Lin of the neural network MLP is composed of a first input layer Lin1 to which the output from ResNet-18 is input, and a second input layer Lin2. This allows the vehicle evaluation system to reflect data other than the evaluation sound data contained in the evaluation data in the evaluation of the target vehicle 10. For example, the second input layer Lin2 is provided with a node for inputting outside air temperature data. Note that by increasing the number of nodes in the second input layer Lin2 according to the amount of input data, it becomes possible to use other data in the evaluation data.

[0032] In this vehicle evaluation system, evaluation data acquired while operating the target vehicle 10 with a measurement operation pattern is input to a learned model, and a generation process is executed to generate operation data indicating a transition of operation information of the target vehicle 10 in the measurement operation pattern as generated data. As described with reference to FIG. 3, the operation information in the measurement operation pattern is the throttle opening and the brake oil pressure. Therefore, the output layer Lout of the neural network MLP is composed of a node that outputs the throttle opening and a node that outputs the brake oil pressure. Note that the types and number of operation information used for evaluation can be increased by increasing the number of nodes in the output layer Lout of the neural network MLP according to the number of data to be output.

[0033] In this way, the trained model used in this vehicle evaluation system is a model that has been transfer-trained into a regression model that outputs throttle opening and brake oil pressure, with the classification output layer of ResNet-18 replaced by an MLP neural network.

[0034] <About model training> Next, a training process for training the model shown in FIG. 5 to obtain a trained model will be described. The model is trained by supervised learning using a large amount of measurement data collected in advance using a reference vehicle that serves as the basis for evaluation. In this example, the reference vehicle is a vehicle that has been subjected to a certain amount of break-in driving after manufacture, and has been thoroughly inspected and confirmed to have no abnormalities. In other words, the reference vehicle is a vehicle in extremely good condition with almost no deterioration.

[0035] <About data acquisition processing> The data acquisition process for acquiring the measurement data is executed by a computer, similar to the vehicle controller 300, which is connected to the vehicle control unit 20 and capable of operating the vehicle.

[0036] Fig. 6 is a flowchart showing the flow of the data acquisition process for acquiring measurement data. As shown in Fig. 6, when the computer starts the data acquisition process, it first starts the measurement operation in the process of step S100 to start measuring data. Then, the computer records sound data with microphone 350 while operating the reference vehicle with the measurement operation pattern, and acquires outside air temperature data. Note that this sound data, which records the sound emitted by the reference vehicle in order to train the model, is training sound data.

[0037] In the process of step S110, the computer determines whether the measurement operation in the measurement operation pattern is completed. If it is determined that the measurement operation in the measurement operation pattern is not completed (step S110: NO), the computer repeats the process of step S110.

[0038] On the other hand, if the computer determines in the process of step S110 that the measurement operation in the measurement operation pattern is completed (S110: YES), the computer proceeds to the process of step S120. Then, in the process of step S120, the computer ends the measurement of the data.

[0039] When the measurement is thus completed, in the process of step S130, the computer records the measurement data acquired while performing the measurement operations in the series of measurement operation patterns and the data of operation information in the measurement operation patterns as one data set in the storage device.

[0040] That is, the computer records the throttle opening data, brake oil pressure transition data, sound data, and outside air temperature data in the measurement pattern as one data set in the storage device. Then, the computer temporarily ends this data acquisition process.

[0041] This completes the acquisition of one data set. Training data to be used in supervised learning is collected by repeating this data acquisition process many times to collect a large data set of measurement data recorded while operating the reference vehicle using the measurement operation patterns.

[0042] The data set of measurement data collected in this way is shaped into data to be input into the model through the data shaping process shown in Figure 7. <About data formatting processing> As described above, the data shaping process is a process in which one data set is cut out while shifting the window Tw and shaped into multiple lists.

[0043] This data shaping process is performed by a computer. The computer that performs this data shaping process may be the same as the computer that executes the data acquisition process, or may be a different computer. For example, a computer with high processing power such as the data center 100 may be used. Note that a large amount of data sets collected by repeatedly executing the data acquisition process are stored in the storage device of the computer that executes the data shaping process.

[0044] When the data shaping process starts, the computer first reads one data set of measurement data in the process of step S200. Next, in the process of step S210, the computer converts the sound data in the data set read in the process of step S200 into a mel spectrogram. Then, the computer proceeds to the process of step S220. In the process of step S210, the computer standardizes the data other than the sound data in the data set.

[0045] Next, in the process of step S230, the computer sets the data extraction start time t to 0. Then, in the process of step S240, the computer extracts the data. That is, the start point of the window Tw is aligned with the extraction start time t, and the data within the window Tw is extracted. Specifically, as shown in FIG. 4, an image within the window Tw range is extracted from the mel spectrogram. Also, as shown in FIG. 3, data within the window Tw range is extracted from the outside air temperature data, throttle opening data, and brake oil pressure data.

[0046] In the process of step S250, the computer calculates representative values ​​of the outside air temperature data, throttle opening data, and brake hydraulic pressure data extracted through the process of step S240. For example, the average value of the data in window Tw is calculated as the representative value in this window Tw. Instead of the average value, a maximum value or a minimum value may be calculated as the representative value. After calculating the representative value of the outside air temperature, throttle opening, and brake hydraulic pressure in the extracted data in this way, the computer proceeds to the process of step S260.

[0047] In the process of step S260, the computer judges whether or not to shift the window Tw by the stride t_st. The data extraction is performed by repeatedly shifting the window Tw by the stride t_st for the data set acquired during the measurement operation pattern. When the window Tw reaches the end of the data set and all the data contained in the data set has been extracted, the window Tw can no longer be shifted by the stride t_st. When the computer is unable to shift the window Tw by the stride t_st in this way, it makes a negative judgment in the process of step S260.

[0048] In the process of step S260, if it is determined that the window Tw is shifted by the stride t_st (step S260: YES), the computer proceeds to the process of step S270. In the process of step S270, the computer stores the data of the cut-out image and each set of representative values ​​in one list. When storing in the list, the computer resizes the image data to a size of 224×224 suitable for input to ResNet-18. Then, the computer updates the cut-out start time t. Specifically, the cut-out start time t is updated by adding the stride t_st to the cut-out start time t as the new cut-out start time t. As a result, the window Tw is shifted by the stride t_st.

[0049] Then, the computer shifts the window Tw by the stride t_st and executes the processes of steps S240 to S260 again. That is, the computer repeats the processes of steps S240 to S280 until the window Tw can no longer be shifted by the stride t_st.

[0050] If the computer determines in the process of step S260 that the window Tw cannot be shifted (step S260: NO), the computer proceeds to step S290. The process of step S290 is the same as the process of step S270. When the computer stores the data in the list in the process of step S290, the computer proceeds to step S300.

[0051] In the process of step S300, the computer determines whether all the data sets prepared for shaping in the process of step S200 have been processed. If it is determined in the process of step S300 that the processing of all the data sets has not been completed (step S300: NO), the computer returns the process to step S200. Then, the computer reads one unprocessed data set and executes the processes from step S210 onward.

[0052] When it is determined in the process of step S300 that all the data sets have been processed (step S300: YES), the computer ends this series of data shaping processes. In this way, the computer shapes the prepared data sets of measurement data into the form of a plurality of lists. In this way, a training process is performed to train a model using a large number of data sets each shaped into a set of a plurality of lists through the data shaping process.

[0053] <About the training process> This training process is performed by a computer. It is preferable to use a computer with high processing power such as the data center 100 as the computer performing this training process. A large amount of data set shaped through a data shaping process is stored as training data in a storage device of the computer that executes the training process. The amount of learning data stored in the storage device is set to an amount that can obtain a trained model with sufficient accuracy based on the results of a verification performed in advance.

[0054] When the training process starts, the computer first reads the training data stored in the storage device in step S400. Then, in the next step S410, the computer reads one data set from the training data that has been read.

[0055] Next, in the process of step S420, the computer reads one list from the data set. Then, the computer inputs the data of the list into the model described with reference to FIG. 5 to calculate the throttle opening and brake oil pressure. When the training process is started, the parameters of the model are in the initial state. That is, the ResNet-18 portion is in a learned state, but the weights and biases of the neural network MLP portion are in the initial values. In this training process, the weights and biases of the neural network MLP portion are updated.

[0056] Specifically, in the process of step S430, the image data Dw resized to a size of 224 x 224 and included in the list is input to ResNet-18 as shown in FIG. 5. Then, a representative value of the outside air temperature data is input to the second input layer Lin2 of the neural network MLP. The feature amount of the image data Dw is extracted through ResNet-18 and input to the first input layer Lin1 of the neural network MLP. Then, the values ​​of the throttle opening and brake oil pressure are output from the output layer Lout of the neural network MLP. After calculating the throttle opening and brake oil pressure in this way, the computer records the values ​​of the throttle opening and brake oil pressure in the process of step S440.

[0057] Next, in the process of step S450, the computer determines whether all the lists included in the data set have been processed. If it is determined in the process of step S450 that the processing of all the lists has not been completed (step S450: NO), the computer returns the process to step S420. Then, the computer reads one unprocessed list and executes the processes from step S430 onward.

[0058] If it is determined in the process of step S450 that all the lists have been processed (step S450: YES), the computer proceeds to the process of step S460. In this manner, the computer calculates the throttle opening value and the brake oil pressure value for each list included in the loaded data set. Then, the computer records the calculated values ​​and generates data showing the transition of the throttle opening degree and the brake oil pressure in the measurement operation pattern, as shown in the time chart of FIG.

[0059] In the process of step S460, the computer calculates an evaluation index value. The evaluation index value is a value indicating the magnitude of deviation between the value calculated through the process of step S430 and the data of the throttle opening and brake oil pressure in the measurement operation pattern included in the data set. The data of the throttle opening and brake oil pressure in the measurement operation pattern included in the data set are correct values.

[0060] Here, the computer calculates the magnitude of deviation between the value calculated through the process of step S430 and the correct value. For example, the computer calculates the square sum error between the throttle opening calculated through the process of S430 and the correct throttle opening, and the square sum error between the brake oil pressure calculated through the process of S430 and the correct brake oil pressure. The computer then calculates the sum of these in the measurement operation pattern for a preset time as an evaluation index value. After calculating the evaluation index value in this way, the computer proceeds to step S470.

[0061] In the process of step S470, the computer performs learning. Specifically, the computer adjusts the weights and biases in the neural network MLP by the error backpropagation method so as to reduce the value of the evaluation index value.

[0062] Then, in the process of step S480, the computer determines whether all the data sets included in the loaded training data have been processed. If, in the process of step S480, it is determined that the processing of all the data sets has not been completed (step S480: NO), the computer returns the process to step S410. Then, the computer reads one unprocessed data set and executes the processes of step S420 and after. The computer thus repeats learning to train the model until the processing of all the data sets is completed. In this way, the computer trains the model so that it can generate operation data, which is data indicating the transition of operation information in the measurement operation pattern, from the training data by supervised learning.

[0063] If it is determined in the process of step S480 that all the data sets have been processed (step S480: YES), the computer proceeds to the process of step S490. In the process of step S490, the computer records the parameters of the model that has completed learning using all the data sets in the storage device. Then, the computer ends this series of training processes. In this way, data of the trained model is obtained through the training process.

[0064] The storage device 120 of the data center 100 stores data of the learned model thus trained through the training process. Next, the data acquisition process, data formatting process, and evaluation process when evaluating the target vehicle 10 using the vehicle evaluation system will be described.

[0065] <Data Acquisition Process by Vehicle Controller 300> When evaluating a vehicle using the vehicle evaluation system, as described above, the vehicle controller 300 is connected to the target vehicle 10, which is the vehicle to be evaluated. In addition, the microphone 350 is installed in the target vehicle 10. Then, the target vehicle 10 is operated in a predetermined measurement operation pattern according to a command from the vehicle controller 300, and the microphone 350 records sound. In addition, the vehicle controller 300 simultaneously acquires outside air temperature data. Specifically, the vehicle controller 300 executes the data acquisition process described with reference to FIG. 6 to acquire one data set as evaluation data. The sound data included in the evaluation data is the evaluation sound data.

[0066] <Data Formatting Process by Vehicle Controller 300> The vehicle controller 300 performs a data shaping process to shape the evaluation data. Specifically, the data shaping process described with reference to FIG. 7 is performed to shape the evaluation data into a plurality of lists. Since only one evaluation data is acquired in the data acquisition process, only one data set is read and shaped in the data shaping process at this time. Through this data shaping process, the evaluation data is shaped into a plurality of lists of image data Dw and data of representative values ​​of outside air temperature.

[0067] After executing the data shaping process to shape the evaluation data, the vehicle controller 300 transmits the shaped evaluation data to the data center 100. <Evaluation process by data center 100> Upon receiving the evaluation data, the data center 100 stores the evaluation data in the storage device 120. Then, the data center 100 executes an evaluation process to evaluate the target vehicle 10 by executing a routine shown in Fig. 9. The routine shown in Fig. 9 is executed by the processing circuit 110 of the data center 100.

[0068] 9, when this routine starts, the data center 100 reads the evaluation data stored in the storage device 120 in the process of step S500. Then, through the processes of steps S510 to S540, the calculation of the throttle opening and the brake oil pressure is repeated using the trained model stored in the storage device 120.

[0069] Specifically, in the process of step S510, the data center 100 reads one list from the evaluation data. Then, in the process of step S520, the data center 100 inputs the data of the list into the trained model to calculate the throttle opening and brake oil pressure, similar to the process of step S430 in the training process. After calculating the throttle opening and brake oil pressure in this way, the data center 100 records the values ​​of the throttle opening and brake oil pressure in the process of step S530.

[0070] Next, in the process of step S540, the data center 100 determines whether all the lists included in the data set have been processed. If it is determined in the process of step S540 that the processing of all the lists has not been completed (step S540: NO), the data center 100 returns the process to step S510. Then, the data center 100 reads one unprocessed list and executes the processes from step S520 onwards. If it is determined in the process of step S540 that all the lists have been processed (step S540: YES), the data center 100 advances the process to step S550.

[0071] In this way, the data center 100 calculates the throttle opening value and the brake oil pressure value for each list included in the loaded evaluation data.The calculated values ​​are then recorded to generate generated data that indicates the transition of the throttle opening degree and the brake oil pressure in the measurement operation pattern.The series of processes from step S510 to step S540 is a generation process that inputs the evaluation data into the trained model and outputs the generated data.

[0072] In the process of step S550, the data center 100 calculates the evaluation index value in the same manner as the process of step S460 in the training process. The trained model is optimized to generate generated data in which operation data is restored from sounds emitted from a reference vehicle. Therefore, when evaluation sound data emitted from a target vehicle 10 in a state different from that of the reference vehicle is input, the operation data cannot be restored correctly. That is, if the state of the target vehicle 10 deviates from the state of the reference vehicle, a deviation occurs between the operation data stored in the data set as correct data and the generated data. The evaluation index value indicates the magnitude of this deviation. That is, the larger the evaluation index value, the more the state of the target vehicle 10 deviates from the state of the reference vehicle. As described above, the reference vehicle is a vehicle in an extremely good state with almost no deterioration. Therefore, in this evaluation system, the smaller the evaluation index value, the closer the state of the target vehicle 10 is to the state of the reference vehicle, and the higher the evaluation is. After calculating the evaluation index value in this manner, the data center 10 advances the process to step S560.

[0073] In the process of step S560, the data center 100 determines the evaluation rank based on the evaluation index value. The data center 100 determines the evaluation rank by selecting an evaluation rank according to the magnitude of the evaluation index value from among four evaluation ranks, S rank, A rank, B rank, and C rank. S rank is the evaluation rank indicating that the vehicle is of the highest level of evaluation among the four evaluation ranks. And C rank is the evaluation rank indicating that the vehicle is of the lowest level of evaluation among the four evaluation ranks. The evaluations decrease in order from S rank to A rank, B rank, and C rank, with C rank being the lowest evaluation rank.

[0074] The data center 100 selects rank S as the evaluation rank when the evaluation index value is equal to or less than the first threshold. The data center 100 selects rank A as the evaluation rank when the evaluation index value is greater than the first threshold and equal to or less than the second threshold. The second threshold is a value greater than the first threshold. The data center 100 selects rank B as the evaluation rank when the evaluation index value is greater than the second threshold and equal to or less than the third threshold. The third threshold is a value greater than the second threshold. The data center 100 selects rank C as the evaluation rank when the evaluation index value is equal to or greater than the third threshold.

[0075] The magnitudes of the first threshold, the second threshold, and the third threshold may be set in advance, for example, with reference to the correlation between vehicle evaluation and price in the used car market. They may also be set based on the results of vehicle durability tests and simulations. After determining the evaluation rank of the target vehicle 10 through the processing of step S560, the data center 100 proceeds to the processing of step S570.

[0076] The process of step S550 and the process of step S560 are evaluation processes in which the operation data is compared with the generated data and the target vehicle 10 is evaluated according to the degree of deviation between the operation data and the generated data.

[0077] In the process of step S570, the data center 100 outputs the evaluation rank by transmitting the evaluation rank to the vehicle controller 300. After outputting the evaluation rank in this manner, the data center 100 ends this routine.

[0078] The vehicle controller 300 that receives the evaluation rank displays the received evaluation rank on the display device 340 as the evaluation rank of the target vehicle 10 . <Action of this embodiment> In this vehicle evaluation system, in order to evaluate the target vehicle 10, the target vehicle 10 is operated by the vehicle controller 300 with a predetermined measurement operation pattern. Then, evaluation data is acquired while operating the target vehicle 10 with the predetermined measurement operation pattern. The vehicle controller 300 transmits the acquired evaluation data to the data center 100.

[0079] The data center 100 executes a generation process using the trained model to generate generated data by restoring operation data from the evaluation data. The trained model is a neural network that uses the feature amount of data extracted from data for a predetermined time period as explanatory variables and operation information at the time point corresponding to the extracted data as objective variables. The data center 100 then executes an evaluation process based on the generated data.

[0080] In the evaluation process, the data center 100 calculates, as an evaluation index value, the sum of the deviations output for a predetermined time period for each piece of data repeatedly extracted while changing the extraction start time t. The data center 100 determines the evaluation rank of the target vehicle 10 based on the evaluation index value. Then, the data center 100 transmits the evaluation rank to the vehicle controller 300.

[0081] The evaluation rank is displayed on the display device 340 of the vehicle controller 300 that has received the evaluation rank. <Effects of this embodiment> (1) The difference between the condition of the target vehicle 10 and the reference vehicle appears in the magnitude of the deviation between the generated data and the operation data. Therefore, the vehicle evaluation system can determine the difference in the condition of the vehicle from the sound data and perform the evaluation.

[0082] (2) By performing the evaluation based on the sound data acquired when operating with a predefined evaluation measurement pattern, the driving conditions of the target vehicle 10 when collecting the sound data used for the evaluation are standardized. Therefore, the above vehicle evaluation system can suppress the influence of variations in sound data due to differences in driving conditions.

[0083] (3) The trained model generates operation data from evaluation data including image data of a spectrogram obtained by frequency-analyzing sound data. That is, the vehicle evaluation system uses image data obtained by frequency-analyzing sound data. Therefore, the vehicle evaluation system can efficiently extract features contained in the sound data and perform evaluation processing.

[0084] (4) The above vehicle evaluation system divides data for a given time into multiple sections and analyzes them. The results are then integrated to calculate the evaluation index value. Therefore, according to the above vehicle evaluation system, the trained model can be smaller than when analyzing data for a given time collectively.

[0085] (5) The vehicle evaluation system outputs the evaluation results by applying them to a preset evaluation rank. Therefore, this evaluation system makes it easier to understand the relative level of the condition of the target vehicle 10 in the used car market.

[0086] (6) Sound data may be affected by differences in the measurement environment even if the vehicle condition is the same. The above vehicle evaluation system includes outside temperature data in the training data and evaluation data. Therefore, the vehicle evaluation system can perform evaluations that reflect the effects of differences in outside temperature.

[0087] (7) The driving operation with the measurement operation pattern when acquiring the training data and the driving operation with the measurement operation pattern when acquiring the evaluation data can be performed by an operator operating the vehicle. However, if there is variation in the driving operation by the operator, the variation in the driving operation will affect the evaluation result. In contrast, the above vehicle evaluation system operates the target vehicle 10 by the vehicle controller 300. Therefore, it is possible to evaluate the target vehicle 10 while eliminating the influence caused by the variation in the driving operation by the operator.

[0088] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined with each other to the extent that there is no technical contradiction.

[0089] Although an example has been given in which the data shaping process for evaluating the target vehicle 10 is executed by the vehicle controller 300, the data shaping process may also be executed by the data center 100. Although an example has been shown in which sound data converted into a mel spectrogram is used, the data used as measurement data and evaluation data does not have to be a mel spectrogram. For example, a spectrogram obtained by wavelet transforming sound data may be used. Also, a spectrogram obtained by short-time Fourier transforming sound data may be used.

[0090] It is not necessary to convert sound data into image data. For example, features may be extracted from the sound data itself and used as measurement data and evaluation data. In that case, the model used for evaluation processing does not need to be ResNet-18, which handles image data. For example, a neural network can be used.

[0091] Although we have shown a model that uses ResNet-18 for transfer learning, the model structure is not limited to this. The trained model only needs to be able to output generated data based on the evaluation data.

[0092] Although an example of the evaluation process in which the evaluation rank is determined based on the evaluation index value has been shown, the evaluation process is not limited to this. For example, the evaluation index value may be output as it is, with the larger the value, the lower the evaluation, and displayed on the display device 340.

[0093] In the above vehicle evaluation system, an example has been given in which a vehicle in extremely good condition is used as the reference vehicle to evaluate the target vehicle 10. It is not necessary to use a vehicle in good condition as the reference vehicle. For example, a vehicle in extremely poor condition with a low rating can also be used as the reference vehicle. The evaluation index value in the above evaluation process is a value indicating the degree of deviation between the condition of the target vehicle 10 and that of the reference vehicle. Therefore, when an aged vehicle with an extremely low rating is used as the reference vehicle, the smaller the evaluation index value, the lower the rating. The target vehicle 10 can also be evaluated using such evaluation index values.

[0094] The operation data may include other operation information. For example, the operation data may include information on the gear ratio. The operation data may also include information on the steering angle. The operation data may also include information on the operating status of the vehicle's air conditioning system. The operation data may also include information on the operation of the vehicle's electrical components such as wipers and power windows.

[0095] Furthermore, operation information of these electrical components may be included in the training data and evaluation data. In addition to the sound data, the training data and the evaluation data may include data on the gear ratio obtained from the transmission control unit 30. The sound data also changes when the gear ratio is changed. In this way, by adding the data on the gear ratio to the training data and the evaluation data, it becomes possible to perform an evaluation that reflects the influence of differences in the gear ratio.

[0096] The training data and the evaluation data may include data on the water temperature THW detected by the water temperature sensor 36. The training data and the evaluation data may include data on the air-fuel ratio detected by the air-fuel ratio sensor 37.

[0097] Although an example has been shown in which the vehicle evaluation system includes the data center 100 and the vehicle controller 300, the configuration of the vehicle evaluation system is not limited to this configuration. For example, the vehicle evaluation system may be configured with only the data center 100 that performs the generation process and the evaluation process. In this case, the vehicle evaluation system is a system that performs the generation process and the evaluation process using the received evaluation data and outputs the evaluation result. That is, in this case, the vehicle evaluation system does not have the function of operating the target vehicle 10.

[0098] In addition, for example, data of the trained model may be stored in the storage device 320 of the vehicle controller 300, and the vehicle evaluation system may be configured using only the vehicle controller 300. In this case, the generation process and the evaluation process are executed in the vehicle controller 300.

[0099] Although an example has been shown in which the vehicle controller 300 is connected to the vehicle control unit 20 of the target vehicle 10 to operate the target vehicle 10, the method of operating the target vehicle 10 is not limited to this. For example, as shown in FIG. 10, the vehicle controller 300 may be provided with an actuator 360 that physically operates the target vehicle 10. In such a configuration, the target vehicle 10 can be operated in a measurement operation pattern by operating the accelerator pedal or brake pedal with the actuator 360. Note that the actuator 360 may be anything that can operate an operation target, and may be a robot, for example.

[0100] Although an example has been shown in which the vehicle controller 300 is provided with the microphone 350, the vehicle controller 300 itself does not have to be provided with the microphone 350. Sound data recorded while performing operations in the measurement operation pattern can be obtained from an external device and subjected to data shaping processing, generation processing, and evaluation processing.

[0101] The generation process and evaluation process may be performed using a plurality of pieces of sound data recorded using a plurality of microphones 350. The operation in the measurement operation pattern may be performed by an operator.

[0102] An example of evaluation using four evaluation ranks has been shown. The number of evaluation ranks does not have to be four. For example, the number of evaluation ranks may be increased. Conversely, the number of evaluation ranks may be decreased. For example, evaluation may be performed using two ranks, that is, vehicles in good condition and vehicles in poor condition.

[0103] In the above embodiment, an example has been described in which the vehicle evaluation system evaluates the target vehicle 10 itself. In contrast, the vehicle evaluation system may evaluate the target vehicle 10 by evaluating a specific unit in the target vehicle 10. For example, the vehicle evaluation system may evaluate the transmission mounted on the target vehicle 10 using sound data recorded from sounds emitted from the transmission.

[0104] In the above embodiment, the vehicle evaluation system includes a processing circuit 110 and a storage device 120 to execute software processing. However, this is merely an example. For example, the vehicle evaluation system may include a dedicated hardware circuit (e.g., ASIC, etc.) that processes at least a part of the software processing executed in the above embodiment. That is, the vehicle evaluation system may have any of the following configurations (A) to (C). (A) The vehicle evaluation system includes an execution device that executes all processing according to a program, and a storage device that stores the program. That is, the vehicle evaluation system includes a software execution device. (B) The vehicle evaluation system includes an execution device that executes a part of the processing according to a program, and a storage device. Furthermore, the vehicle evaluation system includes a dedicated hardware circuit that executes the remaining processing. (c) The vehicle evaluation system includes a dedicated hardware circuit that executes all processing. Here, the software execution device and / or the dedicated hardware circuit may be multiple. That is, the above processing may be executed by a processing circuitry that includes at least one of one or more software execution devices and one or more dedicated hardware circuits. The storage device that stores the program, i.e., the computer-readable medium, includes any available medium that can be accessed by a general-purpose or dedicated computer.

[0105] The above embodiment and modified examples include the configurations described in the following supplementary notes. [Appendix 1] A vehicle evaluation system that evaluates a target vehicle using sound data recorded from the target vehicle, which is a vehicle to be evaluated, comprising a processing circuit and a storage device, wherein the storage device stores data of a learned model trained by supervised learning to generate operation data from the training sound data, the operation data including training sound data recorded while a reference vehicle in a state that serves as a reference for evaluation is operated using a measurement operation pattern over a predetermined period of time, and operation data indicating a transition of operation information in the measurement operation pattern, and the processing circuit executes a generation process of inputting evaluation data including evaluation sound data recorded while operating the target vehicle using the measurement operation pattern into the trained model, and outputting generation data, which is data of the operation information, and an evaluation process of comparing the operation data with the generation data and evaluating the target vehicle depending on the magnitude of deviation between the operation data and the generation data.

[0106] [Appendix 2] The vehicle evaluation system described in [Appendix 1], in which the trained model generates the operation data from evaluation data including image data of a spectrogram obtained by frequency analysis of the sound data.

[0107] [Appendix 3] The vehicle evaluation system described in [Appendix 1] or [Appendix 2], wherein the trained model is a neural network that uses features of data extracted from the data for the specified time period as explanatory variables and the operation information at the time corresponding to the extracted data as a target variable.

[0108] [Appendix 4] The vehicle evaluation system described in [Appendix 3], wherein the evaluation process calculates, as an evaluation index value, the sum of the deviations output for the specified time period for each of the data repeatedly extracted while changing the extraction start time.

[0109] [Appendix 5] The vehicle evaluation system described in [Appendix 4], wherein the evaluation process selects and outputs an evaluation rank corresponding to the magnitude of the evaluation index value from among a plurality of evaluation ranks classified according to the magnitude of the evaluation index value.

[0110] [Appendix 6] A vehicle evaluation system according to any one of [Appendix 1] to [Appendix 5], wherein the operation data includes data on accelerator opening and brake oil pressure. [Appendix 7] A vehicle evaluation system according to any one of [Appendix 1] to [Appendix 6], wherein the training data and the evaluation data include outside air temperature data in addition to the sound data.

[0111] [Appendix 8] A vehicle evaluation system according to any one of [Appendix 1] to [Appendix 7], wherein the training data and the evaluation data include data on a gear ratio in addition to the sound data.

[0112] [Appendix 9] Any one of the vehicle evaluation systems of [Appendix 1] to [Appendix 8], comprising a vehicle controller that operates the target vehicle according to the measurement operation pattern. [Appendix 10] A vehicle evaluation system as described in [Appendix 9], wherein the vehicle controller is provided with an actuator that realizes driving operations of the vehicle.

[0113] [Appendix 11] A vehicle evaluation system as described in [Appendix 9] or [Appendix 10], wherein the vehicle controller is equipped with a microphone for recording the sound data. [Appendix 12] A vehicle evaluation system described in any one of [Appendix 9] to [Appendix 11], including a data center having the processing circuit and the memory device, and the vehicle controller connected to the data center via a communication network. [Explanation of symbols]

[0114] 10…Target vehicle 20…Vehicle control unit 21... Processing circuit 22...Storage device 30...Transmission control unit 31...Throttle opening sensor 32...Brake oil pressure sensor 33...Air flow meter 34...Crank position sensor 35...Vehicle speed sensor 36...Water temperature sensor 37...Air-fuel ratio sensor 100…Data center 110... Processing circuit 120...Storage device 130...Communication equipment 200…Communication networks 300…Vehicle controller 310... Processing circuit 320…Storage device 330...Communication equipment 340…Display device 350…Microphone 360…actuator

Claims

1. A vehicle evaluation system for evaluating a target vehicle using sound data recorded from the target vehicle, the vehicle being an evaluation target, comprising: a processing circuit; and a storage device; The storage device stores data of a trained model trained by supervised learning to generate the operation data from the training sound data, the training data including training sound data recorded while operating a reference vehicle in a state serving as a reference for evaluation with a measurement operation pattern for a predetermined time period and operation data indicating a transition of operation information in the measurement operation pattern; The processing circuit executes a generation process of inputting evaluation data including evaluation sound data recorded while operating the target vehicle with the measurement operation pattern into the trained model and outputting generated data, which is data of the operation information generated by the trained model, and an evaluation process of comparing the operation data included in the training data during the supervised learning with the generated data and evaluating the target vehicle according to the magnitude of deviation between the operation data and the generated data. Vehicle rating system.

2. The trained model generates the operation data from evaluation data including image data of a spectrogram obtained by frequency analysis of the sound data. The vehicle evaluation system according to claim 1 .

3. The trained model is a neural network that uses features of data extracted from the data for the predetermined time period as explanatory variables and uses the operation information at the time point corresponding to the extracted data as a target variable. The vehicle evaluation system according to claim 1 .

4. The evaluation process calculates, as an evaluation index value, a sum of the deviations output for the predetermined time period for each piece of data repeatedly extracted while changing the extraction start time. The vehicle evaluation system according to claim 3 .

5. The evaluation process selects and outputs an evaluation rank according to the magnitude of the evaluation index value from among a plurality of evaluation ranks classified according to the magnitude of the evaluation index value. The vehicle evaluation system according to claim 4.

6. The operation data includes data on accelerator opening and brake oil pressure. The vehicle evaluation system according to claim 1 .

7. The training data and the evaluation data include outside air temperature data in addition to the sound data. The vehicle evaluation system according to claim 1 .

8. The training data and the evaluation data include data on a gear ratio in addition to the sound data. The vehicle evaluation system according to claim 1 .

9. a vehicle controller that operates the target vehicle according to the measurement operation pattern; The vehicle evaluation system according to claim 1 .

10. The vehicle controller includes an actuator for realizing driving operations of the vehicle. The vehicle evaluation system according to claim 9.

11. The vehicle controller is equipped with a microphone for recording the sound data. The vehicle evaluation system according to claim 9.

12. a data center including the processing circuit and the storage device; the vehicle controller connected to the data center through a communication network; The vehicle evaluation system according to claim 9.

Citation Information

Patent Citations

  • Measuring setup

    DE102019219371A1

  • Method of detecting abnormality of moving body

    JP2009294147A

  • Running state reproduction system

    JP2019085059A

  • Abnormal sound determination device and abnormal sound determination method

    JP2021152500A

  • Vehicle noise determination device

    JP2022061072A