Model verification method and device, equipment and storage medium

Abstract

The application relates to a model verification method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: a model verification device determines a fault prediction time when a target vehicle fails according to a preset fault prediction model and operation information of the target vehicle in a first historical time period, the preset fault prediction model is obtained based on weak supervision learning, and the first historical time period is a time period before a fault occurrence time when the target vehicle fails. Further, the model verification device verifies whether the accuracy of the preset fault prediction model meets a preset requirement according to the fault prediction time and the fault occurrence time. Thus, the accuracy of a model trained by weak supervision learning is verified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to the technical field of machine learning, and specifically to a model verification method and device, equipment and a storage medium. BACKGROUND

[0002] In the field of fault prediction in the automotive industry, traditional automotive oscilloscopes, multimeters and other fault detection equipment can only detect faults after they occur, and not all faults have fault codes that can be reported. Due to the inability to obtain the true time of some faults, traditional supervised algorithms are useless for fault prediction problems. At this time, weakly supervised learning is the best algorithm model to solve such inaccurate labeling problems. However, even if the weakly supervised learning model is trained, it cannot be determined whether the prediction result of the weakly supervised learning model is accurate because there is no accurate label data to verify the prediction result. SUMMARY

[0003] One of the purposes of the present application is to provide a model verification method, device, equipment and storage medium for verifying the accuracy of a model trained by weakly supervised learning.

[0004] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0005] According to the first aspect of the present application, a model verification method is provided, comprising: a model verification device determining a fault prediction time of a target vehicle occurring a fault according to a preset fault prediction model and running information of the target vehicle in a first historical time period, the preset fault prediction model being trained based on weakly supervised learning, and the first historical time period being a time period before the fault occurrence time of the target vehicle occurring a fault. Further, the model verification device verifies whether the accuracy of the preset fault prediction model meets a preset requirement according to the fault prediction time and the fault occurrence time.

[0006] According to the above technical means, the model verification method provided by the present application compares the fault prediction time predicted by the preset fault prediction model with the fault occurrence time, determines the accuracy of the preset fault prediction model trained based on weakly supervised learning, and ensures that the prediction result of the preset fault prediction model meets the user's demand.

[0007] In a possible implementation, the model verification device determines whether the accuracy of the preset fault prediction model meets the preset requirement according to the fault prediction time and the fault occurrence time, including: determining a difference between the fault prediction time and the fault occurrence time; in a case where the difference is less than a preset threshold, determining that the accuracy of the preset fault prediction model meets the preset requirement; in a case where the difference is greater than or equal to the preset threshold, determining that the accuracy of the preset fault prediction model does not meet the preset requirement.

[0008] According to the technical means, the application provides a method for determining whether a preset fault prediction model is accurate.

[0009] In a possible implementation, the model verification method further includes: determining a shooting time of the fault scene photo according to the fault scene photo of the target vehicle; and determining the fault occurrence time according to the shooting time.

[0010] According to the technical means, the application provides a method for determining a fault occurrence time, so as to facilitate subsequent verification of a preset fault prediction model.

[0011] In a possible implementation, the model verification device determines the fault occurrence time according to the shooting time, including: obtaining running information of the target vehicle in a second historical time period, the second historical time period being a time period before the shooting time; and in a case where the running information in the second historical time period meets a preset condition, determining the shooting time as the fault occurrence time.

[0012] According to the technical means, the application provides a method for guaranteeing the accuracy of the determined fault occurrence time, so as to improve the accuracy of subsequent verification of a preset fault prediction model.

[0013] In a possible implementation, the fault scene photo carries a time watermark; the model verification device determines the shooting time of the fault scene photo according to the fault scene photo of the target vehicle, including: identifying the time watermark based on a text recognition technology to obtain time information on the fault scene photo; and determining the shooting time according to a time indicated by the time information.

[0014] According to the technical means, the application provides a method for determining a shooting time, for subsequent determination of a fault occurrence time.

[0015] In a possible implementation, the model verification device determines the shooting time according to a time indicated by the time information, including: in a case where the time indicated by the time information is before an upload time, determining the time indicated by the time information as the shooting time, the upload time being a time when a rescue worker uploads the system after shooting the fault scene photo.

[0016] According to the technical means, the method for guaranteeing the accuracy of the determined shooting time is provided, so as to improve the accuracy of the subsequent determination of the fault occurrence time.

[0017] In a possible implementation, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle in the power failure fault, the method further includes: in the case that the running information in the second historical time period indicates that the ignition of the target vehicle is normal in the second historical time period, determining that the running information in the second historical time period meets the preset condition.

[0018] In a possible implementation, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle in the engine fault, the method further includes: in the case that the running information in the second historical time period indicates that the engine of the target vehicle is normally running in the second historical time period, determining that the running information in the second historical time period meets the preset condition.

[0019] In a possible implementation, in the case that the accuracy of the preset fault prediction model does not meet the preset requirement, the model verification method further includes: obtaining the running information of each vehicle in the plurality of vehicles in a first historical time period before the fault occurrence time; and taking the running information of each vehicle in the plurality of vehicles in the first historical time period as a feature, and taking the fault occurrence time of each vehicle in the plurality of vehicles as a label, and continuing to train the preset fault prediction model.

[0020] According to the technical means, the preset fault prediction model is further trained and corrected based on the accurate fault occurrence time in the model verification, so as to improve the accuracy of the preset fault prediction model. In this way, the preset fault prediction model after the training and correction can be further repeatedly verified until the preset fault prediction model can meet the corresponding accuracy requirement.

[0021] According to the second aspect of the present application, a model verification device is provided, including a determination unit and a processing unit. The determination unit is configured to determine a fault prediction time of a target vehicle in a fault according to a preset fault prediction model and running information of the target vehicle in a first historical time period. The preset fault prediction model is obtained based on weak supervision learning, and the first historical time period is a time period before a fault occurrence time of the target vehicle in the fault. The processing unit is configured to verify whether the accuracy of the preset fault prediction model meets a preset requirement according to the fault prediction time and the fault occurrence time.

[0022] In a possible implementation, the processing unit is specifically configured to determine a difference between the fault prediction time and the fault occurrence time; in a case where the difference is less than a preset threshold, determine that the accuracy of the preset fault prediction model meets the preset requirement; and in a case where the difference is greater than or equal to the preset threshold, determine that the accuracy of the preset fault prediction model does not meet the preset requirement.

[0023] In a possible implementation, the determining unit is further configured to determine a photographing time of the fault scene photograph according to the fault scene photograph of the target vehicle, and determine the fault occurrence time according to the photographing time.

[0024] In a possible implementation, the model verification apparatus further includes an obtaining unit, which is configured to obtain running information of the target vehicle in a second historical time period, the second historical time period being a time period before the photographing time. The determining unit is further configured to determine the photographing time as the fault occurrence time in a case where the running information in the second historical time period meets a preset condition.

[0025] In a possible implementation, the fault scene photograph carries a time watermark. The obtaining unit is further configured to obtain time information on the fault scene photograph based on a text recognition technology to recognize the time watermark. The determining unit is further configured to determine the photographing time according to a time indicated by the time information.

[0026] In a possible implementation, the determining unit is further configured to determine the time indicated by the time information as the photographing time in a case where the time indicated by the time information is before an uploading time, the uploading time being a time when the rescue personnel uploads the system after photographing the fault scene photograph.

[0027] In a possible implementation, in a case where the preset fault prediction model is used to predict a fault prediction time of a vehicle in which a battery failure fault occurs, the determining unit is further configured to determine that the running information in the second historical time period meets the preset condition in a case where the running information in the second historical time period indicates that the target vehicle has a normal ignition during the second historical time period.

[0028] In a possible implementation, in a case where the preset fault prediction model is used to predict a fault prediction time of a vehicle in which an engine failure fault occurs, the determining unit is further configured to determine that the running information in the second historical time period meets the preset condition in a case where the running information in the second historical time period indicates that the target vehicle has a normal engine operation during the second historical time period.

[0029] According to a third aspect provided by the present application, a model verification device is provided, which is deployed on a vehicle. The model verification device comprises a memory and a processor, which are coupled; the memory is configured to store computer program codes, the computer program codes comprising computer instructions; when the processor executes the computer instructions, the model verification device performs the model verification method provided by the first aspect and any possible implementation manner thereof.

[0030] According to a fourth aspect provided by the present application, a computer readable storage medium is provided, which stores instructions, when the instructions are run on the model verification device, the model verification device performs the model verification method provided by the first aspect and any possible implementation manner thereof.

[0031] According to a fifth aspect provided by the present application, a computer program product is provided, which comprises computer instructions, when the computer instructions are run on the model verification device, the model verification device performs the model verification method provided by the first aspect and any possible implementation manner thereof.

[0032] Therefore, the above technical features of the present application have the following beneficial effects:

[0033] (1) In the model verification method provided by the present application, the fault prediction time predicted by the preset fault prediction model is compared with the fault occurrence time, so as to determine the accuracy of the preset fault prediction model trained based on weak supervision learning, and ensure that the prediction result of the preset fault prediction model can meet the needs of the user.

[0034] (2) In the model verification method provided by the present application, the accuracy of the determined shooting time is improved, the accuracy of the determined fault occurrence time is improved, and then whether the fault occurrence time is accurate is determined according to the judgment of the fault occurrence time. Further, based on the fault occurrence time with high accuracy, the fault prediction time predicted by the preset fault prediction model trained by weak supervision learning is verified, so as to improve the accuracy of the verification result obtained by verifying the preset fault prediction model.

[0035] It should be noted that the technical effects brought by any implementation manner of the second aspect to the fifth aspect can refer to the technical effects brought by the corresponding implementation manner in the first aspect, which will not be repeated here.

[0036] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a structural schematic diagram of a model verification system according to an exemplary embodiment;

[0038] Figure 2 is a flowchart of a model verification method according to an exemplary embodiment;

[0039] Figure 3 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0040] Figure 4 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0041] Figure 5 is a model training diagram according to an exemplary embodiment;

[0042] Figure 6 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0043] Figure 7 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0044] Figure 8 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0045] Figure 9 is a flowchart of yet another model verification method according to an exemplary embodiment;

[0046] Figure 10 is a block diagram of a model verification device according to an exemplary embodiment;

[0047] Figure 11 is a block diagram of a model verification apparatus according to an exemplary embodiment. DETAILED DESCRIPTION

[0048] Other advantages and novel features of the present application will be readily appreciated as the same becomes better understood by reference to the following details, when considered in connection with the accompanying drawings and preferred embodiment described below. The following detailed description is of the best mode at the time of the patenting of the present application. The application may, however, be carried out in other ways than those specifically set forth herein without departing from the essential characteristics of the application, and, the present embodiments should not be limited to the details given herein but should be limited only by the scope of the claims.

[0049] It should be noted that the drawings included in the following embodiments are only a schematic illustration of the basic idea of the present application, and thus only show components related to the present application, not the actual number, shape and size of the components, and the actual implementation of the components may be arbitrarily changed, and the layout of the components may be more complex.

[0050] In the description of the embodiments, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" herein only describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, "at least one" and "multiple" mean two or more. "First", "second", and the like do not limit the quantity and execution order, and "first", "second", and the like do not necessarily mean different.

[0051] In the related art, in the field of fault prediction in the automobile industry, traditional automobile oscilloscopes, multimeters and other fault detection equipment can only detect faults after the faults occur, and not all faults have fault codes that can be reported. Due to the inability to obtain the true time of some faults, the traditional supervised algorithm is useless. At this time, weakly supervised learning is the best algorithm model to solve such inaccurate labeling problems. However, even if the weakly supervised learning model is well trained, it cannot be determined whether the prediction result of the weakly supervised learning model is accurate because there is no accurate label data to verify the prediction result.

[0052] It should be noted that weakly supervised learning can be divided into three categories, including incomplete supervision, inexact supervision and inaccurate supervision. Among them, incomplete supervision means that only a part of the data in the training data of the model has corresponding labels, and part of the data has no labels. Inexact supervision means that the training data of the model only gives a coarse-grained label. Inaccurate supervision means that the given label is inaccurate, which is only a coarse-grained label.

[0053] In order to solve the above problems, the present application provides a model verification method, device, equipment and storage medium. The model verification device determines the fault prediction time of the target vehicle according to the preset fault prediction model and the running information of the target vehicle in the first historical time period. The preset fault prediction model is obtained based on weakly supervised learning, and the first historical time period is the time period before the fault occurrence time of the target vehicle. Further, the model verification device verifies whether the accuracy of the preset fault prediction model meets the preset requirement according to the fault prediction time and the fault occurrence time.

[0054] In this way, in the model verification method provided by the present application, the fault prediction time predicted by the preset fault prediction model is compared with the fault occurrence time, so as to determine the accuracy of the preset fault prediction model trained based on weakly supervised learning, and to ensure that the prediction result of the preset fault prediction model can meet the user's demand.

[0055] Figure 1A model verification system is shown, and the model verification method provided by the embodiments of the present application can be applied to the model verification system as shown in Figure 1 for verifying the accuracy of a model trained by weakly supervised learning. As shown in Figure 1 The model verification system 10 includes a model verification device 11, a terminal device 12, and a server 13.

[0056] The server 13 is connected to the model verification device 11 and the terminal device 12, respectively. In the above connection relationship, a wired connection or a wireless connection can be used, and the embodiments of the present application do not limit this.

[0057] The model verification device 11 can be used to determine a fault prediction time of a target vehicle based on a preset fault prediction model and running information of the target vehicle in a first historical time period.

[0058] The preset fault prediction model is trained based on weakly supervised learning, and the first historical time period is a time period before a fault occurrence time of the target vehicle.

[0059] The model verification device 11 can also be used to verify whether the accuracy of the preset fault prediction model meets a preset requirement based on the fault prediction time and the fault occurrence time.

[0060] The terminal device 12 can be an electronic device with a shooting function, such as a mobile phone with a shooting function, a camera with a communication function, and the like, and the embodiments of the present application do not limit this.

[0061] In some embodiments, the terminal device 12 is configured to include a shooting time in photo information carried in a shooting fault scene photo.

[0062] In other embodiments, the terminal device 12 is configured to have a time watermark on the shooting fault scene photo, and the time watermark is used to reflect the shooting time of the fault scene photo.

[0063] The terminal device 12 can be carried by a rescue personnel and shoot a fault scene photo of a fault scene.

[0064] The fault scene photo carries a vehicle frame number of the target vehicle, and the vehicle frame number is used to uniquely identify the target vehicle.

[0065] The terminal device 12 can also be used to upload the shooting fault scene photo to the server 13 based on the connection between the terminal device 12 and the server 13.

[0066] The server 13 can be used to name and store the fault scene photo with the vehicle frame number after receiving the fault scene photo uploaded by the terminal device 12.

[0067] The server 13 can also be configured to record the uploading time of the fault scene photos uploaded by the terminal device 12.

[0068] Figure 2 Fig. 1 is a flow diagram of a model verification method according to some example embodiments. In some embodiments, the model verification method described above can be applied to the model verification device 11 in the model verification system 10 as shown in Fig. 1. Hereinafter, the model verification method described above will be explained by taking the model verification method applied to the model verification device 11 as an example. Figure 1

[0069] As shown in Fig. 1, the model verification method provided by the embodiments of the present application comprises the following S201-S202. Figure 2

[0070] S201, the model verification device determines a fault prediction time of a fault of a target vehicle according to a preset fault prediction model and running information of the target vehicle in a first historical time period.

[0071] In the formula, the preset fault prediction model is obtained based on weak supervision learning, and the first historical time period is a time period before a fault occurrence time of the fault of the target vehicle.

[0072] As a possible implementation manner, the model verification device inputs the running information of the target vehicle in the first historical time period as a feature into the preset fault prediction model to obtain the fault prediction time of the fault of the target vehicle predicted by the preset fault prediction model.

[0073] It should be noted that the running information includes features used in the process of training the preset fault prediction model. For example, in the case where the preset fault prediction model is used to predict the fault prediction time of a vehicle running out of power, if in the training process of the preset fault prediction model, the features include vehicle model, vehicle mileage information, highland coefficient, vehicle running state, on-off state of each power-consuming vehicle lamp, battery current, battery voltage, battery temperature, ambient temperature, vehicle abnormal wake-up time, vehicle abnormal wake-up frequency, etc., and the maintenance time in the running-out-of-power maintenance record is taken as a label, then the running information also includes the vehicle model, vehicle mileage information, highland coefficient, vehicle running state, on-off state of each power-consuming vehicle lamp, battery current, battery voltage, battery temperature, ambient temperature, vehicle abnormal wake-up time, vehicle abnormal wake-up frequency, etc.

[0074] It should be noted that the preset fault prediction model can be set in the model verification device by an operation and maintenance personnel of the model verification system in advance, and is used to predict the fault of the vehicle which cannot determine the accurate fault time through the fault code. For example, refer to the subsequent description of the embodiments of the present application, which will not be described here.​​

[0075] It should be noted that the length of the first historical time period can be set in advance in the model verification device by an operation and maintenance personnel of the model verification system, and can be 1 month, 2 months, 3 months, etc. for example. The embodiments of the present application do not make specific limitations thereto.

[0076] In some embodiments, the fault occurrence time can be the fault time fed back by the driver to the vehicle enterprise service platform after perceiving the fault; or can be the fault time reported by the rescue personnel dispatched by the vehicle enterprise. How to determine the accurate fault occurrence time will be described later in the embodiments of the present application, and will not be described here.

[0077] In S202, the model verification device verifies whether the accuracy of the preset fault prediction model meets the preset requirement according to the fault prediction time and the fault occurrence time.

[0078] As a possible implementation manner, in the case that the fault prediction time and the fault occurrence time are in units of days, the model verification device determines the date indicated by the fault prediction time and the date indicated by the fault occurrence time. In the case that the date indicated by the fault prediction time and the date indicated by the fault occurrence time are the same day, it is determined that the accuracy of the preset fault prediction model meets the preset requirement.

[0079] In some embodiments, if the fault prediction time and the fault occurrence time are both used to indicate time points, the model verification device determines the difference between the fault prediction time and the fault occurrence time, and in the case that the difference is less than a preset threshold, it is determined that the accuracy of the preset fault prediction model meets the preset requirement; in the case that the difference is greater than or equal to the preset threshold, it is determined that the accuracy of the preset fault prediction model does not meet the preset requirement.

[0080] It can be understood that in the model verification method provided by the embodiments of the present application, the fault prediction time predicted by the preset fault prediction model is compared with the fault occurrence time, so as to determine the accuracy of the preset fault prediction model trained based on weak supervision learning, and to ensure that the prediction result of the preset fault prediction model can meet the needs of the user.

[0081] In some embodiments, the model verification device verifies the preset fault prediction model based on the fault occurrence time of a plurality of vehicles and the running information of each vehicle in the corresponding first historical time period. In this case, the preset requirement can also be that the fault occurrence time and the fault prediction time are in the same day, or the proportion of the difference between the fault occurrence time and the fault prediction time being less than a preset threshold is greater than a preset proportion, and the preset proportion is exemplarily 90%, 95%, etc.

[0082] Further, if the accuracy of the preset fault prediction model does not meet the preset requirement, the model verification device continues to train the preset fault model based on the running information of each vehicle in the plurality of vehicles in a first historical time period before the fault occurrence time, taking the running information of each vehicle in the plurality of vehicles in the first historical time period as features and taking the fault occurrence time of each vehicle in the plurality of vehicles as labels.

[0083] Optionally, after the preset fault prediction model is trained and corrected based on the running information of each vehicle in the plurality of vehicles in the first historical time period and the fault occurrence time, the model verification device further verifies the preset fault prediction model to determine whether the accuracy of the trained and corrected fault prediction model meets the preset requirement. If the accuracy of the trained and corrected preset fault prediction model still cannot meet the preset requirement, the preset fault prediction model is continuously trained and corrected until the accuracy of the preset fault prediction model meets the preset requirement.

[0084] In one design, to determine the fault occurrence time so as to verify the prediction result of the preset fault prediction model, as shown in FIG. 3, the model verification method provided by the embodiments of the present application further includes S301-S302. Figure 3

[0085] S301, the model verification device determines the shooting time of the fault scene photo according to the fault scene photo of the target vehicle.

[0086] As a possible implementation manner, the model verification device obtains the fault scene photo of the target vehicle from the server based on the vehicle frame number of the target vehicle, and analyzes the fault scene photo to obtain the shooting time of the fault scene photo from the detailed information of the fault scene photo.

[0087] It should be noted that the fault scene photo of the target vehicle can be uploaded to the server by the rescue personnel called by the driver of the target vehicle after shooting the scene photo, and stored in the server with the vehicle frame number of the target vehicle, which is not limited in the embodiments of the present application.

[0088] In some embodiments, if the fault scene photo carries a time watermark, the model verification device determines the shooting time of the fault scene photo, and further includes the following steps S3011-S3012.

[0089] S3011, the model verification device identifies the time watermark based on a text recognition technology to obtain the time information on the fault scene photo.

[0090] ​It can be understood that, by using optical character recognition (OCR) to recognize the time information recorded by the time watermark, the time information on the fault scene photo is obtained, which reduces the manual input of time compared to the traditional manual input system time information, and improves the efficiency of time input.

[0091] In S3012, the model verification device determines the shooting time according to the time indicated by the time information.

[0092] In some embodiments, in order to ensure the accuracy of the shooting time, after obtaining the time information recorded by the time watermark, the model verification device further obtains the uploading time of the personnel uploading the system after shooting the fault scene photo from the server. Further, the model verification device determines whether the time indicated by the time information is before the uploading time, and in the case that the time indicated by the time information is before the uploading time, the time indicated by the time information is determined as the shooting time.

[0093] It can be understood that, although the current OCR recognition technology can achieve high accuracy, in order to reduce the data errors caused by misrecognition, the shooting time and the uploading time of the uploading system are compared. In the case that the shooting time is greater than the uploading time, i.e. the shooting time is later than the uploading time, it indicates that the shooting time recognition is wrong, and the model verification device re-recognizes the fault scene photo to ensure the accuracy of the shooting time, and further ensure the accuracy of the accuracy verification of the preset fault prediction model.

[0094] In S302, the model verification device determines the fault occurrence time according to the shooting time.

[0095] As a possible implementation manner, the model verification device determines the shooting time determined in the above step S301 as the fault occurrence time.

[0096] In some embodiments, if the fault occurrence time is in days, the model verification device determines the date to which the shooting time belongs as the fault occurrence time.

[0097] It can be understood that, in the model verification method provided by the embodiments of the present application, the fault occurrence time is obtained by processing the fault scene photo, which provides support for subsequent accuracy verification of the preset fault prediction model.

[0098] In one design, although the real failure rescue time is obtained in the above-mentioned embodiments of the present application, it does not mean that the real failure rescue time is the real failure time. (For example, the vehicle has failed after a week of use, but the user discovers the failure in the second week after stopping using the vehicle and requests rescue, and at this time, the rescue time cannot be confirmed as the failure occurrence time.) In order to guarantee the accuracy of the determined failure occurrence time, the model verification method provided in the embodiments of the present application, as shown in Figure 4 S401-S402.

[0099] S401, the model verification device obtains the running information of the target vehicle in a second historical time period.

[0100] The second historical time period is a time period before the shooting time.

[0101] As a possible implementation manner, the model verification device determines the second historical time period after determining the shooting time in the above-mentioned step S301. Further, the model verification device obtains the running information of the target vehicle in the second historical time period from the server based on the vehicle frame number of the target vehicle.

[0102] It should be noted that the length of the second historical time period can be set in the model verification device in advance by the operation and maintenance personnel of the model verification system, and for example, can be 8 hours, 16 hours, 24 hours, etc., which is not limited in the embodiments of the present application.

[0103] For example, if the shooting time is A year B month C day D, and the second historical time period is a time period of 24 hours before the shooting time, the model verification device determines the second historical time period as a time period between A year B month C-1 day D and A year B month C day D.

[0104] S402, the model verification device determines the shooting time as the failure occurrence time in the case that the running information in the second historical time period meets a preset condition.

[0105] It should be noted that the preset condition is used to indicate that the running information of the vehicle is normal and no failure occurs.

[0106] As a possible implementation manner, the model verification device determines whether the running information in the second historical time period meets the preset condition. Further, the model verification device determines the shooting time as the failure occurrence time in the case that the running information in the second historical time period meets the preset condition.

[0107] In some embodiments, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle occurring the power shortage fault, the model verification device determines that the running information in the second historical time period meets the preset condition in the case that the running information in the second historical time period indicates that the target vehicle has a normal ignition in the second historical time period.

[0108] Otherwise, the model verification device determines that the fault occurrence time of the target vehicle determined in the above embodiments cannot be used for verification of the preset fault prediction model, and the target vehicle is replaced to verify the preset fault prediction model.

[0109] In some embodiments, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle occurring the engine fault, the model verification device determines that the running information in the second historical time period meets the preset condition in the case that the running information in the second historical time period indicates that the target vehicle has a normal engine operation in the second historical time period.

[0110] Otherwise, the model verification device determines that the fault occurrence time of the target vehicle determined in the above embodiments cannot be used for verification of the preset fault prediction model, and the target vehicle is replaced to verify the preset fault prediction model.

[0111] It can be understood that, in the model verification method provided by the embodiments of the present application, the running information of the target vehicle in the second historical time period before the shooting time of the fault scene photo is used to determine whether the shooting time of the fault scene photo can be used to indicate the fault occurrence time, so as to guarantee the accuracy of the determined fault occurrence time.

[0112] In one design, taking the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle occurring the power shortage fault as an example, the training process of the preset fault prediction model is as follows, including data acquisition, data processing, model training and model screening.

[0113] It should be noted that the data processing and model training are as shown in Figure 5 First, the original data is preprocessed to eliminate data with missing values and abnormal values, and the original data is sorted according to the time sequence. Further, the original data sorted according to the time sequence is segmented into events, and each event is converted into corresponding features. Then, the obtained features and a preset algorithm are used to train the model, and the preset fault prediction model is obtained.

[0114] In the data acquisition process, the operation and maintenance personnel of the model verification system acquires the vehicle occurring the power shortage fault from the data cloud platform of the vehicle manufacturer, and acquires the time sequence data recorded by the vehicle data bus and the maintenance record.

[0115] In the data processing process, the data with missing values and abnormal values in the obtained data are processed, and then some variables in the original data are converted for subsequent model training process.

[0116] For example, if the obtained data includes switch state information, the total time corresponding to the two switch states of 0 and 1 of the switch state is calculated. If the obtained data includes the current signal of the battery sensor, the output current of the battery is determined according to the following formula one.

[0117] I = (value - m) * k Formula one

[0118] Where I is the output current of the battery, value is the current signal of the battery sensor, m is the current signal parameter, which can be 32768 for example, and k is the current signal and current conversion coefficient.

[0119] Further, after cleaning and converting the obtained data, the operation information related to the power loss failure is extracted as the feature in the model training process, and the maintenance record related to the power loss failure is extracted, and the maintenance time is determined as the label in the model training process.

[0120] In the model training process, based on the features and labels obtained in the data processing process, a plurality of survival analysis algorithms are trained to obtain a plurality of power loss prediction models.

[0121] For example, the survival analysis algorithm can be a random survival forest (Random survival forests) algorithm, a deep hit (DeepHit) algorithm, and a deep measurement (DeepSurv) algorithm.

[0122] In the survival analysis algorithm, the functions are described as shown in Table 1.

[0123] Table 1: Function description table

[0124] Function Explanation f(t) Density of death function F(t) Cumulative death distribution function S(t) Survival probability function h(t) Hazard function [h0(t)] Baseline hazard function H(t) Cumulative hazard function

[0125] Where the survival function represents the probability that the event occurs no earlier than t, as shown in the following formula two.

[0126] S(t) = Pr(T ≥ t) Formula two

[0127] It should be noted that the value of the survival function decreases as t increases, and in the case of t = 0, S(t) = 1; in the case of t tends to infinity, S(t) tends to 0.

[0128] The cumulative death distribution function F(t) represents the probability that the event occurs earlier than t, as shown in the following formula three.

[0129] F(t) = 1 - S(t) Equation Three

[0130] The risk function h(t) represents the probability of the occurrence of the event of interest at the instant, as shown in the following Equation Four.

[0131]

[0132] Corresponding to this is the cumulative hazard function, which is the integral of the hazard function over time, as shown in the following Equation Five.

[0133]

[0134] It should be noted that the above is a description of some functions involved in the survival analysis algorithm. In the process of training the preset fault prediction model, the training method in the prior art can be referred to for training the preset fault prediction model, and the embodiments of the present application do not make specific limitations thereto.

[0135] In the model screening process, based on the prediction results and the fault records of the vehicle, a confusion matrix is used to score each model, and the model with the best score is determined as the preset fault prediction model.

[0136] For example, taking the power loss fault as an example, if the vehicle has a power loss event, the label value is 1, and if the vehicle does not have a power loss event, the label value is 0. The precision (Precision) and recall (Recall) are used as performance indicators to determine the score of each model.

[0137] The confusion matrix is shown in Table 2 below.

[0138] Table 2: Confusion Matrix

[0139]

[0140] The calculation formula of Precision is shown in Equation Six.

[0141]

[0142] The calculation formula of Recall is shown in Equation Six.

[0143]

[0144] After performance testing of the multiple models obtained in the model training process based on the confusion matrix shown in Table 2, the performance indicators of different models are obtained as shown in Table 3.

[0145] Table 3: Performance indicators of different models

[0146] Precision Recall Random Survival Forests 0.96 0.80 DeepHit 0.84 0.45 DeepSurv 0.86 0.28

[0147] In this way, based on the performance indicators of the different models in Table 3 above, it is determined that the model trained based on the Random Survival Forests algorithm is the preset fault prediction model.

[0148] In one design, the overall verification process of the model verification method provided by the embodiments of the present application, as shown in Figure 6 , includes S501-S506.

[0149] S501, the model verification device obtains the fault scene photo of the target vehicle.

[0150] S502, the model verification device obtains the fault occurrence time of the target vehicle based on a text recognition technology.

[0151] S503, the model verification device obtains the running information of the target vehicle in a first historical time period.

[0152] S504, the model verification device determines the fault prediction time of the fault occurrence of the target vehicle according to the preset fault prediction model and the running information of the target vehicle in the first historical time period.

[0153] S505, the model verification device verifies the preset fault prediction model according to the fault occurrence time and the fault prediction time.

[0154] S506, the model verification device outputs the model verification result.

[0155] As a possible implementation manner, after determining the verification result of the preset fault prediction model based on the above step S505, the model verification device outputs the model verification result to a display device connected thereto or stores the model verification result in a server.

[0156] It should be noted that the specific implementation manner of the model verification method described in the above steps S501-S506 can refer to the description in the above embodiments of the present application, and will not be described here.

[0157] In one design, the determination process of the shooting moment of the model verification method provided by the embodiments of the present application, as shown in Figure 7 , includes S601-S605.

[0158] S601, the model verification device obtains the fault information of the fault vehicle reported by the rescue personnel.

[0159] The fault information includes a fault scene photo, a vehicle frame number of the fault vehicle, a fault type, a fault description, etc.

[0160] S602, the model verification device determines the uploading time of the fault information.

[0161] S603, the model verification device verifies the shooting time based on the shooting time on the fault scene photo and the uploading time.

[0162] S604, the model verification device determines whether the shooting time verification is successful.

[0163] It should be noted that in the case where the model verification device determines that the shooting time verification is successful, step S605 is performed; in the case where it is determined that the shooting time verification fails, the data is discarded, and the fault information of other fault vehicles is used to verify the preset fault prediction model.

[0164] S605, the model verification device renames the fault scene photo based on the vehicle frame number of the fault vehicle and the shooting time, and stores the renamed fault scene photo.

[0165] For example, if the vehicle frame number of the fault vehicle is vin A , the shooting time is T A , the renamed fault scene photo can be named, for example, vin A +T A .

[0166] It should be noted that the specific implementation of the model verification method described in steps S601-S605 above can refer to the description in the above embodiments of the present application, and will not be described here.

[0167] In one design, to further improve the accuracy of model verification, after determining the shooting time in step S605 above, the process of further determining the fault occurrence time is further determined, as shown in Figure 8 , including S701-S705.

[0168] S701, the model verification device obtains the vehicle frame number of the fault vehicle and the shooting time.

[0169] S702, the model verification device obtains the running information of the fault vehicle in the second historical time period based on the vehicle frame number and the shooting time.

[0170] S703, the model verification device determines whether the running information in the second historical time period meets the preset condition.

[0171] It should be noted that in the case where the model verification device determines that the running information in the second historical time period meets the preset condition, step S704 is performed; in the case where it is determined that the running information in the second historical time period does not meet the preset condition, the data is discarded, and the fault information of other fault vehicles is used to verify the preset fault prediction model.

[0172] S704, in a case where the running information of the model verification device in the second historical time period meets a preset condition, determining the photographing moment as the fault occurrence time.

[0173] S705, the model verification device storing the fault occurrence time of the fault vehicle.

[0174] It should be noted that the specific implementation of the model verification method described in steps S701-S705 can refer to the description in the above embodiments of the present application, and will not be repeated here.

[0175] In one design, after obtaining the fault occurrence time of the fault vehicle, the verification process of the preset fault prediction model trained by the weakly supervised learning in the model verification method provided by the embodiments of the present application includes S801-S803, as shown in the following figure. Figure 9

[0176] S801, the model verification device obtains the running information of the fault vehicle in a first historical time period before the fault occurrence time.

[0177] S802, the model verification device determines a fault prediction time according to the preset fault prediction model and the running information in the first historical time period.

[0178] S803, the model verification device determines a verification result of the preset fault prediction model according to the fault prediction time and the fault occurrence time.

[0179] It should be noted that the specific implementation of the model verification method described in steps S801-S803 can refer to the description in the above embodiments of the present application, and will not be repeated here.

[0180] The above mainly introduces the scheme provided by the embodiments of the present application from the perspective of method. In order to realize the above functions, the model verification device or the model verification equipment contains the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or combination of hardware and computer software. Whether a certain function is realized in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0181] ​The embodiments of the present application can divide the functional modules of the model verification device or the model verification equipment according to the above method. For example, the model verification device or the model verification equipment can include various functional modules corresponding to various functional divisions, or two or more functions can be integrated in one processing module. The above integrated module can be realized in the form of hardware or in the form of a software functional module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. When actually implemented, there can be another division manner.

[0182] Figure 10 A structural schematic diagram of a model verification device provided by the embodiments of the present application is shown in the figure. The model verification device is used to execute the above model verification method. As shown in the figure, the model verification device 90 includes a determination unit 901 and a processing unit 902. Figure 10

[0183] The determination unit 901 is configured to determine a fault prediction time when the target vehicle occurs a fault according to a preset fault prediction model and running information of the target vehicle in a first historical time period. The preset fault prediction model is obtained based on weak supervision learning training, and the first historical time period is a time period before a fault occurrence time when the target vehicle occurs a fault.

[0184] The processing unit 902 is configured to verify whether the accuracy of the preset fault prediction model meets a preset requirement according to the fault prediction time and the fault occurrence time.

[0185] Optionally, the processing unit 902 is specifically configured to determine a difference value between the fault prediction time and the fault occurrence time; in a case where the difference value is less than a preset threshold, determine that the accuracy of the preset fault prediction model meets the preset requirement; and in a case where the difference value is greater than or equal to the preset threshold, determine that the accuracy of the preset fault prediction model does not meet the preset requirement.

[0186] Optionally, the determination unit 901 is further configured to determine a shooting time of the fault scene photo according to the fault scene photo of the target vehicle, and determine the fault occurrence time according to the shooting time.

[0187] Optionally, the model verification device 90 further includes an acquisition unit 903.

[0188] The acquisition unit 903 is configured to acquire running information of the target vehicle in a second historical time period. The second historical time period is a time period before the shooting time.

[0189] The determination unit 901 is further configured to determine the shooting time as the fault occurrence time in a case where the running information in the second historical time period meets a preset condition.

[0190] ​Optionally, in the case that the fault scene photo carries a time watermark, the acquisition unit 903 is further configured to identify the time watermark based on a text recognition technology, and acquire time information on the fault scene photo.

[0191] The determination unit 901 is further configured to determine the shooting time according to the time indicated by the time information.

[0192] Optionally, the determination unit 901 is further configured to determine the time indicated by the time information as the shooting time in the case that the time indicated by the time information is before the uploading time of the rescue personnel after shooting the fault scene photo.

[0193] Optionally, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle in the case of the battery failure, the determination unit 901 is further configured to determine that the running information in the second historical time period meets the preset condition in the case that the running information in the second historical time period indicates that the target vehicle has a normal ignition in the second historical time period.

[0194] Optionally, in the case that the preset fault prediction model is used to predict the fault prediction time of the vehicle in the case of the engine failure, the determination unit 901 is further configured to determine that the running information in the second historical time period meets the preset condition in the case that the running information in the second historical time period indicates that the target vehicle has a normal engine operation in the second historical time period.

[0195] Figure 11 is a block diagram of a model verification device according to an example embodiment. As shown in Figure 11 The model verification device 100 includes, but is not limited to, a processor 1001 and a memory 1002.

[0196] The memory 1002 is configured to store executable instructions of the processor 1001. It can be understood that the processor 1001 is configured to execute the instructions to implement the model verification method in the above embodiments.

[0197] It should be noted that those skilled in the art can understand that the structure of the model verification device shown in Figure 11 does not constitute a limitation on the model verification device. The model verification device can include more or fewer components than those shown in Figure 11 , or combine some components, or different component arrangements.

[0198] The processor 1001 is a control center of the model verification device, connects various parts of the model verification device by using various interfaces and lines, performs various functions of the model verification device and processes data by running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, and thus performs overall monitoring on the model verification device. The processor 1001 can include one or more processing units. Alternatively, the processor 1001 can integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1001.

[0199] The memory 1002 can be used to store software programs and various data. The memory 1002 can mainly include a program storage area and a data storage area, where the program storage area can store an operating system, application programs (such as a determination unit, a processing unit, etc.) required by at least one function module, and the like. In addition, the memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0200] In the example embodiment, a computer readable storage medium including instructions is also provided, for example, the memory 1002 including instructions, and the instructions can be executed by the processor 1001 of the model verification device 100 to implement the model verification method in the above embodiment.

[0201] In actual implementation, Figure 10 The functions of the determination unit 901, the processing unit 902, and the acquisition unit 903 in the model verification device 100 can be implemented by the processor 1001 calling the computer program stored in the memory 1002. The specific execution process can refer to the description of the model verification method in the above embodiment, and will not be described here. Figure 11

[0202] Alternatively, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0203] In the example embodiment, the embodiment of the present application also provides a vehicle including the above model verification device.

[0204] ​In an example embodiment, the embodiments of the present application also provide a computer program product comprising one or more instructions executable by the processor 1001 of the model verification device to perform the model verification method in the above embodiments.

[0205] It should be noted that the instructions in the above computer-readable storage medium or the one or more instructions in the computer program product are executed by the processor of the model verification device to implement each process of the above model verification method embodiments, and achieve the same technical effects as the above model verification method. To avoid repetition, it will not be described here.

[0206] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete the above-described full classification part or part of the function.

[0207] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0208] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0209] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0210] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole classification part or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute the whole classification part or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various storage program codes.

[0211] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A model validation method, characterized in that, The method includes: Based on the preset fault prediction model and the operating information of the target vehicle in the first historical time period, the fault prediction time of the target vehicle is determined. The preset fault prediction model is obtained based on weakly supervised learning training, and the first historical time period is the time period before the fault occurrence time of the target vehicle. Based on the fault prediction time and the fault occurrence time, verify whether the accuracy of the preset fault prediction model meets the preset requirements; The time of the fault occurrence is determined in the following way: Obtain the operating information of the target vehicle during a second historical time period, which is the time period before the fault scene photos were taken. If the operation information within the second historical time period meets preset conditions, the time when the fault scene photo was taken is determined as the fault occurrence time; wherein, the preset conditions include that the vehicle's operation information is normal; the preset conditions correspond to a target fault type; the target fault type is the fault type used by the preset fault prediction model to predict the fault prediction time.

2. The model validation method according to claim 1, characterized in that, The step of verifying whether the accuracy of the preset fault prediction model meets preset requirements based on the fault prediction time and the fault occurrence time includes: Determine the difference between the fault prediction time and the fault occurrence time; If the difference is less than a preset threshold, the accuracy of the preset fault prediction model is determined to meet the preset requirement. If the difference is greater than or equal to the preset threshold, it is determined that the accuracy of the preset fault prediction model does not meet the preset requirement.

3. The model verification method according to claim 1 or 2, characterized in that, The method further includes: Based on the photos of the target vehicle's malfunction scene, determine the time when the photos were taken; The time of the malfunction is determined based on the shooting time.

4. The model verification method according to claim 3, characterized in that, The fault scene photos carry a time watermark; determining the time the fault scene photos were taken based on the fault scene photos of the target vehicle includes: The time watermark is identified using text recognition technology to obtain the time information on the fault scene photo; The shooting time is determined based on the time indicated by the time information.

5. The model verification method according to claim 4, characterized in that, Determining the shooting time based on the time indicated by the time information includes: If the time indicated by the time information is before the upload time, the time indicated by the time information is determined as the shooting time, and the upload time is the time when the rescue personnel upload the photos of the fault scene to the system after taking them.

6. The model validation method according to claim 1, characterized in that, When the preset fault prediction model is used to predict the fault prediction time of a vehicle experiencing a battery drain fault, the method further includes: If the operating information during the second historical time period indicates that the target vehicle ignites normally during the second historical time period, it is determined that the operating information during the second historical time period meets the preset conditions.

7. The model validation method according to claim 1, characterized in that, When the preset fault prediction model is used to predict the fault prediction time of engine failure in a vehicle, the method further includes: If the operating information during the second historical time period indicates that the target vehicle's engine is operating normally during the second historical time period, then it is determined that the operating information during the second historical time period meets the preset conditions.

8. The model verification method according to claim 1 or 2, characterized in that, If the accuracy of the preset fault prediction model does not meet the preset requirements, the method further includes: Obtain the operating information of each of the plurality of vehicles during a first historical time period prior to the time of the fault occurrence; Using the operating information of each of the multiple vehicles within the first historical time period as features and the fault occurrence time of each of the multiple vehicles as a label, the preset fault prediction model is further trained.

9. A model verification device, characterized in that, It includes a determining unit, a processing unit, and an acquiring unit; The determining unit is used to determine the fault prediction time of the target vehicle based on a preset fault prediction model and the operating information of the target vehicle in a first historical time period. The preset fault prediction model is obtained based on weakly supervised learning training, and the first historical time period is the time period before the fault occurrence time of the target vehicle. The processing unit is used to verify whether the accuracy of the preset fault prediction model meets the preset requirements based on the fault prediction time and the fault occurrence time. The acquisition unit is used to acquire the operating information of the target vehicle during a second historical time period, which is the time period before the fault scene photos were taken. The determining unit is further configured to determine the time of taking the fault scene photo as the fault occurrence time when the operation information within the second historical time period meets preset conditions; wherein, the preset conditions include the normal operation information of the vehicle; the preset conditions correspond to the target fault type; the target fault type is the fault type used by the preset fault prediction model to predict the fault prediction time.

10. The model verification apparatus according to claim 9, characterized in that, The processing unit is specifically used to determine the difference between the fault prediction time and the fault occurrence time; If the difference is less than a preset threshold, the accuracy of the preset fault prediction model is determined to meet the preset requirement. If the difference is greater than or equal to the preset threshold, it is determined that the accuracy of the preset fault prediction model does not meet the preset requirement.

11. A model verification device, characterized in that, Deployed in vehicles, including memory and processor; The memory and the processor are coupled; The memory is used to store computer program code, which includes computer instructions; When the processor executes the computer instructions, the model verification device performs the model verification method as described in any one of claims 1-8.

12. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed on the model verification device, the model verification device performs the model verification method as described in any one of claims 1-8.

Images