Fault identification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of vehicles, in particular to a fault recognition method, device and equipment and a storage medium, which can greatly improve the recognition efficiency, save the labor cost and improve the recognition accuracy. The method comprises the following steps: acquiring a first image of a tested vehicle; inputting the first image into a first neural network model to obtain multiple groups of second images in one-to-one correspondence with the multiple types of parts; inputting the plurality of groups of second images into a second neural network model to obtain a plurality of groups of recognition results in one-to-one correspondence with the plurality of types of parts; aiming at each type of part, the corresponding group of identification results comprise identification results in one-to-one correspondence with at least one type of fault of the part; wherein the identification result is used for indicating the type and the maximum severity of the fault existing in the part.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a fault identification method, device, equipment and storage medium. Background Art

[0002] Before leaving the factory, vehicles undergo full vehicle testing to verify performance and optimize their designs based on the test results. These test results are typically recorded in the form of images. After obtaining the test results, technicians typically manually identify component locations, determine the fault type, and assess its severity. While this method can achieve basic fault identification, it suffers from low efficiency and error-proneness when dealing with large amounts of image data, making it difficult to meet the needs of large-scale data analysis. Summary of the Invention

[0003] This application provides a fault identification method, apparatus, device, and storage medium. The technical solutions adopted in this application are as follows:

[0004] In a first aspect, a fault identification method is provided, the method comprising: acquiring a first image of a tested vehicle; the first image comprising multiple types of parts of the vehicle; inputting the first image into a first neural network model to obtain multiple groups of second images corresponding one-to-one to the multiple types of parts; for each type of part, a corresponding group of second images is cropped from the first image according to the outline of the part; inputting the multiple groups of second images into a second neural network model to obtain multiple groups of recognition results corresponding one-to-one to the multiple types of parts; for each type of part, a corresponding group of recognition results comprises a one-to-one recognition result corresponding to at least one type of fault existing in the part; wherein the recognition result is used to indicate the type and maximum severity of the fault existing in the part.

[0005] In this application, there's no need to collect individual images of vehicle parts after testing, nor is there a need to manually identify fault types within each part image. By feeding a first image containing multiple types of parts into a first neural network model, the contours of the parts can be extracted and cropped into multiple sets of second images. These second images are then fed into a second neural network model to obtain recognition results corresponding to each part type. This significantly improves recognition efficiency, saves labor costs, and enhances recognition accuracy.

[0006] In a possible implementation of the first aspect, for a type of part, a corresponding set of second images is cropped from the first image according to the outline of the part that meets a first condition; wherein the first condition is: the confidence of the part is greater than a preset value and / or the confidence of the part is one of the top N confidences among the confidences of parts of the same type, where N is a positive integer; the confidence of the part is used to indicate the probability that the part belongs to the type predicted by the first neural network model.

[0007] For a given part type, this application crops a set of second images from the first image based on the contours of the parts with higher confidence. A higher confidence level indicates a greater probability that the part belongs to the type predicted by the first neural network model. Occlusion or resolution can lead to misclassification of parts. This application crops the second image from the first image using only the contours of the parts with higher confidence, ensuring the accuracy of the second image and thus improving recognition accuracy.

[0008] In a possible implementation of the first aspect, according to the identification result of each type of part and based on a preset judgment method related to the fault type, whether the part is qualified is identified and first information indicating whether the part is qualified is output.

[0009] In a possible implementation of the first aspect, the first information includes: the type of part, the type of fault existing in the part, and a judgment result corresponding to the type of fault existing in the part; wherein the judgment result corresponding to the type of fault is used to indicate whether the part with the fault is qualified.

[0010] This application further analyzes the part recognition results using a pre-set judgment method, quickly determining whether the part meets standard requirements and generating structured information, such as the first information, for subsequent processing. This can improve the automation level of the inspection process, thereby improving overall work efficiency and reducing human error.

[0011] In a possible implementation of the first aspect, based on the identification results of each type of parts and combined with the historical test data of the vehicle, the identification of the target vehicle including the parts and having the same type of fault is identified; combined with the design specifications of the target vehicle, rectification suggestions for the parts are output.

[0012] By incorporating historical test data and design specifications, this application can quickly identify vehicles with similar historical failures after identifying the current component failure and generate targeted corrective action recommendations based on their design specifications. This not only helps to deepen fault analysis but also provides a scientific basis for subsequent repairs and preventive measures.

[0013] In a possible implementation of the first aspect, based on the type of part and the type of fault present in the part, combined with after-sales data of the vehicle and environmental information of the region, environmental information of the part fault is identified; combined with the environmental information of the part fault, a risk warning for the part is output; the risk warning is used to indicate that the part is at risk of having a fault indicated by the fault type in the environment indicated by the environmental information.

[0014] By integrating after-sales data and regional environmental information, this application can predict the likelihood of failure of specific parts in specific environments and generate risk warnings accordingly. Prompting failures before they actually occur helps companies take preventive measures and mitigate potential losses. This shifts the focus from post-processing to pre-emptive prevention, effectively enhancing the intelligence of vehicle testing and after-sales management.

[0015] In a possible implementation of the first aspect, the first neural network model and the second neural network model are constructed based on a faster regional convolutional neural network. The first neural network model includes a sampling network, a region proposal generation network, a region of interest pooling network, and a masking layer network. The second neural network model includes a sampling network, a region proposal generation network, and a region of interest pooling network. The faster regional convolutional neural network has the advantages of high efficiency and high precision. The present application uses a faster regional convolutional neural network for fault identification, which can improve the efficiency and accuracy of identification.

[0016] In a second aspect, a fault identification device is provided for identifying a fault in a part of a tested vehicle. The fault identification device comprises:

[0017] An acquisition module, configured to acquire a first image of a tested vehicle; the first image includes multiple types of parts of the vehicle;

[0018] a first processing module configured to input the first image into a first neural network model to obtain a plurality of sets of second images corresponding to a plurality of types of parts; wherein, for a type of part, a corresponding set of second images is cropped from the first image according to the contour of the part;

[0019] The second processing module is used to input multiple groups of second images into the second neural network model to obtain multiple groups of recognition results corresponding one-to-one to multiple types of parts; wherein, for one type of part, the corresponding group of recognition results includes at least one recognition result corresponding one-to-one to at least one type of fault existing in the part; the recognition result is used to indicate the type and maximum severity of the fault existing in the part.

[0020] In a possible implementation of the second aspect, the fault identification device further includes:

[0021] The first output module is used to identify whether the parts are qualified according to the identification results of each type of parts and based on a preset judgment method, and output first information indicating whether the parts are qualified.

[0022] In a possible implementation of the second aspect, the fault identification device further includes:

[0023] The third processing module is used to identify the identity of the target vehicle that includes the part and has the same type of fault based on the identification results of each type of part in combination with the historical test data of the vehicle; and output rectification suggestions for the part based on the identity of the target vehicle in combination with the design specifications of the target vehicle.

[0024] In a possible implementation of the second aspect, the third processing module is also used to identify environmental information of part failure based on the type of part and the type of failure of the part, combined with after-sales data; and output a risk warning for the part based on the environmental information of the part failure; the risk warning is used to indicate that the part is at risk of failure in the environment indicated by the environmental information.

[0025] In a third aspect, a fault identification device is provided for identifying faults in a component of a tested vehicle. The fault identification device includes one or more processors; and a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the fault identification device implements the first aspect and any possible implementation thereof.

[0026] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer is caused to execute the first aspect and any possible implementation method thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a fault identification method in the prior art;

[0028] Figure 2A This is a flowchart of a fault identification method provided by an embodiment of the present application;

[0029] Figure 2B This is a flowchart of another fault identification method provided by an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of a sample image provided in an embodiment of the present application;

[0031] Figure 4 This is a flowchart of another fault identification method provided by an embodiment of the present application;

[0032] Figure 5 This is a schematic diagram of the recognition process of a first neural network model provided in an embodiment of the present application;

[0033] Figure 6 is a schematic diagram of a fault identification method provided in an embodiment of the present application;

[0034] Figure 7This is a schematic structural diagram of a fault identification device provided in an embodiment of the present application;

[0035] Figure 8 This is a flow chart of building an intelligent system provided by an embodiment of the present application;

[0036] Figure 9 It is a structural diagram of the fault identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.

[0038] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0039] Before leaving the factory, vehicles undergo full vehicle testing to verify performance and optimize their designs. Full vehicle testing includes, but is not limited to, performance, safety, and durability testing. Durability testing includes, but is not limited to, full vehicle corrosion testing and aging testing. Full vehicle test results are typically recorded in the form of images. After obtaining the test results, manual analysis of the images is required to organize the results and output conclusions. Figure 1 A schematic diagram of a prior art fault identification method is shown. For example, the method first requires manual identification of the part name in the image. Based on the part name, the system category, responsible department, part visibility, and inspection nodes are determined. Secondly, the fault type and severity are manually identified, and the fault meets the requirements according to the standard. Finally, the results are output. For a full vehicle test, the number of images recording the test results is enormous, perhaps exceeding 5,000. The image analysis process is time-consuming and labor-intensive, and the judgment of fault severity is subject to human error.

[0040] To this end, the present invention provides a fault identification method that, when a first image of a tested vehicle includes multiple types of parts, can identify the images of the multiple types of parts using a first neural network model and crop the images of the multiple types of parts from the first image. Subsequently, a second neural network model can be used to simultaneously identify the fault type and severity of the multiple types of parts. This significantly improves efficiency, saves labor costs, and enhances identification accuracy.

[0041] The following combination Figure 2A A fault identification method provided in an embodiment of the present application is introduced. The method may include:

[0042] S1, obtaining a first image of a tested vehicle.

[0043] The first image is an image of the tested vehicle. For example, the first image may be an image of the chassis of the tested vehicle. Another example is an image of a first exterior surface of the tested vehicle. The first exterior surface may include the front, rear, left, and right exterior surfaces of the vehicle. The test may be the full vehicle test described above.

[0044] The first image includes images of multiple types of parts. For example, if the first image is of the chassis of a tested vehicle, the parts include, but are not limited to, shock absorbers, springs, tie rods, steering knuckles, drive shafts, fasteners, and guard plates. For another example, if the first image is of the first exterior surface of a tested vehicle, the parts include, but are not limited to, doors, handles, glass, and wheels. The number of parts of the same type in the first image is greater than or equal to one.

[0045] A fault identification method provided in an embodiment of the present application can be executed by a server. The server can interact with an image acquisition device to obtain a first image of a tested vehicle. The image acquisition device is used to acquire the first image.

[0046] In some embodiments, an image acquisition device is equipped with hardware and software modules, including an image sensor, an infrared sensor, an image processing chip, and a communication module. The image sensor is used to capture images. The infrared sensor is used to determine the distance of an object from the lens. The image processing chip is used to process the image. The image acquisition device interacts with other devices, such as a server, through the communication module. For example, the image acquisition device uses a built-in infrared sensor to emit modulated infrared light pulses of a specific wavelength and receive reflected signals. Time-of-flight (ToF) is used to calculate the distance between the image acquisition device and the target object. The image processing chip associates the distance measurement data with the image pixel coordinates to establish a proportional relationship, thereby generating an image with scale information. This method enables accurate restoration of the actual dimensions of parts and faults even when photographed at different distances, improving the accuracy of image analysis. For example, based on the distance measurement results, such as the distance D from the image acquisition device to the target object and the focal length of the lens, the following formula is used to calculate the actual dimensions of each pixel in the image using the principle of similar triangles: Actual size = (pixel size × D) / focal length of the lens.

[0047] Optionally, the image acquisition device can also be connected to a terminal, such as a mobile phone. Users can use the terminal to remotely control camera parameters, such as exposure time and ranging frequency, and view the images captured by the image acquisition device in real time. The method for connecting the image acquisition device to the terminal can be referenced in existing technologies and will not be detailed here.

[0048] Optionally, the image acquisition device is further configured with hardware modules such as a storage unit, a display screen, a battery, an environmental sensor, and corresponding software systems. The above-mentioned hardware modules and corresponding software systems can refer to existing technologies and will not be described in detail here.

[0049] S2: Input the first image into the first neural network model to obtain multiple groups of second images corresponding to multiple types of parts.

[0050] After the first image is input into the first neural network model, multiple groups of second images corresponding one-to-one to multiple types of parts can be obtained, wherein one type of part corresponds to a group of second images, and the second images are images including the part.

[0051] For example, taking a first image of the chassis of a tested vehicle as an example, the first image may include two types of parts, such as a shock absorber and a spring. After inputting the first image into the first neural network model, a set of second images corresponding to the shock absorber and a set of second images corresponding to the spring can be obtained. The second images corresponding to a type of part only include images of parts of that type. For example, the set of second images corresponding to the shock absorber only includes images of the shock absorber. The set of second images corresponding to the spring only includes images of the spring.

[0052] In some embodiments, for each type of part, a corresponding set of images is cropped from the first image according to the outline of the part.

[0053] In other embodiments, for each part type, a corresponding set of images is cropped from the first image based on the outline of the part that meets a first condition. The first condition may be: the part's confidence level is greater than a preset value and / or the part's confidence level is one of the top N confidence levels of parts of the same type, where N is a positive integer. For example, N is in the range [1, 2, 3].

[0054] Exemplarily, a set of second images includes N images, which are cropped from the first image according to the outlines of the top N parts with the highest confidence. The confidence of a part is used to indicate the probability that the part belongs to the type predicted by the first neural network model. It reflects the credibility of the first neural network model. The confidence value is generally between 0 and 1. The closer the confidence value is to 1, the higher the confidence of the first neural network model in the detection result of the target. The closer the confidence value is to 0, the lower the confidence of the first neural network model in the detection result of the target.

[0055] Continuing with the previous example, let's assume that the first image includes images of four shock absorbers and five springs, where N = 3. After inputting the first image into the first neural network model, the first neural network model can identify the four shock absorbers and the corresponding confidence level for each shock absorber. The first neural network model can also identify the five springs and the corresponding confidence level for each spring. For example, the corresponding confidence levels for each shock absorber are 0.95, 0.9, 0.8, and 0.75, respectively. The first neural network model then crops the shock absorbers with the top three confidence levels from the first image based on their outlines, generating a set of second images corresponding to the shock absorbers. This set of second images includes three images: an image of a shock absorber with a confidence level of 0.95, an image of a shock absorber with a confidence level of 0.9, and an image of a shock absorber with a confidence level of 0.8. For example, the corresponding confidence levels for the five springs are 0.9, 0.8, 0.76, 0.75, and 0.7, respectively. The first neural network model clips the springs with the top three confidence levels from the first image based on their outlines, generating a set of second images corresponding to the springs. This set of second images includes three images: an image of a spring with a confidence level of 0.9, an image of a spring with a confidence level of 0.8, and an image of a spring with a confidence level of 0.76. In this embodiment, the value of N corresponding to different types of parts is the same.

[0056] Different types of parts have different corresponding N values. For example, for a spring, N might be 3, while for a shock absorber, N might be 4. Exemplarily, the server stores a mapping between part types and N values. After identifying each part, the server obtains the N value based on this mapping.

[0057] In another exemplary embodiment, a set of second images is cropped from the first image according to the contours of the parts having a confidence level greater than a preset value. In another exemplary embodiment, the preset value may be 0.8 or 0.7.

[0058] In another exemplary embodiment, a set of second images is cropped from the first image based on the contours of the first N parts having the highest confidence values ​​and having confidence values ​​greater than a preset value. In another exemplary embodiment, the preset value may be 0.8 or 0.7.

[0059] S3, inputting multiple groups of second images corresponding one-to-one to the multiple types of parts into the second neural network model to obtain multiple groups of recognition results corresponding one-to-one to the multiple types of parts.

[0060] One type of part corresponds to a set of recognition results.

[0061] For each part type, the corresponding set of identification results includes at least one identification result corresponding to at least one fault type present in the part. The identification result indicates the type and maximum severity of the fault present in the part. In other words, for each part type, each fault type corresponds to one identification result; multiple fault types correspond to multiple identification results.

[0062] The recognition result includes the name of the part, the type of fault of the part, the severity of the part, the location of the fault and other information. The fault types include but are not limited to rust, cracking, wrinkling, blistering, explosion, powdering, mildew, gap changes and other types. In some embodiments, the severity can be, for example, the degree of rust, cracking, wrinkling, blistering, explosion, powdering, mildew and the like. In this embodiment, the severity can be characterized by the ratio of the fault area on the part to the total area of ​​the part. In other embodiments, the severity can be characterized by the size of the fault area, such as the size of the gap. The fault location is used to indicate the location of the fault in the vehicle. In some embodiments, the fault can also be a defect, damage, flaw, damage or abnormality.

[0063] For each type of part, the corresponding set of identification results may include at least one identification result. Exemplarily, for a type of part, when the part has only one type of fault, the corresponding set of identification results may include one identification result. When the part has P types of faults, the corresponding set of identification results may include P identification results, where P is an integer greater than 1. For example, if the part has only one type of fault, such as rust, the corresponding set of identification results may include one identification result, which is used to indicate that the fault of the part is rust and the maximum severity of rust. Another exemplary example is that the part has two types of faults, such as rust and cracking, the corresponding set of identification results may include two identification results. One of the identification results is used to indicate that the fault of the part is rust and the maximum severity of rust. The other identification result is used to indicate that the fault of the part is cracking and the maximum severity of cracking.

[0064] Specifically, for each set of second images corresponding to each part type, the second neural network model can identify the part name, part fault type, and part severity, among other information. For a particular type of fault, the second neural network uses the recognition result corresponding to the highest severity as the final recognition result for that type of fault.

[0065] For example, taking a spring with rust and cracking faults as an example, a set of second images related to the spring output by the first neural network model is input into the second neural network model. For each image in this set of second images, the second neural network model can identify at least one recognition result. The at least one recognition result can be, for example, a recognition result for cracking and / or a recognition result for rusting. That is, for a second image, the second neural network model can identify at least one recognition result for at least one type of fault. For example, if the spring in this second image only has rusting faults, the second neural network model can identify a recognition result for rusting. If the spring in this second image only has cracking faults, the second neural network model can identify a recognition result for cracking. If the spring in this second image has both rusting faults and cracking faults, the second neural network model can identify a recognition result for both rusting and cracking.

[0066] Afterwards, the faults are clustered by type, and a recognition result corresponding to each fault type is output. For example, for each fault type, the recognition result corresponding to the most severe degree among multiple recognition results is output as the final recognition result corresponding to that fault type. For example, for a rust fault, the recognition result with the highest degree of rust among multiple recognition results is output as the recognition result corresponding to rust. For a cracking fault, the recognition result with the highest degree of cracking among multiple recognition results is output as the recognition result corresponding to cracking.

[0067] After executing S1-S3, for a first image including multiple types of parts, a set of recognition results corresponding to the multiple types of parts can be obtained. For one type of part, the set of recognition results includes at least one recognition result corresponding to at least one fault existing in the part.

[0068] It can be seen that the recognition method provided by the embodiment of the present application can recognize images of multiple types of parts through the first neural network model when the first image includes multiple types of parts, and crop the images of multiple types of parts from the first image. Afterwards, the fault type and severity of multiple types of parts can be identified at one time based on the second neural network model. There is no need to manually collect images of individual parts of the tested vehicle. Only a panoramic image needs to be collected, and the image of the part can be obtained based on the first neural network model. There is also no need to manually identify the type of fault in the image of the part one by one. After the image of the part is input into the second neural network model, the second neural network model can output the recognition result of at least one type of fault of a type of part. It can greatly improve efficiency, save labor costs, and improve recognition accuracy.

[0069] Alternatively, as Figure 2B As shown, the method may further include S4.

[0070] S4: Output first information based on the recognition result.

[0071] Based on the identification results of each type of parts, according to the preset judgment method, whether the parts are qualified is identified and the first information is output. In some embodiments, the preset judgment method is related to the type of fault. Different types of faults correspond to different judgment methods. For example, the judgment method for rust is that the degree of rust is less than the threshold A. The judgment method for cracking is that the crack length is less than the threshold B. In other embodiments, the preset judgment method is related to the type of fault and the type of parts. Different types of parts and different types of faults correspond to different judgment methods. For example, for springs, the judgment method for rust is that the degree of rust is less than the threshold A1, and the judgment method for cracking is that the crack length is less than the threshold B1. For shock absorbers, the judgment method for rust is that the degree of rust is less than the threshold A2, and the judgment method for cracking is that the crack length is less than the threshold B2.

[0072] For example, for a part of a certain type, if the part has multiple types of faults, the part's acceptance is determined based on multiple corresponding preset judgment methods. If the part is determined to be acceptable based on all of the multiple preset judgment methods, first information indicating the part's acceptance is output. If the part is determined to be unacceptable based on one or more of the multiple preset judgment methods, first information indicating the part's unacceptability is output. In other words, the first information indicates whether the part is acceptable. The judgment method associated with a particular fault type is used to determine whether the part with that fault is acceptable.

[0073] In some embodiments, the first information may include the part name, at least one type of fault present in the part, a severity level corresponding to the at least one type of fault, and a determination result corresponding to the at least one type of fault. The determination result may include either pass or fail. The determination result corresponding to a type of fault indicates whether the part with that fault is qualified.

[0074] Optionally, the first information may further include a fault location that corresponds one-to-one with at least one type of fault.

[0075] Optionally, the first information may also include a determination result of whether the part is qualified.

[0076] Optionally, the first information may also include the reason why the part is unqualified. For example, take the spring as an example, which includes rust failure and cracking failure. The first information includes two groups of information, one of which is: rust, rust degree 3, unqualified; the other group is: cracking, cracking degree 30%, qualified. The first information also includes the type of part, such as a spring, the judgment result of the part, such as unqualified, and the reason for the unqualified, such as excessive rust. For another example, take the spring as an example, which includes rust failure and cracking failure. The first information includes two groups of information, one of which is: rust, rust degree 1, qualified; the other group is: cracking, cracking degree 30%, qualified. The first information also includes the type of part, such as a spring, and the judgment result of the part, such as qualified.

[0077] Optionally, the fault identification method provided in the embodiment of the present application can also obtain design suggestions and / or risk warnings based on the identification results. Figure 2B As shown, the above method may further include:

[0078] S5, for a type of fault of a type of part, identifying the identifier of a target vehicle that includes the part and has the same type of fault in combination with historical test data of the vehicle.

[0079] Historical test data refers to data recorded during past tests or inspections. This data can reflect vehicle performance and failure scenarios during testing. By analyzing this data, you can identify historical failures similar to those of the currently identified component.

[0080] In some embodiments, for a particular part type, a first database is searched for the identification of a target vehicle based on the part type and fault type. The first database is used to store historical vehicle test data. The first database includes, but is not limited to, six key fields: vehicle identification, part type, fault type, severity, cause of the problem, and corrective measures. The target vehicle is a vehicle that includes the part and, after testing, has the same type of fault. The target vehicle identification can be, for example, a name or code.

[0081] For example, for spring corrosion failure, the first database may be searched for identifiers of all target vehicles whose springs exhibited corrosion failure after testing. For example, if the search finds that vehicle A and vehicle B exhibited spring corrosion failure after testing, then vehicle A and vehicle B are target vehicles.

[0082] In this embodiment of the application, by combining the current recognition results with historical test data, vehicles with the same or similar problems can be accurately identified. This method of combining the current recognition results with historical test data can reduce manual investigation time, allowing potential problem vehicles to be identified more quickly, thereby providing strong support for subsequent rectification work.

[0083] Optionally, after obtaining the identification of target vehicles with the same fault, rectification suggestions can be provided to the R&D personnel of each target vehicle based on the design specifications. Specifically, S6 can be executed.

[0084] S6, based on the identification of the target vehicle and the design specifications of the target vehicle, obtains rectification suggestions for the part.

[0085] Design specifications refer to the technical standards and requirements established during the vehicle design phase, including regulations on material selection, structural design, and manufacturing processes. Corrective recommendations are proposed improvement measures or solutions based on the target vehicle's design specifications and the currently identified component failures. By comparing the target vehicle's design specifications with the currently identified component failures, specific corrective recommendations can be generated to guide subsequent repairs or improvements.

[0086] In some embodiments, based on the target vehicle's identification and part type, a second database is searched for the target vehicle's design specifications for the part. Based on the design specifications, correction recommendations for the part are obtained. The second database stores the design specifications for each vehicle. For example, the second database includes, but is not limited to, six key fields: vehicle identification, part type, material type, coating type, corrosion protection requirements, and clearance range.

[0087] Continuing with the example of a spring part and target vehicles A and B, the second database is searched for vehicle A's spring design specifications, which may include material type, coating type, and corrosion protection requirements. These specifications are then combined into a correction recommendation for vehicle A. The second database is searched for vehicle B's spring design specifications, which may include material type, coating type, and corrosion protection requirements. These specifications are then combined into a correction recommendation for vehicle B. It should be understood that different vehicles may have the same or different design specifications for the same part.

[0088] In this embodiment of the application, by combining the current recognition results with historical test data, vehicles with the same or similar issues can be accurately identified. At the same time, targeted corrective action suggestions can be generated based on the design specifications of the target vehicle. This method can reduce the time required for manual investigation and ensure that the proposed corrective actions meet the design requirements, thereby resolving related issues more quickly and improving vehicle quality, thereby providing strong support for vehicle testing and after-sales maintenance.

[0089] Optionally, the above method may further include S7-S8.

[0090] S7, based on the type of the part and the type of the fault existing in the part, combined with the after-sales data of the vehicle and the environmental information of the region, identifying the environmental information of the fault occurring in the part.

[0091] After-sales data refers to data from historical repair records and / or user feedback, used to analyze the frequency and handling of similar failures in other models or markets. Regional environmental information includes temperature, humidity, rainfall, salt spray deposition, and de-icing salt usage, all of which collectively influence component durability and corrosion propensity. By combining this data with part type and failure type, typical environmental conditions leading to component failure can be more accurately identified, providing a basis for subsequent risk assessment.

[0092] In some embodiments, based on the type of part and the type of failure of the part, the after-sales information of the part is searched in the third database, and the geographical location and mileage of the part where the failure occurred are identified from the after-sales information of the part. The third database is used to store after-sales information. The third database contains but is not limited to eight key fields, including vehicle identification, part feasibility, failure type, failure location, failure year, failure mileage, after-sales processing method, and after-sales processing fee. Afterwards, based on the geographical location and the environmental information of the region, the environmental information of the part where the failure occurred is identified. Among them, the environmental information of the part where the failure occurred includes but is not limited to information such as temperature, humidity, weather information, geographical location, mileage, etc. Specifically, based on the geographical location, the environmental information of the geographical location is searched in the fourth database. The fourth database is used to store environmental information of various regions. Regions include countries, cities, counties, etc. Exemplarily, the fourth database includes but is not limited to nine key fields, including country, city / state, average maximum temperature from January to December, average minimum temperature from January to December, average humidity from January to December, average rainfall from January to December, average sunlight amplitude from January to December, salt spray deposition, and snow-melting salt usage.

[0093] S8, combined with the environmental information of the part failure, outputs the risk warning of the part.

[0094] The risk warning is used to indicate that the part is at risk of experiencing the failure in the environment indicated by the corresponding environmental information. The risk warning is used to inform relevant personnel that a certain type of part may face the risk of failure under specific environmental conditions. Risk warnings usually include key indicators such as failure probability, expected time of occurrence, and scope of impact. In this embodiment, the risk warning is the result obtained by comparing the identified environmental information with historical failure data and combining it with the current design parameters. Risk warnings can be used to guide production adjustments, after-sales service strategy formulation, and product improvement directions. For example, if a certain type of part frequently rusts in a high-humidity and high-salt environment, a warning prompt that this type of part has a higher risk of rust in coastal areas can be automatically generated for engineers' reference.

[0095] In the above embodiment, the first database may be, for example, an experimental database, the second database may be, for example, a design database, the third database may be, for example, an after-sales database, and the fourth database may be, for example, an environment database.

[0096] The following describes the first neural network model and the second neural network model provided in the embodiment of the present application in conjunction with the accompanying drawings. In the embodiment of the present application, the first neural network model has the ability to crop out an image that only includes parts from the sample image based on the confidence of the parts and the contours of the parts according to the input sample image. The second neural network model has the ability to output multiple groups of recognition results corresponding to multiple types of parts based on multiple groups of second images corresponding to multiple types of input. The embodiment of the present application does not specifically limit the type and structure of the first neural network model and the second neural network model. In some embodiments, the first neural network model and the second neural network model are faster region-based convolutional neural networks (Faster R-CNN).

[0097] The following is an example of a first neural network model. The structure of the second neural network model is similar to that of the first neural network model, and can be referred to in the examples below, so it will not be repeated here.

[0098] Exemplarily, the first neural network model includes an image normalization processing network, a sampling network (Backbone), a region proposal generation network (RPN), a region of interest pooling network (RoIPooling) and a masking layer (Mask Head) network. Among them, the image normalization processing network is used to preprocess the input image so that the input image meets the input requirements. The sampling network is used for feature extraction. RPN is used to generate a series of candidate regions that may contain targets. RoIPooling is used to uniformly map candidate regions of different sizes to a fixed-size feature map.

[0099] The following diagram illustrates the training process for the first neural network model. The training process for the second neural network model is similar to that for the first neural network model, and is omitted for clarity. The training process includes training sample generation and model training, each of which is described below.

[0100] Training sample generation includes S101 and S102.

[0101] S101, image preprocessing. The image includes an image of the tested vehicle. Preprocessing includes, but is not limited to, rotation, cropping, scaling, etc. For example, preprocessing may involve scaling the image to a width of 1028 pixels.

[0102] S102, image annotation. For example, annotation software is used to add annotation information to the image to obtain a sample image. The annotation information includes the part area in the image, the type of part, the fault area, the fault type, and the severity. For example, the part area in the image can be the area surrounded by the outline of the part. The fault area can be the area within the smallest rectangular box that includes the fault. For example, Figure 3 Image annotation is performed in the manner shown in FIG. Figure 3 As shown in (a), mark the outline and type of parts such as the steering rod and steering gear dust cover clamp. Figure 3 As shown in (b), the fault area is marked with a rectangle, and the fault type such as rust and the severity level such as level 3 are marked.

[0103] The sample image set can be divided into a training sample set and a validation sample set. For example, the ratio between the number of sample images in the training sample set and the number of sample images in the validation sample set is 2:1.

[0104] Model training includes S201-S207.

[0105] S201: Standardize the sample images in the training sample set and set the maximum and minimum image sizes. For example, the minimum size is 800 pixels and the maximum size is 1333 pixels. The sample images are packaged into data packets (batches), and the number of batches used for each training session is set. Each time a batch is extracted, no fewer than four batches are extracted.

[0106] S202: Extract features of the sample image through the sampling network in the first neural network model.

[0107] The sampling network uses a 50-layer residual network (Resnet50) as the feature extraction layer, with a sampling depth of 1280 bits. The sampling network includes an initial convolution and pooling layer, a mid-level feature extraction layer, a high-level semantic feature extraction layer, a global feature extraction layer, and a final abstract feature layer.

[0108] The initial convolution and pooling layers are used to initially extract low-level features, including edges, color, and texture of the inspected parts. Large convolution kernels and pooling are used to rapidly expand the receptive field. For example, the initial convolution and pooling layers use a 7-pixel × 7-pixel kernel, a 2-pixel sampling stride, and 64-bit output channels.

[0109] The mid-level feature extraction layer extracts mid-level features, such as texture combinations and simple shapes. For example, the mid-level feature extraction layer includes three residual blocks, each of which uses a bottleneck block structure, and all residual blocks maintain the input resolution. Bottleneck is a special structure in residual networks that reduces computation and the number of parameters while maintaining network performance. It achieves efficient feature extraction by using 1×1 convolutional layers to reduce and restore the number of channels in the feature map.

[0110] The high-level semantic feature extraction layer downsamples to further expand the receptive field and reduce computational complexity. For example, the high-level semantic feature extraction layer includes four residual blocks, the first of which downsamples with a step size of 2 pixels, while subsequent blocks maintain resolution. This layer is used to extract high-level semantic features, such as local features of object components and parts.

[0111] The function of the global feature extraction layer is to extract global features, such as complete objects, car parts, etc. Exemplarily, the global feature extraction layer includes 6 residual blocks, wherein the first residual block is used for downsampling with a step size of 2 pixels.

[0112] The final abstract feature layer generates highly abstract features, such as class distinctions. The extremely low-resolution feature maps are suitable for global pooling and classification. For example, the final abstract feature layer includes three residual blocks, the first of which is used for downsampling with a stride of 2 pixels.

[0113] S203: Output a set of candidate regions based on the features of the sample image through RPN.

[0114] Each candidate region can be, for example, a minimum bounding box that includes the part. For example, the minimum bounding box can be represented using coordinate values ​​(x, y, w, h), where (x, y) represents the center coordinates of the bounding box and (w, h) represents the width and height of the bounding box.

[0115] S204: Position the candidate region through RoIPooling.

[0116] In some embodiments, RoIPooling uses a region of interest alignment (RoIAlign) network to locate candidate regions.

[0117] S205 , generating a mask image for the part selected in each candidate area through the masking layer.

[0118] In some embodiments, the masking layer is a lightweight fully convolutional network consisting of multiple convolutional layers and deconvolutional layers. Exemplarily, the masking layer consists of a feature extraction module, an upsampling module, and a classification module. Among them, the feature extraction module consists of 4 convolutional layers, each of which uses a 3-pixel × 3-pixel convolution kernel, followed by a rectified linear unit (ReLU) activation function. The convolution layer is used to extract high-level semantic information from the input feature map and gradually increase the level of abstraction of the features. The upsampling module contains 1 transposed convolution layer (also called a deconvolution layer) using a 2-pixel × 2-pixel transposed convolution kernel. This layer increases the resolution of the feature map from 14 pixels × 14 pixels to 28 pixels × 28 pixels through an upsampling operation to restore some spatial resolution, and is followed by a ReLU activation function to introduce nonlinearity. The classification module contains 1 convolution layer using a 1-pixel × 1-pixel convolution kernel to map the number of channels of the feature map to the number of target categories. The function of this layer is to perform classification at each pixel position and output the probability distribution of each pixel belonging to each category.

[0119] S206, through the fully connected layer, outputs the type of part, the type of fault, and an image that only includes the part.

[0120] S207, calculate the loss. If the loss meets the conditions, the training ends.

[0121] The following describes a fault identification method provided by an embodiment of the present application in conjunction with the structure of the first neural network model. Figure 4 FIG. 1 is a flow chart of another fault identification method provided in an embodiment of the present application. Exemplarily, the method may include:

[0122] S410: Preprocess the first image.

[0123] For the first image, please refer to the introduction in the previous article and will not be repeated here.

[0124] S420: Input the first image into the network to obtain image features of the first image.

[0125] Exemplarily, ResNet-50 is used as a sampling network to perform feature extraction on the input first image, and finally output a 7-pixel×7-pixel feature map with 1280 channels.

[0126] S430: Input the image features of the first image into the RPN to obtain a set of candidate regions and corresponding confidence levels.

[0127] The candidate areas can be referred to in the previous article and will not be described in detail here.

[0128] In some embodiments, the RPN network includes five prediction feature layers for generating candidate regions (ROIs) and confidence scores of the candidate regions.

[0129] S440: Input the image features of the first image and a set of candidate regions into RoIPooling to process the candidate regions.

[0130] The processing can be understood as: projecting the candidate region from the original image coordinate space to the feature map space, extracting fixed-size local features from the feature map by retaining floating-point coordinates and bilinear interpolation, while maintaining spatial alignment accuracy.

[0131] S450: Input the image features, candidate regions and confidence levels of the first image into the masking layer to obtain multiple sets of second images corresponding to multiple types of parts.

[0132] For each type of part, a corresponding set of second images is cropped from the first image according to the outlines of the top N parts with the highest confidence.

[0133] For example, Figure 5 As shown, after the image 501 is input into the first neural network model, the sampling network 502 extracts features from the image 501. RPN 503 obtains a set of candidate regions and corresponding confidence levels based on the image features of the image 501. RoIPooling 504 processes a set of candidate regions. The classifier 505 outputs the probability that the parts selected in each candidate region belong to each category. The regressor 506 outputs the adjusted bounding box parameters. These parameters generally include the coordinate offset and size of the center point of the bounding box. The image features, candidate regions, confidence levels, categories and bounding box parameters of the first image are input into the masking layer 507 to obtain multiple sets of second images corresponding to multiple types of parts. Then, the multiple sets of second images are input into the second neural network model to obtain the type of fault and the maximum severity of the fault. As shown Figure 5 As shown, the second neural network model can output the fault type of the lock pin fastening bolt, such as rust, and the maximum rust level, such as level 3. The second neural network model can also output a second image corresponding to the maximum severity and the confidence level of the second image.

[0134] S460, inputting multiple sets of second images into the second neural network model to obtain multiple sets of recognition results corresponding to multiple types of parts.

[0135] Among them, the multiple sets of recognition results corresponding to various types of parts can be referred to the introduction in the previous article and will not be repeated here.

[0136] In some embodiments, the first neural network model and the second neural network model have the following differences:

[0137] The RPN downsampling ratio of the first neural network model is 16 times, and the RPN downsampling ratio of the second neural network model is 32 times.

[0138] The size of the anchor box generator (anchar generator) in the first neural network model is (32 pixels, 64 pixels, 128 pixels, 256 pixels), and the size of the anchar generator in the second neural network model is (32 pixels, 64 pixels, 128 pixels, 25 pixels, 512 pixels).

[0139] The first neural network model sampling network uses Resnet50 as the feature extraction layer, and the sampling depth is 640 bits. The second neural network model sampling network uses Resnet50 as the feature extraction layer, and the sampling depth is 1280 bits.

[0140] The first neural network model consists of four stages. Each stage uses convolution downsampling with a step size of 2 pixels to gradually reduce the feature map resolution. The fourth stage senses global features to distinguish between part features and generate feature maps of different resolutions. The second neural network model consists of five stages. The fifth stage is used to extract local features and identify fault type and severity.

[0141] like Figure 6 As shown, in an embodiment of the present application, inputting a first image into a first neural network model can generate multiple sets of second images corresponding to various types of parts. Inputting these multiple sets of second images into a second neural network model can generate multiple sets of recognition results corresponding to various types of parts. For each type of part, the corresponding set of recognition results includes at least one recognition result corresponding to at least one fault type present in the part. This significantly improves efficiency, saves labor costs, and enhances recognition accuracy.

[0142] The above text, in conjunction with the accompanying drawings, describes in detail the method embodiments of the present application. Below, in conjunction with the accompanying drawings, the device embodiments of the present application will be described in detail. It should be understood that the description of the device embodiments corresponds to the method embodiments. Therefore, for portions not described in detail, reference can be made to the method embodiments above.

[0143] Figure 7 FIG. 7 is a schematic diagram of a structure of a fault identification device provided in an embodiment of the present application. Exemplarily, the fault identification device 700 includes an acquisition module 710 , a first processing module 720 , and a second processing module 730 .

[0144] An acquisition module 710 is configured to acquire a first image of a tested vehicle; the first image includes various types of vehicle parts;

[0145] A first processing module 720 is configured to input the first image into a first neural network model to obtain a plurality of sets of second images corresponding to a plurality of types of parts. For each type of part, the corresponding set of second images is cropped from the first image according to the outlines of the top N parts with the highest confidence, where N is a positive integer.

[0146] The second processing module 730 inputs multiple groups of second images into the second neural network model to obtain multiple groups of recognition results corresponding to multiple types of parts; for each type of part, the corresponding group of recognition results includes at least one recognition result corresponding to at least one fault type existing in the part; wherein the recognition result is used to indicate the type and maximum severity of the fault existing in the part.

[0147] Optionally, the fault identification device 700 may further include a first output module 740 .

[0148] The first output module 740 is used to identify whether the part is qualified and output first information based on the identification result of each type of part, based on the fault type of the part and a preset judgment method; wherein the first information includes the type of part, the type of fault existing in the part, the maximum severity corresponding to the fault type and the judgment result corresponding to the fault type.

[0149] Optionally, the fault identification device 700 may further include a third processing module 750 .

[0150] The third processing module 750 is used to identify the target vehicle that includes the part and has the same type of fault based on the identification results of each type of part in combination with the historical test data of the vehicle; and output rectification suggestions for the part in combination with the design specifications of the target vehicle.

[0151] The third processing module 750 is also used to identify the environmental information of the part failure based on the type of part and the type of part failure in combination with after-sales data; and output a risk warning for the part based on the environmental information of the part failure; the risk warning is used to indicate that the part is at risk of failure in the environment indicated by the environmental information.

[0152] In some embodiments, the third processing module 750 may be an agent. An agent is an intelligent entity and is a very important concept in the field of artificial intelligence. Any entity that has processing capabilities and can exchange information with other entities can be abstracted as an agent. An agent can be a software entity, a hardware entity, or a combination of software and hardware.

[0153] For example, Figure 8This diagram shows the architecture of an agent system. The agent system consists of a knowledge base and agents. The agent system construction process includes building the knowledge base, building agents, debugging agents, and operating agents.

[0154] Build a knowledge base: Build a knowledge base including experimental database, design database, after-sales database, and environmental database. These knowledge bases will provide the necessary information and data support for the intelligent agent.

[0155] For example, a knowledge base can be constructed using pre-defined tools. For example, for an experimental database, Python can be used to analyze and organize numerical data, such as sorting high-risk vehicle models and analyzing fault types; a model can be used to analyze and organize text data; and annotation software can be used to annotate images to generate sample images for training the first neural network model.

[0156] For environmental databases, gray theory can be used to analyze and organize numerical data. For example, by analyzing parameters such as temperature, humidity, rainfall, chloride ion concentration, snow-melting salt environment, sulfur dioxide concentration, and condensation parameters, the corrosion intensity of the region can be evaluated and the corrosion intensity value can be given.

[0157] For after-sales databases, Python can be used to analyze and organize numerical data, for example, to sort high-risk vehicle models, analyze fault types, and compile statistics and rankings on after-sales repair costs; models can be used to analyze and organize text-type data; and annotation software can be used to process images to generate sample images for training the first neural network model.

[0158] For the design database, Python can be used to analyze and organize numerical data to facilitate the search for the design specifications of the target vehicle.

[0159] Agent building: Building an agent based on the workflow. This step involves integrating data and information from the knowledge base into the agent so that it can understand and handle relevant tasks and problems. For example,

[0160] Agent debugging: After the agent is built, debugging is required. This includes steps such as generating and adding or removing prompt words, setting recall parameters, setting parameters for the large model, and calling the company's large model to ensure the performance and accuracy of the agent. The agent retrieves relevant information from the knowledge base based on the retrieval-augmented generation pipeline (RAGPipeline) and sorts this information to ensure that the most relevant and useful information is output, which helps improve the accuracy and relevance of the agent's search.

[0161] Agent Operation: The operation phase includes activities such as data detection, content evaluation, logging and recording, and response annotation. These activities help monitor the performance of the agent and make adjustments and optimizations based on feedback.

[0162] Figure 9 FIG. 9 is a schematic diagram of a structure of a fault identification device provided in an embodiment of the present application. Exemplarily, the fault identification device 900 includes one or more processors 910 and one or more memories 920 . Figure 9 The fault identification device 900 is used to implement the fault identification method described in the above method embodiment. The fault identification device 900 can be a control unit in a vehicle. The control unit can be, for example, a vehicle controller or an engine controller, or, as a possible implementation method, the control unit can also be a controller independently set up to implement the above control method.

[0163] The processor 910 can support the fault identification device 900 to implement the method described in the above method embodiment.

[0164] The memory 920 stores a program that can be executed by the processor 910 so that the processor 910 performs the method described in the above method embodiment. The memory 920 can be independent of the processor 910 or integrated into the processor 910.

[0165] Optionally, the fault identification device 900 may further include a transceiver 930. The processor 910 may communicate with other devices or chips via the transceiver 930. For example, the processor 910 may transmit and receive data with other devices or chips via the transceiver 930.

[0166] In some embodiments, the fault identification device 900 can be a server. The server is configured with two NVIDIA A100 model graphics processing units, each of which is equipped with 90GB of video memory, which can meet the processing requirements of manuscript resolution images. The server is also configured with a central processing unit model AMD EPYC 7B13. The central processing unit is an ultra-powerful configuration with 64 cores and 128 threads. It can provide excellent computing performance. The server is configured with 512 gigabytes of fourth-generation double data rate synchronous dynamic random access memory (double data rate 4SDRAM, DDR4). The DDR4 supports error correction code (ECC) function. The server is configured with a 4-megabyte non-volatile memory express (NVMe) solid-state drive and a 100-megabyte hard drive redundant array independent disk. The server is configured with dual 10 Gigabit network cards.

[0167] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the method of any of the above embodiments.

[0168] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0169] The processor may be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It is understood that the electronic device that implements the functions of the processor may also be other electronic devices, which are not specifically limited in the embodiments of the present application.

[0170] The computer storage medium / memory may be a read-only memory, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface mount memory, an optical disc, or a compact disc read-only memory (CD ROM).

[0171] An embodiment of the present application provides a computer program, including computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes some or all of the steps for implementing the above method.

[0172] An embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, implements some or all of the steps in the above method. The computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium. In other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0173] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned steps / processes does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0174] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0176] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0177] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0178] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0179] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a vehicle-mounted terminal (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0180] The above is only an embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who can easily think of changes or replacements within the technical scope disclosed in the present application should be included in the protection scope of the present application.

[0181] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art based on the present application are within the protection scope of the present application.

Claims

1. A fault identification method, characterized in that: The method comprises: Acquiring a first image of a tested vehicle; the first image including multiple types of parts of the vehicle; Inputting the first image into a first neural network model to obtain a plurality of sets of second images corresponding to the plurality of types of parts; wherein, for a type of part, a corresponding set of second images is cropped from the first image according to the outline of the part; The multiple groups of second images are input into a second neural network model to obtain multiple groups of recognition results corresponding one-to-one to the multiple types of parts; wherein, for one type of part, the corresponding group of recognition results includes at least one recognition result corresponding one-to-one to at least one type of fault existing in the part; the recognition result is used to indicate the type and maximum severity of the fault existing in the part.

2. The method according to claim 1, characterized in that For a type of part, the corresponding set of second images is cropped from the first image according to the outline of the part that meets the first condition; Among them, the first condition is: the confidence of the part is greater than a preset value and / or the confidence of the part is one of the top N confidences among the confidences of parts of the same type, where N is a positive integer; the confidence of the part is used to indicate the probability that the part belongs to the type predicted by the first neural network model.

3. The method according to claim 1 or 2, characterized in that The method further comprises: According to the identification result of each type of parts, based on a preset judgment method related to the type of fault, it is identified whether the part is qualified and first information indicating whether the part is qualified is output.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Based on the identification results of each type of parts and in combination with historical test data of the vehicle, an identifier of a target vehicle including the part and having the same type of fault is identified; Based on the identification of the target vehicle and in combination with the design specifications of the target vehicle, a rectification suggestion for the part is output.

5. The method according to claim 4, characterized in that The method further comprises: According to the type of the part and the type of the fault existing in the part, combined with after-sales data, identifying the environmental information in which the fault occurs in the part; Based on the environmental information of the part in which the fault occurs, a risk warning for the part is output; the risk warning is used to indicate that the part has a risk of causing the fault in the environment indicated by the environmental information.

6. The method according to claim 3, characterized in that The first information includes: the type of the part, the type of fault existing in the part, and a determination result corresponding one-to-one to the type of fault existing in the part; wherein the determination result corresponding to the type of fault is used to indicate whether the part with the fault is qualified.

7. The method according to any one of claims 1 to 6, characterized in that The first neural network model and the second neural network model are constructed based on a faster regional convolutional neural network; the first neural network model includes a sampling network, a region proposal generation network, a region of interest pooling network and a masking layer network; the second neural network model includes a sampling network, a region proposal generation network, and a region of interest pooling network.

8. A fault identification device, characterized in that: The fault identification device is used to identify a fault of a part of a tested vehicle, and the fault identification device includes: An acquisition module, configured to acquire a first image of a tested vehicle; the first image includes multiple types of parts of the vehicle; a first processing module, configured to input the first image into a first neural network model to obtain a plurality of sets of second images corresponding one-to-one to the plurality of types of parts; wherein, for a type of part, the corresponding set of second images is cropped from the first image according to the contour of the part; The second processing module is used to input the multiple groups of second images into a second neural network model to obtain multiple groups of recognition results corresponding one-to-one to the multiple types of parts; wherein, for one type of part, the corresponding group of recognition results includes at least one recognition result corresponding one-to-one to at least one type of fault existing in the part; the recognition result is used to indicate the type and maximum severity of the fault existing in the part.

9. A fault identification device, characterized in that: The fault identification device is used to identify a fault of a part of a tested vehicle, and the fault identification device includes: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the device to implement the fault identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the fault identification method according to any one of claims 1 to 7.

Images