Vehicle anomaly detection method, device, equipment, medium and product

Abstract

The invention discloses a vehicle anomaly detection method, device and equipment, a medium and a product. The method comprises the following steps: if a target vehicle video is detected, splicing the target vehicle video to obtain a panoramic image of a target detection vehicle; performing identification processing on the panoramic image by adopting a preset deep learning model to determine whether the appearance of the target detection vehicle is abnormal or not, and if yes, determining an abnormal category and an abnormal level corresponding to the abnormal appearance feature; and determining an early warning strategy for the target detection vehicle according to the abnormality category and the abnormality level, and performing abnormality early warning at the corresponding early warning time according to the early warning strategy. According to the technical scheme, the deep learning technology can be combined, abnormal recognition and early warning can be automatically carried out on the vehicle appearance, the vehicle maintenance efficiency and safety are improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle appearance detection, and in particular to a vehicle abnormality detection method, device, equipment, medium and product. Background Art

[0002] With the development of vehicle technology, the demand for vehicle intelligence is gradually increasing. The traditional way of visual inspection of vehicle appearance is inefficient and easy to miss details. In addition, with the development of deep learning technology, automated vehicle appearance anomaly detection has become possible.

[0003] Therefore, how to combine deep learning technology to automatically identify and warn of abnormalities in vehicle appearance, improve the efficiency and safety of vehicle maintenance, and enhance user experience is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides a vehicle anomaly detection method, device, equipment, medium and product, which combine deep learning technology to automatically identify and warn of abnormalities in vehicle appearance, improve the efficiency and safety of vehicle maintenance, and enhance user experience.

[0005] According to one aspect of the present invention, a vehicle abnormality detection method is provided, comprising:

[0006] If the target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detection vehicle;

[0007] Use a preset deep learning model to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal. If so, determine the abnormal category and abnormal level corresponding to the abnormal appearance features;

[0008] According to the abnormality category and abnormality level, the warning strategy for the target detection vehicle is determined, and according to the warning strategy, the abnormality warning is issued at the corresponding warning time.

[0009] According to another aspect of the present invention, there is provided a vehicle abnormality detection device, comprising:

[0010] The obtaining module is used for, if a target vehicle video is detected, stitching the target vehicle video to obtain a panoramic image of the target detected vehicle;

[0011] A judgment module is used to use a preset deep learning model to perform recognition processing on the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, determine the abnormal category and abnormal level corresponding to the abnormal appearance features;

[0012] The warning module is used to determine the warning strategy for the target detection vehicle according to the abnormal category and abnormal level, and to issue an abnormal warning at the corresponding warning time according to the warning strategy.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle abnormality detection method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the vehicle abnormality detection method described in any embodiment of the present invention when executed.

[0018] According to another aspect of the present invention, a computer program product is further provided. The computer program product includes a computer program. When the computer program is executed by a processor, the vehicle abnormality detection method according to any embodiment of the present invention is implemented.

[0019] The technical solution of the embodiment of the present invention is that if the target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detection vehicle; a preset deep learning model is used to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, the abnormal category and abnormal level corresponding to the abnormal appearance features are determined; according to the abnormal category and abnormal level, the early warning strategy for the target detection vehicle is determined, and according to the early warning strategy, an abnormal warning is issued at the corresponding warning time. By combining deep learning technology, abnormal identification and early warning of vehicle appearance can be automatically performed, improving the efficiency and safety of vehicle maintenance and enhancing user experience.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a vehicle abnormality detection method provided by Embodiment 1 of the present invention;

[0023] Figure 2 is a flow chart of a vehicle abnormality detection method provided by Embodiment 2 of the present invention;

[0024] Figure 3 is a structural block diagram of a vehicle abnormality detection device provided by Embodiment 3 of the present invention;

[0025] Figure 4 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", "target", "candidate", "alternative", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0028] Embodiment 1

[0029] Figure 11 is a flow chart of a vehicle anomaly detection method provided in Embodiment 1 of the present invention; this embodiment is applicable to a situation where a vehicle cloud server identifies an abnormality in the appearance of a vehicle based on a video of a target vehicle and issues an early warning, the method can be executed by a vehicle anomaly detection device, the vehicle anomaly detection device can be implemented in the form of hardware and / or software, the vehicle anomaly detection device can be configured in an electronic device, such as a vehicle cloud server, such as Figure 1 As shown, the vehicle abnormality detection method includes:

[0030] S101: If a target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detected vehicle.

[0031] The target vehicle video can be obtained by the target user shooting the target detection vehicle from multiple angles, or by shooting with a camera pre-configured on the target detection vehicle. The target detection vehicle refers to the vehicle for target vehicle abnormality detection, and the panoramic image refers to an image that can characterize the appearance of the entire target detection vehicle. The panoramic image can specifically be an image of the surroundings (side of the vehicle) of the car. The duration of the target vehicle video can be, for example, 30 seconds.

[0032] Optionally, the target user can use a mobile phone to take 360-degree surround shots of the target detection vehicle before getting on the vehicle, and upload the target vehicle video to the vehicle's cloud server through a preset mobile client application to instruct the cloud server to perform vehicle abnormality detection based on the target vehicle video; the vehicle's cloud server can also periodically use a camera pre-configured on the target detection vehicle to collect video of the target detection vehicle based on a preset duration.

[0033] Optionally, after determining the target vehicle video, the change area and repeated area between the front and rear video frames can be determined, the repeated area between the front and rear video frames can be deleted, and the change areas can be spliced ​​in sequence according to the shooting time of each video frame to obtain a panoramic image of the target detection vehicle; vehicle images at different angles can also be identified from the target vehicle video, and the vehicle images at different angles can be fused through a preset image fusion algorithm to obtain a panoramic image of the target detection vehicle.

[0034] S102: Use a preset deep learning model to perform recognition processing on the panoramic image to determine whether the appearance of the target detection vehicle is abnormal. If so, determine the abnormal category and abnormal level corresponding to the abnormal appearance features.

[0035] Among them, the preset deep learning model can be a convolutional neural network model or a one-class classification model (One-Class Classification Models).

[0036] It should be noted that the convolutional neural network model can extract local features through the convolution layer and reduce the feature dimension through the pooling layer, thereby realizing a high-dimensional abstract representation of the features. In vehicle appearance anomaly detection, the convolutional neural network model can perform target detection and image classification to extract features of vehicle appearance images and identify anomalies such as scratches and dents.

[0037] It should be noted that a single-class classification model, such as One-Class SVM (Support Vector Machine), is designed to identify normal patterns in data and identify abnormal points that are significantly different from normal patterns. It does not require abnormal samples for training. In the case of vehicle appearance anomaly detection, a single-class classification model can be used to identify abnormal situations that are significantly different from the normal vehicle appearance.

[0038] Abnormal appearance features refer to features that can characterize abnormal appearance of the target detection vehicle, and can be obtained by performing image segmentation on the panoramic image. Abnormal categories include damage categories and non-damage categories; non-damage categories include stains and / or flat tires; damage categories refer to stains that may cause corrosion to the surface of the vehicle body, and damage categories include at least one of the following: dents, scratches, paint peeling, cracks, wrinkles, and punctures.

[0039] Scratches refer to linear scratches on the vehicle surface, which can be identified by edge detection and texture analysis of panoramic images. Dirt or Stains refer to foreign objects such as bird droppings, shellac, mud, rust, and oil stains attached to the vehicle surface, which can be identified by object detection and object classification of panoramic images. Rust is caused by chemical reactions caused by moisture and other pollutants in the air after the paint film is damaged. Dents refer to dents formed on the vehicle surface due to impact, which can be identified by shape analysis and depth estimation of panoramic images. Cracks refer to cracks formed by cracking on the vehicle surface, which can be identified by texture analysis and edge detection of panoramic images. Creases refer to wrinkles formed on the vehicle surface due to impact or other external forces, which can be identified by shape analysis of panoramic images. Punctures refer to holes formed by penetration of the vehicle surface, which can be identified by depth estimation and edge detection of panoramic images. Paint peeling refers to the deterioration or peeling of the paint film on the car body due to ultraviolet rays, drastic changes in ambient temperature and humidity, stains, aging or mechanical damage.

[0040] Optionally, a preset deep learning model may be used to perform at least one of the following image processing operations on the panoramic image: shape analysis, edge detection, texture analysis, depth estimation, target detection, and target classification.

[0041] Optionally, a preset deep learning model can be used to perform recognition processing on the panoramic image. If foreign matter such as bird droppings, shellac, mud, rust and oil stains are detected on the surface of the vehicle, it can be determined that the appearance of the target detection vehicle is abnormal and the abnormal category is stains in the non-damage category.

[0042] Optionally, a preset deep learning model is used to perform recognition processing on the panoramic image to determine whether there is any abnormality in the appearance of the target detection vehicle, including: using a preset deep learning model to extract features of the panoramic image to obtain appearance features of the target detection vehicle; comparing the appearance features with historical annotation information to obtain comparison results, and determining whether there is any abnormality in the appearance of the target detection vehicle based on the comparison results.

[0043] Among them, the appearance features include at least one of the following: shape features, color features and texture features; historical annotation information refers to the information generated by feature and anomaly annotation of the historical panoramic images of the target detection vehicle, and the historical annotation information may include at least one of the following: shape information of different parts of the target detection vehicle in the historical panoramic images, color information of different parts of the target detection vehicle in the historical panoramic images, texture information of different parts of the target detection vehicle in the historical panoramic images, and historical abnormal features contained in the target detection vehicle in the historical panoramic images and their corresponding abnormal categories and abnormal levels. The comparison result can characterize the comparison of different parts of the target detection vehicle in the historical panoramic images and the currently detected panoramic images.

[0044] Optionally, after obtaining the appearance features of the target detection vehicle, at least one of the following abnormality recognition operations can be performed on the appearance features: shape analysis, edge detection, texture analysis, depth estimation, target detection, and target classification, so as to obtain preliminary abnormality recognition results for each appearance feature, wherein the abnormality recognition results include the abnormal appearance features detected, and based on the preliminary abnormality recognition results, the abnormal appearance features are compared with the corresponding area features in the historical annotation information to determine the final recognition result of whether an abnormality occurs at the corresponding position of the target detection vehicle.

[0045] Specifically, if the appearance features of the abnormality are different from the features of the corresponding area in the historical annotation information, it is determined that an abnormality has occurred at the corresponding position of the target detection vehicle; if the appearance features of the abnormality are the same as the features of the corresponding area in the historical annotation information, it can be considered that an warning has been issued for the abnormality in the area, and it is determined that no warning is issued for the time being for the area, and the number of no warnings recorded in the history for the area is determined, that is, the number of times the abnormality is initially detected and the final decision is made not to issue a warning after re-comparison. If the number of no warnings is greater than the preset warning threshold, it can be determined that an abnormality has occurred at the corresponding position of the target detection vehicle, and the preliminary abnormality recognition results are directly displayed on the display screen of the vehicle for warning.

[0046] Optionally, determining the abnormal category and abnormal level corresponding to the abnormal appearance feature includes: identifying the abnormal appearance feature corresponding to the target detection vehicle to determine the abnormal category corresponding to the abnormal appearance feature; and determining the abnormal level corresponding to the abnormal appearance feature based on the abnormal category and the correlation between the abnormal parameters corresponding to the appearance feature and a preset evaluation threshold.

[0047] The abnormal parameters may include at least one of the following: abnormal area, tire pressure, dent depth, scratch length, paint detachment area, crack length, wrinkle height, and puncture diameter. The preset evaluation threshold may correspond to the abnormal parameter. For example, if the abnormal category is a stain in the non-damage category and the abnormal parameter is the abnormal area, the corresponding preset evaluation threshold may be a first area threshold and a second area threshold, the first area threshold is less than the second area threshold, the first area threshold may be 0.1 square meters, and the second area threshold may be 0.5 square meters.

[0048] Optionally, at least one of the following abnormality recognition operations may be performed on the appearance features: shape analysis, edge detection, texture analysis, depth estimation, target detection, and target classification, so as to obtain abnormal appearance features among the appearance features.

[0049] Optionally, the abnormal appearance feature may be compared with preset category features of different abnormal categories, and the abnormal category corresponding to the abnormal appearance feature may be identified and determined based on the similarity between the abnormal appearance feature and the category features.

[0050] Optionally, the abnormality level corresponding to the abnormal appearance feature is determined according to the abnormality category and the correlation between the abnormal parameters corresponding to the appearance feature and the preset evaluation threshold, including: if the abnormality category is a stain in the non-damage category, then according to the appearance feature, the abnormal area of ​​the abnormal region is determined to obtain the abnormal parameters corresponding to the appearance feature; and according to the relationship between the abnormal area and the stain evaluation threshold in the preset evaluation threshold, the abnormality level corresponding to the abnormal appearance feature is determined.

[0051] The abnormal area refers to the area where the stain is located. When the abnormal category is a stain in the non-damage category, the preset evaluation threshold may be a first area threshold and a second area threshold, the first area threshold is less than the second area threshold, the first area threshold may be 0.1 square meters, and the second area threshold may be 0.5 square meters.

[0052] Optionally, if the abnormal area is smaller than the first area threshold, the abnormal level can be determined as level 1, i.e., a small area of ​​stains, indicating that the stains are easy to clean; if the abnormal area is greater than or equal to the first area threshold and less than or equal to the second area threshold, the abnormal level can be determined as level 2, i.e., a medium area of ​​stains, indicating that certain cleaning work is required; if the abnormal area is greater than the second area threshold, the abnormal level can be determined as level 3, i.e., a large area of ​​stains, indicating that it is difficult to clean and affects the appearance of the vehicle.

[0053] Exemplarily, when the abnormal category is a flat tire in the non-damage category, the corresponding abnormal classification standards can be as follows: Level 1: slight lack of air pressure (the air pressure is within 10% lower than the standard value); Level 2: moderate lack of air pressure (the air pressure is 10% to 20% lower than the standard value); Level 3: severe lack of air pressure or overpressure (the air pressure is more than 20% lower than the standard value or overpressure).

[0054] For example, when the abnormality category is a dent in the damage category, the corresponding abnormality classification standard can be as follows: Level 1: slight dent (depth less than 5mm), which does not affect the structural strength; Level 2: moderate dent (depth 5mm to 15mm), which may affect the structural strength; Level 3: severe dent (depth greater than 15mm), which seriously affects the structural strength.

[0055] For example, when the abnormality category is a scratch in the damage category, the corresponding abnormality grading standard can be as follows: Level 1: minor scratches (less than 10 cm in length and not deep enough to reach the primer); Level 2: moderate scratches (10 to 30 cm in length and with paint peeling but no metal exposed); Level 3: severe scratches (more than 30 cm in length or deep enough to expose metal).

[0056] For example, when the abnormality category is paint peeling in the damage category, the corresponding abnormality classification standards can be as follows: Level 1: small area peeling (area less than 0.1 square meters); Level 2: medium area peeling (area 0.1 to 0.5 square meters); Level 3: large area peeling (area more than 0.5 square meters).

[0057] Exemplarily, when the abnormality category is a crack in the damaging category, the corresponding abnormality grading standard can be as follows: Level 1: surface cracks (less than 10 cm in length, not penetrating the paint layer); Level 2: medium cracks (10 cm to 30 cm in length, penetrating the paint layer but not touching the metal); Level 3: deep cracks (more than 30 cm in length or touching the metal).

[0058] Exemplarily, when the abnormality category is wrinkles in the damaging category, the corresponding abnormality grading standards can be as follows: Level 1: slight wrinkles (wrinkle height less than 5 mm, area less than 0.1 square meters); Level 2: moderate wrinkles (wrinkle height 5 mm to 15 mm, area 0.1 to 0.5 square meters); Level 3: severe wrinkles (wrinkle height exceeds 15 mm or area exceeds 0.5 square meters).

[0059] For example, when the abnormality category is perforation in the damaging category, the corresponding abnormality grading standards may be as follows: Level 1: small hole (less than 5 mm in diameter); Level 2: medium hole (5 to 15 mm in diameter); Level 3: large hole (more than 15 mm in diameter).

[0060] S103: Determine a warning strategy for the target detection vehicle according to the abnormality category and abnormality level, and issue an abnormality warning at a corresponding warning time according to the warning strategy.

[0061] Among them, the early warning strategy includes at least one of the following early warning methods: display early warning, voice early warning and text early warning. The high-level early warning strategy can correspond to the three early warning methods of display early warning, voice early warning and text early warning. The medium-level early warning strategy can correspond to the two early warning methods of text early warning and voice early warning. The low-level early warning strategy can correspond to one early warning method of text early warning. The early warning time refers to the time when the early warning is required in the future. The early warning strategy can correspond to the early warning time one by one. The early warning time represents the timeliness of the early warning, that is, whether an immediate early warning is required.

[0062] Optionally, the warning strategy for the target detection vehicle may be determined based on the correspondence between preset abnormality categories, abnormality levels and candidate warning strategies, combined with the abnormality category and abnormality level of the current target detection vehicle.

[0063] For example, if it is determined based on the abnormality category and abnormality level that the target detection vehicle has a large area of ​​hazardous stains such as shellac, the warning strategy can be determined as a medium-level warning strategy. If it is determined based on the abnormality category and abnormality level that the target detection vehicle has level 2 scratches, the warning strategy can be determined as a high-level warning strategy.

[0064] Optionally, according to the warning strategy, an abnormal warning is issued at a corresponding warning time, including: if the warning strategy is a display warning, determining the vehicle three-dimensional model corresponding to the panoramic image, and annotating the vehicle three-dimensional model according to the abnormal category and abnormal appearance characteristics to determine the annotated three-dimensional model; displaying the annotated three-dimensional model on the vehicle's display screen at the corresponding warning time for display warning.

[0065] Among them, the annotated three-dimensional model can represent the location of the abnormal area on the target detection vehicle, the abnormal category corresponding to the abnormal area, and the related information of the abnormal appearance characteristics.

[0066] The technical solution of the embodiment of the present invention is that if the target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detection vehicle; a preset deep learning model is used to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, the abnormal category and abnormal level corresponding to the abnormal appearance features are determined; according to the abnormal category and abnormal level, the early warning strategy for the target detection vehicle is determined, and according to the early warning strategy, an abnormal warning is issued at the corresponding warning time. By combining deep learning technology, abnormal identification and early warning of vehicle appearance can be automatically performed, improving the efficiency and safety of vehicle maintenance and enhancing user experience.

[0067] Embodiment 2

[0068] Figure 2 is a flow chart of a vehicle abnormality detection method provided in Embodiment 2 of the present invention; based on the above embodiments, this embodiment provides a preferred example of a cloud server interacting with a vehicle to detect abnormalities on the vehicle appearance, specifically, Figure 2 As shown, the method includes:

[0069] S201: If a target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detected vehicle.

[0070] S202: Using a preset deep learning model, extract features from the panoramic image to obtain appearance features of the target detection vehicle.

[0071] S203: Compare the appearance features with the historical annotation information to obtain a comparison result, and determine whether the appearance of the target detection vehicle is abnormal based on the comparison result.

[0072] S204: If yes, identify the abnormal appearance features corresponding to the target detection vehicle to determine the abnormal category corresponding to the abnormal appearance features.

[0073] S205: If the abnormal category is a stain in the non-damage category, determine the abnormal area of ​​the abnormal region according to the appearance feature to obtain abnormal parameters corresponding to the appearance feature.

[0074] S206: Determine the abnormality level corresponding to the abnormal appearance feature according to the relationship between the abnormal area and the stain assessment threshold in the preset assessment threshold.

[0075] S207: Determine a warning strategy for the target detection vehicle according to the abnormality category and abnormality level.

[0076] S208: If the warning strategy is to display a warning, determine the three-dimensional model of the vehicle corresponding to the panoramic image, and annotate the three-dimensional model of the vehicle according to the abnormal category and abnormal appearance characteristics to determine the annotated three-dimensional model.

[0077] S209: Displaying the annotated three-dimensional model on a display screen of the vehicle at a corresponding warning time to provide a warning display.

[0078] The technical solution of the present invention can realize the rapid and accurate detection of abnormal appearance of the car, and promptly remind the car owner, thereby improving the efficiency and safety of vehicle maintenance. The continuous learning ability of the deep learning model enables the system to be continuously optimized and adapted to different vehicles and abnormal situations.

[0079] Embodiment 3

[0080] Figure 3 : is a structural block diagram of a vehicle anomaly detection device provided in Embodiment 3 of the present invention; this embodiment can be applied to a situation where a cloud server of a vehicle recognizes an abnormality in the appearance of a vehicle based on a video of a target vehicle and issues an early warning; the vehicle anomaly detection device provided in the embodiment of the present invention can execute the vehicle anomaly detection method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method; the vehicle anomaly detection device can be implemented in the form of hardware and / or software, and configured in an electronic device with a vehicle anomaly detection function, such as a cloud server of a vehicle, such as Figure 3 As shown, the vehicle abnormality detection device specifically includes:

[0081] The obtaining module 301 is used for performing splicing processing on the target vehicle video if the target vehicle video is detected to obtain a panoramic image of the target detection vehicle;

[0082] The judgment module 302 is used to use a preset deep learning model to perform recognition processing on the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, determine the abnormal category and abnormal level corresponding to the abnormal appearance feature;

[0083] The warning module 303 is used to determine the warning strategy for the target detection vehicle according to the abnormality category and abnormality level, and to issue an abnormality warning at a corresponding warning time according to the warning strategy.

[0084] The technical solution of the embodiment of the present invention is that if the target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detection vehicle; a preset deep learning model is used to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, the abnormal category and abnormal level corresponding to the abnormal appearance features are determined; according to the abnormal category and abnormal level, the early warning strategy for the target detection vehicle is determined, and according to the early warning strategy, an abnormal warning is issued at the corresponding warning time. By combining deep learning technology, abnormal identification and early warning of vehicle appearance can be automatically performed, improving the efficiency and safety of vehicle maintenance and enhancing user experience.

[0085] Furthermore, the determination module 302 is specifically used for:

[0086] Using a preset deep learning model, feature extraction is performed on the panoramic image to obtain appearance features of the target detection vehicle; the appearance features include at least one of the following: shape features, color features, and texture features;

[0087] The appearance features are compared with the historical annotation information to obtain a comparison result, and based on the comparison result, it is determined whether there is any abnormality in the appearance of the target detection vehicle.

[0088] Furthermore, the determination module 302 further includes:

[0089] A category determination unit, used to identify abnormal appearance features corresponding to the target detection vehicle to determine the abnormal category corresponding to the abnormal appearance features;

[0090] The level determination unit is used to determine the abnormal level corresponding to the abnormal appearance feature according to the abnormal category and the correlation between the abnormal parameters corresponding to the appearance feature and the preset evaluation threshold.

[0091] Furthermore, the target vehicle video is obtained by the target user shooting the target detection vehicle from multiple angles, or by shooting with a camera pre-configured on the target detection vehicle; the abnormality categories include damage categories and non-damage categories; the non-damage categories include stains and / or flat tires; the damage categories include at least one of the following: dents, scratches, paint peeling, cracks, wrinkles and punctures.

[0092] Furthermore, the level determination unit is specifically used for:

[0093] If the abnormal category is a stain in the non-damage category, then the abnormal area of ​​the abnormal region is determined according to the appearance characteristics to obtain the abnormal parameters corresponding to the appearance characteristics;

[0094] The abnormal level corresponding to the abnormal appearance feature is determined according to the size relationship between the abnormal area and the stain evaluation threshold in the preset evaluation threshold.

[0095] Furthermore, the early warning module 303 is specifically used for:

[0096] If the warning strategy is to display the warning, determine the vehicle three-dimensional model corresponding to the panoramic image, and annotate the vehicle three-dimensional model according to the abnormal category and abnormal appearance characteristics to determine the annotated three-dimensional model;

[0097] At the corresponding warning time, the annotated three-dimensional model will be displayed on the display screen of the vehicle to provide a warning display.

[0098] Embodiment 4

[0099] Figure 4 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a vehicle abnormality detection method.

[0103] In some embodiments, the vehicle abnormality detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the vehicle abnormality detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the vehicle abnormality detection method in any other appropriate manner (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0110] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and the computer program implements the vehicle abnormality detection method of any embodiment of the present invention when executed by a processor.

[0111] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle abnormality detection method, characterized in that: include: If the target vehicle video is detected, the target vehicle video is spliced ​​to obtain a panoramic image of the target detection vehicle; Use a preset deep learning model to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal. If so, determine the abnormal category and abnormal level corresponding to the abnormal appearance features; According to the abnormality category and abnormality level, the warning strategy for the target detection vehicle is determined, and according to the warning strategy, the abnormality warning is issued at the corresponding warning time.

2. The method according to claim 1, characterized in that The preset deep learning model is used to identify and process the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, including: Using a preset deep learning model, feature extraction is performed on the panoramic image to obtain appearance features of the target detection vehicle; the appearance features include at least one of the following: shape features, color features, and texture features; The appearance features are compared with the historical annotation information to obtain a comparison result, and based on the comparison result, it is determined whether there is any abnormality in the appearance of the target detection vehicle.

3. The method according to claim 1, characterized in that Determine the abnormal category and abnormal level corresponding to the abnormal appearance features, including: Identify the abnormal appearance features corresponding to the target detection vehicle to determine the abnormal category corresponding to the abnormal appearance features; According to the abnormality category, combined with the correlation between the abnormal parameters corresponding to the appearance features and the preset evaluation threshold, the abnormality level corresponding to the abnormal appearance features is determined.

4. The method according to claim 1, characterized in that: in, The target vehicle video is obtained by the target user shooting the target detection vehicle from multiple angles, or by shooting with a camera pre-configured on the target detection vehicle; the abnormality categories include damage categories and non-damage categories; the non-damage category includes stains and / or flat tires; the damage category includes at least one of the following: dents, scratches, paint peeling, cracks, wrinkles and punctures.

5. The method according to claim 3, characterized in that: According to the abnormal category, combined with the correlation between the abnormal parameters corresponding to the appearance features and the preset evaluation threshold, the abnormal level corresponding to the abnormal appearance features is determined, including: If the abnormal category is a stain in the non-damage category, then the abnormal area of ​​the abnormal region is determined according to the appearance characteristics to obtain the abnormal parameters corresponding to the appearance characteristics; The abnormal level corresponding to the abnormal appearance feature is determined according to the size relationship between the abnormal area and the stain evaluation threshold in the preset evaluation threshold.

6. The method according to claim 1, characterized in that According to the early warning strategy, abnormal warnings are issued at the corresponding warning time, including: If the warning strategy is to display the warning, determine the vehicle three-dimensional model corresponding to the panoramic image, and annotate the vehicle three-dimensional model according to the abnormal category and abnormal appearance characteristics to determine the annotated three-dimensional model; At the corresponding warning time, the annotated three-dimensional model will be displayed on the display screen of the vehicle to provide a warning display.

7. A vehicle abnormality detection device, characterized in that: include: The obtaining module is used for, if a target vehicle video is detected, stitching the target vehicle video to obtain a panoramic image of the target detected vehicle; A judgment module is used to use a preset deep learning model to perform recognition processing on the panoramic image to determine whether the appearance of the target detection vehicle is abnormal, and if so, determine the abnormal category and abnormal level corresponding to the abnormal appearance features; The warning module is used to determine the warning strategy for the target detection vehicle according to the abnormal category and abnormal level, and to issue an abnormal warning at the corresponding warning time according to the warning strategy.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle abnormality detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the vehicle abnormality detection method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the vehicle abnormality detection method according to any one of claims 1 to 6.

Images