Video-based car change identification method and device, electronic equipment and storage medium

Abstract

The invention provides a video-based car change identification method and device, electronic equipment and a storage medium, and relates to the technical field of image processing and the field of financial science and technology. The method comprises the steps of obtaining a target vehicle surrounding video of a target vehicle in response to a target business request of a target object; performing shot division on the target vehicle surrounding video to obtain shot vehicle surrounding video segments; carrying out key frame extraction on each lens vehicle surrounding video segment to obtain a target vehicle surrounding key frame; vehicle area proportion detection is carried out on each target vehicle surrounding key frame, a target vehicle area proportion is obtained, and the target vehicle area proportion is the proportion of the area of the target vehicle in the target vehicle surrounding key frame; determining the number of abnormal frames in the target vehicle surrounding key frames according to the target vehicle area proportion; and according to the number of the abnormal frames, vehicle abnormity evaluation is carried out to obtain a vehicle risk category, and the vehicle risk category is used for representing that vehicle change exists or does not exist in the target business. According to the invention, the identification efficiency and the identification accuracy of car changing can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology and financial technology, and in particular to a video-based vehicle change recognition method and device, electronic equipment, and storage medium. Background Art

[0002] In the field of financial technology, car replacement risk refers to the risk to the service provider (such as an insurance company) when a user uses another vehicle instead of his own vehicle to perform a certain business (such as purchasing auto insurance, auto insurance claims, etc.). For example, in the insurance industry, there may be a situation where the insured vehicle when the user purchases insurance is inconsistent with the claim vehicle when applying for claims, that is, there is a risk that the user uses another vehicle to apply for auto insurance claims to obtain the claim premium. Therefore, car replacement identification is required.

[0003] In the existing car replacement identification technology, users usually need to shoot a video of the car and upload it to the server, and then the claims specialist will manually review the video to determine whether the user has changed the car. However, this method has at least the disadvantages of low efficiency and low accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a video-based vehicle change recognition method and device, electronic equipment, and storage medium, which can improve the recognition efficiency and recognition accuracy of vehicle changes.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a vehicle change recognition method based on video, the method comprising:

[0006] In response to a target service request of a target object, obtaining a target surround video of a target vehicle;

[0007] Divide the target surround-car video into shots to obtain the surround-car video segments under each shot;

[0008] Extracting key frames from each of the camera's surround video segments to obtain target surround key frames;

[0009] Performing vehicle area ratio detection on each of the target vehicle-surrounding key frames to obtain the target vehicle area ratio; wherein the target vehicle area ratio is the ratio of the area of ​​the target vehicle to the target vehicle-surrounding key frame;

[0010] According to the target vehicle area ratio, determine the number of abnormal frames in the target vehicle key frame;

[0011] A vehicle abnormality assessment is performed according to the number of abnormal frames to obtain a vehicle risk category; wherein the vehicle risk category is used to characterize whether a vehicle change exists or does not exist in a target business.

[0012] In some embodiments, performing vehicle abnormality assessment according to the number of abnormal frames to obtain a vehicle risk category includes:

[0013] Determine the number of key frames according to each of the target loop key frames;

[0014] Calculate the ratio of abnormal frames according to the number of abnormal frames and the number of key frames to obtain the abnormal frame ratio;

[0015] If the abnormal frame ratio is greater than a preset abnormal ratio threshold, generating the vehicle risk category for characterizing that a vehicle change exists in the target business;

[0016] If the abnormal frame ratio is less than or equal to the preset abnormal ratio threshold, the vehicle risk category for characterizing that there is no vehicle change in the target business is generated.

[0017] In some embodiments, the target vehicle surround video is obtained by the target object shooting the target vehicle according to the guidance of a preset vehicle surround shooting component;

[0018] After performing vehicle abnormality assessment according to the number of abnormal frames to obtain a vehicle risk category, the method further includes:

[0019] If the vehicle risk category indicates that there is no vehicle change in the target business, vehicle component detection is performed on each of the target loop key frames to obtain a vehicle component sequence;

[0020] Extract two adjacent vehicle components from the vehicle component sequence to obtain a first vehicle component and a second vehicle component; wherein the timing of the first vehicle component is before the timing of the second vehicle component;

[0021] Searching for the next component according to the first vehicle component for the vehicle-surrounding photographing component guide to obtain a reference vehicle component;

[0022] A component abnormality assessment is performed based on the second vehicle component and the reference vehicle component to obtain a component risk category; wherein the component risk category indicates whether a vehicle replacement exists for the target business or not.

[0023] In some embodiments, performing component anomaly assessment according to the second vehicle component and the reference vehicle component to obtain a component risk category includes:

[0024] If the second vehicle component is different from the reference vehicle component, generating the component risk category for characterizing that the target business has a vehicle replacement risk;

[0025] If the second vehicle component is the same as the reference vehicle component, the component risk category is generated to characterize that there is no vehicle replacement risk for the target business.

[0026] In some embodiments, after performing component anomaly assessment according to the second vehicle component and the reference vehicle component to obtain a component risk category, the method further includes:

[0027] If the component risk category indicates that the target business does not have a vehicle replacement risk, then obtaining a target object shooting trajectory; wherein the target object shooting trajectory is a trajectory formed by shooting the target vehicle;

[0028] A trajectory anomaly assessment is performed according to the captured trajectory of the target object to obtain a trajectory risk category; wherein the trajectory risk category is used to characterize whether a vehicle change exists or does not exist in the target business.

[0029] In some embodiments, performing trajectory anomaly assessment according to the target object shooting trajectory to obtain a trajectory risk category includes at least one of the following:

[0030] Perform walking direction detection on the captured trajectory of the target object to obtain walking direction detection data, and if the walking direction detection data indicates that two adjacent walking directions are opposite, generate the trajectory risk category for characterizing that the target business has a risk of changing vehicles;

[0031] Performing a trajectory deviation detection on the target object shooting trajectory and the preset expected trajectory to obtain trajectory deviation detection data, and if the trajectory deviation detection data is greater than a preset deviation threshold, generating the trajectory risk category for characterizing that the target business has a risk of vehicle switching;

[0032] The distance between the trajectory starting point of the target object shooting trajectory and the trajectory starting point of the expected trajectory is calculated to obtain the starting point distance. If the starting point distance is greater than a preset distance threshold, the trajectory risk category is generated to characterize the risk of vehicle change for the target business.

[0033] In some embodiments, if the component risk category indicates that the target business does not have a vehicle replacement risk, obtaining a target object shooting trajectory includes:

[0034] If the component risk category indicates that the target business does not have a vehicle replacement risk, obtaining posture information and loop detection information at different times during the shooting process; wherein the shooting process refers to the process of shooting the target vehicle;

[0035] A target object shooting trajectory is generated according to the posture information and the loop detection information at each moment.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a vehicle change recognition device based on video, the device comprising:

[0037] A video acquisition module, used for acquiring a target surround video of a target vehicle in response to a target service request of a target object;

[0038] A shot division module is used to divide the target surround video into shots to obtain the surround video segments under each shot;

[0039] A frame extraction module is used to extract key frames from each of the camera's surround video segments to obtain a target surround key frame;

[0040] A proportion detection module is used to perform vehicle area proportion detection on each of the target vehicle key frames to obtain the target vehicle area proportion; wherein the target vehicle area proportion is the proportion of the area of ​​the target vehicle occupying the target vehicle key frame;

[0041] An abnormality determination module, used to determine the number of abnormal frames in the target vehicle-surrounding key frames according to the area ratio of the target vehicle;

[0042] The vehicle anomaly assessment module is used to perform vehicle anomaly assessment according to the number of abnormal frames to obtain a vehicle risk category; wherein the vehicle risk category is used to characterize whether a vehicle change exists or does not exist in a target business.

[0043] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the video-based vehicle change recognition method described in the first aspect.

[0044] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the video-based vehicle change recognition method described in the first aspect.

[0045] The video-based vehicle change identification method and device, electronic device, and storage medium proposed in the present application first divide the target vehicle-surrounding video into shots and then extract key frames to obtain target vehicle-surrounding key frames, and each target vehicle-surrounding key frame can effectively summarize the main content of the video. Secondly, the target key frame is subjected to vehicle area ratio detection and then the number of abnormalities is determined to obtain the number of abnormal frames, which can represent the number of key frames with abnormalities in the target vehicle-surrounding video. Then, the vehicle abnormality assessment is performed based on the number of abnormal frames, and the vehicle risk category can be obtained. In this way, there is no need to process each frame in the target vehicle-surrounding video, which can improve processing efficiency and has high accuracy.

[0046] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a vehicle change identification method based on video provided in an embodiment of the present application;

[0048] Figure 2 yes Figure 1 Flow chart of step 106 in;

[0049] Figure 3 is a flow chart of a vehicle change identification method based on video provided by another embodiment of the present application;

[0050] Figure 4 is a flow chart of a vehicle change identification method based on video provided by another embodiment of the present application;

[0051] Figure 5 yes Figure 4 Flow chart of step 401 in;

[0052] Figure 6 It is a module structure block diagram of a video-based vehicle change identification device provided in an embodiment of the present application;

[0053] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] First, some nouns involved in this application are analyzed:

[0058] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information image processing, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0060] In the existing auto insurance claims process, users generally need to follow the instructions to shoot a video around the car and upload it to the server. Then the claims specialist will manually review the video to determine whether the user has the risk of changing the car. However, this method has the following major shortcomings: 1. Inefficiency: The manual review process is very time-consuming, especially when the number of claims applications is large, it may take hours or even days to complete the review. This not only affects the efficiency of claims, but may also cause users to wait too long. 2. Error-prone: During the manual review process, human factors may cause missed detection or misjudgment. For example, the claims specialist may ignore some important details in the video due to fatigue, lack of concentration, etc. 3. Unable to respond in real time: Since the manual review process cannot be carried out in real time, after the user uploads the video, it takes a while to get the review result. This delay may affect the user's claims experience. 4. Low accuracy: Although the existing automated risk identification technology can improve the efficiency of the review, its accuracy is usually not high and the probability of false detection is high. This may result in some cases that actually involve risk of vehicle replacement being misjudged as risk-free, or some risk-free cases being misjudged as involving risk.

[0061] Based on the above problems, the embodiments of the present application provide a method and device for identifying vehicle swaps based on video, an electronic device, and a storage medium. The core of the present application is to improve the efficiency of identification and the accuracy of risk identification while reducing the probability of false detection through vehicle anomaly assessment, component anomaly assessment, and trajectory anomaly assessment.

[0062] The video-based vehicle change identification method provided in the embodiment of the present application can be applied to the terminal and the server side, and can also be software running on the server side. The server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video-based vehicle change identification method, but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: server computers, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0064] The embodiments of the present application provide a video-based vehicle change identification method and a video-based vehicle change identification device, an electronic device, and a computer-readable storage medium, which are specifically explained through the following embodiments. First, the video-based vehicle change identification method in the embodiments of the present application is described.

[0065] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0066] Reference Figure 1 , Figure 1 It is an optional flowchart of the video-based vehicle change identification method provided in an embodiment of the present application, which may include but is not limited to steps 101 to 106.

[0067] Step 101, in response to a target service request of a target object, obtaining a target surround video of a target vehicle;

[0068] Step 102, dividing the target surround video into shots to obtain the surround video segments under each shot;

[0069] Step 103, extracting key frames from each shot's car-around video segment to obtain a target car-around key frame;

[0070] Step 104, performing vehicle area ratio detection on each target vehicle key frame to obtain the target vehicle area ratio; wherein the target vehicle area ratio is the ratio of the area of ​​the target vehicle to the target vehicle key frame;

[0071] Step 105, determining the number of abnormal frames in the target vehicle surrounding key frames according to the target vehicle area ratio;

[0072] Step 106, performing vehicle anomaly assessment according to the number of abnormal frames to obtain a vehicle risk category; wherein the vehicle risk category is used to characterize whether a vehicle change exists or does not exist in the target business.

[0073] Step 101 to step 106 shown in the embodiment of the present application, first the target car-surrounding video is subjected to shot division and then key frame extraction is performed to obtain the target car-surrounding key frame, and each target car-surrounding key frame can effectively summarize the main content of the video. Secondly, the target key frame is subjected to vehicle area proportion detection and then the abnormal number is determined to obtain the abnormal frame number, and the abnormal frame number can represent the number of abnormal key frames in the target car-surrounding video. Then, vehicle abnormality assessment is performed according to the abnormal frame number, and vehicle risk category can be obtained. In this way, each frame in the target car-surrounding video need not be processed, and processing efficiency can be improved, and accuracy is higher.

[0074] In step 101 of some embodiments, in response to a target service request of a target object, a target surround video of a target vehicle is obtained. A target service request refers to a request to use a target vehicle to perform a target service. For example, in the insurance industry, if a target object wishes to apply for a car insurance claim with vehicle C01, a target service request is generated on the client side with vehicle C01 as the target vehicle and the case number corresponding to the car insurance claim as the target service. In addition to the above-mentioned car insurance claim service, the target service may also include services such as purchasing car insurance and renewing car insurance.

[0075] The target vehicle-surrounding video is a video obtained by shooting the target vehicle. For example, the user captures the target vehicle through a mobile phone to obtain the target vehicle-surrounding video. When the above-mentioned video-based vehicle-changing recognition method is applied to the server side, the server side receives the target vehicle-surrounding video from the terminal (such as a mobile phone).

[0076] In one embodiment, the video-based vehicle exchange identification method is applied to the server side, and step 101 includes: responding to the target service request of the target object, parsing the case number from the target service request; searching the identification database according to the case number to obtain the vehicle identification of the target vehicle; and obtaining the target vehicle-swap video corresponding to the vehicle identification from the vehicle database. In this way, the server can quickly locate the target vehicle-swap video in the vehicle database storing data of multiple vehicles.

[0077] In step 102 of some embodiments, the target surround video is divided into shots to obtain the surround video segments under each shot. Different shots are different at least in shooting angle and posture. The surround video segment is a video segment belonging to a certain shot in the target surround video.

[0078] In one embodiment, step 102 may include: performing frame detection on two adjacent frames of the target surround video based on histogram difference to obtain adjacent frame difference data; if the adjacent frame difference data is greater than a preset difference threshold, determining the lens edge according to the two adjacent frames; and segmenting the target surround video according to each lens edge to obtain lens surround video segments under each lens. In this way, the segmentation method is not only simple but also flexible.

[0079] In step 103 of some embodiments, key frames are extracted from each shot of the car-around video to obtain a target car-around key frame. The target car-around key frame refers to a frame that can summarize the important content of the target car-around video.

[0080] In one embodiment, step 103 may include: extracting the first frame, the middle frame and the last frame of each pan-around video segment to obtain the target key frame. In this way, the extraction method is easy to design and the calculation is simple, which is particularly suitable for the case where the video content is simple or the scene is fixed.

[0081] In step 104 of some embodiments, a vehicle area ratio detection is performed on each target vehicle key frame to obtain the target vehicle area ratio. The target vehicle area ratio is the ratio of the area of ​​the target vehicle to the target vehicle key frame. If the target vehicle area ratio is smaller, it means that the target vehicle photographed is smaller, and the possibility of changing vehicles is higher.

[0082] In one embodiment, step 104 may include: 1. Image preprocessing, including: Image cleaning: removing noise, adjusting brightness and contrast to improve the effect of subsequent processing. Scaling: If the image size is large, it can be scaled to a suitable size to increase the processing speed. 2. Target detection, including: using a target detection algorithm to identify vehicles in the image, and commonly used methods include: Classical methods: such as Haar feature cascade classifier, HOG (Histogram of Oriented Gradients), etc. Deep learning: using convolutional neural networks (CNN) and ready-made models such as YOLO (You Only Look Once), FasterR-CNN, SSD (Single Shot Multibox Detector), etc. to detect vehicles. 3. Calculating area share, including: multiplying the width of the target vehicle key frame by the height to obtain the total area of ​​the target vehicle key frame; determining the vehicle occupied area based on the detected vehicle bounding box or segmentation mask; dividing the vehicle occupied area by the total area to obtain the target vehicle area share.

[0083] In one embodiment, after the above-mentioned target detection, for scenarios requiring higher accuracy, an instance segmentation algorithm (such as Mask R-CNN) can be used to segment out the pixels of the target vehicle to further improve the accuracy of determining the area occupied by the vehicle.

[0084] In step 105 of some embodiments, the number of abnormal frames in the target vehicle surrounding key frames is determined according to the target vehicle area ratio.

[0085] In one embodiment, if the target vehicle area ratio is less than the ratio threshold, the target key frame is determined as an abnormal frame; the number of abnormal frames is determined based on each abnormal frame. The ratio threshold can be set according to demand, for example, the average value is calculated based on the area ratio of each target vehicle to obtain the average ratio; the average ratio is determined as the ratio threshold.

[0086] In step 106 of some embodiments, vehicle anomaly assessment is performed based on the number of abnormal frames to obtain a vehicle risk category. The vehicle risk category is used to characterize whether a target business has a vehicle change or not.

[0087] In one embodiment, referring to Figure 2 , step 106 may include:

[0088] Step 201, determining the number of key frames according to each target loop key frame;

[0089] Step 202, calculating the ratio of abnormal frames based on the number of abnormal frames and the number of key frames to obtain the abnormal frame ratio;

[0090] Step 203: if the abnormal frame ratio is greater than a preset abnormal ratio threshold, a vehicle risk category is generated to characterize the existence of vehicle replacement in the target business;

[0091] Step 204: If the abnormal frame ratio is less than or equal to a preset abnormal ratio threshold, a vehicle risk category is generated to characterize that there is no vehicle change in the target business.

[0092] The advantages of the above-mentioned embodiment of step 201 to step 204 are that the determination logic is simple, and the abnormal ratio threshold can be adjusted according to demand, which is highly flexible.

[0093] The above steps 202 to 204 are performed when the number of key frames is greater than the key number threshold. In one embodiment, after step 201, step 106 may include: if the number of key frames is less than or equal to the key number, comparing the number of abnormal frames with the abnormal number threshold; if the number of abnormal frames is greater than the abnormal number threshold, generating a vehicle risk category for characterizing that the target business has a vehicle change; if the number of abnormal frames is less than or equal to the abnormal number threshold, generating a vehicle risk category for characterizing that the target business does not have a vehicle change. In this way, another way to generate vehicle risk categories is provided, which is highly diverse.

[0094] In one embodiment, after step 106, refer to Figure 3 The video-based vehicle change recognition method provided in this embodiment may further include:

[0095] Step 301: If the vehicle risk category indicates that the target business does not have a vehicle change, vehicle component detection is performed on each target vehicle key frame to obtain a vehicle component sequence;

[0096] Step 302, extracting two adjacent vehicle parts from the vehicle part sequence to obtain a first vehicle part and a second vehicle part; wherein the time of the first vehicle part is before the time of the second vehicle part;

[0097] Step 303, searching for the next component according to the first vehicle component for the vehicle-surrounding photographed component guide, and obtaining a reference vehicle component;

[0098] Step 304 , performing component anomaly assessment based on the second vehicle component and the reference vehicle component to obtain a component risk category; wherein the component risk category indicates whether a vehicle replacement exists for the target business or not.

[0099] In step 301, vehicle parts refer to the various components that constitute a vehicle. The vehicle parts include: front bumper, left front fender, left front door, left rear door, left rear fender, rear bumper, right rear fender, right rear door, right front door, right front fender. A vehicle part sequence refers to a sequence containing at least one vehicle part. Any two vehicle parts in the vehicle part sequence are not the same. For example, there will not be two left front doors in the vehicle part sequence.

[0100] In one embodiment, the above-mentioned step of detecting vehicle parts for each target car-loop key frame to obtain a vehicle parts sequence may include: 1. Data preparation, including: collecting image data sets containing various vehicles and their parts to ensure the diversity of the data set; using annotation tools (such as LabelImg, VGG Image Annotator, etc.) to annotate the vehicle parts in the image and generate a label file (such as Pascal VOC, COCO format). 2. Select a model and select a suitable target detection or instance segmentation model. Commonly used deep learning models include: YOLO (You Only Look Once), Faster R-CNN, Mask R-CNN. 3. Model training, including: data preprocessing: converting the image into input format and normalizing it; dividing the data set: dividing the data set into a training set, a validation set, and a test set; training the model: using a deep learning framework (such as TensorFlow, PyTorch) to train the model. Pre-trained models can be used for transfer learning to accelerate the training process; validation model: evaluating model performance on the validation set, monitoring loss values ​​and accuracy, and adjusting hyperparameters. 4. Model testing, including: using the test set to evaluate the performance of the model to ensure that it has good vehicle part detection accuracy; calculating common evaluation indicators of the model, such as mean average precision (mAP), recall rate, etc. 5. Image detection, including: inputting the target surrounding car keyframe into the trained model, and the model outputs the category, location (bounding box) and confidence of each detected vehicle part.

[0101] In step 302, the vehicle component sequence includes at least two vehicle components arranged in time order. Two adjacent vehicle components are extracted from the vehicle component sequence to obtain a first vehicle component and a second vehicle component, and the time of the first vehicle component is before the time of the second vehicle component. For example, the first vehicle component is the left front door and the time is t1, and the second vehicle component is the left rear door and the time is t2, and t1 is before t2.

[0102] In step 303, the car-surrounding shooting guide refers to the operating instructions for guiding the target object to shoot the target vehicle. The target car-surrounding video is obtained by the target object shooting the target vehicle according to the preset car-surrounding shooting component guide. For example, the car-surrounding shooting guide is "Please shoot the target vehicle in the following order: front bumper, left front fender, left front door, left rear door, left rear fender, rear bumper, right rear fender, right rear door, right front door, right front fender". For example, the first vehicle component is the front bumper, and after searching the above-mentioned car-surrounding shooting guide, the reference vehicle component is the left front door. For example, the first vehicle component is the right rear door, and after searching the above-mentioned car-surrounding shooting guide, the reference vehicle component is the right front door.

[0103] In step 304, component abnormality assessment is performed based on the second vehicle component and the reference vehicle component to obtain a component risk category. The component risk category indicates whether the target business has a vehicle replacement or not.

[0104] In one embodiment, step 304 may include: if the second vehicle component is different from the reference vehicle component, generating a component risk category for characterizing that the target business has a risk of vehicle replacement; if the second vehicle component is the same as the reference vehicle component, generating a component risk category for characterizing that the target business does not have a risk of vehicle replacement. In this way, whether the target object is actually photographed in accordance with the vehicle photography guide can be measured based on whether the components are the same, thereby improving the accuracy of vehicle replacement recognition.

[0105] In one embodiment, after step 304, refer to Figure 4 The video-based vehicle change recognition method provided in this embodiment may further include:

[0106] Step 401, if the component risk category indicates that the target business does not have a vehicle replacement risk, then obtain a target object shooting trajectory;

[0107] Step 402 , performing trajectory anomaly assessment based on the captured trajectory of the target object to obtain a trajectory risk category; wherein the trajectory risk category is used to characterize whether a vehicle change exists in the target business or not.

[0108] In step 401, the target object shooting trajectory is a trajectory formed by shooting the target vehicle.

[0109] In one embodiment, referring to Figure 5 , step 401 may include:

[0110] Step 501, if the component risk category indicates that the target business does not have a vehicle replacement risk, obtain the posture information and loop detection information at different times during the shooting process; wherein the shooting process refers to the process of shooting the target vehicle;

[0111] Step 502: Generate a target object shooting trajectory based on the posture information and loop detection information at each moment.

[0112] Specifically, when generating the target object shooting trajectory, SLAM (simultaneous localization and mapping) technology can be introduced. A complete SLAM technology generally includes sensor data, front-end, back-end, loop detection and mapping. Among them, the front-end abstracts the sensor data into a model suitable for estimation, and loop detection determines whether the user passes through a known location. The back-end accepts the posture and loop detection information measured by the front-end at different times and optimizes them to obtain a globally consistent trajectory.

[0113] The benefit of the embodiment of the above steps 501 to 502 is that the accuracy of trajectory acquisition is improved.

[0114] In one embodiment, the acquisition process of the shooting trajectory of the above-mentioned target object may include: 1. Prepare equipment, including: GPS device or smart phone: make sure your device has GPS function and can record location information; camera: can be any camera device, mobile phone camera, digital SLR camera, etc. 2. Record the location during shooting, including: Use GPS application: install an application that can record GPS trajectory (such as Google Maps, Strava, GPX Logger, etc.) on the smart phone, start the application to record the trajectory before shooting, and the application will continue to record your location information; camera records location information: if you are using a GPS-supported camera, turn on the GPS function of the camera, and the camera will automatically record the location information to the EXIF ​​data of each photo when shooting. 3. Take photos: perform shooting activities while recording the location data, and ensure that the shooting time of each photo is synchronized with the time recorded by GPS. 4. Get and export track data, including: Export data from GPS applications: The recorded track data (such as GPX, KML, etc.) can be exported to your computer or cloud storage; Extract location information from the camera's EXIF ​​data: Use EXIF ​​data extraction tools (such as ExifTool, PhotoME, etc.) to view and extract photo metadata containing latitude and longitude information. 5. Visualize tracks, including: Import the exported track data into map applications (such as Google Earth, GPSVisualizer, etc.) to view and analyze the shooting track; If necessary, you can use programming libraries (such as Python's Matplotlib, Folium, etc.) to visualize the track data as a map.

[0115] In one embodiment, step 402 may include:

[0116] Perform walking direction detection on the target object's shooting trajectory to obtain walking direction detection data;

[0117] If the walking direction detection data indicates that two adjacent walking directions are opposite, a trajectory risk category is generated to characterize that there is a risk of vehicle change for the target business.

[0118] The benefit of this embodiment is that, by detecting the walking direction, it is possible to determine whether the target object is walking in the opposite direction during the shooting process, thereby generating a trajectory risk category, and the generation method is simple and highly applicable.

[0119] In one embodiment, step 402 may further include:

[0120] Performing trajectory deviation detection on the target object shooting trajectory and the preset expected trajectory to obtain trajectory deviation detection data;

[0121] If the trajectory deviation detection data is greater than a preset deviation threshold, a trajectory risk category is generated to characterize the risk of vehicle change for the target business.

[0122] The benefit of this embodiment is that trajectory detection can be used to determine whether the target object has a trajectory deviation that is too large during the shooting process, thereby generating a trajectory risk category, and the generation method is simple and highly applicable.

[0123] In one embodiment, step 402 may further include:

[0124] Calculate the distance between the starting point of the trajectory of the target object and the starting point of the trajectory of the expected trajectory to obtain the starting point distance;

[0125] If the distance between the starting points is greater than a preset distance threshold, a trajectory risk category is generated to characterize the risk of vehicle change for the target business.

[0126] The benefit of this embodiment is that, by distance calculation, it is possible to determine whether the target object has a starting point that is too far away during the shooting process, thereby generating a trajectory risk category, and the generation method is simple and highly applicable.

[0127] In one embodiment, step 402 may also include: if the walking direction detection data indicates that two adjacent walking directions are not opposite, if the trajectory offset detection data is less than or equal to a preset offset threshold, and if the starting point distance is less than or equal to a preset distance threshold, then a trajectory risk category is generated to characterize that there is no risk of vehicle change for the target business.

[0128] In summary of the above embodiments, the present application has at least the following beneficial effects: 1. Improving the ability of automatic vehicle change identification: Through vehicle anomaly assessment, vehicle component anomaly assessment and trajectory anomaly assessment, the method can automatically identify whether there is a vehicle change, greatly improving the efficiency of vehicle change identification. 2. Improving the recognition accuracy of vehicle change: By performing a detailed analysis of each key frame in the video, it is possible to more accurately identify whether there is a vehicle change, thereby improving the accuracy of recognition. 3. Reducing the probability of false detection: By using SLAM technology to generate trajectories, it is possible to accurately determine whether the user has changed the vehicle, thereby reducing the probability of false detection. 4. Improving the efficiency of business request response and customer satisfaction: Since it is possible to identify whether there is a vehicle change in real time, it is possible to greatly improve the efficiency of responding to business requests, shorten the user's waiting time, and thus improve customer satisfaction. 5. Promoting technological progress in the industry: The implementation of this method will promote technological progress in related industries (such as the auto insurance industry) and lay the foundation for future development. This will help enhance the competitiveness of the entire industry, thereby bringing greater economic benefits.

[0129] It should be noted that the video-based car replacement identification method provided in this application is mainly applicable to the scenario of auto insurance claims, especially when dealing with claims involving car replacement risks. By analyzing the target car-surrounding video uploaded by the user, the car replacement risk can be automatically identified, thereby improving the efficiency and accuracy of claims. In addition, the method can also be applied to other scenarios that require car replacement identification, such as vehicle sales, vehicle leasing, etc. As long as the user can provide a car-surrounding video, the method can be used to identify potential risks. In general, the scope of application of this method is very wide, and it can be widely used in the auto insurance industry and other related fields.

[0130] See also Figure 6 The embodiment of the present application also provides a video-based vehicle change recognition device, which can implement the above-mentioned video-based vehicle change recognition method. Figure 6 The module structure block diagram of the video-based vehicle change identification device provided in an embodiment of the present application includes: a video acquisition module 601, a shot division module 602, a frame extraction module 603, a proportion detection module 604, an anomaly determination module 605 and a vehicle anomaly assessment module 606. Among them, the video acquisition module 601 is used to respond to the target business request of the target object and obtain the target surround video of the target vehicle; the shot division module 602 is used to perform shot division on the target surround video to obtain the lens surround video segments under each shot; the frame extraction module 603 is used to extract the key frame of each lens surround video segment to obtain the target surround key frame; the proportion detection module 604 is used to perform vehicle area proportion detection on each target surround key frame to obtain the target vehicle area proportion; wherein the target vehicle area proportion is the proportion of the area of ​​the target vehicle in the target surround key frame; the abnormality determination module 605 is used to determine the number of abnormal frames in the target surround key frame according to the target vehicle area proportion; the vehicle abnormality assessment module 606 is used to perform vehicle abnormality assessment according to the number of abnormal frames to obtain the vehicle risk category; wherein the vehicle risk category is used to characterize whether there is a vehicle change or not in the target business.

[0131] In one embodiment, after performing a vehicle abnormality assessment based on the number of abnormal frames to obtain a vehicle risk category, the video-based vehicle change identification device also includes: a component abnormality assessment module, which is used to: if the vehicle risk category indicates that there is no vehicle change in the target business, then perform vehicle component detection on each target vehicle-loop key frame to obtain a vehicle component sequence; extract two adjacent vehicle components from the vehicle component sequence to obtain a first vehicle component and a second vehicle component; wherein the moment of the first vehicle component is before the moment of the second vehicle component; perform a next component search based on the first vehicle component to guide the vehicle-loop photographed component to obtain a reference vehicle component; perform a component abnormality assessment based on the second vehicle component and the reference vehicle component to obtain a component risk category; wherein the component risk category indicates whether there is a vehicle change or not in the target business.

[0132] In one embodiment, after performing a component abnormality assessment based on the second vehicle component and the reference vehicle component to obtain a component risk category, the video-based vehicle change identification device also includes: a trajectory abnormality assessment module, which is used to: if the component risk category characterizes that there is no vehicle change risk for the target business, obtain a target object shooting trajectory; wherein the target object shooting trajectory is a trajectory formed by shooting the target vehicle; perform a trajectory abnormality assessment based on the target object shooting trajectory to obtain a trajectory risk category; wherein the trajectory risk category is used to characterize the presence or absence of a vehicle change for the target business.

[0133] It should be noted that the specific implementation of the video-based vehicle change identification device is basically the same as the specific implementation of the video-based vehicle change identification method described above, and will not be repeated here.

[0134] The embodiment of the present application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and when the program is executed by the processor, the above-mentioned video-based vehicle change identification method is realized. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0135] See also Figure 7 , Figure 7 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0136] The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0137] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 calls and executes the video-based vehicle change recognition method of the embodiment of this application;

[0138] Input / output interface 703, used to implement information input and output;

[0139] Communication interface 704, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0140] A bus 705 that transmits information between the various components of the device (e.g., the processor 701, the memory 702, the input / output interface 703, and the communication interface 704);

[0141] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0142] The embodiment of the present application also provides a computer-readable storage medium for computer-readable storage, the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned video-based vehicle change recognition method.

[0143] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The video-based vehicle change identification method and device, electronic device, and storage medium proposed in the present application first divide the target vehicle-surrounding video into shots and then extract key frames to obtain target vehicle-surrounding key frames, and each target vehicle-surrounding key frame can effectively summarize the main content of the video. Secondly, the target key frame is subjected to vehicle area ratio detection and then the number of abnormalities is determined to obtain the number of abnormal frames, which can represent the number of key frames with abnormalities in the target vehicle-surrounding video. Then, the vehicle abnormality assessment is performed based on the number of abnormal frames, and the vehicle risk category can be obtained. In this way, there is no need to process each frame in the target vehicle-surrounding video, which can improve processing efficiency and has high accuracy.

[0145] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0146] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0147] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0149] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0150] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0155] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A video-based vehicle change recognition method, characterized in that: The method comprises: In response to a target service request of a target object, obtaining a target surround video of a target vehicle; Divide the target surround-car video into shots to obtain the surround-car video segments under each shot; Extracting key frames from each of the camera's surround video segments to obtain target surround key frames; Performing vehicle area ratio detection on each of the target vehicle-surrounding key frames to obtain the target vehicle area ratio; wherein the target vehicle area ratio is the ratio of the area of ​​the target vehicle to the target vehicle-surrounding key frame; According to the target vehicle area ratio, determine the number of abnormal frames in the target vehicle key frame; A vehicle abnormality assessment is performed according to the number of abnormal frames to obtain a vehicle risk category; wherein the vehicle risk category is used to characterize whether a vehicle change exists or does not exist in a target business.

2. The method according to claim 1, characterized in that The vehicle abnormality assessment is performed according to the number of abnormal frames to obtain a vehicle risk category, including: Determine the number of key frames according to each of the target loop key frames; Calculate the ratio of abnormal frames according to the number of abnormal frames and the number of key frames to obtain the abnormal frame ratio; If the abnormal frame ratio is greater than a preset abnormal ratio threshold, generating the vehicle risk category for characterizing that a vehicle change exists in the target business; If the abnormal frame ratio is less than or equal to the preset abnormal ratio threshold, the vehicle risk category for characterizing that there is no vehicle change in the target business is generated.

3. The method according to claim 1, characterized in that The target vehicle surround video is obtained by the target object shooting the target vehicle according to the guidance of the preset vehicle surround shooting component; After performing vehicle abnormality assessment according to the number of abnormal frames to obtain a vehicle risk category, the method further includes: If the vehicle risk category indicates that there is no vehicle change in the target business, vehicle component detection is performed on each of the target loop key frames to obtain a vehicle component sequence; Extract two adjacent vehicle components from the vehicle component sequence to obtain a first vehicle component and a second vehicle component; wherein the timing of the first vehicle component is before the timing of the second vehicle component; Searching for the next component according to the first vehicle component for the vehicle-surrounding photographing component guide to obtain a reference vehicle component; A component abnormality assessment is performed based on the second vehicle component and the reference vehicle component to obtain a component risk category; wherein the component risk category indicates whether a vehicle replacement exists for the target business or not.

4. The method according to claim 3, characterized in that: The performing component abnormality assessment according to the second vehicle component and the reference vehicle component to obtain a component risk category includes: If the second vehicle component is different from the reference vehicle component, generating the component risk category for characterizing that the target business has a vehicle replacement risk; If the second vehicle component is the same as the reference vehicle component, the component risk category is generated to characterize that there is no vehicle replacement risk for the target business.

5. The method according to claim 3 or 4, characterized in that: After performing component anomaly assessment according to the second vehicle component and the reference vehicle component to obtain a component risk category, the method further includes: If the component risk category indicates that the target business does not have a vehicle replacement risk, then obtaining a target object shooting trajectory; wherein the target object shooting trajectory is a trajectory formed by shooting the target vehicle; A trajectory anomaly assessment is performed according to the captured trajectory of the target object to obtain a trajectory risk category; wherein the trajectory risk category is used to characterize whether a vehicle change exists or does not exist in the target business.

6. The method according to claim 5, characterized in that The performing trajectory anomaly assessment according to the target object shooting trajectory to obtain a trajectory risk category includes at least one of the following: Perform walking direction detection on the captured trajectory of the target object to obtain walking direction detection data, and if the walking direction detection data indicates that two adjacent walking directions are opposite, generate the trajectory risk category for characterizing that the target business has a risk of changing vehicles; Performing a trajectory deviation detection on the target object shooting trajectory and the preset expected trajectory to obtain trajectory deviation detection data, and if the trajectory deviation detection data is greater than a preset deviation threshold, generating the trajectory risk category for characterizing that the target business has a risk of vehicle switching; The distance between the trajectory starting point of the target object shooting trajectory and the trajectory starting point of the expected trajectory is calculated to obtain the starting point distance. If the starting point distance is greater than a preset distance threshold, the trajectory risk category is generated to characterize the risk of vehicle change for the target business.

7. The method according to claim 5, characterized in that If the component risk category indicates that the target business does not have a vehicle replacement risk, obtaining a target object shooting trajectory includes: If the component risk category indicates that the target business does not have a vehicle replacement risk, obtaining posture information and loop detection information at different times during the shooting process; wherein the shooting process refers to the process of shooting the target vehicle; A target object shooting trajectory is generated according to the posture information and the loop detection information at each moment.

8. A video-based vehicle change recognition device, characterized in that: The device comprises: A video acquisition module, used for acquiring a target surround video of a target vehicle in response to a target service request of a target object; A shot division module is used to divide the target surround video into shots to obtain the surround video segments under each shot; A frame extraction module is used to extract key frames from each of the camera's surround video segments to obtain a target surround key frame; A proportion detection module is used to perform vehicle area proportion detection on each of the target vehicle key frames to obtain the target vehicle area proportion; wherein the target vehicle area proportion is the proportion of the area of ​​the target vehicle occupying the target vehicle key frame; An abnormality determination module, used to determine the number of abnormal frames in the target vehicle-surrounding key frames according to the area ratio of the target vehicle; The vehicle anomaly assessment module is used to perform vehicle anomaly assessment according to the number of abnormal frames to obtain a vehicle risk category; wherein the vehicle risk category is used to characterize whether a vehicle change exists or does not exist in a target business.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images