Vehicle damaged part image acquisition method and device, mobile terminal and storage medium

Abstract

The embodiment of the invention provides a vehicle damaged part image acquisition method and device, a mobile terminal and a storage medium, and belongs to the technical field of financial science and technology and artificial intelligence. The method comprises the steps of loading a vehicle segmentation model based on a model operation tool library; obtaining a target tensor of the video frame data of the vehicle; damage segmentation processing is carried out on the target tensor through a vehicle segmentation model to obtain a damage segmentation image, and the damage segmentation image comprises at least two parts of the vehicle; according to the contour area of each component, screening out damaged components of the vehicle from the at least two components; and shooting the damaged part based on the contour area and the pixels of the contour rectangle of the damaged part and the total area and the total pixels of the damaged segmented image to obtain a damaged part image of the vehicle. According to the embodiment of the invention, the high-quality vehicle damaged part image meeting the damage assessment requirement can be obtained, and the obtaining reliability, efficiency and convenience of the vehicle damaged part image are improved.

Description

Technical Field

[0001] The present application relates to the fields of financial technology and artificial intelligence technology, and in particular to a method, device, mobile terminal and storage medium for acquiring images of damaged vehicle parts. Background Art

[0002] In the vehicle damage assessment scenario in the field of financial technology, if the insured vehicle is involved in a collision, scratch or other accident, the insurance agency's damage assessor needs to conduct an on-site inspection and upload the accident image of the insured vehicle to assess the damage to the insured vehicle. This requires that the accident image of the insured vehicle can clearly reflect the damage to the damaged vehicle.

[0003] However, obtaining accident images that meet the requirements for damage assessment usually relies on the professionalism and experience of the damage assessor, with low reliability. In addition, it is often necessary to manually screen accident images that meet the damage assessment requirements from multiple accident images taken and uploaded by the damage assessor, which is time-consuming and labor-intensive, resulting in low acquisition efficiency and convenience. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a method, device, mobile terminal and storage medium for acquiring images of damaged vehicle parts, aiming to quickly and automatically acquire high-quality images of damaged vehicle parts that can meet the requirements of damage assessment, and to improve the reliability, efficiency and convenience of acquiring images of damaged vehicle parts.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a method for acquiring an image of a damaged vehicle component, the method comprising:

[0006] Based on the model running tool library, load the pre-trained vehicle segmentation model;

[0007] Get the target tensor of the vehicle's video frame data;

[0008] Performing damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same;

[0009] Obtaining the outline area of ​​each of the components;

[0010] Screening out damaged parts of the vehicle from at least two of the parts according to the contour area;

[0011] Obtaining a contour rectangle of the damaged component;

[0012] Based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmented image, the damaged component is photographed to obtain a damaged component image of the vehicle.

[0013] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for acquiring an image of a damaged vehicle component, the device comprising:

[0014] Model loading module, used to load pre-trained vehicle segmentation models based on the model running tool library;

[0015] A tensor acquisition module, used to acquire a target tensor of the video frame data of the vehicle;

[0016] A damage segmentation module, configured to perform damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same;

[0017] An area acquisition module, used to acquire the contour area of ​​each of the components;

[0018] A component screening module, used for screening out damaged components of the vehicle from at least two of the components according to the contour area;

[0019] A rectangle acquisition module, used to acquire the outline rectangle of the damaged component;

[0020] The image shooting module is used to shoot the damaged part based on the contour area of ​​the damaged part and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image, to obtain the damaged part image of the vehicle.

[0021] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a mobile terminal, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0022] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0023] The present application proposes a method, device, mobile terminal and storage medium for acquiring images of damaged vehicle parts. The method for acquiring images of damaged vehicle parts loads a pre-trained vehicle segmentation model based on a model operation tool library; acquires a target tensor of video frame data of the vehicle; performs damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image contains at least two parts of the vehicle, and any two parts are not the same; acquires the contour area of ​​each part; selects the damaged parts of the vehicle from at least two parts based on the contour area; acquires the contour rectangle of the damaged part; and photographs the damaged part based on the contour area of ​​the damaged part and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image to obtain an image of the damaged part of the vehicle. In this way, based on the model operation tool library, the vehicle segmentation model can be used to quickly and accurately identify the vehicle's damaged segmentation image and at least two components contained in the damaged segmentation image, so as to identify the damaged component from at least two components through the contour area of ​​each component, and then based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image, the damaged component is automatically photographed to obtain a high-quality vehicle damaged component image that can meet the damage assessment requirements, which simplifies user operations, can improve the reliability, efficiency and convenience of obtaining vehicle damaged component images, and provide an accurate basis for subsequent vehicle damage assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a method for acquiring images of damaged vehicle parts provided in an embodiment of the present application;

[0025] Figure 2 yes Figure 1 A detailed flowchart of step S101 in FIG.

[0026] Figure 3 This is an example diagram of encrypting and decrypting a pre-trained vehicle segmentation model using an AES encryption algorithm provided in an embodiment of the present application;

[0027] Figure 4 yes Figure 3 A detailed flowchart of step S107 in FIG.

[0028] Figure 5 is a schematic block diagram of a vehicle damaged parts image acquisition device provided in an embodiment of the present application;

[0029] Figure 6 It is a schematic block diagram of the structure of the mobile terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0033] First, some nouns involved in this application are analyzed:

[0034] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0035] Model running tool library: a tool library with the function of running machine learning models, such as tensorflow.js. tensorflow.js is an open source JavaScript library that supports GPU hardware acceleration. You only need to import the tensorflow.js library file to run it in the browser and mini-program environment without installing any software or drivers. It allows machine learning models to be imported into the browser and mini-program environment, so that machine learning models can be predicted directly in the browser and mini-program environment without sending data to the server. This local computing method makes the prediction process faster and is particularly suitable for real-time image recognition application scenarios, and is particularly suitable for processing sensitive data, ensuring privacy protection.

[0036] ArrayBuffer: It is an object used in JavaScript to represent a general, fixed-length raw binary data buffer. It is a byte array itself, but unlike ordinary JavaScript arrays, ArrayBuffer cannot be read and written directly, but needs to be operated through views (such as TypedArray or DataView).

[0037] tf.loadGraphModel function: It is a function in the model execution tool library that is used to load machine learning models that have been converted to the model execution tool library graph model format.

[0038] findcontourByJS function: It is obtained by modifying the contour detection (findContours) function in OpenCV. It can implement functions similar to findContours in OpenCV in the JavaScript environment.

[0039] Based on this, the embodiments of the present application provide a method, device, mobile terminal and storage medium for acquiring images of damaged vehicle parts, aiming to quickly and automatically acquire high-quality images of damaged vehicle parts that can meet the requirements of damage assessment, and improve the reliability, efficiency and convenience of acquiring images of damaged vehicle parts.

[0040] The vehicle damaged parts image acquisition method, device, mobile terminal and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the vehicle damaged parts image acquisition method in the embodiments of the present application is described.

[0041] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0042] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0043] The vehicle damaged parts image acquisition method provided in the embodiment of the present application relates to the fields of financial technology and artificial intelligence technology. The vehicle damaged parts image acquisition method provided in the embodiment of the present application can be applied to a mobile terminal, or it can be software running in a terminal or server. The mobile terminal can be a smart phone, tablet computer, PDA, etc.; the software can be a small program end that implements the vehicle damaged parts image acquisition method, but is not limited to the above forms.

[0044] Figure 1 is an optional flowchart of the method for acquiring images of damaged vehicle parts provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0045] Step S101, based on the model running tool library, load the pre-trained vehicle segmentation model.

[0046] Taking the vehicle damage assessment scenario as an example, when the damage assessor and other users arrive at the scene of the vehicle accident, they can trigger a request to obtain images of damaged vehicle parts on the mini program end of the mobile terminal, and hold the mobile terminal and keep the camera of the mobile terminal pointed at the vehicle.

[0047] In response to the request, the mini program side loads the pre-trained vehicle segmentation model based on the model running tool library in the mini program side environment.

[0048] The pre-trained vehicle segmentation model is used to predict the damage segmentation image of the vehicle and the components corresponding to the damaged area contained in the damage segmentation image.

[0049] Considering the limited computing resources of the mini program, in order to reduce the consumption of computing resources, the pre-trained lightweight MobileNet series can be used as the pre-trained vehicle segmentation model.

[0050] Among them, MobileNet is a lightweight convolutional neural network suitable for visual tasks on mobile terminals. MobileNet includes multiple versions, such as MobileNet V1, MobileNet V2 and MobileNet V3.

[0051] For example, taking the pre-trained vehicle segmentation model as a pre-trained MobileNet V3, the pre-trained MobileNet V3 optimizes the depthwise separable convolution. The optimized depthwise separable convolution can reduce the number of parameters and the amount of computation, thereby improving the performance while ensuring its light weight. Therefore, the pre-trained MobileNet V3 can also show superior performance on the mini program side.

[0052] Step S102, obtaining a target tensor of the vehicle's video frame data.

[0053] When the mini program responds to a request to obtain an image of a damaged part of a vehicle, the model-based running tool library can call the camera of the mobile terminal to capture the video stream data of the vehicle, where the video stream data of the vehicle is composed of multiple video frame data.

[0054] In order to facilitate the pre-trained vehicle segmentation model to perform image segmentation processing, a single video frame data is extracted from the vehicle video stream data to obtain the target tensor of the video frame data.

[0055] Step S103, performing damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image contains at least two components of the vehicle, and any two components are not the same.

[0056] The target tensor is input into a pre-trained vehicle segmentation model for damage segmentation processing, and a damage segmentation image output by the pre-trained vehicle segmentation model is obtained, wherein the damage segmentation image includes at least two components, and any two components are not the same.

[0057] It should be noted that pixel labels can be used to represent vehicle parts. The pre-trained vehicle segmentation model can accurately predict multiple types of pixel labels. Taking the pre-trained MobileNet V3 as an example, it can accurately distinguish up to 43 types of pixel labels, including background, left front door, right front door, etc.

[0058] For example, if the dimension of the target tensor is 257*257, the pre-trained vehicle segmentation model can predict that the dimension of the output pixel label sequence is also 257*257, and the pixel label sequence contains the pixel label of each pixel on the target tensor.

[0059] Step S104, obtaining the contour area of ​​each component.

[0060] The contour of each component in the damage segmentation image is detected to obtain the contour area of ​​each component.

[0061] Step S105 , screening out damaged parts of the vehicle from at least two parts according to the contour area.

[0062] The contour area of ​​each component is sorted from large to small, and the component with the largest contour area is determined, and then the component with the largest contour area is selected from at least two components as the damaged component. In this way, the damaged components of the vehicle can be accurately selected according to the contour area of ​​the components.

[0063] Step S106, obtaining the outline rectangle of the damaged component.

[0064] The outline rectangle of the damaged component may also be obtained based on the outline of the damaged component, wherein the outline rectangle of the damaged component may be the minimum circumscribed rectangle of the outline of the damaged component.

[0065] Step S107 , based on the contour area and pixels of the contour rectangle of the damaged component, and the total area and total pixels of the damage segmented image, the damaged component is photographed to obtain an image of the damaged component of the vehicle.

[0066] The pixels of the outline rectangle of the damaged part, as well as the total area and total pixels of the damaged segmented image are obtained. Finally, based on the outline area of ​​the damaged part and the pixels of the outline rectangle, as well as the total area and total pixels of the damaged segmented image, the damaged part is quickly and automatically photographed to obtain the damaged part image of the vehicle. In this way, the automatic photography of the damaged part image can be realized on the mini program end.

[0067] In this way, users can quickly and automatically obtain high-quality vehicle damaged parts images that can meet the damage assessment requirements through the mini program on the mobile terminal, which simplifies user operations and improves the reliability, efficiency and convenience of obtaining vehicle damaged parts images.

[0068] In steps S101 to S107 shown in the embodiment of the present application, a pre-trained vehicle segmentation model is loaded based on a model operation tool library; a target tensor of the vehicle's video frame data is obtained; the target tensor is subjected to damage segmentation processing through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same; the contour area of ​​each component is obtained; based on the contour area, the damaged component of the vehicle is screened out from at least two components; the contour rectangle of the damaged component is obtained; based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image, the damaged component is photographed to obtain a damaged component image of the vehicle. In this way, based on the model operation tool library, the vehicle segmentation model can be used to quickly and accurately identify the vehicle's damaged segmentation image and at least two components contained in the damaged segmentation image, so as to identify the damaged component from at least two components through the contour area of ​​each component, and then based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image, the damaged component is automatically photographed to obtain a high-quality vehicle damaged component image that can meet the damage assessment requirements, which simplifies user operations, can improve the reliability, efficiency and convenience of obtaining vehicle damaged component images, and provide an accurate basis for subsequent vehicle damage assessment.

[0069] See also Figure 2In some embodiments, step S101 may include but is not limited to steps S1011 to S1012.

[0070] Step S1011, obtaining the encrypted vehicle segmentation model.

[0071] It is worth noting that in order to improve training efficiency and training effect, the initial vehicle segmentation model can be pre-trained on a device with data processing function with higher performance and computing resources, such as a personal computer or server, to obtain a trained vehicle segmentation model.

[0072] Among them, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN), as well as big data and data analysis platforms.

[0073] Exemplarily, taking the training of the initial vehicle segmentation model on a cloud server as an example, the training process includes: obtaining vehicle video frame samples, and converting the vehicle video frame samples into target tensor samples; then inputting the target tensor samples into the initial vehicle segmentation model for damage segmentation processing to obtain a predicted damage segmentation image; then calculating the loss value between the predicted damage segmentation image and the actual damage segmentation image of the target tensor sample, and iteratively updating the parameters of the initial vehicle segmentation model according to the loss value until the number of iterations is greater than the preset iteration number threshold, stopping the update, and obtaining a trained vehicle segmentation model.

[0074] In order to ensure the security of the pre-trained vehicle segmentation model, the cloud server can also encrypt the pre-trained vehicle segmentation model to obtain an encrypted vehicle segmentation model.

[0075] For example, the pre-trained vehicle segmentation model can be encrypted by a symmetric encryption algorithm such as the AES encryption algorithm. The AES encryption algorithm is both convenient and highly secure, and the same key is used for encryption and decryption.

[0076] See also Figure 3 , Figure 3 This is an example diagram of encrypting and decrypting a pre-trained vehicle segmentation model using the AES encryption algorithm. Figure 2 In the encryption part shown, assume that the AES encryption function is E, C=E(K, P), where P represents the pre-trained vehicle segmentation model, K represents the key, and C represents the encrypted vehicle segmentation model. The cloud server uses the pre-trained vehicle segmentation model P and the key K as the input of the encryption function E, and the encryption function E will output the encrypted vehicle segmentation model C.

[0077] The cloud server can store the encrypted vehicle segmentation model locally on the cloud server.

[0078] In this way, when the mini program responds to a request to obtain an image of a damaged vehicle part, it can obtain the encrypted vehicle segmentation model from the cloud server through an https request.

[0079] Step S1012, based on the modified tf.loadGraphModel function in the model running tool library, the encrypted vehicle segmentation model is decrypted to obtain the vehicle segmentation model, and the vehicle segmentation model is loaded.

[0080] The mini program end decrypts the encrypted vehicle segmentation model based on the model running tool library, obtains the pre-trained vehicle segmentation model, and loads the pre-trained vehicle segmentation model.

[0081] Among them, the tf.loadGraphModel function for loading the graph model in the model running tool library can be customized in advance to obtain the modified tf.loadGraphModel function. Based on the modified tf.loadGraphModel function, the model running tool library can support decryption of the encrypted vehicle segmentation model on the mini program end.

[0082] For example, please refer to 3. Figure 3 In the decryption part shown, the mini program can construct an AES decryption function through javascript code. Suppose the AES decryption function is D, P=D(K, C), and the encrypted vehicle segmentation model C and key K are used as the input of the decryption function D. The decryption function D will output the pre-trained vehicle segmentation model P.

[0083] Among them, after the mini program completes the decryption operation, it can also convert the pre-trained vehicle segmentation model into arraybuffer format, which is convenient for loading the pre-trained vehicle segmentation model.

[0084] In the steps S1021 to S1022 shown in the embodiment of the present application, the encrypted vehicle segmentation model is obtained; based on the modified tf.loadGraphModel function in the model operation tool library, the encrypted vehicle segmentation model is decrypted to obtain the vehicle segmentation model, and the vehicle segmentation model is loaded. In this way, the mini program end can not only ensure the security of the vehicle segmentation model, but also realize the rapid call of the vehicle segmentation model by completing the decryption of the vehicle segmentation model based on the model operation tool library locally and loading the vehicle segmentation model.

[0085] In step S102 of some embodiments, video frame data of the vehicle may be acquired; the video frame data may be converted to obtain a tensor of the video frame data; and the tensor may be scaled to obtain a target tensor.

[0086] The mini program can obtain the video frame data of the vehicle, then convert the video frame data to obtain the tensor of the video frame data, and then scale the tensor of the video frame data, such as scaling the image information of the tensor of the video frame data to a size of 257*257 to obtain the target tensor of the video frame data. The target tensor of the video frame data is used as the input of the trained vehicle segmentation model.

[0087] Through the above method, the mini program end can convert the video frame data captured by the mobile terminal camera into a target tensor suitable for processing by the trained vehicle segmentation model.

[0088] In step S104 of some embodiments, binary image data of each component may be extracted from the damage segmentation image; contour detection processing may be performed on the binary image data to obtain the contour of each component; and area detection processing may be performed on the contour to obtain the contour area of ​​each component.

[0089] The mini program can extract the binary image data of each component from the damage segmentation image, and then perform contour detection on the binary image data based on the findcontourByJS function to obtain the contour of each component, and then perform area detection on the contour of each component to obtain the contour area of ​​each component.

[0090] Through the above method, the contour area of ​​each component in the damaged segmentation image can be quickly determined, providing an accurate analysis basis for the subsequent screening of damaged components.

[0091] In step S106 in some embodiments, a minimum circumscribed rectangle of the outline of the damaged component may be constructed to obtain an outline rectangle.

[0092] The mini program can run the tool library based on the model to construct the minimum circumscribed rectangle of the outline of the damaged part and obtain the outline rectangle, which is used to provide an accurate basis for taking images of the damaged part.

[0093] See also Figure 4 In some embodiments, step S107 may include but is not limited to steps S1071 to S1073.

[0094] Step S1071, calculating the ratio of the contour area of ​​the damaged component to the total area.

[0095] The ratio between the contour area of ​​the damaged part and the total area of ​​the damaged segmented image can be calculated.

[0096] It should be noted that the ratio is used to determine whether the distance between the damaged component and the camera of the mobile terminal is appropriate.

[0097] Step S1072, calculating the difference between the pixels of the outline rectangle and the total pixels.

[0098] The difference between the pixels of the contour rectangle and the total pixels of the lesion segmentation image may also be calculated.

[0099] It should be noted that the difference is used to determine the integrity of the damaged component in the picture taken by the mobile terminal camera.

[0100] Step S1073: photograph the damaged component based on the ratio and the difference to obtain an image of the damaged component.

[0101] Finally, based on the ratio, the distance between the damaged component and the mobile terminal camera is judged, and based on the difference, the integrity of the damaged component in the picture taken by the mobile terminal camera is judged. When the distance and integrity are both appropriate, it means that the mobile terminal camera focuses on the damaged component accurately, and the mobile terminal camera is controlled to automatically shoot the damaged component to obtain a higher quality image of the damaged component.

[0102] In this way, the focusing accuracy of the mobile terminal camera on the damaged component can be indirectly determined through the ratio and difference, which helps to control the camera to capture higher quality images of the damaged component.

[0103] In step S1073 of some embodiments, it can be determined whether the ratio is within a preset ratio range, and whether the difference is greater than or equal to a preset difference threshold; when the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, the damaged component is photographed to obtain an image of the damaged component; when the ratio is not within the preset ratio range or the difference is less than the preset difference threshold, a prompt message is output, and the prompt message is used to prompt the damaged component to be focused until the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, and the damaged component is photographed to obtain an image of the damaged component.

[0104] Specifically, it is determined whether the ratio is within a preset ratio range, and whether the difference is greater than or equal to a preset difference threshold. The preset ratio range and the preset difference threshold can be flexibly set according to actual conditions. For example, the preset ratio range can be 0.2 to 0.8, and the preset difference threshold can be 2 pixels or 3 pixels.

[0105] When the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, it is determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the shooting picture of the mobile terminal camera are both appropriate, indicating that the mobile terminal camera focuses on the damaged component accurately. Controlling the mobile terminal camera to automatically shoot the damaged component can obtain a higher quality image of the damaged component.

[0106] When the ratio is not within the preset ratio range or the difference is less than the preset difference threshold, it is determined that the distance between the damaged component and the mobile terminal camera or the integrity of the damaged component in the captured image of the mobile terminal camera is not appropriate, indicating that the focus of the camera on the damaged component is not accurate enough, and a corresponding prompt message can be output to prompt the user to adjust the mobile terminal to trigger the camera to focus on the damaged component again, and continue to loop steps S101 to S107 until the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, that is, until it is determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the captured image of the mobile terminal camera are appropriate, the damaged component is photographed to obtain an image of the damaged component.

[0107] For example, if the ratio is between 0.2 and 0.8 and the difference is greater than or equal to 3 pixels, it can be determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the shooting picture of the mobile terminal camera are both appropriate, indicating that the camera focuses on the damaged component accurately. By controlling the camera to automatically shoot the damaged component, higher quality images of the damaged component can be obtained.

[0108] If the ratio is less than 0.2 and the difference is greater than or equal to 3 pixels, it is determined that the distance between the damaged component and the mobile terminal camera is inappropriate while the integrity of the damaged component in the shooting picture of the mobile terminal camera is appropriate, indicating that the damaged component is too far away from the mobile terminal, and a prompt message such as "Please get closer to the vehicle" is generated to prompt the user to adjust the distance between the vehicle and the mobile terminal camera closer, thereby triggering the mobile terminal camera to focus on the damaged component again, and continue to loop steps S101 to S107 until the ratio is between 0.2 and 0.8 and the difference is greater than or equal to 3 pixels, that is, until it is determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the shooting picture of the mobile terminal camera are both appropriate, the damaged component is photographed to obtain an image of the damaged component.

[0109] If the ratio is greater than 0.8 and the difference is greater than or equal to 3 pixels, it is determined that the distance between the damaged component and the mobile terminal camera is inappropriate while the integrity of the damaged component in the shooting picture of the mobile terminal camera is appropriate, indicating that the damaged component is too close to the mobile terminal camera, and a prompt message such as "Please stay away from the vehicle" is generated to prompt the user to adjust the distance between the vehicle and the mobile terminal camera to a greater distance, thereby triggering the mobile terminal camera to focus on the damaged component again, and continuing to loop steps S101 to S107 until the ratio is between 0.2 and 0.8 and the difference is greater than or equal to 3 pixels, that is, until it is determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the shooting picture of the mobile terminal camera are both appropriate, the damaged component is photographed to obtain an image of the damaged component.

[0110] If the ratio is between 0.2 and 0.8 and the difference is less than 3 pixels, it can be determined that the distance between the damaged component and the mobile terminal camera is appropriate, but the integrity of the damaged component in the picture taken by the mobile terminal camera is not appropriate, indicating that the camera of the mobile terminal may not be aimed at the damaged part, and a prompt message such as "Please move the mobile terminal left or right or up and down" is generated to prompt the user to adjust the mobile terminal so that its camera is aimed at the damaged part, thereby triggering the camera of the mobile terminal to focus on the damaged part again, and continue to loop steps S101 to S107 until the ratio is between 0.2 and 0.8 and the difference is greater than or equal to 3 pixels, that is, until it is determined that the distance between the damaged component and the mobile terminal camera and the integrity of the damaged component in the picture taken by the mobile terminal camera are appropriate, the damaged component is photographed to obtain an image of the damaged component.

[0111] Through the above method, it is possible to ensure that the distance between the damaged part and the mobile terminal camera and the integrity of the damaged part in the shooting picture of the mobile terminal camera are appropriate, thereby ensuring the accuracy of the camera focusing on the damaged part, and then automatically capturing higher quality images of the damaged parts, thereby improving the reliability of obtaining images of damaged parts of the vehicle.

[0112] It should be noted that the non-Company software tools or components appearing in the embodiments of the present application are merely examples and do not represent actual use.

[0113] See also Figure 5 , Figure 5 It is a schematic block diagram of a vehicle damaged part image acquisition device provided in an embodiment of the present application. The vehicle damaged part image acquisition device can be configured in a computer device to execute the aforementioned vehicle damaged part image acquisition method.

[0114] like Figure 5As shown, the vehicle damaged component image acquisition device 500 includes: a model loading module 501, a tensor acquisition module 502, a damage segmentation module 503, an area acquisition module 504, a component screening module 505, a rectangle acquisition module 506 and an image shooting module 507.

[0115] A model loading module 501 is used to load a pre-trained vehicle segmentation model based on a model operation tool library;

[0116] A tensor acquisition module 502 is used to acquire a target tensor of the video frame data of the vehicle;

[0117] A damage segmentation module 503 is used to perform damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image contains at least two components of the vehicle, and any two components are not the same;

[0118] An area acquisition module 504 is used to acquire the contour area of ​​each of the components;

[0119] A component screening module 505, configured to screen out damaged components of the vehicle from at least two components according to the contour area;

[0120] A rectangle acquisition module 506 is used to acquire a contour rectangle of the damaged component;

[0121] The image capturing module 507 is used to capture the damaged component based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmented image, to obtain an image of the damaged component of the vehicle.

[0122] In one embodiment, the image capturing module 507 is further used for:

[0123] Calculating a ratio of the contour area of ​​the damaged component to the total area;

[0124] Calculate the difference between the pixels of the outline rectangle and the total pixels;

[0125] Based on the ratio and the difference, the damaged component is photographed to obtain an image of the damaged component.

[0126] In one embodiment, the image capturing module 507 is further used for:

[0127] Determining whether the ratio is within a preset ratio range, and determining whether the difference is greater than or equal to a preset difference threshold;

[0128] When the ratio is within a preset ratio range and the difference is greater than or equal to the preset difference threshold, photographing the damaged component to obtain an image of the damaged component;

[0129] When the ratio is not within the preset ratio range or the difference is less than the preset difference threshold, a prompt message is output, wherein the prompt message is used to prompt focusing processing on the damaged component until the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, and the damaged component is photographed to obtain the image of the damaged component.

[0130] In one embodiment, the area acquisition module 504 is further used to:

[0131] extracting binary image data of each of the components from the damage segmentation image;

[0132] Performing contour detection processing on the binary image data to obtain the contour of each of the components;

[0133] Performing area detection processing on the contour to obtain the contour area of ​​each of the components.

[0134] In one embodiment, the rectangle acquisition module 506 is further used to:

[0135] The minimum circumscribed rectangle of the outline of the damaged component is constructed to obtain the outline rectangle.

[0136] In one embodiment, the model loading module 501 is further used for:

[0137] Acquire video frame data of the vehicle;

[0138] Performing conversion processing on the video frame data to obtain a tensor of the video frame data;

[0139] The tensor is scaled to obtain the target tensor.

[0140] In one embodiment, the tensor acquisition module 502 is further used to:

[0141] Acquire video frame data of the vehicle;

[0142] Performing conversion processing on the video frame data to obtain a tensor of the video frame data;

[0143] The tensor is scaled to obtain the target tensor.

[0144] Among them, each module in the above-mentioned vehicle damaged part image acquisition device 500 corresponds to each step in the above-mentioned vehicle damaged part image acquisition method embodiment, and its functions and implementation processes are no longer repeated here one by one.

[0145] The vehicle damaged parts image acquisition device 500 can execute the vehicle damaged parts image acquisition method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the vehicle damaged parts image acquisition method provided in the embodiment of the present application. Please refer to the previous embodiments for details and will not be repeated here.

[0146] The method and apparatus of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0147] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Figure 6 Runs on the computer device shown.

[0148] See also Figure 6 , Figure 6 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0149] See also Figure 6 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0150] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0151] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any method for acquiring images of damaged vehicle parts.

[0152] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0153] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0154] Based on the model running tool library, load the pre-trained vehicle segmentation model;

[0155] Get the target tensor of the vehicle's video frame data;

[0156] Performing damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same;

[0157] Obtaining the outline area of ​​each of the components;

[0158] Screening out damaged parts of the vehicle from at least two of the parts according to the contour area;

[0159] Obtaining a contour rectangle of the damaged component;

[0160] Based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmented image, the damaged component is photographed to obtain a damaged component image of the vehicle.

[0161] In one embodiment, when the processor photographs the damaged component based on the contour area of ​​the damaged component and the pixels of the contour rectangle, and the total area and total pixels of the damage segmented image to obtain the damaged component image of the vehicle, the processor is used to implement:

[0162] Calculating a ratio of the contour area of ​​the damaged component to the total area;

[0163] Calculate the difference between the pixels of the outline rectangle and the total pixels;

[0164] Based on the ratio and the difference, the damaged component is photographed to obtain an image of the damaged component.

[0165] In one embodiment, when the processor photographs the damaged component based on the ratio and the difference to obtain the damaged component image, the processor is used to implement:

[0166] Determining whether the ratio is within a preset ratio range, and determining whether the difference is greater than or equal to a preset difference threshold;

[0167] When the ratio is within a preset ratio range and the difference is greater than or equal to the preset difference threshold, photographing the damaged component to obtain an image of the damaged component;

[0168] When the ratio is not within the preset ratio range or the difference is less than the preset difference threshold, a prompt message is output, wherein the prompt message is used to prompt focusing processing on the damaged component until the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, and the damaged component is photographed to obtain the image of the damaged component.

[0169] In one embodiment, when the processor implements the acquiring of the contour area of ​​each component based on the acquiring of the contour area of ​​each component, the processor implements:

[0170] extracting binary image data of each of the components from the damage segmentation image;

[0171] Performing contour detection processing on the binary image data to obtain the contour of each of the components;

[0172] Performing area detection processing on the contour to obtain the contour area of ​​each of the components.

[0173] In one embodiment, the processor, when implementing the step of obtaining the outline rectangular form of the damaged component, is used to implement:

[0174] The minimum circumscribed rectangle of the outline of the damaged component is constructed to obtain the outline rectangle.

[0175] In one embodiment, when the processor implements the model-based running tool library and loads the pre-trained vehicle segmentation model, it is used to implement:

[0176] Obtaining the encrypted vehicle segmentation model;

[0177] Based on the modified tf.loadGraphModel function in the model running tool library, the encrypted vehicle segmentation model is decrypted to obtain the vehicle segmentation model, and the vehicle segmentation model is loaded.

[0178] In one embodiment, when the processor implements the acquisition of the target tensor of the video frame data of the vehicle, it is used to implement:

[0179] Acquire video frame data of the vehicle;

[0180] Performing conversion processing on the video frame data to obtain a tensor of the video frame data;

[0181] The tensor is scaled to obtain the target tensor.

[0182] The computer device can execute the vehicle damaged parts image acquisition method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the vehicle damaged parts image acquisition method provided in the embodiment of the present application. Please refer to the previous embodiment for details and will not be repeated here.

[0183] An embodiment of the present application also provides a computer-readable storage medium.

[0184] The computer-readable storage medium of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for acquiring images of damaged vehicle parts are implemented.

[0185] The computer-readable storage medium may be an internal storage unit of the vehicle damaged component image acquisition device or computer device described in the aforementioned embodiment, such as a hard disk or memory of the vehicle damaged component image acquisition device or computer device. The computer-readable storage medium may also be an external storage device of the vehicle damaged component image acquisition device or computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SDCard), a flash card, etc., equipped on the vehicle damaged component image acquisition device or computer device.

[0186] Since the computer program stored in the computer-readable storage medium can execute any one of the vehicle damaged parts image acquisition methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the vehicle damaged parts image acquisition methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0187] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0188] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0189] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0190] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.

Claims

1. A method for acquiring images of damaged vehicle parts, characterized in that: The method comprises: Based on the model running tool library, load the pre-trained vehicle segmentation model; Get the target tensor of the vehicle's video frame data; Performing damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same; Obtaining the outline area of ​​each of the components; Screening out damaged parts of the vehicle from at least two of the parts according to the contour area; Obtaining a contour rectangle of the damaged component; Based on the contour area of ​​the damaged component and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmented image, the damaged component is photographed to obtain a damaged component image of the vehicle.

2. The method according to claim 1, characterized in that The method of photographing the damaged component based on the contour area of ​​the damaged component and the pixels of the contour rectangle, and the total area and total pixels of the damage segmented image to obtain the damaged component image of the vehicle includes: Calculating a ratio of the contour area of ​​the damaged component to the total area; Calculate the difference between the pixels of the outline rectangle and the total pixels; Based on the ratio and the difference, the damaged component is photographed to obtain an image of the damaged component.

3. The method according to claim 2, characterized in that The step of photographing the damaged component based on the ratio and the difference to obtain the damaged component image comprises: Determining whether the ratio is within a preset ratio range, and determining whether the difference is greater than or equal to a preset difference threshold; When the ratio is within a preset ratio range and the difference is greater than or equal to the preset difference threshold, photographing the damaged component to obtain an image of the damaged component; When the ratio is not within the preset ratio range or the difference is less than the preset difference threshold, a prompt message is output, wherein the prompt message is used to prompt focusing processing on the damaged component until the ratio is within the preset ratio range and the difference is greater than or equal to the preset difference threshold, and the damaged component is photographed to obtain the image of the damaged component.

4. The method according to claim 1, characterized in that The obtaining of the contour area of ​​each of the components comprises: extracting binary image data of each of the components from the damage segmentation image; Performing contour detection processing on the binary image data to obtain the contour of each of the components; Performing area detection processing on the contour to obtain the contour area of ​​each of the components.

5. The method according to claim 4, characterized in that The step of obtaining the outline rectangle of the damaged component comprises: The minimum circumscribed rectangle of the outline of the damaged component is constructed to obtain the outline rectangle.

6. The method according to any one of claims 1 to 5, characterized in that: The model-based running tool library loads a pre-trained vehicle segmentation model, including: Obtaining the encrypted vehicle segmentation model; Based on the modified tf.loadGraphModel function in the model running tool library, the encrypted vehicle segmentation model is decrypted to obtain the vehicle segmentation model, and the vehicle segmentation model is loaded.

7. The method according to any one of claims 1 to 5, characterized in that: The target tensor of obtaining the video frame data of the vehicle includes: Acquiring video frame data of the vehicle; Performing conversion processing on the video frame data to obtain a tensor of the video frame data; The tensor is scaled to obtain the target tensor.

8. A device for acquiring images of damaged vehicle parts, characterized in that: The device comprises: Model loading module, used to load pre-trained vehicle segmentation models based on the model running tool library; A tensor acquisition module, used to acquire a target tensor of the video frame data of the vehicle; A damage segmentation module, configured to perform damage segmentation processing on the target tensor through the vehicle segmentation model to obtain a damage segmentation image, wherein the damage segmentation image includes at least two components of the vehicle, and any two components are not the same; An area acquisition module, used to acquire the contour area of ​​each of the components; A component screening module, used for screening out damaged components of the vehicle from at least two of the components according to the contour area; A rectangle acquisition module, used to acquire the outline rectangle of the damaged component; The image shooting module is used to shoot the damaged part based on the contour area of ​​the damaged part and the pixels of the contour rectangle, as well as the total area and total pixels of the damage segmentation image, to obtain the damaged part image of the vehicle.

9. A mobile terminal, characterized in that: The mobile terminal includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle damaged parts image acquisition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for acquiring an image of a damaged vehicle part according to any one of claims 1 to 7 is implemented.

Images