Vehicle Damage Identification Method, Apparatus, Electronic Device, and Storage Medium

Abstract

The present invention provides a vehicle damage identification method and apparatus including: dividing the entire appearance surface of a target vehicle into a predetermined N regions; respectively acquiring images for each of the N regions by a preset image acquisition model to obtain N original images corresponding to the N regions; performing vehicle part identification on each of the N original images to obtain a vehicle part position identification result; cutting each of the N original images into M sub-images of a predetermined size based on a preset cutting model; performing damage identification on each of the N original images and the corresponding M sub-images to obtain a damage identification result; and fusing the vehicle part position identification result and the damage identification result to obtain a vehicle part damage result of the target vehicle. By standardized image acquisition and image preprocessing, the number of image acquisitions is reduced, the speed of the entire damage identification is accelerated, and the efficiency of damage identification is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence in the automotive aftermarket, and particularly to a vehicle damage identification method, apparatus, electronic device, and storage medium based on deep learning.

Background Art

[0002] Currently, in the field of the automotive aftermarket, the technology for identifying and evaluating vehicle damage is mainly used by companies and third-party organizations that conduct appraisals and evaluations. When conducting damage evaluation, the damage evaluator takes and uploads an external image of the vehicle damage site via a mobile phone, and then the system automatically identifies the damaged parts and damage types, thereby improving the efficiency of damage evaluation claims for small-amount cases or obtaining a general damage evaluation value for visible damage. As can be seen from this, the conventional technologies in the current automotive aftermarket all evaluate damage to the visible damage sites of the vehicle and have a certain degree of subjective judgment, and there is also ambiguity in the definition of damage. In addition, there is still no solution to realize damage evaluation for damage that cannot be easily judged and is difficult to notice. For example, in the car rental industry, when a customer returns a rental car to the car rental company, some damage to the vehicle exterior is not easily judged by the naked eye and is difficult to notice. Therefore, there is a need to propose new technical means for identifying and evaluating such damage.

[0003] In addition, in the conventional identification of vehicle damage, in order to identify fine damage, it is necessary to take both a close-up photo and a long-distance photo. Here, the close-up photo is used for detailed identification, and the long-distance photo is used for vehicle body position identification. Although such an image acquisition and recognition process is accurate, since the user needs to take photos many times, it affects the user experience and time efficiency, and the efficiency becomes poor.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention has been made in view of the above circumstances, and uses an end-to-end standardized damage identification process to convert vehicle damage identification from subjective judgment to objective judgment, reduce the dependence on the user's vehicle expertise, have broad versatility and compatibility, and provide a vehicle damage identification method, device, electronic device, and storage medium that improve the identification efficiency of fine damage to vehicles. Further, the present invention adopts a standardized image acquisition process and image preprocessing process to reduce the number of image acquisitions and accelerate the image acquisition process without affecting the identification accuracy of damage, thereby accelerating the overall speed of damage identification and improving the efficiency of damage identification.

Means for Solving the Problems

[0005] According to a first aspect of the present invention, the entire outer surface of a target vehicle is partitioned into a predetermined N regions, where N is a positive integer, and image acquisition is performed on each of the N regions according to a preset image acquisition model to obtain N original images corresponding to the N regions, performing vehicle part identification on each of the N original images to obtain a vehicle part position identification result, cutting each of the N original images into M sub-images of a predetermined size based on a preset cutting model, where M is a positive integer, performing damage identification on each of the N original images and the corresponding M sub-images to obtain a damage identification result, and fusing the vehicle part position identification result and the damage identification result to obtain a vehicle part damage result of the target vehicle. A vehicle damage identification method for identifying damage to a target vehicle is provided.

[0006] According to this embodiment, by standardizing the damage identification process, vehicle damage identification is converted from subjective judgment to objective judgment, reducing the dependence on the user's vehicle expertise, having wide versatility and compatibility, and achieving the technical effect of improving the identification efficiency of minor vehicle damage. Further, by adopting the standardized image acquisition process and image preprocessing process, without affecting the accuracy of damage identification, the number of image acquisitions can be reduced, and by accelerating the image acquisition process, the overall speed of damage identification can be accelerated and the efficiency of damage identification can be improved.

[0007] As an embodiment, the image acquisition model can perform image acquisition on the N regions of the target vehicle at a preset shooting angle so as to acquire the N original images each having an aspect ratio of a:b.

[0008] According to this embodiment, by performing image acquisition on each region partitioned at a predetermined shooting angle, the acquired original images can be standardized. Therefore, on the premise of not affecting the accuracy of damage identification, the technical effect of reducing the number of image acquisitions and accelerating the image acquisition process can be obtained. Further, the influence of the user's subjective shooting on the original images can be reduced, the application efficiency of image acquisition can be improved, the coverage rate of the images for the vehicle parts of the vehicle can be improved, and furthermore, the efficiency of damage identification can be improved.

[0009] As an embodiment, the cutting model can divide each of the N original images into a equal parts in the horizontal direction and divide the original images into b equal parts in the vertical direction to obtain a×b sub-images, where a×b = M.

[0010] According to this embodiment, by performing a normalization cutting process on the original image obtained using a preset cutting model, square sub-images of the same size can be obtained. As a result, the size of the cut image can be made to approximately match the size of the training image, avoiding problems such as poor implicit ability of invariance occurring in convolution, and eliminating the need for operations such as size adjustment. Thus, the technical effect of being able to preserve all the original features of the entire original image without changing the aspect ratio of the image can be achieved.

[0011] As one embodiment, performing damage identification on each of the N original images and the corresponding M sub-images respectively to obtain damage identification results includes: performing damage identification on each of the N original images respectively to obtain the overall damage identification results of each original image; performing damage identification on each of the M sub-images in each original image respectively to obtain local damage identification results; based on the positions of the M sub-images in their corresponding original images respectively, performing coordinate transformation on the local damage identification results to convert the coordinates of the local damage identification results from the coordinates in the sub-images to the coordinates in the corresponding original images, and obtaining the transformed local damage identification results; and fusing the transformed local damage identification results and the overall damage identification results to obtain the damage identification results.

[0012] According to this embodiment, damage identification can be performed on the original image and the sub-image respectively, and the technical effect of improving the accuracy and precision of damage identification can be achieved.

[0013] As one embodiment, it can further include outputting and displaying the vehicle part damage results.

[0014] According to this embodiment, the display result can be directly displayed to the photographed user, and the effect that the user can obtain the display information of the vehicle damage result within a short time (basically within 5 minutes) after taking the photo can be achieved.

[0015] As one embodiment, partitioning the entire exterior surface of the target vehicle into a predetermined N regions includes partitioning the entire exterior surface of the target vehicle into 14 regions, and these 14 regions can include the upper front side, lower front side, left front part, right front part, left front side part, right front side part, left middle part, right middle part, left rear part, right rear part, left rear side, right rear side, upper rear side and lower rear side of the target vehicle.

[0016] According to this embodiment, by performing the above-mentioned region partitioning and image acquisition for the partitioned regions, members within each region can be repeatedly presented in a plurality of acquired images, and a technical effect of ensuring that damage can be detected in at least one or more images can be obtained. Therefore, through standardized partitioning and image acquisition, the influence of the user's subjective shooting on the original image can be reduced, the application efficiency of image acquisition can be improved, the coverage rate of vehicle parts by the images can be improved, and furthermore, the efficiency of damage identification can be improved.

[0017] According to a second aspect of the present invention, there is provided a partition module for partitioning the entire exterior surface of a target vehicle into a predetermined N regions, where N is a positive integer; an original image acquisition module for respectively acquiring images for each of the N regions according to a preset image acquisition model to obtain N original images corresponding to the N regions; a vehicle part position identification module for performing vehicle part identification on each of the N original images to obtain a vehicle part position identification result; an original image cutting module for cutting each of the N original images into M sub-images of a predetermined size according to a preset cutting model, where M is a positive integer; a damage identification module for respectively performing damage identification on each of the N original images and the corresponding M sub-images to obtain a damage identification result; and a vehicle part damage fusion module for fusing the vehicle part position identification result and the damage identification result to obtain a vehicle part damage result of the target vehicle. A vehicle damage identification device for identifying damage to a target vehicle is provided.

[0018] As one embodiment, the image acquisition model can perform image acquisition on each of the N regions of the target vehicle at a preset shooting angle so as to acquire the N original images each having an aspect ratio of a:b.

[0019] As one embodiment, the cutting model can divide each of the N original images into a equal parts in the horizontal direction and divide the original images into b equal parts in the vertical direction to obtain a×b sub-images, where a×b = M.

[0020] As one embodiment, the damage identification module includes: an overall damage identification unit for performing damage identification on each of the N original images to obtain an overall damage identification result of each original image; a local damage identification unit for performing damage identification on each of the M sub-images in each original image to obtain a local damage identification result; a coordinate conversion unit for performing coordinate conversion on the local damage identification result based on the position in the corresponding original image of each of the M sub-images, so as to convert the coordinates of the local damage identification result from the coordinates in the sub-image to the coordinates in the corresponding original image and obtain a converted local damage identification result; and a damage fusion unit for fusing the converted local damage identification result and the overall damage identification result to obtain the damage identification result.

[0021] As one embodiment, the device can include a result output module for outputting and displaying the vehicle part damage result.

[0022] As one embodiment, the partitioning module partitions the entire appearance surface of the target vehicle into 14 regions, and these 14 regions can include the upper front part, the lower front part, the left front part, the right front part, the left front side part, the right front side part, the left middle part, the right middle part, the left rear part, the right rear part, the left rear part, the right rear part, the upper rear part and the lower rear part of the target vehicle.

[0023] Each of the above embodiments of the vehicle damage identification device according to the second aspect can obtain basically the same technical effects as those of the corresponding embodiments of the damage identification method, and thus the description thereof is omitted here.

[0024] According to a third aspect of the present invention, there is provided an electronic device including a memory storing a computer program, a processor configured to execute the computer program to perform the steps of the method according to the first aspect, an imaging device for acquiring an image, and a display device for displaying.

[0025] According to the electronic device of the third aspect, an end-to-end standardized damage identification process can be realized, vehicle damage identification can be converted from subjective judgment to objective judgment, the dependence on the user's vehicle expertise can be reduced, and on the premise of not affecting the accuracy of damage identification, the number of image acquisitions can be reduced, the image acquisition process can be accelerated, thereby accelerating the overall speed of damage identification, reducing the labor required for damage identification (which can be reduced within 5 minutes), and thereby reducing the training cost of personnel.

[0026] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the steps of the method according to the first aspect to be performed.

[0027] Hereinafter, the technical solution of the present invention will be described in more detail with reference to the drawings and preferred embodiments of the present invention, and the beneficial effects of the present invention will become more apparent.

Brief Description of the Drawings

[0028]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0029] The drawings described herein are provided for a further understanding of the present invention, constitute a part of the present invention, but are only for interpreting the present invention and do not unduly limit the present invention.

[0030] Hereinafter, the technical solution of the present invention will be clearly and completely described with reference to specific embodiments of the present invention and corresponding drawings. Obviously, the described embodiments are only some preferred embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the present invention.

[0031] Hereinafter, a vehicle damage identification method for identifying damage to a target vehicle according to an embodiment of the present invention will be described with reference to FIGS. 1 to 3.

[0032] FIG. 1 is a schematic flowchart of a vehicle damage identification method according to a preferred embodiment of the present invention. As shown in FIG. 1, the vehicle damage identification method of the present invention includes the following steps S101 to S106. Hereinafter, each step will be described in detail.

[0033] Step S101: Partitioning step.

[0034] The entire appearance surface of the target vehicle is partitioned into a predetermined N regions, where N is a positive integer.

[0035] As an example, for example, the entire appearance surface of the target vehicle can be partitioned into 14 regions, and these 14 regions can include the upper front part, lower front part, left front part, right front part, left front side part, right front side part, left middle part, right middle part, left rear part, right rear part, left rear side part, right rear side part, upper rear part, and lower rear part of the target vehicle.

[0036] It should be noted that the above partitioning method of the present invention is only an example, and the appearance surface of the target vehicle can be reasonably partitioned into multiple regions by other partitioning methods. In addition, among the above 14 regions, adjacent regions may have overlapping parts with each other.

[0037] Step S102: Original image acquisition step.

[0038] Image acquisition is respectively performed on each of the N regions by a preset image acquisition model, and N original images corresponding to the N regions are obtained.

[0039] Specifically, the image acquisition model can perform image acquisition on each of the N regions of the target vehicle at a preset shooting angle so as to obtain the N original images each having an aspect ratio of a:b.

[0040] The preset shooting angle, for example, taking the above-mentioned 14 regions as an example, the shooting angles corresponding to the 14 regions will be described in detail below.

[0041] Upper Front Side: Taking a picture of the front of the target vehicle directly from the front, with the two left and right lights on the front side of the target vehicle and the lower edge of the front bumper as the main positioning reference objects. Specifically, for example, the two left and right lights on the front side can be positioned at both left and right edges of the image, and the lower edge of the front bumper can be positioned at the lower edge of the image.

[0042] Lower Front Side: Taking a picture of the front of the target vehicle directly from the front, with the two left and right lights on the front side of the target vehicle, the lower edge of the front bumper, and the roof as the main positioning reference objects. Specifically, for example, the two left and right lights on the front side can be positioned at both left and right edges of the image, the lower edge of the front bumper can be positioned approximately at the center of the image, and the roof can be positioned at the upper edge of the image.

[0043] Left Front Part: Taking a picture of the left front diagonal of the target vehicle such that the image takes the front bumper as the positioning reference object, with the license plate included on the left side of the image and the entire fender included on the right side. For example, the front bumper can be positioned at approximately the middle position in the vertical direction on the left side of the image, with the license plate included on the left side of the image and the entire fender included on the right side.

[0044] Right Front Part: Taking a picture of the right front diagonal of the target vehicle such that the image takes the front bumper as the positioning reference object, with the license plate included on the right side of the image and the entire fender included on the left side. For example, the front bumper can be positioned at approximately the middle position in the vertical direction on the right side of the image, with the license plate included on the right side of the image and the entire fender included on the left side.

[0045] Left Front: By taking a picture of the left front of the target vehicle, the left front light is included on the left side of the image, and as much of the left side body of the vehicle as possible is captured on the right side of the image (see, for example, Figure 4).

[0046] Right Front: By taking a picture of the right front of the target vehicle, the right front light is included on the right side of the image, and as much of the right side body of the vehicle as possible is captured on the left side of the image.

[0047] Left middle: Photograph the left side of the target vehicle so that the handover location of the front and rear doors is at the central position in the left - right direction of the image. Align the upper side of the image with the roof, and capture as many front and rear doors as possible on both the left and right sides of the image.

[0048] Right middle: Photograph the right side of the target vehicle so that the handover location of the front and rear doors is at the central position in the left - right direction of the image. Align the upper side of the image with the roof, and capture as many front and rear doors as possible on both the left and right sides of the image.

[0049] Left rear: Photograph the left rear of the target vehicle so that the left backlight is included on the right side of the image, and capture as much of the left - hand side body of the vehicle as possible on the left side of the image.

[0050] Right rear: Photograph the right rear of the target vehicle so that the right backlight is included on the left side of the image, and capture as much of the left - hand side body of the vehicle as possible on the right side of the image.

[0051] Left rear part: Photograph the left - rear diagonal of the target vehicle. Set the image with the rear bumper as the positioning reference target. Include the rear license plate on the right side of the image and the entire fender on the left side. For example, the rear bumper can be positioned at approximately the middle position in the up - down direction on the right side of the image, with the license plate included on the right side of the image and the entire fender included on the left side.

[0052] Right rear part: Photograph the right - rear diagonal of the target vehicle. Set the image with the rear bumper as the positioning reference target. Include the rear license plate on the left side of the image and the entire fender on the right side. For example, the rear bumper can be positioned at approximately the middle position in the up - down direction on the left side of the image, with the license plate included on the left side of the image and the entire fender included on the right side.

[0053] Rear upper: Photograph directly behind the target vehicle with the two left - and - right lights on the rear side and the lower edge of the rear bumper as the main positioning reference targets. Specifically, for example, the two left - and - right lights on the rear side can be positioned at the left - and - right edges of the image, and the lower edge of the rear bumper can be positioned at the lower edge of the image.

[0054] Lower rear side: Taking a picture directly behind the target vehicle, with the two left and right lights on the rear side of the target vehicle, the lower edge of the rear bumper, and the roof as the main positioning reference targets. Specifically, for example, the two left and right lights on the rear side can be positioned at the left and right edges of the image, the lower edge of the rear bumper can be positioned approximately in the center of the image, and the roof can be positioned at the upper edge of the image.

[0055] Through the above standardized area division and image acquisition, the members in each area can appear repeatedly in the multiple acquired images, and it can be guaranteed that damage can be detected by at least one or more images. Therefore, through the standardized division and image acquisition, the influence of the user's subjective shooting on the original image can be reduced, the application efficiency of image acquisition can be improved, the coverage rate of vehicle parts by the image can be improved, and the efficiency of damage identification can be improved. It should be noted that the vehicle parts used as the above positioning reference targets are not limited to those described above. As long as the parts in each area can appear repeatedly in the multiple acquired images so that damage can be detected in at least one of the multiple acquired images, they can be set appropriately.

[0056] Also, the image acquisition model may perform image acquisition with fixed pixels. For example, in the horizontal shooting method, image acquisition can be performed with 4032*3024 fixed pixels. Thereby, an original image with a general aspect ratio of a:b = 4:3 can be obtained.

[0057] It should be noted that the above image acquisition angle, pixels, aspect ratio, etc. are only examples and can be set appropriately according to needs.

[0058] Step S103: Vehicle part position identification step.

[0059] Perform vehicle part identification on each of the N original images to obtain vehicle part position identification results.

[0060] Specifically, in this embodiment, using a pre-trained vehicle part detection model, vehicle part detection is performed on each original image based on the original images of each acquired region, and a vehicle part position identification result corresponding to each original image is obtained.

[0061] Step S104: Original image cutting step.

[0062] Based on a preset cutting model, each of the N original images is cut into M sub-images of a predetermined size, where M is a positive integer. Preferably, the sizes of the M sub-images are exactly the same.

[0063] Specifically, the segmentation model may, for example, use a sliding window and a cut algorithm with overlap = 0. For example, when the aspect ratio of the acquired original image is a:b, in each of the N acquired original images, the original image is divided into a equal parts horizontally and the original image is divided into b equal parts vertically to obtain a×b sub-images, where a×b = M.

[0064] As an example, as shown in FIG. 4, the pixels of the original image are 4032*3024 and the aspect ratio is 4:3. For each original image, the above cutting model is set such that the overlap is 0 (overlap = 0) and the step width is 1008 (step = 1008), and the original image is divided into 4 equal parts horizontally and 3 equal parts vertically to obtain 12 square sub-images with 1008*1008 pixels.

[0065] In the training set of object detection in the prior art, the size of images is usually between 600 and 1000, because the implicit ability of the convolution network in terms of size, rotation, and translation invariance is low. Also, for conventional object detection, in order to accelerate the processing of images in a batch, the pre-processing of images includes processes such as resizing and cropping into squares.

[0066] On the other hand, the above-mentioned cutting method of the present invention makes the size of the cut image basically match the size of the training image, avoids problems that may occur in the above-mentioned convolution, and does not require size adjustment, etc., so all original features of the entire original image can be preserved without changing the aspect ratio of the image.

[0067] Step S105: Damage identification step.

[0068] Perform damage identification on each of the N original images and the corresponding M sub-images respectively, and obtain damage identification results.

[0069] Specifically, as an example, the above step S105 includes the following steps S201 - S204. Hereinafter, each of the above steps S201 - S204 of the embodiment of the present invention will be described with reference to FIG. 2.

[0070] S201: Overall damage identification step.

[0071] Perform damage identification on each of the N original images respectively, and obtain the overall damage identification results of each original image.

[0072] Specifically, for the acquired original images, they are sent to a pre-trained vehicle damage detection model (for example, a vehicle damage object detection AI system), and overall damage identification is performed on each original image to obtain an overall damage identification result corresponding to each original image.

[0073] S202: Local damage identification step.

[0074] Damage identification is performed on each of the M sub-images in each of the original images to obtain a local damage identification result.

[0075] Specifically, for the M sub-images obtained by splitting each original image, each sub-image is sent to the aforementioned vehicle damage detection model, local damage identification is performed on each sub-image, and a local damage identification result corresponding to each sub-image is obtained.

[0076] S203: Coordinate transformation step.

[0077] Based on the position of each of the M sub-images in the corresponding original image, coordinate transformation is performed on the local damage identification result, so that the coordinates of the local damage identification result are transformed from the coordinates in the sub-image to the coordinates in the corresponding original image, and a transformed local damage identification result is obtained.

[0078] Specifically, since the sub-images in this application are images obtained by standardized cutting, an offset value can be calculated based on the position in the original image corresponding to each sub-image, whereby the local coordinates of the local damage identification result in the sub-image are transformed into the coordinates in the original image based on the offset value, and a transformed local damage identification result is obtained.

[0079] S204: Damage fusion step.

[0080] Fuse the local damage identification result after the transformation and the overall damage identification result to obtain the damage identification result.

[0081] Specifically, since the local damage identification result after the transformation has already been coordinate-transformed and is in the same coordinate system as the overall damage identification result, the two can be fused to obtain a damage identification result in which the local damage identification result and the overall damage identification result corresponding to each original image are fused.

[0082] As described above, the flow of the damage identification step has been explained. However, by using the above steps S201 - S204, damage identification can be performed on the original image and the sub-image respectively, improving the accuracy and precision of the damage identification.

[0083] Step S106: Vehicle component damage fusion step.

[0084] Fuse the vehicle component position identification result obtained in step S103 and the damage identification result obtained in step S105 to obtain the vehicle component damage result of the target vehicle.

[0085] Specifically, in the prior art, coordinate matching is generally performed using the ratio of the common set and the union set of the bounding boxes (bounding box intersection over union).

[0086] In the present invention, due to the property that the damage box is small, if the above coordinate matching method is adopted, the matching effect will be poor. Therefore, for the relative position between the vehicle damage and the vehicle component in the present invention, coordinate matching is performed using the ratio of the common set of the bounding boxes and the damage area (bounding box intersection over damage area) to obtain the result of the vehicle component damage, and the ratio of the common set of the bounding boxes and the damage area is expressed by the following formula.

[0087]

Equation

[0088] Also, as shown in FIG. 3, the vehicle damage identification method according to an embodiment of the present invention may further include a result output step, that is, step S107, which outputs and displays the vehicle part damage result obtained by fusion.

[0089] As described above, the vehicle damage identification method of the present invention has been described. However, by using a standardized image acquisition process and image preprocessing process, vehicle damage identification is converted from subjective judgment to objective judgment, reducing the dependence on the user's vehicle expertise, having wide versatility and compatibility, improving the identification efficiency of minor vehicle damage, realizing true AI intelligent damage evaluation, and accelerating the identification speed of AI. Also, by adopting a standardized image acquisition process and image preprocessing process, without affecting the accuracy of damage identification, the number of image acquisitions is reduced, the image acquisition process is accelerated, thereby accelerating the overall speed of damage identification, reducing the labor required for damage identification (it can be reduced within 5 minutes), and reducing the training cost of personnel.

[0090] As described above, the damage identification method according to an embodiment of the present invention has been described. However, the embodiment of the present invention further provides a damage identification device. As shown in the figure, the damage identification device 100 according to an embodiment of the present invention includes modules 101-106. Hereinafter, the damage identification device 100 according to an embodiment of the present invention will be described with reference to FIG. 5.

[0091] Module 101: Partitioning module.

[0092] The partitioning module 101 is for partitioning the entire appearance surface of the target vehicle into a predetermined N regions, where N is a positive integer.

[0093] As an example, for instance, the partitioning module 101 can partition the entire exterior surface of the target vehicle into 14 regions, and these 14 regions can include the upper front part, lower front part, left front part, right front part, left front side part, right front side part, left middle part, right middle part, left rear part, right rear part, left rear side part, right rear side part, upper rear part, and lower rear part of the target vehicle.

[0094] It should be noted that the above partitioning method of the present invention is only an example, and the exterior surface of the target vehicle can be reasonably partitioned into multiple regions by other partitioning methods. Also, among the above 14 regions, adjacent regions may have overlapping parts with each other.

[0095] Module 102: Original image acquisition module.

[0096] The original image acquisition module 102 is for respectively acquiring images for each of the N regions according to a preset image acquisition model, and acquiring N original images corresponding to the N regions.

[0097] Specifically, the image acquisition model can respectively perform image acquisition on the N regions of the target vehicle at a preset shooting angle so as to acquire the N original images each having an aspect ratio of a:b.

[0098] Regarding the preset shooting angle, for example, taking the above-mentioned 14 regions as an example, the shooting angles corresponding to the 14 regions will be described in detail below.

[0099] Upper front part: Taking the left and right two lights on the front side of the target vehicle and the lower edge of the front bumper as the main positioning reference targets, shoot at the true front of the target vehicle. Specifically, for example, the left and right two lights on the front side can be positioned at the left and right edges of the image, and the lower edge of the front bumper can be positioned at the lower edge of the image.

[0100] Lower front side: Taking a photo of the front of the target vehicle directly from the front, with the two left and right lights on the front side, the lower edge of the front bumper, and the roof of the target vehicle as the main positioning reference objects. Specifically, for example, the two left and right lights on the front side can be positioned at both left and right edges of the image, the lower edge of the front bumper can be positioned at approximately the center of the image, and the roof can be positioned at the upper edge of the image.

[0101] Upper left front: Taking a photo of the left front diagonal of the target vehicle with the image using the front bumper as the positioning reference object, where the license plate is included on the left side of the image and the entire fender is included on the right side. For example, the front bumper can be positioned at approximately the middle position in the vertical direction on the left side of the image, with the license plate included on the left side of the image and the entire fender included on the right side.

[0102] Upper right front: Taking a photo of the right front diagonal of the target vehicle with the image using the front bumper as the positioning reference object, where the license plate is included on the right side of the image and the entire fender is included on the left side. For example, the front bumper can be positioned at approximately the middle position in the vertical direction on the right side of the image, with the license plate included on the right side of the image and the entire fender included on the left side.

[0103] Left front side: Taking a photo of the left front of the target vehicle, so that the left front light is included on the left side of the image and as much of the left side body of the vehicle as possible is captured on the right side of the image (see, for example, Figure 4).

[0104] Right front side: Taking a photo of the right front of the target vehicle, so that the right front light is included on the right side of the image and as much of the right side body of the vehicle as possible is captured on the left side of the image.

[0105] Left middle side: Taking a photo of the left side of the target vehicle so that the handover location of the front and rear doors is positioned at the center in the left - right direction of the image, the upper side of the image is aligned with the roof, and as many front and rear doors as possible are captured on both left and right sides of the image.

[0106] Right middle: Photograph the right side of the target vehicle so that the handover location of the front and rear doors is at the central position in the left-right direction of the image. Align the upper side of the image with the roof, and capture as many front and rear doors as possible on both the left and right sides of the image.

[0107] Left rear: Photograph the left rear of the target vehicle so that the left backlight is included on the right side of the image, and capture as much of the left side body of the vehicle as possible on the left side of the image.

[0108] Right rear: Photograph the right rear of the target vehicle so that the right backlight is included on the left side of the image, and capture as much of the left side body of the vehicle as possible on the right side of the image.

[0109] Left rear part: Photograph the left oblique rear of the target vehicle. The image takes the rear bumper as the positioning reference target, includes the rear license plate on the right side of the image, and includes the entire fender on the left side. For example, the rear bumper can be positioned at approximately the middle position in the up-down direction on the right side of the image, the license plate is included on the right side of the image, and the entire fender is included on the left side.

[0110] Right rear part: Photograph the right oblique rear of the target vehicle. The image takes the rear bumper as the positioning reference target, includes the rear license plate on the left side of the image, and includes the entire fender on the right side. For example, the rear bumper can be positioned at approximately the middle position in the up-down direction on the left side of the image, the license plate is included on the left side of the image, and the entire fender is included on the right side.

[0111] Rear upper: Photograph directly behind the target vehicle with the two left and right lights on the rear side and the lower edge of the rear bumper as the main positioning reference targets. Specifically, for example, the two left and right lights on the rear side can be positioned at the left and right edges of the image, and the lower edge of the rear bumper can be positioned at the lower edge of the image.

[0112] Lower Rear Side: Taking a picture directly behind the target vehicle with the two left and right lights on the rear side of the target vehicle, the lower edge of the rear bumper, and the roof as the main positioning reference targets. Specifically, for example, the two left and right lights on the rear side can be positioned at both left and right edges of the image, the lower edge of the rear bumper can be positioned at approximately the center of the image, and the roof can be positioned at the upper edge of the image.

[0113] Through the above standardized area division and image acquisition, the components within each area can appear repeatedly in the multiple acquired images, ensuring that damage can be detected by at least one or more images. Therefore, through the standardized division and image acquisition, it is possible to reduce the influence of the user's subjective shooting on the original image, improve the application efficiency of image acquisition, improve the coverage rate of vehicle parts in the image, and improve the efficiency of damage identification. It should be noted that the vehicle parts used as the above positioning reference targets are not limited to those described above. As long as the parts within each area can appear repeatedly in the multiple acquired images so that damage can be detected in at least one of the multiple acquired images, they can be set as appropriate.

[0114] Also, the image acquisition model may perform image acquisition with fixed pixels. For example, in the horizontal shooting mode, image acquisition can be performed with 4032*3024 fixed pixels. Thereby, an original image with a general aspect ratio of a:b = 4:3 can be obtained.

[0115] It should be noted that the above image acquisition angle, pixels, aspect ratio, etc. are only examples and can be set as appropriate according to needs.

[0116] Module 103: Vehicle Part Position Identification Module.

[0117] The vehicle part position identification module 103 is used to perform vehicle part identification on each of the N original images and obtain the vehicle part position identification result.

[0118] Specifically, in this embodiment, the vehicle component position identification module 103 uses a pre-trained vehicle component detection model to perform vehicle component detection on each of the acquired original images of each region based on the original images of each region, and obtains a vehicle component position identification result corresponding to each original image.

[0119] Module 104: Original image cutting module.

[0120] The original image cutting module 104 cuts each of the N original images into M sub-images of a predetermined size based on a preset cutting model, where M is a positive integer. Preferably, the sizes of the M sub-images are exactly the same.

[0121] Specifically, the segmentation model may, for example, use a sliding window and a cut algorithm with overlap = 0. For example, when the aspect ratio of the acquired original image is a:b, in each of the N acquired original images, the original image is divided into a equal parts in the horizontal direction and the original image is divided into b equal parts in the vertical direction to obtain a×b sub-images, where a×b = M.

[0122] As an example, as shown in FIG. 4, the pixels of the original image are 4032*3024 and the aspect ratio is 4:3. For each original image, the above cutting model is set such that the overlap is 0 (overlap = 0) and the step width is 1008 (step = 1008), and the original image is divided into 4 equal parts in the horizontal direction and the original image is divided into 3 equal parts in the vertical direction to obtain 12 square sub-images with 1008*1008 pixels.

[0123] In the training set of object detection in the prior art, the size of the images is usually between 600 and 1000, and the implicit ability of the convolution network in terms of size, rotation, and translation invariance is low. Also, in conventional object detection, in order to accelerate the processing of images in a batch, the preprocessing of images includes operations such as resizing and cropping into squares.

[0124] On the other hand, the above-mentioned cutting method of the present invention makes the size of the cut images basically match the size of the training images, avoids problems that may occur in the above-mentioned convolution, and there is no need to perform size adjustment, etc. Therefore, all the original features of the entire original image can be preserved without changing the aspect ratio of the image.

[0125] Module 105: Damage identification module.

[0126] The damage identification module 105 is for performing damage identification on each of the N original images and the corresponding M sub-images respectively, and obtaining damage identification results.

[0127] Specifically, as an example, the above module 105 includes the following units 201-204. Hereinafter, each of the above units 201-204 of the embodiment of the present invention will be described with reference to FIG. 6.

[0128] 201: Overall damage identification unit.

[0129] The overall damage identification unit 201 is for performing damage identification on each of the N original images respectively, and obtaining the overall damage identification results of the respective original images.

[0130] Specifically, the overall damage identification unit 201 sends the acquired original image to a vehicle damage detection model (for example, a vehicle damage object detection AI system) that has been pre-trained, performs overall damage identification on each original image, and obtains an overall damage identification result corresponding to each original image.

[0131] 202: Local damage identification unit.

[0132] The local damage identification unit 202 is for performing damage identification on each of the M sub-images in each of the original images respectively to obtain a local damage identification result.

[0133] Specifically, the local damage identification unit 202 sends each of the M sub-images obtained by cutting each original image to the vehicle damage detection model described above, performs local damage identification on each sub-image, and obtains a local damage identification result corresponding to each sub-image.

[0134] Unit 203: Coordinate transformation unit.

[0135] The coordinate transformation unit 203 performs coordinate transformation on the local damage identification result based on the position in the corresponding original image of each of the M sub-images, thereby converting the coordinates of the local damage identification result from the coordinates in the sub-image to the coordinates in the corresponding original image, and is for obtaining the local damage identification result after transformation.

[0136] Specifically, since the sub-images in the present application are images obtained by standardized cutting, the coordinate transformation unit 203 can calculate an offset value based on the position in the original image corresponding to each sub-image, whereby the local coordinates of the local damage identification result in the sub-image are converted to the coordinates in the original image based on the offset value, and the local damage identification result after transformation is obtained.

[0137] 204: Damage fusion unit.

[0138] The damage fusion unit 204 is for fusing the converted local damage identification result and the overall damage identification result to obtain the damage identification result.

[0139] Specifically, since the converted local damage identification result has already been coordinate-transformed and is in the same coordinate system as the overall damage identification result, the damage fusion unit 204 can fuse the two to obtain a damage identification result in which the local damage identification result and the overall damage identification result corresponding to each original image are fused.

[0140] As described above, each unit of the damage identification module has been explained. By using the above units 201 - 204, damage identification can be performed on the original image and the sub-image respectively, and the accuracy and precision of damage identification can be improved.

[0141] Module 106: Vehicle part damage fusion module.

[0142] Fuse the vehicle part position identification result obtained by module 103 and the damage identification result obtained by module 105 to obtain the vehicle part damage result of the target vehicle.

[0143] Specifically, in the prior art, the coordinate matching of the relative positions of vehicle damage and vehicle parts is generally performed using the ratio of the common set and the union set of the bounding boxes (bounding box intersection over union).

[0144] In the present invention, due to the property that the damage box is small, if the above coordinate matching method is adopted, the matching effect will deteriorate. Therefore, the relative position between the vehicle damage and the vehicle parts in the present invention performs coordinate matching using the ratio of the common set of bounding boxes to the damage area (bounding box intersection over damage area), obtains the result of vehicle part damage, and the ratio of the common set of the bounding boxes to the damage area is expressed by the following formula.

[0145]

Equation

[0146] Also, as shown in FIG. 7, the vehicle damage identification device according to an embodiment of the present invention may further include a module 107 which is a result output module, that is, a module that outputs and displays the vehicle part damage result obtained by fusion.

[0147] As described above, the vehicle damage identification device of the present invention has been described. By using a standardized image acquisition process and an image preprocessing process, vehicle damage identification is converted from subjective determination to objective determination, reducing the dependence on the user's vehicle expertise, having wide generality and compatibility, improving the identification efficiency of minor vehicle damage, realizing true AI intelligent damage evaluation, and accelerating the identification speed of AI. Also, by adopting a standardized image acquisition process and an image preprocessing process, without affecting the accuracy of damage identification, the number of image acquisitions is reduced, the image acquisition process is accelerated, thereby accelerating the overall speed of damage identification, reducing the labor required for damage identification (it can be reduced within 5 minutes), and reducing the training cost of personnel.

[0148] Embodiments of the present invention provide a system architecture to which a vehicle damage identification method or a vehicle damage identification device according to embodiments of the present invention can be applied. FIG. 8 shows an exemplary system architecture 800 to which a vehicle damage identification method or a vehicle damage identification device according to embodiments of the present invention can be applied.

[0149] As shown in FIG. 8, the system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805 (this architecture is only an example, and the components included in the specific architecture can be adjusted according to the specific situation of the application). The network 804 is used to provide a medium for communication links between the terminal devices 801, 802, 803, and the server 805. The network 804 may include various connection types such as wired, wireless communication links, or optical fiber cables.

[0150] Users can use the terminal devices 801, 802, 803 to access the server 805 via the network 804 and receive or send messages and the like. Various communication client applications such as shopping applications, web browser applications, query applications, instant messaging tools, mailbox clients, social platform software (just examples) may be installed on the terminal devices 801, 802, 803.

[0151] The terminal devices 801, 802, 803 may be various electronic devices having a display and capable of browsing web pages, including but not limited to smartphones, tablet computers, laptop portable computers, desktop computers, etc.

[0152] Server 805 may be a server that provides various services, for example, a background management server (merely an illustration) that supports a shopping web site browsed by users using terminal devices 801, 802, and 803. The background management server can perform processes such as analysis on data such as received product information query requests, and feedback the processing results (for example, target push information, product information - merely an illustration) to the terminal devices.

[0153] Note that the vehicle damage identification method according to the embodiments of the present invention is generally executed by terminal devices 801, 802, and 803. Correspondingly, the vehicle damage identification device is generally installed in terminal devices 801, 802, and 803.

[0154] It should be understood that the numbers of terminal devices, networks, and servers in FIG. 8 are merely illustrations. Any number of terminal devices, networks, and servers may be provided as necessary.

[0155] )]] Also, FIG. 9 is a structural schematic diagram of a computer system 900 for realizing the terminal device according to the embodiments of the present invention. The terminal device shown in FIG. 9 is merely an example and does not impose any limitations on the functions and usage ranges of the embodiments of the present invention.

[0156] As shown in FIG. 9, the computer system 900 includes a central processing unit (CPU) 901 that can execute various appropriate operations and processes according to a program stored in a ROM (Read Only Memory) 902 or a program loaded from a storage unit 908 to a RAM (Random Access Memory) 903. Various programs and data necessary for the operation of the system 900 are also stored in the RAM 903. The CPU 901, ROM 902, and RAM 903 are interconnected by a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0157] The I / O interface 905 is connected to an input unit 906 including a keyboard, a mouse, etc., an output unit 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc., a storage unit 908 including a hard disk, etc., and a communication unit 909 including a network interface card such as a LAN card, a modem, etc. The communication unit 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as required. Removable media 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. are mounted on the drive 910 as required, and the computer program read therefrom is installed in the storage unit 908 as required.

[0158] FIG. 10 shows an example of an operation flow of a terminal device for realizing an embodiment of the present invention. The terminal device may be the terminal devices 801, 802, 803, etc. described above. As shown in FIG. 10, in the present invention, the terminal device includes at least an imaging device for acquiring an image and a display device for displaying, and each step of vehicle damage identification of the present invention, for example, processing and identification of an image, is executed by the background (processor) of the terminal device.

[0159] According to the above terminal device, an end-to-end standardized damage identification process can be realized, vehicle damage identification can be converted from subjective judgment to objective judgment, dependence on the user's vehicle expertise can be reduced, and on the premise of not affecting the accuracy of damage identification, the number of image acquisitions can be reduced, the image acquisition process can be accelerated to accelerate the overall speed of damage identification, the labor required for damage identification can be reduced (it can be reduced within 5 minutes), thereby reducing the training cost of personnel.

[0160] In particular, according to the embodiments of the present disclosure, the steps described with reference to the above flowchart diagrams may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium including program code for executing the method shown in the flowchart diagrams. In such an embodiment, the computer program may be downloaded and installed from the network via the communication unit 909 and / or installed from the removable media 911. When this computer program is executed by the central processing unit (CPU) 901, the above-described limited functions in the system of the present invention are executed.

[0161] Note that the computer-readable medium shown in the present invention may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that includes or records a program, and the program may be used in or in combination with a command execution system, apparatus, or device. In the present invention, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, on which computer-readable program code is carried. Such a propagated data signal may adopt various forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can transmit, propagate, or transmit a program used in or in combination with a command execution system, apparatus, or device. The program code included in the computer-readable medium can be transmitted by any suitable medium including, but not limited to, wireless, wired, fiber optic cable, RF, or any suitable combination of the above.

[0162] The flowcharts and block diagrams in the figures illustrate the possible system architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. From this perspective, each block in the flowchart or block diagram can represent a module, program segment, or part of code that includes one or more executable instructions for implementing a predetermined logical function. Note that, alternatively, the functions described in the blocks may occur in an order different from the order described in the drawings. For example, two consecutively shown blocks may actually be executed substantially in parallel or in the reverse order, as determined by the functions. Also, each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware system for performing a predetermined function or operation, or may be implemented by a combination of dedicated hardware and computer instructions.

[0163] The modules described in the embodiments of the present invention may be implemented in software or in hardware. The described modules may be provided in a processor. For example, the processor may be described as including a partitioning module, an original image acquisition module, a vehicle part position identification module, an original image cutting module, a damage identification module, and a vehicle part damage fusion module. Here, the names of these modules do not limit the modules themselves. For example, the original image acquisition module may be described as a "shooting module for acquiring an original image".

[0164] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiments, may exist alone, or may not be incorporated into the device. The computer-readable medium records one or more programs, and when the one or more programs are executed by a device, the device executes the following steps.

[0165] Divide the entire external surface of the target vehicle into a predetermined N regions, where N is a positive integer, and For each of the N regions, acquire an image according to a preset image acquisition model, and acquire N original images corresponding to the N regions, and For each of the N original images, perform vehicle part identification to obtain a vehicle part position identification result, and Based on a preset cutting model, cut each of the N original images into M sub-images of a predetermined size, where M is a positive integer, and For each of the N original images and the corresponding M sub-images, perform damage identification respectively to obtain a damage identification result, and Fuse the vehicle part position identification result and the damage identification result to obtain a vehicle part damage result of the target vehicle.

[0166] The present invention is applied to an algorithm / model for the size of a fixed-class neural network, and uses an end-to-end standardized damage identification process to convert vehicle damage identification from subjective judgment to objective judgment, reduce the dependence on the user's vehicle expertise, have wide versatility and compatibility, and provide a vehicle damage identification method, device, electronic device, and storage medium that improve the identification efficiency of fine damage to the vehicle. Further, the present invention adopts a standardized image acquisition process and image preprocessing process, which can reduce the number of image acquisitions and accelerate the image acquisition process without affecting the accuracy of damage identification, thereby accelerating the overall speed of damage identification and improving the efficiency of damage identification.

[0167] The above is only an example of the present invention and is not intended to limit the present invention, and various modifications and deformations are possible for those skilled in the art. Changes, equivalent substitutions, improvements, etc. made within the scope of the spirit and principle of the present invention shall be included in the scope of the claims of the present invention.

Claims

1. Divide the entire exterior surface of the target vehicle into a predetermined number N of regions, where N is a positive integer, Perform image acquisition for each of the N regions by means of a preset image acquisition model, and obtain N original images corresponding to the N regions, Perform vehicle part identification for each of the N original images to obtain a vehicle part position identification result, Based on a preset cutting model, cut each of the N original images into M sub-images of a predetermined size, where M is a positive integer, Perform damage identification for each of the N original images and the corresponding M sub-images, and obtain a damage identification result, Fuse the vehicle part position identification result and the damage identification result to obtain a vehicle part damage result of the target vehicle, A vehicle damage identification method for identifying damage to a target vehicle, characterized by including the above steps.

2. The image acquisition model is configured to perform image acquisition for each of the N regions of the target vehicle at a preset shooting angle so as to obtain the N original images, each of which has an aspect ratio of a:b. The method according to claim 1.

3. The cutting model is configured to divide each of the N original images into a equal parts in the horizontal direction and b equal parts in the vertical direction in each of the N original images, so as to obtain a×b sub-images, where a×b = M. The method according to claim 2.

4. Performing damage identification for each of the N original images and the corresponding M sub-images, and obtaining a damage identification result includes: Performing damage identification for each of the N original images to obtain an overall damage identification result for each original image, Performing damage identification for each of the M sub-images in each original image to obtain a local damage identification result, Based on the position of each of the M sub-images in the corresponding original image, performing coordinate transformation on the local damage identification result to transform the coordinates of the local damage identification result from the coordinates in the sub-image to the coordinates in the corresponding original image, and obtaining a transformed local damage identification result, Fusing the transformed local damage identification result and the overall damage identification result to obtain the damage identification result. The method according to any one of claims 1 to 3, characterized by including

5. The method according to claim 4, characterized by including outputting and displaying the vehicle component damage result.

6. Partitioning the entire external surface of the target vehicle into a predetermined N regions, The method according to claim 5, characterized by including partitioning the entire external surface of the target vehicle into 14 regions, and these 14 regions include the upper front side, lower front side, left front part, right front part, left front side part, right front side part, left middle part, right middle part, left rear part, right rear part, left rear side, right rear side, upper rear side and lower rear side of the target vehicle.

7. A partitioning module for partitioning the entire external surface of a target vehicle into a predetermined N regions, where N is a positive integer, the partitioning module; An original image acquisition module for respectively acquiring images for each of the N regions by using a preset image acquisition model, and acquiring N original images corresponding to the N regions; A vehicle component position identification module for performing vehicle component identification on each of the N original images and acquiring a vehicle component position identification result; An original image cutting module for cutting each of the N original images into M sub-images of a predetermined size based on a preset cutting model, where M is a positive integer, the original image cutting module; A damage identification module for performing damage identification on each of the N original images and the corresponding M sub-images respectively and acquiring a damage identification result; A vehicle component damage fusion module for fusing the vehicle component position identification result and the damage identification result to obtain a vehicle component damage result of the target vehicle; A vehicle damage identification device for identifying damage to a target vehicle, characterized by including

8. The device according to claim 7, characterized in that the image acquisition model acquires the N original images each having an aspect ratio of a:b by respectively performing image acquisition on the N regions of the target vehicle at a preset shooting angle.

9. The device according to claim 8, characterized in that the cutting model horizontally divides the original image into a equal parts and vertically divides the original image into b equal parts in each of the N original images to obtain a×b sub-images, where a×b = M.

10. The damage identification module is An overall damage identification unit that performs damage identification on each of the N original images and obtains an overall damage identification result for each of the original images; A local damage identification unit that performs damage identification on each of the M sub-images in each of the original images and obtains a local damage identification result; A coordinate conversion unit that performs coordinate conversion on the local damage identification result by converting the coordinates of the local damage identification result from the coordinates in the sub-image to the coordinates in the corresponding original image based on the position in the corresponding original image of each of the M sub-images, thereby obtaining a converted local damage identification result; A damage fusion unit that fuses the converted local damage identification result and the overall damage identification result to obtain the damage identification result; The apparatus according to any one of claims 7 to 9, characterized by including the above.

11. The apparatus according to claim 10, characterized by including a result output module that outputs and displays a vehicle part damage result.

12. The partitioning module partitions the entire outer surface of the target vehicle into 14 regions, and these 14 regions include the upper front part, lower front part, left front part, right front part, left front side part, right front side part, left middle part, right middle part, left rear part, right rear part, left rear part, right rear part, upper rear part, and lower rear part of the target vehicle. The apparatus according to claim 11, characterized by this.

13. A memory storing a computer program; A processor that executes the computer program to execute the steps of the method according to any one of claims 1 to 6; An imaging device for acquiring an image; A display device for displaying; An electronic device, characterized by including the above.

14. A computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, causing the steps of the method according to any one of claims 1 to 6 to be executed. Characterized by this.

Images