[0056] In order to make the objects, technical solutions, and advantages of the present application, the technical solutions in the present application embodiment will be described in connext of the present application embodiment, and It is a part of the present application, not all of the embodiments. Based on the embodiments in the present application, one of ordinary skill in the art is in the scope of the claims of the present application without creative labor premistence.
[0057] figure 1 A flowchart of the image enhancement method. The method is accomplished by a reinforcing device of the vehicle damage image, including component division modules, reflective, and shaded detection modules, reflective removal modules, and shaded removal modules.
[0058] The enhancement method of the image specifically includes the steps of:
[0059] Step 1, get the vehicle image to be loaded;
[0060] Step 2, partition the various components of the vehicle from the vehicle image to be determined, and the component region of the interested component is determined from the respective components of the divided vehicle to be used as an area interested in the AI loss link;
[0061] This step is implemented by the component segmentation module. Figure 2A The first part of the vehicle appearance component is schematically diagram; Figure 2b A second part of the vehicle appearance component divided into the area. In the present invention, the exterior parts of the vehicle are divided into 59 regions, and 59 part areas are classified as follows:
[0062] Front bumper skin, post bumper skin, front leaf plate (right), front leaf plate (left), medium network, front door shell (right), front door shell (left), back door shell (right), rear door shell (left) , Rear leaf plate (right), rear leaf plate (left), front windshield, rear windshield, reverse mirror (right), reverse mirror (left), hood, luggage cover, front headlight (right), before Badlight (left), internal light (right), end light (left), tail light (right), tail light (left), taillight (right), taillight (left), bottom large edge (right), bottom large side (Left), steel ring, lift door shell, lift door glass; front bumper skin, post bumper skin, front leaf plate (right), front leaf plate (left), China network, front door shell (right), Front door shell (left), rear door shell (right), rear door shell (left), rear leaf plate (right), rear leaf plate (left), front windshield, rear windshield, reverse mirror (right), reverse mirror (left ), Hood, luggage cover, front headlight (right), front headlight (left), internal light (right), internal tail light (left), tail light (right), tail light (left), taillight (right ), Taillight (left), bottom large side (right), bottom large edge (left), steel ring, lift door shell, lift door glass.
[0063] The purpose of dividing the appearance of the vehicle into 59 regions is mainly in the region where the AI fixed loss link is interested in the image. The present invention selects 30 of the part of interest in 59 regions. Specific as follows:
[0064] Front bumper skin, post bumper skin, front leaf plate (right), front leaf plate (left), medium network, front door shell (right), front door shell (left), back door shell (right), rear door shell (left) , Rear leaf plate (right), rear leaf plate (left), front windshield, rear windshield, reverse mirror (right), reverse mirror (left), hood, luggage cover, front headlight (right), before Badlight (left), internal light (right), end light (left), tail light (right), tail light (left), taillight (right), taillight (left), bottom large edge (right), bottom large side (Left), steel ring, lift door shell, lift door glass.
[0065] The above 30-mentioned part of the component area is interested in the region of the AI loss link. On only 30 images of the component segmentation module in the processing of the subsequent modules are processed.
[0066] Among them, an instance division image is divided by an instance segmentation algorithm mASK-RCNN method to divide various components of the vehicle. Mask-rcnn is a very flexible framework that can increase different branches to complete different tasks, can complete the target classification, target detection, semantic split, instance segmentation, human gesture identification.
[0067] MASK-RCNN is a framework or FASTER-RCNN framework, which can be said to have added a full-connection division subnet after the basic feature network, which is changed from the original task (classification + regression) for three tasks (classification + return + segmentation). MASKR-CNN uses two stages of FASTERR-CNN: The first stage has the same first layer (ie, RPN), scanned images and generates proposals (Proposals, possibly containing a target); second stage In addition to predicting types and bbox regression, a full-volume network branch is added, and each ROI predicts the corresponding binarymask to illustrate whether the given pixel is part of the target. The so-called binary MASK is that when the pixel belongs to all positions of the target, the identifier is 1, and the other position identifies is 0. The detailed process of the MASK-RCNN method is not the core content of the present invention, which is not described in detail here.
[0068] image 3 A schematic diagram of the effect of dividing the various components of the vehicle. In this figure, MASK of the various components of the vehicle is obtained, and only the area inside the MASK in the process of the subsequent reflective and shaded detection module and the treatment of reflective elimination shadows, because when the drawing is taken, the image image often complicated, The background area is also processed in the background area, which is easy to cause image changes in the background area.
[0069] Step 3, respectively perform a reflective region detection, a shaded region detection, and if there is a reflective area in the vehicle image to be determined, if there is a reflective area in the vehicle image to be determined, it is determined that the mASK of the reflective area is detected. The shadow area determines the mask of the shaded area; if the reflective area and the shadow area are not detected, step 5 is performed;
[0070] Among them, in step 3, the reflective area detection, the shadow area detection, includes:
[0071] The Mask-RCNN method is used to semantically divide the image, and the Mask-RCNN adds a semantic division branch to detect the reflective area and the shaded area of the image;
[0072] This step is achieved by the reflective and shaded area detection modules that give Mask of the Mask and the shadow area of the reflective region, which is used for subsequent reflective removal modules and shaded removal modules. Figure 4A-1 , Figure 4A-2 A schematic diagram of the detection results of the reflective region and the shaded area of the first image, respectively, Figure 4B-1 , Figure 4B-2 Schematic diagram of the detection result of the reflective area and shaded area of the second image, wherein Figure 4A-1 , Figure 4B-1 The detection result of the reflective region; Figure 4A-2 , Figure 4B-2 The results of the shadow area.
[0073] Step 4, if there is a reflective region in the vehicle image to be determined, then according to the determined MASK and the vehicle image to be fixed, the image to be determined after the reflectance is obtained, the image after the reflectance is eliminated. Give the AI losaged server for the backend;
[0074] If there is a shaded area in the vehicle image to be determined, the image to be determined after the shadow is obtained according to the Mask of the shaded region and the vehicle image to be fixed, and the image after the shadow is obtained. End AI losaged server;
[0075] The elimination of reflective processing is realized by reflective removal modules, and the reflective removal flow schematic Figure 5 As shown, the specific steps are as follows:
[0076] Step 401, using a multi-scale Laplas Convolutionary Laplace Kernel to perform characteristic extraction, to obtain Laplaste;
[0077] Step 402, using a normal volume nuclear module to perform feature extraction of the vehicle image to be loaded to obtain a vehicle image characteristic to be lossed;
[0078] Step 403, extracting and transmitting suppression processing of reflection characteristics, respectively, respectively, respectively, to obtain reflection probability And after suppressing Laplaste
[0079] Step 404, according to the vehicle image characteristics to be lossed, reflecting probability And after the suppressed Laplaste is reflecting reflection;
[0080] Step 405, use 1 minus reflection probability map To obtain a transmissive probability map;
[0081] Step 406, the vehicle image, reflection and transmission probability map to be loses, using the context-based automatic encoder CONTEXTUAL AutoEncoder to obtain an image after the reflective image;
[0082] Among them, reflection probability map For a single channel, the value of each pixel point is 0-1, which indicates the probability of reflected light in the pixel position. The pixel value of this point indicates the probability of light received in the pixel position;
[0083] The Contextual AutoEncoder section contains 6 CONV + RELU + CBAM structures and 6 DCONV + RELU structures, each of the reduction samples and the rear on the resolution of the resolution, and the output of the ContextualautoEncoder is the image after the reflective image. .
[0084]Among them, CONV + RELU is a way of extracting image features by filtering, and CBAM is a focus module for focusing the attention of neural networks primarily in the target area. DCONV + RELU is a reverse convezol module, which is aimed at raising the current image, such as The figure, after a step of 2 anti-convolution modules will become Diagram;
[0085] During the processing of the Contextual AutoEncoder section, a picture will change the size of the CONV + RELU + CBAM module to the original 1/2, the structure is 6 conv + relu + CBAM, each passing into The original 1/2 size, there are 6 DCONV + RELU structures after each DCONV + RELU, the size will become 2 times the original, so the result and the original Figure one Sample size, but the intermediate will appear 2 times 1/2, 4/1 ... 1/32, the front and back size is the same, the hopping is connected, and the hopping is connected to the sample. / 2 diagram of the size, spliced to 1/2 of the upper sampled.
[0086] The Contextual AutoEncoder module contains two phases of the neural network, and there is two phases of the training phase.
[0087] 1) Training stage: As an input as an image, each image is collected by a mobile camera and an optical lens (light that can be filtered out of the specular reflection), including a picture containing the reflective region and the diagram of the reflective region, with The diagram of the reflective area is an input of the neural network, and an image without a reflective area as a training label of the neural network, and the loss of training is between corresponding pixels. loss in Representing image area, Represents pixels in the label image, The image after the anti-reflectance of the neural network output. The neural network is optimized by random gradient drop or other methods until the loss is less than the specified value, and the weight at this time is saved for its optimal weight.
[0088] 2) Innerness phase: The best weight obtained by the training stage is used as the initial weight of the neural network, and inputs an image with a reflective area, and the reason is to eliminate the image after the reflective region.
[0089] In step 403, further comprising:
[0090] Step 4031, the vehicle image to be loaded is scaled to 1, 1/2, 1/4, 1/8 size of the original size, respectively, using the Laplas Convolutionary Laplace Kernel for feature extraction, and then perform according to the session 0, 1, 2, 4 times to sample the size of the vehicle image to be fixed to obtain a vehicle image after each sample;
[0091] Step 4032, the above-described vehicle image is spliced at the channel layer, and then the three SE (SQUEEZE-AND-EXCITATION) modules are subsequently extracted, and the reflection feature after the extracted reflection is then connected to a convolution layer and Sigmoid layer gets reflection probability map;
[0092] Step 4033, after the extracted reflection features, then 3 SE modules, to suppress the Laplaste of the transmission region;
[0093] Step 4034, by reflection probability And after the suppressed Laplaste, Laplaste for the final reflection portion;
[0094] Step 4035, using four SE modules to extract the characteristics of the vehicle image to be lossed , Pull the reflective part of Laplaste And the characteristics of the vehicle image to be fixed Stitching at the channel layer, then get a reflection reflection by LSTM + CONV + RELU;
[0095] Among them, "LSTM + CONV + RELU" is a common connection method of neural network. Reflective features.
[0096] The elimination shadow processing in this step is realized by the shadow removal module, and the flow of the shadow removal is Image 6 As shown, the specific steps are as follows:
[0097] Step 401 ', the vehicle image to be losaged and the MASK of the shaded region is dotted at the pixel layer to obtain a picture of the shaded area;
[0098] Wherein, the size of the shaded area is the same as the vehicle image to be loaded, but the pixels other than the shadow area are all 0 values;
[0099] Step 402 ', by the shadow removal of the network to repair the figure of the shaded area to obtain a shaded area diagram after compensation;
[0100] The shadow removing the network adopts the UNET structure, including 6 CONV + RELU + CBAM structure and 6 DCONV + RELU structures, and there is a hopping between each mining sample layer and the rear on the resolution of the resolution. The illumination of the shaded area has been compensated;
[0101] Step 403 ', at the channel layer spliced the MASK of the shadow area, the vehicle image to be fixed, and the compensated shaded area map, through the full graph repair network to obtain a complete elimination of shadow;
[0102] Wherein, the grid repair network contains three SE modules.
[0103] The shadow removal module includes two phases of the neural network, three stages of training and reasoning, the input of the training stage is a three-way group consisting of 3 maps (including a shadow, Mask map, excluding shadow Figure), these samples can be fixed with a fixed camera, to obtain a picture containing the shadow, remove the shadow object to acquire the shadow, and label the Mask map containing the shadow area, can also be trained A large amount of shaded image is generated in the formation of a large amount of shadow image.
[0104] The loss of the training stage is divided into two parts, part of the shadow removal of the network loss, using the corresponding pixel in the shadow area Loss ,in Represents the scope of the shadow area, Represents pixels in the label image, On behalf of the image of the elimination shadow of the neural network, the other part is the loss of the entire map repair network, using the whole picture corresponding pixel loss ,in Except for the entire map area Outside the area, Represents pixels in the label image, In order to repair the output of the network, the loss of the entire module is: in with Represents the weight of the two parts, 1 and 0.5, respectively. The neural network is trained using a random gradient drop or other optimized method until it is less than the specified value, and the weight at this time is the best weight.
[0105] The best weight obtained by the training stage is used as the initial weight of the neural network, input to an image with a shaded area and a MASK obtained from the second stage, and the result is an image after the shaded area.
[0106] Step 5, the AI losaged server of the rear end performs the damage processing.
[0107] The rear end AI losaged server eliminates the image after the image after the reflective image is eliminated, and the vehicle image to be loses is fixed.
[0108] Alternatively, the component splitting module and the reflective and shaded region detecting module use the MASK-RCNN method to perform example segmentation, and the object is to divide MASK and the reflective and shaded area of the respective components. MASK-RCNN is only One of the existing instance segmentation methods, there are also some examples of example segmentation, such as FASTER-RCNN, BlendMask, and other methods, can also achieve approximation in the present invention.
[0109] In the process of acquiring the reflection map in the reflective removing module, the reflection feature is extracted, and then the transmission characteristic is suppressed to inhibit the characteristics of the final combination of the original map, the order of the two processes can be adjusted during this process, not Influence effect, the scheme of multi-scale Laplas special nuclear volume can be exchanged to a single scale, and the final effect may be a bit. In the shadow elimination module, the shadow removal of the network and then connected a full map repair network, which has limited impact on the entire module.
[0110] In summary, the method of the present invention is directed to the two cases in the vehicle loss image, first dividing the various component regions in the vehicle appearance, and fixes the reflective shaded area in the component region, whether it is subsequent AI or The accuracy of the artificial nuclear loss will be greatly improved, which can save the insurance company to effectively improve the accuracy of the AI loss link.
[0111] It is apparent to those skilled in the art that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention without departing from the spirit or essential characteristics of the present invention. Therefore, in any case, it should be regarded as an exemplary, and non-limiting, and the scope of the invention is defined by the appended claims. Therefore, it is intended to fall in accordance with the claims All changes in the meaning and range of equivalents are covered in the present invention. Any reference numerals in the claims should not be regarded as the claims. In addition, it is apparent that the term "includes" does not exclude other units or steps, and the single number does not exclude the plural. The plurality of units or means described in the device claim can also be implemented by software or hardware by a unit or device. First, the second et al. Is used to represent the name, and does not represent any particular order.
