Defect detection method and device applied to vehicle chassis

Abstract

The embodiment of the invention provides a defect detection method and device applied to a vehicle chassis, and the method comprises the steps: obtaining a plurality of local images of the vehicle chassis, and enabling different local images to represent images shot at different points of the vehicle chassis; combining the plurality of local images according to the feature points in the plurality of local images to obtain an overall image of the vehicle chassis; segmenting an area including a detection object from the overall image through a target neural network to serve as a detection image; and detecting the defect of the detection object based on the detection image. According to the technical scheme provided by the embodiment of the invention, the accuracy of detecting the defects of the detected object on the vehicle chassis can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of component defect identification, and in particular, to a defect detection method and device applied to a vehicle chassis. Background Art

[0002] With the rapid development of automobile technology, using cars as a means of transportation has become one of the most commonly used modes of travel.

[0003] In the related art, during the manufacturing of a vehicle, defects may occur in the vehicle components due to the manufacturing process used, which may make the subsequent loading and unloading process impossible to continue or reduce the overall safety factor of the vehicle. For example, during the welding process of the vehicle chassis, the sparks generated during welding may induce abnormal defects such as deformation and skewness of the surrounding bolts. In order to detect defects in vehicle components, the overall image of the vehicle is usually captured by a camera, and then the defects of each component on the vehicle are detected and searched based on the overall image.

[0004] However, due to the large overall size of the vehicle, in order to capture the entire image of the vehicle, the relative distance between the shooting equipment and the vehicle is extended, which causes the image details of smaller parts on the vehicle to be easily lost, resulting in a decrease in the accuracy of component defect recognition. Summary of the invention

[0005] In order to solve the above technical problems, the embodiments of the present application provide a defect detection method and device for a vehicle chassis.

[0006] According to one aspect of an embodiment of the present application, a defect detection method applied to a vehicle chassis is provided, comprising: acquiring multiple local images of a vehicle chassis, wherein different local images represent images taken at different points on the vehicle chassis; merging the multiple local images according to feature points in the multiple local images to obtain an overall image of the vehicle chassis; segmenting a region including a detection object from the overall image as a detection image through a target neural network; and detecting defects of the detection object based on the detection image.

[0007] According to one aspect of an embodiment of the present application, there is provided a defect detection device for a vehicle chassis, comprising: a shooting module, configured to acquire a plurality of local images of a vehicle chassis, wherein different local images represent images captured at different points of the vehicle chassis; a merging module, configured to merge the plurality of local images according to feature points in the plurality of local images to obtain an overall image of the vehicle chassis; a segmentation module, configured to segment a region including a detection object from the overall image as a detection image through a target neural network; and a detection module, configured to detect defects of the detection object based on the detection image.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the merging module is further configured to: determine a local image to be processed from the multiple local images before merging the multiple local images according to feature points in the multiple local images; obtain lighting information corresponding to the local image to be processed; determine a target lighting balance scheme based on the relationship between the lighting information and preset lighting information; and adjust the lighting information of the local image to be processed through the target lighting balance scheme.

[0009] In some embodiments of the present application, based on the aforementioned scheme, the merging module is further configured to: after determining the target illumination balance scheme based on the relationship between the illumination information and the preset illumination information, determine the position information of the detection object in the local image to be processed; update the target illumination balance scheme according to the position information, so as to adjust the illumination information of the local image to be processed through the updated target illumination balance scheme.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the merging module is further configured to: obtain feature descriptions corresponding to each feature point in the multiple local images; determine similar feature points in the multiple local images based on the feature description of each feature point; and merge the multiple local images based on similar feature points in the multiple local images.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the detection module is further configured to: before merging the multiple local images according to the feature points in the multiple local images, obtain the landmark components in the multiple local images; detect the landmark feature points corresponding to the landmark components in each of the local images; and use the landmark feature points as the feature points in the multiple local images.

[0012] In some embodiments of the present application, based on the aforementioned scheme, the detection module is further configured to: extract feature information of the detection object in the detection image according to multiple dimensions; input the feature information of the multiple dimensions into the defect recognition model; obtain the defect prediction result output by the defect recognition model based on the feature information of the multiple dimensions, and use it as the defect detection result of the detection object.

[0013] In some embodiments of the present application, based on the aforementioned scheme, the segmentation module is further configured to: before segmenting the area including the detection object from the overall image as the detection image through the target neural network, determine the associated neural network from a preset neural network library according to at least one of the parameter information of the detection object and the model information of the vehicle; and use the determined neural network as the target neural network.

[0014] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the defect detection method applied to a vehicle chassis as described in the above embodiment.

[0015] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the defect detection method applied to a vehicle chassis as described in the above-mentioned embodiments.

[0016] In the technical solution of the embodiment of the present application, multiple partial images of the vehicle chassis are obtained to shorten the relative distance between the shooting device and the vehicle so that the details of the vehicle chassis in the image are not easily lost, and then the multiple partial images are merged according to the feature points in the multiple partial images to obtain the overall image of the vehicle chassis, and then the area including the detection object is segmented from the overall image as the detection image through the target neural network, so as to detect the defects of the detection object based on the detection image, thereby improving the clarity of the detection object in the overall image of the vehicle chassis through the details in the partial images, avoiding distortion of the detection object, and thus ensuring that the accuracy of detecting defects in the detection object is not easily reduced. In addition, the overall image obtained by merging the feature points in the multiple partial images can also avoid the problem of repeated detection of the same detection object on the vehicle chassis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a flow chart of a defect detection method applied to a vehicle chassis shown in an exemplary embodiment of the present application;

[0019] Figure 2 yes Figure 1 A flowchart in an exemplary embodiment before step S120 in the illustrated embodiment;

[0020] Figure 3 yes Figure 2 A flowchart in an exemplary embodiment after step S230 in the illustrated embodiment;

[0021] Figure 4 yes Figure 1 The flowchart of step S120 in the illustrated embodiment in an exemplary embodiment;

[0022] Figure 5 yes Figure 1 A flowchart in an exemplary embodiment before step S120 in the illustrated embodiment;

[0023] Figure 6 is a block diagram of a defect detection device applied to a vehicle chassis shown in an exemplary embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0026] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0029] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0030] The technical solution of the embodiment of the present application proposes a defect detection method for a vehicle chassis, which is specifically referred to as Figure 1 As shown. The method can be executed by a control unit in a defect detection device. The method at least includes steps S110 to S140, which are described in detail as follows:

[0031] In step S110 , a plurality of partial images of the vehicle chassis are acquired.

[0032] In an embodiment of the present application, in order to identify defects in a vehicle chassis, multiple local images of the vehicle chassis can be acquired first, wherein different local images represent images taken at different points on the vehicle chassis, so that when the vehicle chassis is large and it is inconvenient for inspection personnel to photograph the overall state of the vehicle chassis, image information that can map the overall state of the vehicle chassis can be acquired.

[0033] For example, the vehicle chassis is photographed from the southeast, northwest, and northeast directions of the vehicle respectively, and correspondingly, four partial images of the vehicle chassis are obtained, that is, images representing the images photographed at the southeast, northwest, and northeast points of the vehicle chassis.

[0034] In addition, after acquiring multiple local images of the vehicle chassis, in an embodiment of the present application, image denoising may be performed on the multiple local images to reduce the impact of image noise on the state of the vehicle chassis represented in the image.

[0035] For example, the pixel value of each pixel in the image can be adjusted by Gaussian weighting. Secondly, the current pixel value to be denoised can be further adjusted by the local pixel mean and variance to avoid distortion of the current pixel value to be denoised. At the same time, the edge adaptability of the filter can be adjusted in combination with the local gradient features in the image to improve the degree of refinement in the process of processing the edge area of ​​the image to ensure that the edge area of ​​the current local image to be denoised is not easily distorted. The specific calculation method can refer to the following formula:

[0036]

[0037] Wherein, I'(x) is the pixel value corresponding to the pixel point with pixel coordinate x in the current local image to be denoised after denoising; I(x') is the pixel value corresponding to the pixel point with pixel coordinate x' in the current local image to be denoised before denoising; Ω is the filtering neighborhood corresponding to the pixel point with pixel coordinate x in the current local image to be denoised; N is the total number of pixels in the filtering neighborhood Ω; is the gradient value corresponding to the pixel point with pixel coordinate x in the current local image to be denoised; I(x) is the pixel value corresponding to the pixel point with pixel coordinate x in the current local image to be denoised before denoising.

[0038] In step S120, the multiple local images are merged according to the feature points in the multiple local images to obtain an overall image of the vehicle chassis.

[0039] In an embodiment of the present application, after acquiring multiple local images of the vehicle, the multiple local images can be merged according to feature points in the multiple local images to obtain an overall image of the vehicle chassis, thereby removing repeated parts in the multiple local images and avoiding repeated identification of defects in partial areas of the vehicle chassis.

[0040] Among them, the feature points in the multiple local images include but are not limited to Harris corner points, extreme points, SIFT (Scale-invariant feature transform) feature points, FAST (Features from Accelerated Segment Test) corner points, etc.

[0041] The method of merging multiple partial images according to feature points in the multiple partial images can be flexibly set as needed. In one example, common feature points can be determined from feature points in the multiple partial images, wherein the common feature points are feature points contained in each partial image, and then the multiple partial images are merged based on the common feature points in the multiple partial images to reduce the repeated parts in the partial images, thereby obtaining an overall image of the vehicle chassis.

[0042] In another example, common components in multiple local images can be determined first, wherein the common components are vehicle chassis components contained in each local image, and then feature points corresponding to the common components are determined from feature points in the multiple local images. Then, the multiple local images are merged based on the feature points corresponding to the common components in each local image, so as to reduce the processing amount required to obtain an overall image of the vehicle chassis based on the merging of feature points in multiple local images while ensuring that the overall image of the merged vehicle chassis is not easily distorted.

[0043] In addition, in the process of merging multiple local images based on feature points in the multiple local images, some embodiments of the present application can use a bilinear interpolation method to perform interpolation processing based on the feature points to obtain a merged overall image of the vehicle chassis, and use the Poisson equation to smooth the fused and merged overall image of the vehicle chassis, so as to achieve the purpose of seamlessly merging multiple local images into an overall image of the vehicle chassis.

[0044] In step S130, a region including the detection object is segmented from the overall image by a target neural network as a detection image.

[0045] In an embodiment of the present application, after obtaining an overall image of the vehicle chassis, the area including the detection object can be segmented from the overall image through the target neural network as a detection image, so as to eliminate the area in the overall image that does not contain the current detection object, thereby avoiding interference when identifying defects of the current detection object on the vehicle chassis.

[0046] The inspection objects may include components on the vehicle chassis such as bolts, exhaust, suspension, etc. The specific inspection objects can be flexibly adjusted according to the needs of the inspectors and are not limited here.

[0047] Secondly, the neural network may include a UNeXt network, a residual network, a U-Net network or other networks for segmenting images, and the target neural network for determining the method of segmenting the area including the detection object in the overall image can be flexibly set as needed. In the implementation of the present application, considering that the chassis complexity of different vehicle models is different, and the lower the chassis complexity of the vehicle chassis, the lower the difficulty in segmenting the area including the detection object, and correspondingly, the recognition accuracy required by the neural network in order to achieve the purpose of segmenting the area including the detection object from the overall image is also lower. In addition, considering that the size and clarity of different components in the vehicle chassis are different, and the smaller the shape of the component corresponding to the detection object or the lower the clarity, the higher the difficulty in segmenting the area including the detection object from the overall image, and correspondingly, the recognition accuracy required by the neural network in order to achieve the purpose of segmenting the area including the detection object in the overall image is also higher. Based on this, the associated neural network can be determined from the preset neural network library according to at least one of the parameter information of the detection object and the model information of the vehicle, so as to use the determined neural network as the target neural network, thereby shortening the time consumed in determining the detection image from the overall image while ensuring that the recognition accuracy of the determined target neural network can segment the area including the detection object from the overall image of the vehicle chassis. Among them, the parameter information of the detection object includes but is not limited to the name, color and shape parameters of the detection object; the model information of the vehicle includes but is not limited to the name, product serial number and vehicle identification code of the vehicle.

[0048] For example, when the vehicle model information or parameter information of the detection object is obtained, the neural network associated with the obtained information can be determined from the preset neural network library, and the determined neural network can be used as the target neural network, thereby ensuring that the recognition accuracy of the determined target neural network is able to segment the area including the detection object from the overall image of the vehicle chassis.

[0049] When the vehicle model information and parameter information of the detection object are obtained, the neural network associated with the model information and parameter information can be determined from the preset neural network library, and the determined neural network can be used as the target neural network, thereby further ensuring that the recognition accuracy of the determined neural network is capable of segmenting the area including the detection object in the overall image.

[0050] In step S140 , defects of the inspection object are inspected based on the inspection image.

[0051] In an embodiment of the present application, after the target neural network segments the area including the detection object from the overall image and makes it a detection image, defects of the detection object can be detected based on the detection image to obtain defect conditions of the detection object on the vehicle chassis.

[0052] Among them, the method of detecting defects of the inspection object based on the inspection image can be flexibly set according to needs. In one example, the feature information of the inspection object in the inspection image can be extracted according to multiple dimensions, and then the feature information of multiple dimensions can be input into the defect recognition model. Thereafter, the defect prediction result output by the defect recognition model based on the feature information of multiple dimensions is obtained and used as the defect detection result of the inspection object, so as to achieve the purpose of obtaining the defect situation of the inspection object on the vehicle chassis.

[0053] The multiple dimensions include but are not limited to color, geometric shape, texture, etc. Correspondingly, the feature information of the detection object in multiple dimensions is the color information, geometric shape information, texture information, etc. of the detection object. The defect recognition model includes but is not limited to a multi-classification support vector machine model, a random forest model or other models based on multi-dimensional feature output prediction results.

[0054] For example, when the inspection object is a bolt and the multiple dimensions include color, geometric shape and texture, the feature information of the bolt in the inspection image can be extracted separately according to the multiple dimensions, that is, the color information, geometric shape information and texture information of the bolt in the inspection image can be extracted, and then the color information, geometric shape information and texture information can be input into the defect recognition model, so that the defect recognition model outputs a defect prediction result as the defect detection result of the inspection object based on the input color information, geometric shape information and texture information of the inspection object.

[0055] Among them, the training process of the defect recognition model can be changed accordingly according to the different detection objects to ensure that the defect recognition model has a high prediction accuracy.

[0056] In another example, feature information of the detection object in the detection image can be extracted according to multiple dimensions respectively, and then a feature level group is determined based on the extracted feature information of the multiple dimensions, wherein the feature level group includes the feature level corresponding to the feature information of the detection object in each dimension, and an associated defect recognition model is determined from a preset defect recognition model library according to the feature level group, and the determined defect recognition model is used as a target defect recognition model, and then the feature information of the multiple dimensions is input into the target defect recognition model, and then the defect prediction result output by the target defect recognition model based on the feature information of the multiple dimensions is obtained and used as the defect detection result of the detection object, so as to ensure that the defect situation of the detection object on the vehicle chassis is obtained, and then the best defect recognition model can be determined according to the extracted feature information of the multiple dimensions, thereby improving the detection accuracy of the detection object.

[0057] Referring to the above example, when the inspection object is a bolt, and multiple dimensions include color, geometric shape and texture, and the color information, geometric shape information and texture information of the bolt in the inspection image have been extracted according to multiple dimensions, if the feature level of the color information of the bolt is 3, the feature level of the geometric shape information is 5, and the level of the texture information is 4, then the feature level group determined based on the extracted feature information of multiple dimensions is (3, 5, 4). Correspondingly, in the process of determining the associated defect recognition model from the preset defect recognition model library according to the feature level group, a defect recognition model with a color recognition capability of level 3, a geometric shape recognition capability of level 5, and a texture recognition capability of level 4 can be determined from the preset defect recognition model library as the target defect recognition model to ensure that the determined target defect recognition model can output defect prediction results with higher accuracy based on the feature information of multiple dimensions.

[0058] Through the above implementation, multiple partial images of the vehicle chassis are obtained to shorten the relative distance between the shooting device and the vehicle, so that the details of the vehicle chassis in the image are not easily lost, and then the multiple partial images are merged according to the feature points in the multiple partial images to obtain the overall image of the vehicle chassis, and then the area including the detection object is segmented from the overall image as the detection image through the target neural network, so as to detect the defects of the detection object based on the detection image, thereby improving the clarity of the detection object in the overall image of the vehicle chassis through the details in the partial image, avoiding distortion of the detection object, and thus ensuring that the accuracy of the detection of defects in the detection object is not easily reduced. In addition, the overall image obtained by merging the feature points in the multiple partial images can also avoid the problem of repeated detection of the same detection object on the vehicle chassis.

[0059] See also Figure 2 , Figure 2FIG. 1 is a defect detection method for a vehicle chassis according to another exemplary embodiment. Figure 2 As shown, in Figure 1 Before step S120 in the illustrated embodiment, the method may further include steps S210 to S240, which are described in detail as follows:

[0060] In step S210, a partial image to be processed is determined from a plurality of partial images.

[0061] First of all, it should be explained that, considering that different local images are taken at different points on the vehicle chassis, and during the image shooting process, the lighting direction and lighting intensity in the image are bound to change due to the switching of the shooting points, which leads to the lighting direction and lighting intensity corresponding to the subsequent overall image obtained based on the merging of multiple local images being relatively chaotic, and the lighting intensity of some areas in the overall image cannot reach the lighting intensity required for segmenting the inspection image containing the current inspection object and detecting the defects of the current inspection object, which increases the difficulty of segmenting the area including the inspection object from the overall image and the difficulty of detecting the defects of the inspection object.

[0062] In an embodiment of the present application, after acquiring multiple local images of the vehicle chassis, a local image to be processed can be determined from the multiple local images, wherein the local image to be processed is the local image of the multiple local images whose lighting information needs to be adjusted.

[0063] The method of determining the local image to be processed from multiple local images can be flexibly set as needed. In one example, the image numbers corresponding to the multiple local images can be obtained first, where the image numbers represent the shooting order of the current local images, and the local images with image numbers greater than 1 are taken as the local images to be processed.

[0064] In another example, the illumination intensity corresponding to each partial image may be obtained first, and the partial images other than the partial image with the highest illumination intensity among the multiple partial images are used as the partial images to be processed.

[0065] In another example, after obtaining the light intensity corresponding to each local image, a preset light intensity can also be determined based on the inspection object, wherein the preset light intensity represents the light intensity required to segment the inspection image containing the current inspection object and detect defects of the current inspection object, and the similarity between the light intensity of each local image and the preset light intensity is obtained, and the local images among multiple local images whose similarity between the light intensity and the preset light intensity is lower than the preset similarity are used as local images to be processed.

[0066] In another example, the illumination direction corresponding to each local image can also be obtained, and then a preset illumination direction can be determined based on the detection object, wherein the preset illumination direction represents the illumination direction required for segmenting the detection image containing the current detection object and detecting defects of the current detection object, and other local images among multiple local images except for the local images having the same illumination direction as the preset illumination direction are used as local images to be processed.

[0067] In step S220, the illumination information corresponding to the local image to be processed is obtained.

[0068] In an embodiment of the present application, after determining a local image to be processed from a plurality of local images, illumination information corresponding to the local image to be processed may be obtained, wherein the illumination information includes but is not limited to illumination intensity, illumination direction, and the like.

[0069] In step S230, a target lighting balance solution is determined based on the relationship between the lighting information and the preset lighting information.

[0070] In an embodiment of the present application, in order to adjust the illumination information of the local image to be processed, a target illumination balance scheme can be first determined based on the relationship between the illumination information and the preset illumination information, wherein the target illumination balance scheme is an illumination balance scheme capable of adjusting the illumination information of the local image to be processed to the preset illumination information.

[0071] The preset illumination information represents the illumination information required for segmenting the inspection image containing the current inspection object and detecting defects of the current inspection object. The method for determining the preset illumination information can be flexibly set as needed. In one example, the associated preset illumination information can be directly obtained from a preset memory based on the current inspection object, that is, each inspection object can be pre-associated with the illumination information required for segmenting the inspection image containing each inspection object and detecting defects of each inspection object in the preset memory.

[0072] In another example, considering that the lighting information of any local image among the multiple local images except the local image to be processed can reach the lighting information required for segmenting the inspection image containing the current inspection object and detecting defects of the current inspection object, the lighting information of any local image among the multiple local images except the local image to be processed can be used as preset lighting information to ensure that the target lighting balance scheme can adjust the lighting information of the local image to be processed to the lighting information required for segmenting the inspection image containing the current inspection object and detecting defects of the current inspection object.

[0073] In step S240, the illumination information of the partial image to be processed is adjusted by a target illumination balance solution.

[0074] In an embodiment of the present application, after determining the target illumination balance scheme, the illumination information of the local image to be processed can be adjusted according to the target illumination balance scheme, so that the illumination information of each local image in the multiple local images can reach the illumination information required for segmenting the detection image containing the current detection object and detecting the defects of the current detection object. Correspondingly, the illumination information of the multiple local images can be unified, thereby reducing the probability of chaotic illumination direction and illumination intensity in the overall image subsequently obtained based on the merging of multiple local images, thereby reducing the difficulty of segmenting the area including the detection object from the overall image and the difficulty of detecting the defects of the detection object.

[0075] The method of adjusting the illumination information of the local image to be processed by the target illumination balance scheme can be flexibly set as needed. In some embodiments of the present application, the illumination information of each pixel in the local image to be processed can be directly adjusted based on the illumination information adjustment amplitude indicated by each pixel in the local image to be processed in the target illumination balance scheme, that is, the illumination information of each pixel in the local image to be processed is directly adjusted according to the corresponding illumination information adjustment amplitude, so as to achieve the purpose of adjusting the illumination information of the local image to be processed; or, the HSV (Hue, Saturation, Value; hue, saturation, brightness) color space model can be used for implementation, that is, the illumination information of the local image to be processed is separated by the HSV color space model, and then the separated illumination information is corrected for illumination balance based on the target illumination balance scheme by using histogram equalization and gamma correction method, so as to achieve the purpose of adjusting the illumination information of the local image to be processed; or, other illumination correction models can be used to adjust the illumination information of the local image to be processed based on the target illumination balance scheme, which is not limited here.

[0076] See also Figure 3 , Figure 3 FIG. 1 is a flow chart showing a defect detection method applied to a vehicle chassis according to another exemplary embodiment. Figure 3 As shown, in Figure 2 After step S230 in the illustrated embodiment, the method may further include steps S310 to S320, which are described in detail as follows:

[0077] In step S310, the position information of the detection object in the partial image to be processed is determined;

[0078] In an embodiment of the present application, in order to further reduce the difficulty of segmenting the area including the inspection object from the overall image and the difficulty of detecting defects of the inspection object, after determining the target illumination balance scheme based on the relationship between the illumination information and the preset illumination information, the position information of the inspection object in the local image to be processed can be first determined.

[0079] Among them, the method of determining the position information of the detection object in the local image to be processed can be flexibly set as needed. In some embodiments of the present application, the position information of the detection object in the local image to be processed can be identified and output through the YOLO (You Only Look Once) model; or, the position information of the detection object in the local image to be processed can be identified and output through a multimodal model; or, other models for image recognition can also be used to identify and output the position information of the detection object in the local image to be processed, which is not limited here.

[0080] In step S320, the target illumination balance solution is updated according to the position information.

[0081] In an embodiment of the present application, after determining the position information of the detection object in the local image to be processed, the target illumination balance scheme can be updated according to the position information, that is, the illumination information of the local image to be processed at the position information is further adjusted in the target illumination balance scheme, so as to adjust the illumination information of the local image to be processed through the updated target illumination balance scheme, so that the illumination information of the position of the detection object in the local image to be processed is not prone to the abnormal situation that the illumination intensity cannot reach the illumination intensity required for segmenting the detection image containing the current detection object and detecting the defects of the current detection object after the adjustment based on the target illumination balance scheme, thereby improving the clarity of the detection object and achieving the purpose of further reducing the difficulty of segmenting the area including the detection object from the overall image and the difficulty of detecting the defects of the detection object.

[0082] See also Figure 4 , Figure 4 is Figure 1 The flowchart of step S120 in the embodiment shown is in an exemplary embodiment. Figure 4 As shown, the process of merging multiple local images according to feature points in the multiple local images may include steps S410 to S430, which are described in detail as follows:

[0083] In step S410, feature descriptions corresponding to feature points in a plurality of local images are obtained.

[0084] In the implementation of the present application, in order to merge multiple local images to obtain the overall image of the vehicle chassis, the feature descriptions corresponding to the feature points in the multiple local images can be obtained first. The feature description of the feature point includes but is not limited to the pixel value of the current feature point, the Euclidean distance between the current feature point and other surrounding feature points, cosine similarity, Gaussian distance, Gaussian kernel distance, or the gradient direction, grayscale change, covariance matrix, etc. of the neighborhood where the current feature point is located.

[0085] In step S420, similar feature points in a plurality of local images are determined according to the feature description of each feature point.

[0086] In an embodiment of the present application, after obtaining the feature descriptions corresponding to the feature points in multiple local images, similar feature points in the multiple local images can be determined based on the feature descriptions of each feature point, that is, similar points in the multiple local images are determined.

[0087] Among them, the method of determining similar feature points in multiple local images based on the feature description of each feature point, in some embodiments of the present application, the similarity between each feature point in each local image and each feature point in other local images can be determined based on the feature description, and the similarity is the similarity between the feature description of each feature point in each local image and the feature description of each feature point in other local images, and the feature points whose similarity with other feature points exceeds the preset similarity are determined as similar feature points in multiple local images.

[0088] In addition, considering that there are multiple feature items in the feature description of the feature point, and the degree of similarity reflected by different feature items is also different, based on this, in some embodiments of the present application, the similarity between each feature point in each local image and each feature point in other local images can be determined based on each feature item of the feature description and the weight value corresponding to each feature item, so as to adjust the flexibility of the feature point matching process and thereby improve the robustness when merging multiple local images subsequently.

[0089] In step S430, the multiple partial images are merged based on similar feature points in the multiple partial images.

[0090] In an embodiment of the present application, feature points in multiple local images are obtained, similar feature points in the multiple local images are determined based on the feature description of each feature point, and then the multiple local images are merged based on the similar feature points in the multiple local images to obtain an overall image of the vehicle chassis, thereby avoiding feature points in the multiple local images that have no correlation with other local images from interfering with the merging process of the multiple local images, thereby improving the accuracy of the merged overall image of the vehicle chassis.

[0091] See also Figure 5 , Figure 5 FIG. 1 is a flow chart showing a defect detection method applied to a vehicle chassis according to another exemplary embodiment. Figure 5 As shown, in Figure 1 Before step S120 in the illustrated embodiment, the method may further include steps S510 to S530, which are described in detail as follows:

[0092] In step S510, landmark components in a plurality of local images are obtained.

[0093] Considering that each of the multiple local images may have some non-overlapping areas that do not overlap with other local areas, the feature points in the non-overlapping areas are less important to the merging process of the multiple local images, and are prone to errors in the merging of the multiple local images. In addition, the process of acquiring the feature points in the non-overlapping areas will further extend the time required to merge the multiple local images. Based on this, in an embodiment of the present application, before merging the multiple local images according to the feature points in the multiple local images, in order to shorten the merging time, the landmark components in the multiple local images may be acquired first; wherein the landmark components may be components of the vehicle chassis included in each local image; or, they may be components of the vehicle chassis that are pre-set. Secondly, the landmark components may be different components on one or more vehicle chassis, which are not limited here.

[0094] For example, when the iconic component is a component of the vehicle chassis included in each partial image, the components included in each partial image can be first obtained through the image recognition model, and then the common component can be determined based on the components included in each partial image. The common component is the component included in each partial image, and the common component can be used as the iconic component.

[0095] When the iconic component is a pre-set component of the vehicle chassis, the components included in each partial image can also be obtained through the image recognition model, and then the components included in each partial image that are identical to the preset components can be used as the iconic component. The preset component is the pre-set iconic component.

[0096] In step S520, the landmark feature points corresponding to the landmark components in each partial image are detected.

[0097] In an embodiment of the present application, after acquiring the iconic components in multiple local images, the iconic feature points corresponding to the iconic components in each local image can be detected, wherein the iconic feature points can also include Harris corner points, extreme points, SIFT feature points, FAST corner points, etc., and the method of detecting the iconic feature points corresponding to the iconic components in each local image can be acquired according to the detection method corresponding to different types of feature points.

[0098] In step S530, the marker feature points are used as feature points in a plurality of local images.

[0099] In the implementation of the present application, by first acquiring the landmark components in multiple local images, then detecting the landmark feature points corresponding to the landmark components in each local image, and then using the landmark feature points as feature points in multiple local images, in the subsequent process of merging multiple local images according to the feature points in the multiple local images to obtain the overall image of the vehicle chassis, the interference of the feature points in the non-overlapping area on the merging process is reduced, and the overall image of the vehicle chassis obtained by merging is further ensured to be less distorted. At the same time, there is no need to detect the feature points in the non-overlapping area, which effectively reduces the time spent on merging multiple local images based on the feature points of multiple local images.

[0100] The following describes an apparatus embodiment of the present application, which can be used to execute the defect detection method applied to the vehicle chassis in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the defect detection method applied to the vehicle chassis of the present application.

[0101] Figure 6 A block diagram of a defect detection device 600 applied to a vehicle chassis according to an embodiment of the present application is shown.

[0102] Reference Figure 6 As shown, a defect detection device 600 applied to a vehicle chassis according to an embodiment of the present application includes: a shooting module 610, configured to acquire multiple local images of a vehicle chassis, wherein different local images represent images captured at different points on the vehicle chassis; a merging module 620, configured to merge multiple local images according to feature points in the multiple local images to obtain an overall image of the vehicle chassis; a segmentation module 630, configured to segment a region including a detection object from the overall image as a detection image through a target neural network; and a detection module 640, configured to detect defects of the detection object based on the detection image.

[0103] In some embodiments of the present application, based on the aforementioned scheme, the merging module 620 is also configured to: determine a local image to be processed from multiple local images before merging multiple local images according to feature points in the multiple local images; obtain lighting information corresponding to the local image to be processed; determine a target lighting balance scheme based on the relationship between the lighting information and preset lighting information; and adjust the lighting information of the local image to be processed through the target lighting balance scheme.

[0104] In some embodiments of the present application, based on the aforementioned scheme, the merging module 620 is also configured to: after determining the target illumination balance scheme based on the relationship between the illumination information and the preset illumination information, determine the position information of the detection object in the local image to be processed; update the target illumination balance scheme according to the position information, so as to adjust the illumination information of the local image to be processed through the updated target illumination balance scheme.

[0105] In some embodiments of the present application, based on the aforementioned scheme, the merging module 620 is further configured to: obtain feature descriptions corresponding to feature points in multiple local images; determine similar feature points in multiple local images based on the feature description of each feature point; and merge multiple local images based on similar feature points in multiple local images.

[0106] In some embodiments of the present application, based on the aforementioned scheme, the detection module 640 is also configured to: obtain landmark components in multiple local images before merging multiple local images according to feature points in the multiple local images; detect landmark feature points corresponding to the landmark components in each local image; and use the landmark feature points as feature points in multiple local images.

[0107] In some embodiments of the present application, based on the aforementioned scheme, the detection module 640 is also configured to: extract feature information of the detection object in the detection image according to multiple dimensions; input the feature information of multiple dimensions into the defect recognition model; obtain the defect prediction result output by the defect recognition model based on the feature information of multiple dimensions, and use it as the defect detection result of the detection object.

[0108] In some embodiments of the present application, based on the aforementioned scheme, the segmentation module 630 is also configured to: before segmenting the area including the detection object from the overall image as a detection image through the target neural network, determine the associated neural network from a preset neural network library according to at least one of the parameter information of the detection object and the model information of the vehicle; and use the determined neural network as the target neural network.

[0109] It should be noted that the defect detection device 800 applied to the vehicle chassis provided in the above embodiment and the defect detection method applied to the vehicle chassis provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here.

[0110] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the defect detection method applied to the vehicle chassis as described above is implemented.

[0111] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0112] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0113] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703, such as executing the method described in the above embodiment. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, ROM 702 and RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0114] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0115] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 709, and / or installed from a removable medium 711. When the computer program is executed by a central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0116] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0117] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0119] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0120] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0121] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0122] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0123] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A defect detection method applied to a vehicle chassis, characterized in that: The method comprises: Acquire multiple local images of a vehicle chassis, wherein different local images represent images taken at different points of the vehicle chassis; Merging the plurality of partial images according to feature points in the plurality of partial images to obtain an overall image of the vehicle chassis; Segmenting a region including a detection object from the overall image as a detection image through a target neural network; Defects of the inspection object are inspected based on the inspection image.

2. The method according to claim 1, characterized in that Before merging the multiple partial images according to the feature points in the multiple partial images, the method further includes: Determining a partial image to be processed from the multiple partial images; Acquire illumination information corresponding to the local image to be processed; Determine a target lighting balance solution based on the relationship between the lighting information and preset lighting information; The illumination information of the local image to be processed is adjusted by the target illumination balance scheme.

3. The method according to claim 2, characterized in that After determining the target illumination balance solution based on the relationship between the illumination information and the preset illumination information, the method further includes: Determine the position information of the detection object in the partial image to be processed; The target illumination balance scheme is updated according to the position information, so as to adjust the illumination information of the to-be-processed local image through the updated target illumination balance scheme.

4. The method according to claim 1, characterized in that The merging of the multiple partial images according to the feature points in the multiple partial images comprises: Obtaining feature descriptions corresponding to the feature points in the multiple local images; Determine similar feature points in the plurality of partial images according to the feature description of each feature point; The plurality of partial images are merged based on similar feature points in the plurality of partial images.

5. The method according to claim 1, characterized in that Before merging the multiple partial images according to the feature points in the multiple partial images, the method further includes: Acquire landmark components in the multiple local images; Detecting the landmark feature points corresponding to the landmark component in each of the partial images; The marked feature points are used as feature points in the multiple local images.

6. The method according to claim 1, characterized in that The detecting defects of the detection object based on the detection image includes: Extracting feature information of the detection object in the detection image according to multiple dimensions respectively; Inputting the feature information of the multiple dimensions into a defect recognition model; The defect prediction result output by the defect recognition model based on the feature information of the multiple dimensions is obtained and used as the defect detection result of the detection object.

7. The method according to claim 1, characterized in that Before segmenting the region including the detection object from the whole image as the detection image by the target neural network, the method further includes: Determining an associated neural network from a preset neural network library according to at least one of parameter information of the detection object and model information of the vehicle; The determined neural network is used as the target neural network.

8. A defect detection device for a vehicle chassis, characterized in that: include: A shooting module is configured to obtain a plurality of partial images of a vehicle chassis, wherein different partial images represent images shot at different points of the vehicle chassis; a merging module, configured to merge the multiple local images according to feature points in the multiple local images to obtain an overall image of the vehicle chassis; a segmentation module, configured to segment a region including a detection object from the overall image as a detection image through a target neural network; The detection module is configured to detect defects of the detection object based on the detection image.

9. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the defect detection method applied to a vehicle chassis as described in any one of claims 1-7.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the defect detection method applied to a vehicle chassis as described in any one of claims 1 to 7.

Images