Vehicle component detection method and device, electronic equipment and readable storage medium

Abstract

The present disclosure provides a vehicle component detection method, device, electronic equipment and computer readable storage medium, relating to the technical field of vehicle component detection, and the method comprises: acquiring a target image of a part to be detected in a vehicle component, acquiring a preset part template corresponding to the target image, determining the similarity between the target image and the preset part template, comparing the similarity with a preset similarity threshold to obtain a comparison result, and determining whether the part to be detected is installed correctly according to the comparison result. The present disclosure realizes the detection of missing and wrong parts of vehicle components, and improves the application range of the detection method of missing and wrong parts of vehicle components.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle component testing technology, and in particular to a vehicle component testing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Error and missing parts inspection items can be applied to the inspection of vehicle floor nuts, bolt holes, small parts, and other components, primarily addressing the issues of incorrect or missing assembly of vehicle parts. Traditional visual quality inspection is handled by dedicated manual quality inspectors. Due to the high production frequency and large volume of vehicles, manual inspection of vehicle parts can lead to visual fatigue among inspectors, increasing the likelihood of missed inspections and compromising overall vehicle quality. Therefore, error and missing part detection models have emerged in the market. These models detect incorrect or missing parts by recognizing images of vehicle components. However, these models require extensive training with numerous labeled production images, are prone to overfitting, have long development cycles, and cannot be reused when new error and missing parts appear, resulting in their inability to detect new errors and missing parts.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, electronic device, and computer-readable storage medium for testing vehicle components, which can at least improve the applicability of the defective product testing method to a certain extent.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a method for inspecting vehicle components is provided, the method comprising:

[0007] Acquire target images of the parts to be inspected in vehicle components;

[0008] Obtain the preset part template corresponding to the target image; wherein the installation position of the part to be detected corresponds to the preset part in the preset part template;

[0009] Determine the similarity between the target image and the preset part template;

[0010] The above similarity is compared with the preset similarity threshold to obtain the comparison result, and the correctness of the installation of the part to be tested is determined based on the comparison result.

[0011] Optionally, the step of determining the similarity between the target image and the preset part template includes: extracting a first feature vector of the part to be detected in the target image, and extracting a second feature vector of the preset part in the preset part template; determining the cosine distance between the first feature vector and the second feature vector; and determining the cosine distance as the similarity.

[0012] Optionally, the steps of extracting the first feature vector of the part to be detected from the target image and extracting the second feature vector of the preset part from the preset part template include: using a pre-trained convolutional network model to extract features from the target image to obtain the first feature vector of the part to be detected; and using the convolutional network model to extract features from the preset part template to obtain the second feature vector of the preset part.

[0013] Optionally, the step of obtaining the target image of the part to be inspected in the vehicle component includes: obtaining the original image of the vehicle component; covering the background area other than the vehicle component in the original image to obtain a mask image; determining the image to be inspected based on the mask image; extracting the position information of the part to be inspected based on the image to be inspected; and cropping the image to be inspected based on the extracted position information to obtain the target image of the part to be inspected.

[0014] Optionally, the step of covering the background area other than the vehicle component in the original image to obtain a mask image includes: obtaining a preset region of interest mask, wherein the preset region of interest mask has the same size information as the original image; multiplying the original image with the preset region of interest mask to obtain a region of interest image; and using the region of interest image as the mask image.

[0015] Optionally, the step of determining the image to be detected based on the mask image includes: determining whether the vehicle component in the mask image is distorted; and if the vehicle component in the mask image is not distorted, determining the mask image as the image to be detected.

[0016] Optionally, the step of determining the image to be detected based on the mask image includes: determining whether the vehicle component in the mask image is distorted; if the vehicle component in the mask image is distorted, obtaining a preset mask image corresponding to the mask image; obtaining first grayscale value change information of the mask image and second grayscale value change information of the preset mask image; selecting n key pixels from the pixels of the mask image and the preset mask image respectively, where n is a positive integer and n is greater than or equal to 4, based on the first grayscale value change information and the second grayscale value change information; generating m key pixel pairs based on the selected n key pixel pairs, and generating a transformation matrix based on the m key pixel pairs, where m is a positive integer greater than n; and correcting the mask image using the transformation matrix to obtain the image to be detected.

[0017] According to another aspect of this disclosure, a vehicle component testing apparatus is provided, the vehicle component testing apparatus comprising:

[0018] The first image acquisition module is used to acquire target images of the parts to be inspected in the vehicle components;

[0019] The second image acquisition module is used to acquire the preset part template corresponding to the target image; wherein the installation position of the part to be detected corresponds to the installation position of the preset part in the preset part template.

[0020] A similarity calculation module is used to determine the similarity between the target image and the preset part template.

[0021] The result detection module is used to compare the above similarity with the preset similarity threshold, obtain the comparison result, and determine whether the above-mentioned part to be detected is installed correctly based on the comparison result.

[0022] According to another aspect of this disclosure, an electronic device is provided, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle component detection method as described in the above embodiments.

[0023] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the vehicle component detection method as described in the above embodiments.

[0024] The vehicle component testing method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this disclosure have the following technical effects:

[0025] This disclosure employs a technical means to acquire a target image of a part to be inspected in a vehicle component, obtain a preset part template corresponding to the target image (the installation position of the part to be inspected corresponds to that of a preset part in the preset part template), determine the similarity between the target image and the preset part template, compare the similarity with a preset similarity threshold, obtain a comparison result, and determine whether the part to be inspected is installed correctly based on the comparison result. By comparing the similarity between the target image of the part to be inspected and the corresponding preset part template, the correct installation of the part to be inspected can be detected, i.e., the detection of defective or faulty vehicle components. Compared with existing defective or faulty product detection models, this vehicle component detection method does not require sample image annotation and model training before model training. The concept of this vehicle component detection method can be applied to the detection of defective or faulty components in other areas, thus expanding the applicability of the vehicle component defective or faulty product detection method.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0028] Figure 1 A flowchart illustrating a vehicle component inspection method in an exemplary embodiment of this disclosure is shown.

[0029] Figure 2 An exemplary flowchart corresponding to step S130 in the vehicle component inspection method of this disclosure is shown;

[0030] Figure 3 An exemplary flowchart corresponding to step S110 in the vehicle component inspection method of this disclosure is shown;

[0031] Figure 4 A schematic diagram of the structure of a vehicle component detection device according to an exemplary embodiment of the present disclosure is shown;

[0032] Figure 5 A schematic diagram of the structure of an electronic device in an exemplary embodiment of the present disclosure is shown. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0034] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0035] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0036] The following are embodiments of the vehicle component testing method provided in this disclosure. Wherein, Figure 1 A flowchart illustrating a vehicle component inspection method according to an exemplary embodiment of this disclosure is shown. Figure 1 As shown, an embodiment of the method disclosed herein provides a vehicle component inspection method applied to a vehicle component inspection equipment. This equipment is used on a vehicle manufacturing production line and can inspect whether small parts in a vehicle are correctly installed. The vehicle component inspection method includes the following steps:

[0037] Step S110: Obtain the target image of the part to be inspected in the vehicle components.

[0038] In an exemplary embodiment, after the vehicle assembly is completed, a target image of the part to be inspected in the vehicle components is acquired. The vehicle components can be understood as the large components used to assemble the vehicle, such as the vehicle floor; the part to be inspected can be understood as a small component in the vehicle components, such as nuts, bolts, etc.; the target image can be understood as a physical image of the part to be inspected in the vehicle components, which includes the part to be inspected and the installation area, but does not include other background. The target image serves as the basis for determining whether the part to be inspected in the vehicle components is misassembled or missing.

[0039] Step S120: Obtain the preset part template corresponding to the above target image.

[0040] In one exemplary embodiment, a large number of preset part templates can be stored in advance. These preset part templates are images of correctly installed and undamaged small parts in vehicle components, which can be understood as reference images. The installation position of the part to be detected in the target image is the same as the installation position of the preset part in the preset part template. The part to be detected corresponds to the installation position of the preset part in the preset part template. A preset part refers to a component in the preset part template that is correctly installed in its installation position according to installation specifications. The preset part and the part to be detected are the same type of component in the same position on the same vehicle body component. After acquiring the target image, the preset part template corresponding to the target image is obtained from the large number of stored preset part templates. It is understood that in the large number of stored preset part templates, the installation position of the preset part in each preset part template is pre-marked with a first identifier. After acquiring the target image, a second identifier of the installation position of the part to be detected in the target image can be obtained. Then, the preset part template corresponding to the first identifier that is identical to the second identifier is found from the large number of stored preset part templates; that is, the found preset part template is the preset part template corresponding to the target image.

[0041] In another scenario, a large number of preset vehicle component templates can be stored in advance. These templates are images of vehicle components, including both the actual vehicle component and a background. The background refers to images of other objects besides the vehicle component in the template. In the actual vehicle component image, the small parts within the component are correctly installed and undamaged. After acquiring the target image, the corresponding preset vehicle component template is retrieved from the stored templates. Each of these templates includes multiple installation locations, each containing a preset part, and each location is pre-marked with a first identifier. After acquiring the target image, a second identifier for the installation location of the part to be detected is obtained. Then, the template containing the same first identifier as the second identifier is searched from the stored templates; the retrieved template is the one corresponding to the target image. For example, the second identifier of the installation position of the part to be detected in the target image is X1. Among the large number of preset vehicle part templates stored, the preset vehicle part template A0 includes 4 installation positions, namely Y1, Y2, Y3 and Y4. Among them, Y1 is the same as X1. Then, the preset vehicle part template A0 is the preset vehicle part template corresponding to the target image.

[0042] After obtaining the preset vehicle component template corresponding to the target image, the background of the obtained preset vehicle component template is then overlaid to obtain a preset mask image. The preset mask image includes the actual image of the vehicle component, and the background is covered. After obtaining the preset mask image, it is corrected so that the vehicle component in the preset mask image is restored to the correct position. The preset mask image is generated from a previously captured image of the vehicle component. The previously captured image of the vehicle component may be misaligned due to the actual vehicle component being placed incorrectly, causing the bottom edge of the actual vehicle component image in the captured image to be offset, resulting in an incorrect image of the actual vehicle component. Therefore, the preset mask image derived from the previously captured vehicle component image needs to be corrected. To correct a preset mask image, one can use one side of the preset mask image, such as the bottom edge, as a reference. By using the pixels in the actual image of the vehicle component, the angle between the actual image of the vehicle component and the bottom edge of the preset mask image can be calculated, which is the deflection angle of the actual image of the vehicle component. Then, the actual image of the vehicle component is rotated in the opposite direction of the deflection angle, thereby achieving the correction of the preset mask image.

[0043] Next, the positional information of each small component in the vehicle part is obtained from the corrected preset mask image. The corrected preset mask image is then segmented using the obtained positional information to obtain preset part templates for each small component. Then, the preset part template corresponding to the target image is obtained from the preset part templates of each small component. The stored preset part templates or preset vehicle part templates can be added. When a new small component appears in the vehicle part, the preset part template or preset vehicle part template corresponding to the new small component can be added to the storage, facilitating subsequent installation and inspection of the new small component.

[0044] Step S130: Determine the similarity between the target image and the preset part template.

[0045] In one exemplary embodiment, a preset part template corresponding to the target image is obtained, the similarity between the target image and the obtained preset part template is calculated, and the similarity between the two is used to determine whether the part to be detected is installed correctly.

[0046] Step S140: Compare the above similarity with the preset similarity threshold to obtain the comparison result, and determine whether the above-mentioned part to be tested is installed correctly based on the above comparison result.

[0047] In an exemplary embodiment, after calculating the similarity between the target image and the acquired preset part template, the similarity is compared with a preset similarity threshold to obtain a comparison result. If the comparison result is that the similarity is greater than the preset similarity threshold, it indicates that the target image and the acquired preset part template have a high similarity, and the part to be detected in the target image is the same as the preset part in the preset part template. In this case, the part to be detected is considered to be correctly installed. If the comparison result is that the similarity is less than or equal to the preset similarity threshold, it indicates that the target image and the acquired preset part template have a low similarity, and the part to be detected in the target image is different from the preset part in the preset part template. Alternatively, the part to be detected in the target image may not exist, meaning that the part to be detected in the target image is actually an exposed area of ​​the installation position. In this case, the part to be detected is considered to be incorrectly installed. That is, an incorrect installation of the part to be detected may be due to the part being incorrectly installed or a part being missing.

[0048] This embodiment, based on a technical solution, employs a method of acquiring a target image of the part to be detected in a vehicle component, obtaining a preset part template corresponding to the target image, determining the similarity between the target image and the preset part template, comparing the similarity with a preset similarity threshold, obtaining a comparison result, and determining whether the part to be detected is installed correctly based on the comparison result. By comparing the similarity between the target image of the part to be detected and the corresponding preset part template, the detection of whether the part to be detected is installed correctly can be achieved, i.e., the detection of defective or faulty parts in vehicle components. Compared with existing defective or faulty part detection models, this vehicle component detection method does not require sample image annotation and model training before model training. The concept of this vehicle component detection method can be applied to the detection of defective or faulty parts in other things, improving the applicability of the vehicle component defective or faulty part detection method. When a small part is added to a vehicle component, the template corresponding to the new small part can be added to the stored preset part template, thus expanding the stored preset part template. This enables the detection of defective or faulty parts for both existing and newly added small parts in the vehicle component. The false positive rate and recall rate of the defective or faulty part detection can meet the business detection requirements.

[0049] For example, Figure 2 An exemplary flowchart corresponding to step S130 in the vehicle component inspection method of this disclosure is shown. Optionally, based on the above method embodiment, step S130 includes the following:

[0050] Step S131: Extract the first feature vector of the part to be detected from the target image and extract the second feature vector of the preset part from the preset part template;

[0051] Step S132: Determine the cosine distance between the first eigenvector and the second eigenvector;

[0052] Step S133: Determine the cosine distance as the similarity.

[0053] In an exemplary embodiment, after obtaining the target image of the part to be detected and the corresponding preset part template, a first feature vector of the part to be detected is extracted from the target image, and a second feature vector of the preset part is extracted from the preset part template. The first and second feature vectors have the same dimension; for example, both are 512-dimensional feature vectors. After obtaining the first and second feature vectors, feature comparison is performed, i.e., the cosine distance between the first and second feature vectors is calculated. The cosine distance is determined as the similarity between the target image and the preset part template, and the difference between the target image and the preset part template is measured by the similarity. The cosine distance is also called cosine similarity.

[0054] Optionally, step S131 includes: using a pre-trained convolutional network model to extract features from the target image to obtain a first feature vector of the part to be detected; using the convolutional network model to extract features from the preset part template to obtain a second feature vector of the preset part.

[0055] It should be understood that a convolutional network model is pre-trained and used for image feature extraction. Specifically, the target image is input into the convolutional network model for feature extraction, yielding the first feature vector of the part to be detected. Similarly, a pre-set part template is input into the convolutional network model for feature extraction, yielding the second feature vector of the pre-set part. The convolutional network model uses ResNet (Deep Residual Network) as its skeleton and employs Metric-FC (Metric Learning), relying on a pre-trained model based on large datasets from ImageNet. Fine-tuning of the convolutional network model is also possible, allowing for minor adjustments to its parameters without retraining the model.

[0056] For example, Figure 3 An exemplary flowchart corresponding to step S110 in the vehicle component inspection method of this disclosure is shown. Optionally, based on the above method embodiment, step S110 includes the following:

[0057] Step S111: Obtain the original image of the above-mentioned vehicle component.

[0058] In one exemplary embodiment, a camera or camera captures raw images of vehicle components. The raw images include physical images of the vehicle components and background areas, such as images of conveyor belts, robotic arms, etc.

[0059] Step S112: Cover the background area other than the vehicle components in the original image to obtain a mask image.

[0060] After obtaining the original image of the vehicle component, since the original image includes a background area other than the actual image of the vehicle component, this background area can interfere with the installation detection of small parts within the vehicle component. Therefore, the background area other than the vehicle component in the original image is covered to obtain a mask image. In the mask image, the background area is covered, and only the actual image of the vehicle component is included.

[0061] Optionally, step S112 includes the following options:

[0062] Obtain a preset region of interest mask, the preset region of interest mask having the same size information as the original image;

[0063] Multiply the original image above with the preset region of interest mask above to obtain the region of interest image;

[0064] The region of interest image described above is used as the mask image described above.

[0065] It should be understood that the preset region of interest (ROI) mask is pre-set and has the same size information as the original image. Pixels in the preset ROI mask correspond one-to-one with pixels in the original image. For example, the first pixel in the top left corner of the original image corresponds to the first pixel in the preset ROI mask. Multiplying the original image with the preset ROI mask yields the ROI image, which includes a real image of the vehicle component. The grayscale values ​​of the vehicle component image in the ROI image are the same as those in the original image, while the grayscale values ​​of the background region in the ROI image become fixed values, for example, all grayscale values ​​of the background region in the ROI image are 0. Finally, the ROI image is used as the mask image to generate the mask image.

[0066] Step S113: Determine the image to be detected based on the above mask image.

[0067] After obtaining the mask image, the image to be inspected is determined based on the mask image. The image to be inspected is the basis for generating the target image of the part to be inspected, and it must be ensured that the actual image of the vehicle part in the image to be inspected is not distorted. If distortion occurs, it will lead to inaccurate detection results regarding whether the part to be inspected is installed correctly. Therefore, determining the image to be inspected based on the mask image includes two cases.

[0068] In the first case, it is determined whether the vehicle parts in the mask image are distorted. If the vehicle parts in the mask image are not distorted, the mask image is determined as the image to be detected.

[0069] The second scenario involves determining whether vehicle components in the mask image are distorted. If distortion is found, the mask image is corrected. This correction includes: acquiring first grayscale value change information of the mask image and second grayscale value change information of the preset mask image; selecting n key pixels from the pixels of both the mask image and the preset mask image based on the first and second grayscale value change information, where n is a positive integer greater than or equal to 4; generating m key pixel pairs from the selected n key pixels, and generating a transformation matrix from the m key pixel pairs, where m is a positive integer greater than n; and applying the transformation matrix to correct the mask image to obtain the image to be detected.

[0070] The system pre-stores a large number of preset mask images, each with a unique identifier. Each preset mask image's identifier corresponds to multiple preset markers for different installation locations. When the original image is acquired, it contains multiple installation locations, each with its own identifier. After acquiring the original image, these identifiers can be extracted. Based on the correspondence between the preset mask image's identifier and these identifiers, the preset mask image corresponding to the original image can be retrieved from the large number of stored preset mask images.

[0071] The first grayscale value change information refers to the grayscale value change information of each pixel in the mask image relative to its surrounding pixels. The second grayscale value change information refers to the grayscale value change information of each pixel in the preset mask image relative to its surrounding pixels. After obtaining the first and second grayscale value change information, pixels whose grayscale value change information differs from the first and second grayscale value change information are extracted from both the mask image and the preset mask image. Then, key pixels of the mask image are extracted from the pixels extracted from the mask image. These key pixels are generally selected from corner pixels, image contour pixels, etc., of the mask image. Similarly, key pixels of the preset mask image are extracted from the pixels extracted from the preset mask image. These key pixels are also generally selected from corner pixels, image contour pixels, etc., of the preset mask image. n key pixels are extracted from the pixels of both the mask image and the preset mask image.

[0072] For example, since each pixel in the mask image corresponds one-to-one with each pixel in the preset mask image, assume that the pixels in the mask image are pixels 01-20, and the pixels in the preset mask image are pixels 01-20. The first grayscale value changes of pixels 01, 05, 10, 15, and 20 in the mask image are different from the second grayscale value changes of pixels 01, 05, 10, 15, and 20 in the preset mask image, while the first grayscale value changes of the remaining pixels in the mask image are the same as the second grayscale value changes of the remaining pixels in the preset mask image. After identification, pixels 01, 05, 10, and 15 in the mask image are identified as key pixels. Similarly, pixels 05, 10, 15, and 20 in the preset mask image are identified as key pixels. Furthermore, pixels 01, 05, 10, and 15 in the mask image correspond one-to-one with pixels 05, 10, 15, and 20 in the preset mask image. Therefore, pixel 01 in the mask image corresponds to pixel 20 in the preset mask image. Pixel 05 in the image is a key pixel pair. Pixel 05 in the mask image and pixel 10 in the preset mask image form a key pixel pair. Pixel 10 in the mask image and pixel 15 in the preset mask image form a key pixel pair. Pixel 15 in the mask image and pixel 20 in the preset mask image form a key pixel pair, resulting in four key pixel pairs. A transformation matrix is ​​generated using these four key pixel pairs, where each key pixel pair is a value in the transformation matrix. After generating the transformation matrix, a projection operation is performed between the transformation matrix and the original matrix of the mask image to obtain the original matrix of the corrected mask image, thus achieving mask image correction.

[0073] Step S114: Based on the above image to be detected, extract the position information of the part to be detected.

[0074] Step S115: Based on the extracted location information, crop the image to be detected to obtain the target image of the part to be detected.

[0075] After obtaining the image to be detected, the actual image of the vehicle component in the image is undistorted. The positional information of the component to be detected is then extracted from the image. The image is then cropped according to this extracted positional information to obtain multiple image blocks, each representing a target image of the component to be detected. Obtaining the positional information of the component to be detected includes: acquiring the coordinates of all pixels of the component in the image; extracting the coordinates of the pixels corresponding to the component's contour from these coordinates, known as contour pixel coordinates; using these contour pixel coordinates as the positional information of the component; and then cropping the image according to this positional information, i.e., cropping the image based on the contour pixel coordinates of the component to obtain the target image of the component to be detected.

[0076] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0077] in, Figure 4 A schematic diagram of a vehicle component inspection apparatus to which an embodiment of this disclosure can be applied is shown. Please refer to... Figure 4 The vehicle component detection device shown in the figure can be implemented as all or part of an electronic device through software, hardware, or a combination of both, or it can be integrated as an independent module into an electronic device or server.

[0078] The vehicle component testing device 400 in this embodiment includes:

[0079] The first image acquisition module 410 is used to acquire the target image of the part to be detected in the vehicle component;

[0080] The second image acquisition module 420 is used to acquire the preset part template corresponding to the target image; wherein the installation position of the part to be detected corresponds to the installation position of the preset part in the preset part template.

[0081] The similarity calculation module 430 is used to determine the similarity between the target image and the preset part template.

[0082] The result detection module 440 is used to compare the above similarity with a preset similarity threshold, obtain the comparison result, and determine whether the above-mentioned part to be detected is installed correctly based on the comparison result.

[0083] In an exemplary embodiment, based on the foregoing scheme, the similarity calculation module 430 includes:

[0084] The feature extraction unit is used to extract the first feature vector of the part to be detected in the target image and to extract the second feature vector of the preset part in the preset part template.

[0085] The data calculation unit is used to determine the cosine distance between the first feature vector and the second feature vector, and to determine the cosine distance as the similarity.

[0086] In an exemplary embodiment, based on the foregoing scheme, the feature extraction unit, in extracting the first feature vector of the part to be detected in the target image and the second feature vector of the preset part in the preset part template, is specifically used to perform feature extraction on the target image using a pre-trained convolutional network model to obtain the first feature vector of the part to be detected; and to perform feature extraction on the preset part template using the convolutional network model to obtain the second feature vector of the preset part.

[0087] In an exemplary embodiment, based on the foregoing scheme, the first image acquisition module 410 includes:

[0088] The first image acquisition unit is used to acquire the original image of the aforementioned vehicle component;

[0089] The second image acquisition unit is used to cover the background area other than the vehicle components in the original image to obtain a mask image;

[0090] The third image acquisition unit is used to determine the image to be detected based on the above-mentioned mask image;

[0091] The position information extraction unit is used to extract the position information of the part to be detected based on the image to be detected.

[0092] The image cropping unit is used to crop the image to be detected based on the extracted location information to obtain the target image of the part to be detected.

[0093] In an exemplary embodiment, based on the foregoing scheme, the second image acquisition unit includes:

[0094] The mask acquisition subunit is used to acquire a preset region of interest mask, the preset region of interest mask having the same size information as the original image.

[0095] The image processing subunit is used to multiply the original image with the preset region of interest mask to obtain a region of interest image, and use the region of interest image as the mask image.

[0096] In an exemplary embodiment, based on the foregoing scheme, the third image acquisition unit includes:

[0097] The first judgment subunit is used to determine whether the vehicle component in the above-mentioned mask image is distorted;

[0098] The first image determination subunit is used to determine the mask image as the image to be detected when the vehicle component in the mask image is not distorted.

[0099] In an exemplary embodiment, based on the foregoing scheme, the third image acquisition unit further includes:

[0100] The second judgment subunit is used to determine whether the vehicle component in the mask image is distorted, and if the vehicle component in the mask image is distorted, to obtain a preset mask image corresponding to the mask image.

[0101] The change information acquisition subunit is used to acquire the first gray value change information of the above-mentioned mask image and the second gray value change information of the above-mentioned preset mask image;

[0102] The feature point acquisition subunit is used to select n key pixels from the pixels of the mask image and the preset mask image respectively, based on the first gray value change information and the second gray value change information, where n is a positive integer and n is greater than or equal to 4.

[0103] The feature point pair acquisition subunit is used to generate m key pixel point pairs based on the n key pixel points obtained by screening, and to generate a transformation matrix based on the m key pixel point pairs, where m is a positive integer greater than n.

[0104] The second image determination subunit is used to correct the mask image using the transformation matrix to obtain the image to be detected.

[0105] It should be noted that the vehicle component testing device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the vehicle component testing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle component testing device and the vehicle component testing method embodiments provided in the above embodiments belong to the same concept. Therefore, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the vehicle component testing method of this disclosure, which will not be repeated here.

[0106] The sequence numbers of the embodiments disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0107] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0108] This disclosure also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of any of the methods described above.

[0109] Figure 5 A schematic diagram of the electronic device is shown. Please refer to [link / reference]. Figure 5 As shown, the electronic device 500 includes a processor 501 and a memory 502.

[0110] In this embodiment, the processor 501 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor 501 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 501 can be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0111] In this embodiment of the disclosure, the processor 501 is specifically used to: acquire a target image of a part to be inspected in a vehicle component; acquire a preset part template corresponding to the target image; wherein the installation position of the part to be inspected corresponds to the installation position of the preset part in the preset part template; determine the similarity between the target image and the preset part template; compare the similarity with a preset similarity threshold to obtain a comparison result, and determine whether the part to be inspected is installed correctly based on the comparison result.

[0112] Furthermore, the processor 501 is also configured to: extract a first feature vector of the part to be detected from the target image, and extract a second feature vector of the preset part from the preset part template; determine the cosine distance between the first feature vector and the second feature vector; and determine the cosine distance as the similarity.

[0113] Furthermore, the processor 501 is also used to: extract features from the target image using a pre-trained convolutional network model to obtain a first feature vector of the part to be detected; and extract features from the preset part template using the convolutional network model to obtain a second feature vector of the preset part.

[0114] Furthermore, the processor 501 is also configured to: acquire an original image of the vehicle component; cover the background area outside the vehicle component in the original image to obtain a mask image; determine an image to be detected based on the mask image; extract the position information of the part to be detected based on the image to be detected; and crop the image to be detected based on the extracted position information to obtain a target image of the part to be detected.

[0115] Furthermore, the processor 501 is also configured to: obtain a preset region of interest mask, wherein the preset region of interest mask has the same size information as the original image; multiply the original image with the preset region of interest mask to obtain a region of interest image; and use the region of interest image as the mask image.

[0116] Furthermore, the processor 501 is also configured to: determine whether the vehicle component in the mask image is distorted; and if the vehicle component in the mask image is not distorted, determine the mask image as the image to be detected.

[0117] Furthermore, the processor 501 is also configured to: determine whether the vehicle component in the mask image is distorted; if the vehicle component in the mask image is distorted, obtain a preset mask image corresponding to the mask image; obtain first grayscale value change information of the mask image and second grayscale value change information of the preset mask image; based on the first grayscale value change information and the second grayscale value change information, select n key pixels from the pixels of the mask image and the preset mask image respectively, where n is a positive integer and n is greater than or equal to 4; generate m key pixel pairs based on the n key pixel pairs respectively selected, and generate a transformation matrix based on the m key pixel pairs, where m is a positive integer greater than n; and use the transformation matrix to correct the mask image to obtain the image to be detected.

[0118] Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this disclosure, the non-transitory computer-readable storage media in memory 502 is used to store at least one instruction, which is executed by processor 501 to implement the methods in the embodiments of this disclosure.

[0119] In some embodiments, the electronic device 500 further includes a peripheral device interface 503 and at least one peripheral device. The processor 501, memory 502, and peripheral device interface 503 are connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a display screen 504, a camera 505, and an audio circuit 506.

[0120] Peripheral device interface 503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 501 and memory 502. In some embodiments of this disclosure, processor 501, memory 502, and peripheral device interface 503 are integrated on the same chip or circuit board; in other embodiments of this disclosure, any one or two of processor 501, memory 502, and peripheral device interface 503 can be implemented on separate chips or circuit boards. This disclosure does not specifically limit the scope of the embodiments.

[0121] Display screen 504 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 504 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 501 for processing. In this case, display screen 504 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of this disclosure, there may be one display screen 504, which serves as the front panel of the electronic device 500; in other embodiments, there may be at least two display screens 504, respectively disposed on different surfaces of the electronic device 500 or in a folded design; in still other embodiments, display screen 504 may be a flexible display screen, disposed on a curved or folded surface of the electronic device 500. Furthermore, display screen 504 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 504 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0122] Camera 505 is used to capture images or videos. Optionally, camera 505 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device 500, and the rear-facing camera is located on the back of the electronic device 500. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusing the main camera and the depth-sensing camera, panoramic shooting by fusing the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments of this disclosure, camera 505 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0123] The audio circuit 506 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 501 for processing. For stereo sound acquisition or noise reduction purposes, there may be multiple microphones, each located in a different part of the electronic device 500. The microphone may also be an array microphone or an omnidirectional microphone.

[0124] Power supply 507 is used to supply power to the various components in electronic device 500. Power supply 507 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 507 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0125] The structural block diagram of the electronic device 500 shown in the embodiments of this disclosure does not constitute a limitation on the electronic device 500. The electronic device 500 may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.

[0126] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the object characteristics, interactive behavior characteristics, and user information involved in this specification were all obtained under full authorization.

[0127] In the description of this disclosure, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances. Furthermore, in the description of this disclosure, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0128] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, equivalent variations made in accordance with the claims of this disclosure are still within the scope of this disclosure.

Claims

1. A vehicle component detection method characterized by, The vehicle component testing method includes: Obtain raw images of vehicle components; Obtain a preset region of interest mask, wherein the preset region of interest mask has the same size information as the original image, and the pixels in the preset region of interest mask correspond one-to-one with the pixels in the original image; The original image is multiplied by the preset region of interest mask to obtain a region of interest image, and the region of interest image is used as the mask image; Determine whether the vehicle component in the mask image is distorted; When the vehicle component in the mask image is distorted, a preset mask image corresponding to the mask image is obtained; Obtain the first grayscale value change information of the mask image and the second grayscale value change information of the preset mask image; Pixels whose grayscale value change information is different from the first grayscale value change information are extracted from the mask image and the preset mask image, respectively; Extract n key pixels of the mask image from the pixels extracted from the mask image, and extract n key pixels of the preset mask image from the pixels extracted from the preset mask image. The key pixels of the mask image and the key pixels of the preset mask image include corner pixels and image contour pixels. n is a positive integer and n is greater than or equal to 4. Based on the n key pixels of the mask image and the n key pixels of the preset mask image, generate m key pixel pairs, and generate a transformation matrix based on the m key pixel pairs, where each key pixel pair is a value in the transformation matrix, and m is a positive integer greater than n. The transformation matrix is ​​projected onto the original matrix of the mask image to correct the mask image and obtain the image to be detected. Based on the image to be detected, extract the position information of the part to be detected; The image to be detected is cropped based on the extracted location information to obtain the target image of the part to be detected; Obtain a preset part template corresponding to the target image; wherein the part to be detected corresponds to the installation position of the preset part in the preset part template; Determine the similarity between the target image and the preset part template; The similarity is compared with a preset similarity threshold to obtain a comparison result, and the correctness of the part to be detected is determined based on the comparison result.

2. The vehicle component testing method as described in claim 1, characterized in that, The step of determining the similarity between the target image and the preset part template includes: Extract the first feature vector of the part to be detected from the target image, and extract the second feature vector of the preset part from the preset part template; Determine the cosine distance between the first feature vector and the second feature vector; The cosine distance is determined as the similarity.

3. The vehicle component testing method as described in claim 2, characterized in that, The steps of extracting the first feature vector of the part to be detected in the target image and extracting the second feature vector of the preset part in the preset part template include: A pre-trained convolutional network model is used to extract features from the target image to obtain the first feature vector of the part to be detected; The convolutional network model is used to extract features from the preset part template to obtain the second feature vector of the preset part.

4. The vehicle component testing method as described in claim 1, characterized in that, After determining whether the vehicle component in the mask image is distorted, the vehicle component detection method further includes: Determine whether the vehicle component in the mask image is distorted; If the vehicle component in the mask image is not distorted, the mask image is determined as the image to be detected.

5. A vehicle component testing device, characterized in that, The vehicle component detection device includes: A first image acquisition module is used to acquire an original image of a vehicle component; acquire a preset region of interest (ROI) mask, wherein the preset ROI mask has the same size information as the original image, and the pixels in the preset ROI mask correspond one-to-one with the pixels in the original image; multiply the original image with the preset ROI mask to obtain a ROI image, and use the ROI image as a mask image; determine whether the vehicle component in the mask image is distorted; if the vehicle component in the mask image is distorted, acquire a preset mask image corresponding to the mask image; acquire first grayscale value change information of the mask image and second grayscale value change information of the preset mask image; extract pixels from the mask image and the preset mask image where the first grayscale value change information and the second grayscale value change information are different; and extract pixels from the pixels extracted from the mask image. The mask image contains n key pixels, and the preset mask image contains n key pixels extracted from pixels extracted from the preset mask image. The key pixels of the mask image and the preset mask image include corner pixels and image contour pixels, where n is a positive integer and is greater than or equal to 4. Based on the n key pixels of the mask image and the preset mask image, m key pixel pairs are generated, and a transformation matrix is ​​generated based on the m key pixel pairs. Each key pixel pair is a value in the transformation matrix, where m is a positive integer greater than n. The transformation matrix is ​​projected onto the original matrix of the mask image to correct the mask image and obtain the image to be detected. Based on the image to be detected, the position information of the part to be detected is extracted. The image to be detected is cropped based on the extracted position information to obtain the target image of the part to be detected. The second image acquisition module is used to acquire a preset part template corresponding to the target image; wherein the part to be detected corresponds to the installation position of the preset part in the preset part template; A similarity calculation module is used to determine the similarity between the target image and the preset part template; The result detection module is used to compare the similarity with a preset similarity threshold, obtain the comparison result, and determine whether the part to be detected is installed correctly based on the comparison result.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the vehicle component detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle component detection method as described in any one of claims 1 to 4.

Images