Part detection method and device

Abstract

The embodiment of the invention provides a part detection method and device. The part detection method comprises the steps that current position information corresponding to a to-be-detected vehicle is acquired; determining a current to-be-detected item of the to-be-detected vehicle at the current position information, the current to-be-detected item being used for detecting a current to-be-detected part, and the current to-be-detected part being a to-be-detected part of the to-be-detected vehicle at the current position information; receiving a to-be-detected part image sent by target shooting equipment for the current to-be-detected part; and inputting the to-be-detected part image into a part detection model, and obtaining a part detection result output by the part detection model. The current detection item of the vehicle is determined through the position information of the vehicle, then the current detection item is automatically detected by using the part detection model, and automatic detection of the part corresponding to the to-be-detected vehicle is realized.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a parts detection method. Background Art

[0002] With the development of automated manufacturing technology, automatic assembly of vehicles has become possible, which not only significantly improves production efficiency, but also greatly reduces the need for manual intervention, making the detection of the installation process particularly important.

[0003] Currently, manual inspection technology still plays a role in installation defect detection, and can handle some complex situations and provide flexible judgments. However, manual inspection has the disadvantages of being slow, difficult to process large amounts of data and real-time analysis, and operators are prone to fatigue when performing repetitive work for a long time, resulting in missed inspections or misjudgments. High-quality inspections rely on specialized skills and experience, and personnel changes require time to learn and adapt. In order to address the above shortcomings, a more efficient parts inspection method is needed. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a part detection method. One or more embodiments of this specification also relate to a part detection device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a parts detection method is provided, comprising:

[0006] Obtain the current position information corresponding to the vehicle to be detected;

[0007] Determine the current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect the current part to be detected, and the current part to be detected is the part to be detected of the vehicle to be detected at the current position information;

[0008] Receiving an image of a part to be inspected sent by a target photographing device for the current part to be inspected;

[0009] The image of the part to be detected is input into a part detection model to obtain a part detection result output by the part detection model.

[0010] According to a second aspect of the embodiments of this specification, there is provided a parts detection device, comprising:

[0011] An acquisition module is configured to acquire current position information corresponding to the vehicle to be detected;

[0012] A determination module is configured to determine a current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect a current part to be detected, and the current part to be detected is a part to be detected of the vehicle to be detected at the current position information;

[0013] A receiving module is configured to receive an image of a part to be detected sent by a target shooting device for the current part to be detected;

[0014] The detection module is configured to input the image of the part to be detected into the part detection model and obtain the part detection result output by the part detection model.

[0015] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0016] Memory and processor;

[0017] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned parts detection method are implemented.

[0018] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned parts detection method are implemented.

[0019] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned parts detection method when executed by a processor.

[0020] One embodiment of the present specification implements the steps of obtaining the current position information corresponding to the vehicle to be detected; determining the current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect the current part to be detected, and the current part to be detected is the part to be detected of the vehicle to be detected at the current position information; receiving an image of the part to be detected sent by a target shooting device for the current part to be detected; inputting the image of the part to be detected into a part detection model, and obtaining a part detection result output by the part detection model.

[0021] By applying the solution of the embodiments of this specification, the current inspection item of the vehicle to be inspected is determined through the vehicle's position information, and then the image of the part to be inspected is collected by the target shooting device corresponding to the current inspection item. After that, the image of the part to be inspected is input into the part detection model to automatically detect the current inspection item, thereby realizing automatic detection of the parts corresponding to the vehicle to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a parts detection method provided by an embodiment of this specification;

[0023] Figure 2 is a structural schematic diagram of an image segmentation module provided by an embodiment of this specification;

[0024] Figure 3 is a process flow chart of a vehicle assembly detection method provided by an embodiment of this specification;

[0025] Figure 4 It is a structural schematic diagram of a parts detection device provided by an embodiment of this specification;

[0026] Figure 5 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0027] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0028] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In this specification, a parts detection method is provided. This specification also relates to a parts detection device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0032] In the production process of vehicles, after the production of various parts on the vehicle is completed, the various parts need to be assembled on the assembly line to complete the production of the vehicle. When the parts are assembled on the vehicle, the parts on the vehicle need to be inspected. However, manual inspection has the disadvantages of slow speed, difficulty in processing large amounts of data and real-time analysis, and operators are prone to fatigue when performing repetitive work for a long time, resulting in missed inspections or misjudgments. Therefore, a method for automatically inspecting items installed on the vehicle during the vehicle production process is needed.

[0033] See also Figure 1 , Figure 1 A flow chart of a parts detection method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0034] Step 102: Obtain the current position information corresponding to the vehicle to be detected.

[0035] In practical applications, the vehicle to be detected is a vehicle on an assembly line during vehicle production, and the current position information represents the position of the vehicle to be detected on the assembly line.

[0036] Exemplarily, the vehicle to be inspected can be understood as a vehicle that is undergoing assembly, that is, a vehicle that has not yet been fully assembled on the assembly line. Considering the need to ensure the quality and safety of the assembled vehicle when assembling the vehicle, the parts assembled on the vehicle to be inspected still need to be inspected and adjusted.

[0037] Step 104: Determine the current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect the current part to be detected, and the current part to be detected is the part to be detected of the vehicle to be detected at the current position information.

[0038] In actual applications, the current item to be inspected is the item to be inspected on the current parts to be inspected of the vehicle to be inspected, the current parts to be inspected are the parts assembled on the assembly line at the current position, and the parts to be inspected are the parts that need to be assembled on the vehicle to be inspected.

[0039] Exemplarily, the current items to be tested can be understood as inspection tasks for key components or systems that have not been inspected in the current vehicle assembly state, which involve the corresponding parts to be inspected, such as engines, exhaust pipes, battery packs, doors, etc. These items to be tested conduct detailed analysis of the specific inspection requirements of each part of the vehicle to ensure that the installation, connection, and function of each component meet the predetermined standards. The execution of the current items to be tested is not only to verify the quality and performance of these key components, but also to discover and correct potential installation defects, thereby ensuring the safety and reliability of the entire vehicle assembly and laying the foundation for the qualified delivery of the final product.

[0040] It should be noted that since multiple parts need to be assembled on the vehicle assembly line, each part is assembled at a different position on the assembly line. Therefore, the current position of the vehicle to be inspected can be determined to determine its corresponding current part to be inspected, and then determine its corresponding current item to be inspected.

[0041] It should be noted that the method for determining the current item to be inspected of the vehicle to be inspected at the current position information can be to directly determine the current item to be inspected by matching the current position information of the vehicle to be inspected with the preset inspection station position range; it can also be determined by obtaining the current assembly progress of the vehicle to be inspected, and selecting the inspection task corresponding to the current position according to the list of items to be inspected corresponding to the progress; it can also be determined by combining the vehicle type and position information to make a comprehensive judgment and select the item to be inspected that matches it. These methods are only for example description, and this manual does not impose any restrictions on this.

[0042] Further, determining the current item to be detected of the vehicle to be detected at the current position information includes:

[0043] Acquire at least one item to be detected corresponding to the vehicle to be detected, and acquire location range information of each item to be detected;

[0044] In the case that the current position information belongs to the target position range information, it is determined that the item to be detected corresponding to the target position range information is the current item to be detected, wherein the target position range information is any one of the position range information.

[0045] In practical applications, the position range information represents the position of the item to be inspected on the assembly line.

[0046] For example, the location range information can be understood as the physical location interval or range of the item to be inspected on the assembly line. This information is used to identify and locate the inspection area corresponding to each part to be inspected on the assembly line, ensuring that the system can accurately match the corresponding inspection task and the part to be inspected according to the current location of the vehicle, thereby achieving effective inspection and quality control of each component.

[0047] It should be noted that there are multiple options for obtaining at least one item to be tested corresponding to the vehicle to be tested, which can be done specifically through the following methods. For example, according to the type of vehicle to be tested, the preset list of test items can be automatically matched to obtain the corresponding items to be tested; the relevant items to be tested can also be screened out by reading the production batch or configuration of the vehicle and combining the specific test requirements of the batch or configuration; or the list of items to be tested can be dynamically generated based on the historical test records and current status of the vehicle to ensure the comprehensiveness and effectiveness of the test. These methods are only used as examples and are not limited in this specification.

[0048] It should be noted that the method of obtaining the location range information of the item to be detected can be to determine the location range of the item to be detected on the assembly line according to the installation position of the parts corresponding to the item to be detected, through the preset distance threshold and the category of the detection item; it can also refer to historical data and combine the location range of similar items to deduce the possible location interval of the current item to be detected; it can also use the real-time positioning system on the production line to directly obtain the precise location range of each item to be detected, so as to ensure the accuracy and timeliness of the detection. These methods are only used as examples, and this manual does not impose any restrictions on this.

[0049] By obtaining the location range information of each item to be inspected and combining it with the current location information of the vehicle to be inspected to determine the current inspection item, the accuracy and pertinence of the inspection task can be ensured, and interference from irrelevant items can be avoided, thereby improving the efficiency and accuracy of the inspection. At the same time, it also reduces unnecessary time consumption and waste of resources, making the entire inspection process smoother and more reasonable, which helps to improve the overall efficiency and quality control level of the production line.

[0050] Further, obtaining at least one item to be detected corresponding to the vehicle to be detected includes:

[0051] Obtaining the vehicle type corresponding to the vehicle to be detected;

[0052] At least one to-be-detected item corresponding to the to-be-detected vehicle is determined according to the vehicle type.

[0053] In practical applications, the vehicle type is the type of vehicle to be detected. The vehicle type can be understood as the classification category of the vehicle to be detected, for example, it can be different vehicle types such as small cars, SUVs or trucks; it can also be different drive types such as electric vehicles, hybrid vehicles or traditional fuel vehicles; it can also be a vehicle for a specific purpose, such as a police car, ambulance or school bus.

[0054] For example, determining at least one item to be inspected corresponding to the vehicle to be inspected according to the vehicle type can be understood as matching the corresponding inspection tasks according to different vehicle classifications. For example, the inspection items including chassis and suspension system can be determined for SUV models; the inspection items including battery pack and electronic control system can be determined for electric vehicles; and the inspection items including alarm devices and communication equipment can be determined for police cars. In this way, it is ensured that each type of vehicle can cover its unique key components and systems during the inspection process.

[0055] Furthermore, the location range information of each item to be detected is obtained, including:

[0056] Determine a part detection method and part installation position information corresponding to a reference item to be detected, wherein the reference item to be detected is any one of the items to be detected;

[0057] In the case where the part detection method is part defect detection, the starting position information of the position range information corresponding to the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the ending position information of the position range information corresponding to the reference item to be detected;

[0058] In the case where the part detection method is part installation detection, the end position information of the corresponding position range information of the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the starting position information of the corresponding position range information of the reference item to be detected.

[0059] In actual applications, the part installation position information refers to the installation position of the corresponding part to be inspected on the installation production line. The part inspection method is that the part defect detection is a detection method used to detect whether the part to be inspected corresponding to the item to be inspected meets the standards. The part installation detection is a detection method used to detect whether the installation position of the part to be inspected corresponding to the item to be inspected meets the standards.

[0060] Exemplarily, part defect detection can be understood as testing the integrity, flatness, and other data that affect the quality of the parts to be installed on the vehicle to be tested. For example, it is possible to detect whether there are physical defects such as cracks and dents on the surface of the parts; it is also possible to detect whether there are burrs or irregularities on the edges of the parts to ensure their precise fit; it is also possible to detect whether the parts are installed incorrectly. These tests are only for illustrative purposes and this specification does not impose any restrictions on them.

[0061] Parts installation inspection can be understood as the inspection of key parameters such as the installation position, angle, and alignment of parts that have been installed on the vehicle to be inspected. For example, it can be inspected whether the parts are correctly aligned with the specified installation position to avoid misalignment; it can also be inspected whether the installation angle of the parts meets the design requirements to ensure that they function normally during vehicle operation; it can also be inspected whether the installed parts are firmly fixed to prevent loosening or falling off. These inspections are only for illustrative purposes and this manual does not impose any restrictions on them.

[0062] It should be noted that since the parts defect detection needs to be performed before the parts are installed on the vehicle to be inspected, if the parts detection method for the item to be inspected is parts defect detection, its corresponding position range information needs to be before the parts installation position information, that is, the starting position information of its corresponding position range information is the position before the parts installation position information, and the ending position information of its corresponding position range information is the position where the parts are installed on the vehicle, that is, the parts installation position information.

[0063] Since the parts installation inspection is performed after the parts have been installed on the vehicle, if the part inspection method for the item to be inspected is parts installation inspection, its corresponding position range information needs to be after the part installation position information, that is, the part installation position information, that is, the starting position information of its corresponding position range information is the position where the part is installed on the vehicle, that is, the part installation position information, and the ending position information of its corresponding position range information is the position information after the part is installed on the vehicle.

[0064] It should be noted that the detection distance threshold is the preset distance required to complete the part inspection project, that is, the detection distance threshold can be understood as the minimum or maximum distance to ensure that the inspection equipment can accurately identify and evaluate the state of the part. For example, it can be understood as the optimal working distance between the deployment location of the inspection project and the installation location of the part to be inspected; it can also be understood as the maximum detection distance allowed without affecting the inspection accuracy; it can also be understood as the minimum distance to ensure that the inspection field of view covers the entire part surface. These explanations are only for example purposes and this manual does not impose any restrictions on this.

[0065] In one embodiment provided in this specification, taking the part to be detected as an automobile exhaust pipe as an example, the corresponding part detection method is the to-be-detected item A of the part defect detection and the corresponding part detection method is the to-be-detected item B of the part installation detection, and the installation position of the automobile exhaust pipe is 100, and the preset detection distance threshold is 5, then the position range information corresponding to the to-be-detected item A is determined to be 95-100, and the position range information corresponding to the to-be-detected item B is determined to be 100-105. Among them, the to-be-detected item A is to detect whether the rubber ring in the automobile exhaust pipe is installed, and the specific detection item of the to-be-detected item B is to detect whether the installation position of the exhaust pipe is correct.

[0066] By determining the items to be inspected for parts defect detection and parts installation detection according to the installation positions of the parts to be inspected, the detection process can be targeted and comprehensive, accurately covering the quality assessment of parts before installation and position verification after installation, avoiding missing any key inspection steps, thereby improving the accuracy and reliability of the overall inspection and effectively ensuring the quality and safety of vehicle assembly.

[0067] Step 106: receiving the image of the part to be inspected sent by the target shooting device for the current part to be inspected.

[0068] In practical applications, the target shooting device is a shooting device that obtains an image of the part to be inspected. Exemplarily, the target shooting device can be understood as a camera used to capture the image of the part to be inspected; it can also be understood as an automated inspection camera installed on a production line, used to monitor the status of the part in real time; it can also be understood as a mobile device with a multi-angle shooting function, used to capture images of different positions of the part. These examples are for reference only and this manual does not impose any restrictions on them.

[0069] It should be noted that due to the different specifications of the parts to be inspected and the different inspection aspects of the items to be inspected corresponding to the parts to be inspected, the items to be inspected also have different choices for the rolling target shooting equipment, and different types of target shooting equipment will obtain different images of the parts to be inspected.

[0070] Taking the inspection item of detecting whether the parts are defective as an example, the target shooting equipment with different specifications of the parts obtains the images of the parts to be inspected, and the obtained part inspection images are the images of the parts taken on the spot. For example, for larger parts, such as car doors, antenna glue or four-door belts, industrial cameras can be used in combination with IoT devices to ensure full coverage and high-precision recognition; for medium-sized parts, such as surface quality inspection of battery casing welds or sealing rings, conventional cameras can be selected with lighting devices for shooting to ensure image clarity and accurate presentation of details; for tiny parts, such as surface coating defect inspection of welded battery cells or QR codes on chips, precision cameras can be used in combination with 3D scanning equipment to obtain high-resolution images for refined inspection to ensure that the parts are free of defects or flaws.

[0071] Taking the example of detecting whether the part is the type of part to be installed, the part image taken on site can be used as the part image to be detected. The QR code image in the image captured by the one-dimensional code or two-dimensional code scanning device can also be used as the part image to be detected to further confirm the accuracy and matching of the parts, thereby ensuring that the part type during the installation process is consistent with expectations and avoiding assembly problems caused by incorrect parts.

[0072] Taking the thermal effect or temperature distribution of the parts to be tested as an example, when the target shooting device is selected as a thermal imaging device, the temperature distribution map and thermal anomalies of the parts can be obtained. For example, for larger parts, such as the thermal conductivity of car doors or antenna glue, the distribution map of the parts at different temperatures can be captured by thermal imaging equipment to ensure its uniformity and integrity; for medium-sized parts, such as battery casings or solder joints, thermal imaging equipment can be used to detect whether their operating temperature meets the design standards and to promptly discover abnormal heating areas; for tiny parts, such as the heat distribution of welded battery cells or the operating temperature of chips, high-resolution thermal imaging equipment can be used to obtain detailed thermal images for accurate detection to ensure the reliability and safety of the parts in actual work.

[0073] By receiving the images of the parts to be inspected sent by the target shooting device for the target project for part inspection, the accuracy and reliability of the inspection process can be ensured, errors and missed inspections caused by manual operations can be avoided, and the most suitable shooting equipment can be automatically selected according to different inspection requirements, thereby improving inspection efficiency and accuracy. It also supports real-time monitoring and rapid feedback, thereby effectively ensuring product quality.

[0074] Step 108: Input the image of the part to be detected into a part detection model, and obtain the part detection result output by the part detection model.

[0075] In practical applications, the part detection model is a deep learning model that performs part detection on the image of the part to be detected, and the part detection result is the result obtained by the image of the part to be detected corresponding to the item to be detected.

[0076] Exemplarily, the part detection model can be understood as an algorithm model that learns and recognizes part features through partial training data. For example, a model that extracts and classifies part images through a convolutional neural network; a model that detects and analyzes the shape, size, and defects of parts through a multi-layer neural network architecture; or an intelligent detection system that can automatically identify and mark abnormalities in part images. These explanations are only for illustrative purposes and are not limited in this specification.

[0077] The part inspection result can be understood as the quality assessment and judgment information obtained by analyzing the part image. For example, it can be the result of identifying the surface defects of the part to determine whether the part has cracks, dents, etc.; it can also be the result of measuring the size and shape of the part to determine whether the part meets the predetermined specifications and tolerances; it can also be the result of checking the installation position of the part to verify whether the part is correctly installed on the vehicle to ensure its normal function. These explanations are only for example purposes and this manual does not impose any restrictions on this.

[0078] It should be noted that due to the different types of target shooting equipment, there are different types of images collected for parts to be inspected. The part detection model recorded in this solution can process the above-mentioned different types of images, thereby realizing accurate feature extraction and anomaly detection regardless of whether it is high-definition images, thermal imaging images or three-dimensional scanning images. This not only improves the applicability and flexibility of detection, but also ensures the detection accuracy in different scenarios and conditions, significantly improving the efficiency and effectiveness of overall production quality control.

[0079] Furthermore, the image of the part to be detected is input into a part detection model to obtain a part detection result output by the part detection model, including:

[0080] Inputting the image of the part to be detected into the image segmentation module of the part detection model, and obtaining a close-up image of the part output by the image segmentation module;

[0081] The part close-up image is input into the part identification module of the part detection model to obtain the part detection result output by the part identification module.

[0082] In practical applications, the image segmentation module is a deep learning network module that extracts images with a high degree of correlation with the corresponding parts to be detected from the images of the parts to be detected. The close-up images of the parts are images with a high degree of correlation with the corresponding parts to be detected. The part discrimination module is a deep learning network module that detects the parts to be detected.

[0083] Exemplarily, the image segmentation module can be understood as a module for identifying and separating the parts to be detected from the background processing through a neural network. The specific method of using the neural network to identify and separate the parts to be detected from the background can be a convolutional neural network to achieve separation of the parts to be detected from the background, or it can be an edge detection algorithm to extract the contour of the part from a complex image, or it can be an intelligent analysis unit that automatically generates candidate areas according to the characteristics of different parts and focuses on key parts, etc. This specification does not impose any restrictions on this.

[0084] The part identification module can be understood as a processing module that extracts and classifies features of close-up images of parts to be detected through a neural network. The specific method of using a neural network to detect the parts to be detected in close-up images of parts can be a convolutional neural network for identifying the shape, size and defects of parts, or a multi-layer perceptron for classifying and distinguishing the material and surface features of parts, or a deep learning algorithm can be combined to automatically determine whether the parts are qualified according to the historical data and training model of the parts to ensure the accuracy and reliability of the detection. This manual does not impose any restrictions on this.

[0085] It should be noted that, considering the cost of implementing the automatic detection method, the preferred image segmentation module can be obtained by training any image that includes multiple entities, and the training samples do not include images of the parts to be detected corresponding to each vehicle to be detected. However, since the part identification module is used to obtain the part detection results corresponding to each part, the part identification module needs to be trained using the images of each part to be detected and the detection results corresponding to each image.

[0086] By obtaining images with a high degree of correlation between the parts to be inspected through the image segmentation module and then obtaining the detection results of the corresponding parts to be inspected through the part identification module, the accuracy and efficiency of detection can be effectively improved, and the accurate separation and feature extraction of parts in complex backgrounds can be achieved, ensuring the reliability and consistency of the detection results. At the same time, it can adapt to the detection needs of various types of parts, thereby improving the overall quality control level and reducing the risk of misjudgment and missed detection.

[0087] Furthermore, the image of the part to be detected is input into the image segmentation module of the part detection model, and a close-up image of the part output by the image segmentation module is obtained, including:

[0088] The image of the part to be detected is input into the first image feature extraction layer of the image segmentation module, and the entity feature image output by the first image feature extraction layer is obtained;

[0089] Inputting the entity feature image into the candidate region generation layer of the image segmentation module, obtaining at least one candidate region output by the candidate region generation layer, and a bounding box and a target score corresponding to each candidate region;

[0090] Input the target score corresponding to each candidate region into the candidate region generation layer, and obtain the entity category corresponding to each candidate region output by the classification recognition layer;

[0091] A part close-up image is determined in the part image to be detected according to the entity category corresponding to each candidate area.

[0092] In practical applications, the first image feature extraction layer is a deep learning layer that extracts the main feature information of each region from the image of the part to be detected. The entity feature image is the image data containing the main feature information of each region after being processed by the first image feature extraction layer. The candidate region generation layer is a deep learning layer that divides the entity feature image into multiple regions containing the same entity. The candidate region is the image region that includes the same entity in the image of the part to be detected. The bounding box is the boundary range of each entity in the image marked by the candidate region generation layer. The target score is the score value of the candidate region generation layer for evaluating the possibility of the entity category corresponding to each candidate region. The classification and recognition layer is an analysis layer that classifies the entities in the candidate region and determines the specific entity category. The entity category is the entity category corresponding to the entity in each candidate region output by the classification and recognition layer.

[0093] Exemplarily, the method of determining the close-up image of a part in the image of the part to be detected according to the entity category corresponding to each candidate area can select the area containing the target part according to the entity category of the candidate area, and crop the area to generate a close-up image; it can also combine multiple entity categories, aggregate related candidate areas, and form a close-up image containing multiple part features; it can also perform weighted processing on candidate areas of different entity categories through weight allocation to generate a clearer and more accurate close-up image. These methods are only used as examples, and this specification does not impose any restrictions on them.

[0094] In one embodiment provided in this specification, the structure of the image segmentation module is as follows: Figure 2As shown in the figure, the image of the part to be detected is first input into the first image feature extraction layer, which is used to extract the main feature information of each position in the image and generate an entity feature image. The entity feature image is then transmitted to the candidate region generation layer, which divides multiple candidate regions by analyzing the image features, and generates the bounding box and corresponding target score of each candidate region. Then, the target scores of these candidate regions are input into the classification recognition layer, the entities in the candidate regions are classified, and the entity category of each candidate region is output.

[0095] The first image feature extraction layer extracts the main feature information from the image of the part to be detected, and generates an entity feature image containing the key details of the part. Then, the candidate area generation layer is used to divide multiple candidate areas that may contain the same entity based on the entity feature image, and mark the bounding box and target score of each candidate area. Then, the classification and recognition layer is used to analyze and classify these candidate areas, and determine the specific entity category in each area, ensuring the accuracy of subsequent part close-up images, thereby improving the accuracy and efficiency of overall part detection, and realizing accurate recognition and classification of different areas in the image of the part to be detected, ensuring the accuracy of subsequent determination of the part close-up images, thereby improving the accuracy and efficiency of part detection.

[0096] Further, determining a close-up image of a part in the image of the part to be detected according to the entity category corresponding to each candidate area includes:

[0097] Obtain the entity category of the current part to be detected corresponding to the current part to be detected;

[0098] Determine the target candidate area according to the entity category corresponding to each candidate area and the entity category of the current part to be detected;

[0099] A part close-up image is determined in the to-be-detected part image based on a bounding box corresponding to the target candidate area.

[0100] In practical applications, the entity category of the current part to be detected is the category corresponding to the current part to be detected, and the target candidate area is an area whose entity category is the same as the category corresponding to the current part to be detected.

[0101] Exemplarily, the method of determining the close-up image of the part in the image of the part to be detected based on the bounding box corresponding to the target candidate area can directly use the coordinates of the bounding box to crop the image, thereby generating a close-up image containing the target part; it is also possible to expand a certain pixel range on the basis of the bounding box to generate a larger range of close-up images to ensure the integrity of the target part; it is also possible to merge multiple overlapping or adjacent bounding boxes to generate a close-up image containing multiple related features, and so on. This specification does not impose any restrictions on this.

[0102] By determining in the image of the part to be inspected that the image corresponding to the candidate area of ​​the same category as the current entity category of the part to be inspected is the close-up image of the part, it can be ensured that the close-up image accurately reflects the key features of the target part, reducing the influence of interference information, thereby improving the accuracy and efficiency of detection, and effectively ensuring the reliability of subsequent detection and analysis processes, ultimately achieving precise control and optimization of part quality.

[0103] Considering that the overall quality of the vehicle can be evaluated using the inspection results of various parts on the vehicle, after obtaining the parts inspection results output by the parts inspection model, the method further includes:

[0104] Obtaining the parts detection results corresponding to each part to be detected of the vehicle to be detected;

[0105] The quality inspection result corresponding to the vehicle to be inspected is generated according to the inspection results of each part.

[0106] In practical applications, the quality inspection result is the result obtained by evaluating the overall quality of the vehicle to be inspected using the inspection results of each part on the vehicle to be inspected. For example, the quality inspection result can be understood as the judgment information on whether the vehicle quality is qualified or not obtained after comprehensive analysis of the inspection results of each part.

[0107] In one embodiment provided in this specification, the parts to be inspected corresponding to the vehicle to be inspected include key parts such as doors, engines, and transmissions. After evaluating the appearance, size, defects, and installation positions of the above parts by calling the parts inspection model through multiple items to be inspected, the system summarizes the inspection results of each part and generates an overall quality inspection report for the vehicle. If the qualified scores of the inspection results of all key parts exceed the high-quality threshold, the vehicle is judged to be of qualified quality; if the qualified scores of the parts inspection results are low, the system will generate detailed quality inspection results and prompt the production lines corresponding to the parts with low qualified scores for timely adjustments.

[0108] Quality inspection results can be used to identify possible defects or problems in the vehicle's production and assembly process, thereby providing a basis for further quality improvement and control to ensure the reliability and safety of the final product.

[0109] Considering that the part detection result may be unqualified, after obtaining the part detection result output by the part detection model, the method further includes:

[0110] When the inspection result of the part corresponding to the current item to be inspected is failed, the current part to be inspected corresponding to the item to be inspected is reinstalled; and the step of receiving the image of the part to be inspected sent by the target shooting device for the current part to be inspected is performed.

[0111] For example, a failure in the part inspection result can be understood as a pass score of the part inspection result being lower than the pass threshold, for example, the dimensional deviation of the part exceeds the tolerance range required by the design, there are obvious scratches or defects on the surface of the part, the installation position of the part is offset and cannot be correctly aligned with the design position, or the type of the part does not match the preset part type, etc. These situations will cause the part inspection result to fail, thereby triggering the reinstallation and inspection process. This specification does not impose any restrictions on this.

[0112] By reinstalling parts that fail inspection, quality problems caused by improper installation or position deviation can be corrected to ensure that the parts meet the design requirements. At the same time, it helps to improve the accuracy of overall inspection and avoid the quality of the entire vehicle being affected by problems with individual parts. It can also reduce unnecessary scrapping of parts, save production costs and improve production efficiency, thereby improving the reliability and consistency of the final product.

[0113] Applying the solution of the embodiment of this specification, first determine the current inspection item of the vehicle to be inspected through the vehicle's position information. After determining the current inspection item, the system will automatically call the corresponding target shooting device to capture the image of the part to form an image of the part to be inspected. Then, the system inputs the image of the part to be inspected into the part detection model for processing. Through the image segmentation module of the model, a close-up image of the part is first generated, and then various features in the close-up image are analyzed to realize automatic detection of the parts involved in the current inspection item, thereby comprehensively evaluating the various key components of the vehicle to be inspected and ensuring the accuracy and efficiency of the detection.

[0114] The following combination Figure 3 Taking the application of the parts detection method provided in this specification in automobile assembly detection as an example, the parts detection method is further described. Figure 3 A processing flow chart of an automobile assembly detection method provided in an embodiment of the present specification is shown, which specifically includes the following steps.

[0115] Step 302: receiving the vehicle type of the currently assembled vehicle, and determining the items to be inspected of the currently assembled vehicle according to the vehicle type of the vehicle.

[0116] Step 304: receiving the current position information of the currently assembled vehicle, and determining the current item to be inspected of the currently assembled vehicle according to the current position information.

[0117] Step 306: sending an image acquisition instruction to the camera corresponding to the current item to be inspected, and receiving the image of the part to be inspected acquired by the camera corresponding to the current item to be inspected.

[0118] Step 308: Input the image of the part to be detected into the image segmentation module of the part detection model, and obtain the close-up image of the part output by the image segmentation module.

[0119] Step 310: Input the above-mentioned close-up image of the part into the part identification module of the part detection model to obtain the part detection result output by the part identification module.

[0120] Step 312: Determine whether the above-mentioned part inspection result is passed, if not, execute step 314, if so, execute step 316.

[0121] Step 314: Reinstall the parts to be inspected corresponding to the current item to be inspected, and return to step 306.

[0122] Step 316: Continue to assemble the currently assembled vehicle, and determine whether the current item to be detected is the last item to be detected of the currently assembled vehicle. If not, return to step 304; if yes, execute step 318.

[0123] Step 318: Determine that the assembly of the currently assembled vehicle is completed.

[0124] By applying the solution of the embodiments of this specification, the entire vehicle inspection process achieves efficient, accurate and comprehensive quality assurance through accurate determination of the items to be inspected, automated image acquisition and processing, intelligent feedback of inspection results, and smooth connection between assembly and inspection. This process not only reduces manual intervention and improves the accuracy and consistency of inspection, but also ensures seamless connection between inspection and assembly, thereby significantly improving the overall quality of the final assembly of the vehicle and the work efficiency of the production line.

[0125] Corresponding to the above method embodiment, this specification also provides a parts detection device embodiment, Figure 4 FIG. 1 is a schematic diagram showing the structure of a parts detection device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0126] The acquisition module 402 is configured to acquire the current position information corresponding to the vehicle to be detected;

[0127] The determination module 404 is configured to determine the current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect the current part to be detected, and the current part to be detected is the part to be detected of the vehicle to be detected at the current position information;

[0128] The receiving module 406 is configured to receive the image of the part to be inspected sent by the target shooting device for the current part to be inspected;

[0129] The detection module 408 is configured to input the image of the part to be detected into a part detection model, and obtain the part detection result output by the part detection model.

[0130] Optionally, the determining module 404 is further configured to:

[0131] Acquire at least one item to be detected corresponding to the vehicle to be detected, and acquire location range information of each item to be detected;

[0132] In the case that the current position information belongs to the target position range information, it is determined that the item to be detected corresponding to the target position range information is the current item to be detected, wherein the target position range information is any one of the position range information.

[0133] Optionally, the determining module 404 is further configured to:

[0134] Obtaining the vehicle type corresponding to the vehicle to be detected;

[0135] At least one to-be-detected item corresponding to the to-be-detected vehicle is determined according to the vehicle type.

[0136] Optionally, the determining module 404 is further configured to:

[0137] Determine a part detection method and part installation position information corresponding to a reference item to be detected, wherein the reference item to be detected is any one of the items to be detected;

[0138] In the case where the part detection method is part defect detection, the starting position information of the position range information corresponding to the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the ending position information of the position range information corresponding to the reference item to be detected;

[0139] In the case where the part detection method is part installation detection, the end position information of the corresponding position range information of the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the starting position information of the corresponding position range information of the reference item to be detected.

[0140] Optionally, the detection module 408 is further configured to:

[0141] Inputting the image of the part to be detected into the image segmentation module of the part detection model, and obtaining a close-up image of the part output by the image segmentation module;

[0142] The part close-up image is input into the part identification module of the part detection model to obtain the part detection result output by the part identification module.

[0143] Optionally, the detection module 408 is further configured to:

[0144] The image of the part to be detected is input into the first image feature extraction layer of the image segmentation module, and the entity feature image output by the first image feature extraction layer is obtained;

[0145] Inputting the entity feature image into the candidate region generation layer of the image segmentation module, obtaining at least one candidate region output by the candidate region generation layer, and a bounding box and a target score corresponding to each candidate region;

[0146] Input the target score corresponding to each candidate area into the classification recognition layer, and obtain the entity category corresponding to each candidate area output by the classification recognition layer;

[0147] A part close-up image is determined in the part image to be detected according to the entity category corresponding to each candidate area.

[0148] Optionally, the detection module 408 is further configured to:

[0149] Obtain the entity category of the current part to be detected corresponding to the current part to be detected;

[0150] Determine the target candidate area according to the entity category corresponding to each candidate area and the entity category of the current part to be detected;

[0151] A part close-up image is determined in the to-be-detected part image based on a bounding box corresponding to the target candidate area.

[0152] Optionally, the parts detection device further includes a vehicle quality inspection module configured to:

[0153] Obtaining the parts detection results corresponding to each part to be detected of the vehicle to be detected;

[0154] The quality inspection result corresponding to the vehicle to be inspected is generated according to the inspection results of each part.

[0155] Optionally, the parts detection device further includes a parts reassembly module configured to:

[0156] When the inspection result of the part corresponding to the current item to be inspected is failed, the current part to be inspected corresponding to the item to be inspected is reinstalled; and the step of receiving the image of the part to be inspected sent by the target shooting device for the current part to be inspected is performed.

[0157] Applying the solution of the embodiment of this specification, first determine the current inspection item of the vehicle to be inspected through the vehicle's position information. After determining the current inspection item, the system will automatically call the corresponding target shooting device to capture the image of the part to form an image of the part to be inspected. Then, the system inputs the image of the part to be inspected into the part detection model for processing. Through the image segmentation module of the model, a close-up image of the part is first generated, and then various features in the close-up image are analyzed to realize automatic detection of the parts involved in the current inspection item, thereby comprehensively evaluating the various key components of the vehicle to be inspected and ensuring the accuracy and efficiency of the detection.

[0158] The above is a schematic scheme of a part detection device of this embodiment. It should be noted that the technical scheme of the part detection device and the technical scheme of the above-mentioned part detection method belong to the same concept, and the details not described in detail in the technical scheme of the part detection device can be referred to the description of the technical scheme of the above-mentioned part detection method.

[0159] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0160] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0161] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0162] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 may also be a mobile or stationary server.

[0163] The processor 520 is used to execute the following computer executable instructions, which implement the steps of the above-mentioned parts detection method when executed by the processor.

[0164] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned parts detection method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned parts detection method.

[0165] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned parts detection method when executed by a processor.

[0166] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned parts detection method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned parts detection method.

[0167] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned parts detection method when executed by a processor.

[0168] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned parts detection method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned parts detection method.

[0169] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0170] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0171] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0173] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A parts detection method, characterized in that: include: Obtain the current position information corresponding to the vehicle to be detected; Determine the current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect the current part to be detected, and the current part to be detected is the part to be detected of the vehicle to be detected at the current position information; Receiving an image of a part to be inspected sent by a target photographing device for the current part to be inspected; The image of the part to be detected is input into a part detection model to obtain a part detection result output by the part detection model.

2. The method according to claim 1, characterized in that Determining the current to-be-detected item of the to-be-detected vehicle at the current position information includes: Acquire at least one item to be detected corresponding to the vehicle to be detected, and acquire location range information of each item to be detected; In the case that the current position information belongs to the target position range information, it is determined that the item to be detected corresponding to the target position range information is the current item to be detected, wherein the target position range information is any one of the position range information.

3. The method according to claim 2, characterized in that Acquiring at least one item to be detected corresponding to the vehicle to be detected, including: Obtaining the vehicle type corresponding to the vehicle to be detected; At least one to-be-detected item corresponding to the to-be-detected vehicle is determined according to the vehicle type.

4. The method according to claim 2, characterized in that Get the location range information of each item to be detected, including: Determine a part detection method and part installation position information corresponding to a reference item to be detected, wherein the reference item to be detected is any one of the items to be detected; In the case where the part detection method is part defect detection, the starting position information of the position range information corresponding to the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the ending position information of the position range information corresponding to the reference item to be detected; In the case where the part detection method is part installation detection, the end position information of the corresponding position range information of the reference item to be detected is determined according to the preset detection distance threshold and the part installation position information, and the part installation position information is determined as the starting position information of the corresponding position range information of the reference item to be detected.

5. The method according to claim 1, characterized in that Inputting the image of the part to be detected into a part detection model, and obtaining the part detection result output by the part detection model, comprises: Inputting the image of the part to be detected into the image segmentation module of the part detection model, and obtaining a close-up image of the part output by the image segmentation module; The part close-up image is input into the part identification module of the part detection model to obtain the part detection result output by the part identification module.

6. The method according to claim 5, characterized in that Inputting the image of the part to be detected into the image segmentation module of the part detection model, and obtaining the close-up image of the part output by the image segmentation module, comprises: The image of the part to be detected is input into the first image feature extraction layer of the image segmentation module, and the entity feature image output by the first image feature extraction layer is obtained; Inputting the entity feature image into the candidate region generation layer of the image segmentation module, obtaining at least one candidate region output by the candidate region generation layer, and a bounding box and a target score corresponding to each candidate region; Input the target score corresponding to each candidate area into the classification recognition layer, and obtain the entity category corresponding to each candidate area output by the classification recognition layer; A part close-up image is determined in the part image to be detected according to the entity category corresponding to each candidate area.

7. The method according to claim 6, characterized in that Determining a close-up image of a part in the image of the part to be detected according to the entity category corresponding to each candidate area includes: Obtain the entity category of the current part to be detected corresponding to the current part to be detected; Determine the target candidate area according to the entity category corresponding to each candidate area and the entity category of the current part to be detected; A part close-up image is determined in the to-be-detected part image based on a bounding box corresponding to the target candidate area.

8. The method according to claim 1, characterized in that After obtaining the part detection result output by the part detection model, the method further includes: Obtaining the parts detection results corresponding to each part to be detected of the vehicle to be detected; The quality inspection result corresponding to the vehicle to be inspected is generated according to the inspection results of each part.

9. The method according to claim 1, characterized in that After obtaining the part detection result output by the part detection model, the method further includes: When the inspection result of the part corresponding to the current item to be inspected is failed, the current part to be inspected corresponding to the item to be inspected is reinstalled; and the step of receiving the image of the part to be inspected sent by the target shooting device for the current part to be inspected is performed.

10. A parts detection device, characterized in that: include: An acquisition module is configured to acquire current position information corresponding to the vehicle to be detected; A determination module is configured to determine a current item to be detected of the vehicle to be detected at the current position information, wherein the current item to be detected is used to detect a current part to be detected, and the current part to be detected is a part to be detected of the vehicle to be detected at the current position information; A receiving module is configured to receive an image of a part to be detected sent by a target shooting device for the current part to be detected; The detection module is configured to input the image of the part to be detected into the part detection model and obtain the part detection result output by the part detection model.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing a computer program / instruction, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images