Automobile part detection method, device and system and storage medium
Through automatic detection methods, the problems of low efficiency and poor quality of down-line inspection of traditional automotive parts stations are solved, efficient and accurate inspection is achieved, defects of manual inspection are avoided, and the reliability and data recording of inspection are ensured.
Patent Information
- Application Number
- CN202411108108.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-26
AI Technical Summary
The offline inspection of traditional automotive parts stations relies on manual visualization, which is inefficient and prone to fatigue and omissions. It is impossible to intuitively judge the detection items, and it is impossible to record and detect images and perform data analysis, resulting in quality and safety hazards.
Using automatic detection method, by detecting whether preset events have occurred, querying the sort of detection items, controlling the camera to take pictures and obtain images, and calling the identification method of the target detection items to determine whether the automobile parts are qualified, avoiding manual visual inspection, and improving detection efficiency and quality .
Automatic detection is realized, the error rate is reduced, the detection quality is improved, the fatigue and omission of manual detection is avoided, and the accuracy and traceability of detection is ensured.
Smart Images

Figure CN119027389A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automatic detection technology, and in particular to an automobile component detection method, device, system and storage medium. Background Art
[0002] In the production process of automobile parts and assemblies, the last link of the production process is the off-line inspection, which means that the assembly and inspection of automobile parts have been completed. It can be regarded as that the product has reached the factory standard and is ready to be delivered to customers. This process involves multiple complex inspection links. The traditional off-line inspection of automobile parts relies on manual visual inspection, which has many complicated processes. Not only is the work efficiency low, but also because the workpiece is large, the inspection items around the surface of the workpiece need to be counted one by one and repeated continuously, which is easy to cause fatigue and omissions. It is also impossible to intuitively and quantitatively judge inspection items such as pins, barcodes, characters, etc. In addition, it is impossible to record the inspection images manually, and it is impossible to trace the source and analyze the data. If there are problems, it will cause quality and safety accidents.
[0003] Therefore, how to provide an automobile parts inspection method to improve the inspection efficiency and inspection quality of automobile parts off-line has become a technical problem that needs to be solved urgently. Summary of the invention
[0004] The present application provides an automobile component detection method, device, system and storage medium, which are used to improve the detection efficiency and detection quality of automobile component offline workstations.
[0005] The present application provides an automobile component detection method, comprising:
[0006] Detect whether a preset event occurs;
[0007] When a preset event occurs, query the preset detection item ranking according to the preset event to determine the current task;
[0008] When the current task is to execute a target detection item, control the camera to move to a photographing position corresponding to the target detection item;
[0009] Controlling the camera to acquire an image at the photographing position;
[0010] Calling the recognition method corresponding to the target detection item number;
[0011] The image is identified by the identification method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified.
[0012] The beneficial effects of the present application are: it can automatically detect whether a preset event occurs; when a preset event occurs, the preset detection item ranking is queried according to the preset event to determine the current task; when the current task is to execute a target detection item, the camera is controlled to move to the photo-taking position corresponding to the target detection item; the camera is controlled to obtain the image at the photo-taking position; the recognition method corresponding to the target detection item number is called; the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automotive component is qualified. Since the method provided by the present application avoids manual visual detection, it not only improves the detection efficiency, but also detects the target detection item based on image recognition and preset recognition methods, reduces the error rate, and improves the detection quality.
[0013] In one embodiment, the detecting whether a preset event occurs includes:
[0014] When it is detected that the automobile parts are transported from the roller conveyor to the inspection station, it is determined that a preset event occurs;
[0015] The step of querying a preset detection item sequence according to the preset event to determine the current task includes:
[0016] The first detection item in the detection item ranking is queried, and a current task is determined to be executing the first detection item.
[0017] In one embodiment, the detecting whether a preset event occurs includes:
[0018] When it is detected that the current detection item has been executed, it is determined that a preset event has occurred;
[0019] The step of querying a preset detection item sequence according to the preset event to determine the current task includes:
[0020] Query whether the current detection item in the detection item ranking is the last detection item;
[0021] When the current detection item is not the last detection item, the current task is determined to be executing the next detection item of the current detection item.
[0022] In one embodiment, the detecting whether a preset event occurs includes:
[0023] When it is detected that the current detection item has been executed, it is determined that a preset event has occurred;
[0024] The step of querying a preset detection item sequence according to the preset event to determine the current task includes:
[0025] Query whether the current detection item in the detection item ranking is the last detection item;
[0026] When the current inspection item is the last inspection item, it is determined that the current task is to judge whether the automobile component meets the requirements of the offline workstation according to the inspection results of all inspection items.
[0027] In one embodiment, when the target detection item is a plug detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified, including:
[0028] Determine the pin outline and the center coordinates of the pin in the plug type detection;
[0029] Calculate the positional relationship of each pin according to the center coordinates and contours of multiple pins;
[0030] Determine whether the pins are skewed based on the positional relationship of each pin;
[0031] When the pin is not skewed, it is determined that the target detection item is qualified.
[0032] In one embodiment, when the target detection item is a screw hole detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified, including:
[0033] The image is enhanced and the thread image is stretched into parallel stripes through regional correction;
[0034] Determining the number of parallel line segments in the thread image in the screw hole detection;
[0035] When the ratio of the number of parallel line segments in the thread image to the actual number of thread turns meets a preset ratio, it is determined that the target detection item is qualified.
[0036] In one embodiment, when the target detection item is a component detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile component is qualified, including:
[0037] Determine the target area where the parts are located;
[0038] Performing region cutting on the image according to the target region, and retaining only the target region;
[0039] Determine the target color of the component to be inspected by the correspondence between the preset area and the component color;
[0040] Selecting a corresponding channel filter according to the target color to obtain an image to be processed that highlights the target color;
[0041] Calculating the area of the target color in the image to be processed;
[0042] When the area of the target color is larger than the preset area, it is determined that the component has passed the inspection;
[0043] When all parts have passed the inspection, it is determined that the target inspection item is qualified.
[0044] The present application also provides an automobile component detection device, comprising:
[0045] A detection module, used to detect whether a preset event occurs;
[0046] A first determination module is used to query a preset detection item ranking according to the preset event to determine a current task when a preset event occurs;
[0047] A first control module, used for controlling the camera to move to a photographing position corresponding to the target detection item when the current task is to execute the target detection item;
[0048] A second control module, used for controlling the camera to acquire the image at the photographing position;
[0049] A calling module, used to call the recognition method corresponding to the target detection item number;
[0050] The judging module is used to identify the image by using the recognition method corresponding to the target detection item number to judge whether the target detection item of the automobile part is qualified.
[0051] In one embodiment, the detection module includes:
[0052] The first determination submodule is used to determine that a preset event occurs when it is detected that the automobile component is transported from the roller conveyor to the inspection station;
[0053] The first determining module includes:
[0054] The first query submodule is used to query the first detection item in the detection item ranking and determine that the current task is to execute the first detection item.
[0055] In one embodiment, the detection module includes:
[0056] A second determination submodule is used to determine the occurrence of a preset event when it is detected that the current detection item has been executed;
[0057] The first determining module includes:
[0058] A second query submodule is used to query whether the current detection item in the detection item ranking is the last detection item;
[0059] The third determination submodule is used to determine that the current task is to execute the next detection item of the current detection item when the current detection item is not the last detection item.
[0060] In one embodiment, the detection module includes:
[0061] A fourth determination submodule is used to determine the occurrence of a preset event when it is detected that the current detection item has been executed;
[0062] The first determining module includes:
[0063] A third query submodule is used to query whether the current detection item in the detection item ranking is the last detection item;
[0064] The fifth determination submodule is used to determine, when the current detection item is the last detection item, whether the current task is to judge whether the automobile component meets the requirements of the offline workstation according to the detection results of all detection items.
[0065] In one embodiment, when the target detection item is plug type detection, the judgment module includes:
[0066] A sixth determination submodule, used to determine the pin outline and the center coordinates of the pin in the plug type detection;
[0067] A first calculation submodule, used for calculating the positional relationship of each pin according to the center coordinates and contours of the plurality of pins;
[0068] A judgment submodule, used to judge whether the pins are skewed according to the position relationship of each pin;
[0069] The seventh determination submodule is used to determine that the target detection item is qualified when the pin is not skewed.
[0070] In one embodiment, when the target detection item is screw hole detection, the judgment module includes:
[0071] A correction submodule is used to enhance the image and stretch the thread image into parallel stripes through regional correction;
[0072] An eighth determination submodule, used to determine the number of parallel line segments in the thread image in the screw hole detection;
[0073] The ninth determination submodule is used to determine that the target detection item is qualified when the ratio of the number of parallel line segments in the thread image to the actual number of thread turns meets a preset ratio.
[0074] In one embodiment, when the target detection item is component detection, the judgment module includes:
[0075] The tenth determination submodule is used to determine the target area where the parts are located;
[0076] A cutting submodule, used for performing region cutting on the image according to the target region, and only retaining the target region;
[0077] An eleventh determination submodule is used to determine the target color of the component to be detected by a correspondence between a preset area and the color of the component;
[0078] A selection submodule, used for selecting a corresponding channel filter according to the target color to obtain an image to be processed that highlights the target color;
[0079] A second calculation submodule, used for calculating the area of the target color in the image to be processed;
[0080] A twelfth determination submodule, configured to determine that the component has passed the inspection when the area of the target color is larger than a preset area;
[0081] The thirteenth determination submodule is used to determine that the target detection item is qualified when all parts have passed the detection.
[0082] The present application also provides an automobile component detection system, comprising:
[0083] at least one processor; and,
[0084] a memory communicatively connected to the at least one processor; wherein,
[0085] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the automobile component detection method recorded in any of the above embodiments.
[0086] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the automobile component detection system, the automobile component detection system can implement the automobile component detection method recorded in any of the above embodiments.
[0087] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.
[0088] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0089] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:
[0090] Figure 1 This is a flow chart of a method for detecting automobile parts in one embodiment of the present application;
[0091] Figure 2 This is a structural schematic diagram of an automobile component detection device in one embodiment of the present application;
[0092] Figure 3 The figure is a schematic diagram of the hardware structure of an automobile component detection system in one embodiment of the present application. DETAILED DESCRIPTION
[0093] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application.
[0094] Figure 1 Flow chart of a method for detecting automobile parts in one embodiment of the present application. Figure 1 As shown, the method can be implemented as the following steps S101-S106:
[0095] In step S101, detecting whether a preset event occurs;
[0096] In step S102, when a preset event occurs, a preset detection item ranking is queried according to the preset event to determine the current task;
[0097] In step S103, when the current task is to execute a target detection item, the camera is controlled to move to a photographing position corresponding to the target detection item;
[0098] In step S104, the camera is controlled to acquire an image at the photographing position;
[0099] In step S105, the recognition method corresponding to the target detection item number is called;
[0100] In step S106, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified.
[0101] In the present application, it is detected whether a preset event occurs, wherein the preset event is the time marking the detection progress of each detection item of the automobile component, such as a trigger event, an end event, etc. Specifically, it can be when the automobile component is transported from the roller to the detection station, when the current detection item is completed, or when a preset execution instruction is received.
[0102] When a preset event occurs, the preset detection item ranking is queried according to the preset event to determine the current task. In the present application, the preset event of the automobile component station offline and the detection items to be performed are stored in advance, and the corresponding relationship between the preset event and each detection item, as well as the detection order of each detection item are stored. Then, when the preset event occurs, the preset detection item ranking can be queried according to the preset event to determine the current task. For example, when it is detected that the automobile component is transported from the roller to the detection station, it is determined that the preset event occurs. At this time, the first detection item in the detection item ranking is queried, and the current task is determined to execute the first detection item. For another example, when it is detected that the current detection item has been executed, it is determined that the preset event occurs; at this time, the current detection item in the detection item ranking is queried to determine whether it is the last detection item; when the current detection item is not the last detection item, the current task is determined to execute the next detection item of the current detection item; when the current detection item is the last detection item, it is determined that the current task is to determine whether the automobile component meets the requirements of the offline station according to the detection results of all detection items.
[0103] It is understandable that, due to different inspection standards for different automobile parts, in one embodiment of the present application, when a preset event is detected, before querying the preset inspection item ranking according to the preset event to determine the current task, the camera is first controlled to obtain the automobile part identification code to determine the corresponding current task for the automobile part, that is, querying the preset inspection item ranking according to the automobile part identification code and the preset event to determine the current task. The automobile part identification code can be in various forms such as data, character string, QR code, etc., which is not limited by the present application.
[0104] When the current task is to execute a target detection item, the camera is controlled to move to the photo position corresponding to the target detection item, wherein the target detection item is to detect whether each component of the automobile component is qualified. That is to say, when the current detection item is not the last detection item, each detection item needs to be detected in sequence according to the preset detection item sequence. When performing the detection, since the detection items and the detection item sequence required for the automobile component are predetermined, the corresponding photo position can be determined according to each detection item, the preset motion trajectory of the camera can be determined, and the camera can be controlled to move to the photo position corresponding to the target detection item according to the preset motion trajectory.
[0105] When the camera moves to the photographing position corresponding to the target detection item, the camera is controlled to obtain the image at the photographing position. It is understandable that when the current camera has multiple photographing tasks at the photographing position, for example, it is necessary to take multiple feature images of the same position, at this time, the multiple photographing tasks can be stored in the independent thread queue of the camera according to the current photographing position and the detection order of the target detection item, and the camera is called up and the feature image is taken after the queue is released in turn.
[0106] Call the identification method corresponding to the target detection item number. The target detection item number can be the part number of the corresponding part. Since the detection methods for different parts are different, the detection methods can be stored in advance, and the corresponding identification method can be called by the target detection item number. The identification method includes but is not limited to plugs, screw holes, regular shape parts, scale size measurement, QR code and other detections, such as pin detection, thread detection, parts detection, oil cap detection, etc.
[0107] The image is identified by the identification method corresponding to the target detection item number to determine whether the target detection item of the automotive component is qualified. Specifically, when the target detection item is a plug detection, the pin contour and the center coordinates of the pin in the plug detection are determined; the positional relationship of each pin is calculated based on the center coordinates and contours of multiple pins; whether the pin is skewed based on the positional relationship of each pin; when the pin is not skewed, the target detection item is determined to be qualified. When the target detection item is a screw hole detection, the image is enhanced, and the thread image is stretched into parallel stripes through regional correction; the number of parallel line segments in the thread image in the screw hole detection is determined; when the ratio of the number of parallel line segments in the thread image to the actual number of threads meets the preset ratio, the target detection item is determined to be qualified. When the target detection item is component detection, wherein the component may be a dust cap, determine the target area where the component is located; perform region cutting on the image according to the target area, and retain only the target area; determine the target color of the component to be detected by the correspondence between the preset area and the component color; select the corresponding channel filter according to the target color to obtain the image to be processed that highlights the target color; calculate the area of the target color in the image to be processed; when the area of the target color is larger than the preset area, determine that the component detection has passed; when all components have passed the detection, determine that the target detection item is qualified.
[0108] In a specific embodiment, a highly complex robot vision inspection system has been developed to achieve the multi-type error-proofing detection, QR code recognition, OCR recognition, and image storage and traceability requirements of various workpieces required for quality inspection before the finished product goes off the line. At the same time, the robot carries a visual camera to increase the flexibility of imaging. The robot realizes real-time communication with the visual system through kernel language development. The visual system captures, processes, stores images, and other multi-threaded asynchronous executions to achieve high-definition, high-speed, and high-complexity visual inspection applications. Specifically, the machine vision inspection system includes a visual part, a control part, and an execution part. The visual part is mainly composed of an industrial camera, a lens, and a light source. The control part includes an industrial computer, an image acquisition card, and a light source controller. The execution part can be a robot.
[0109] Among them, the industrial camera of the visual part can be a high-definition digital camera, for example, a 500W high-pixel color industrial CCD camera can be selected, and in order to ensure image quality and processing efficiency, a photosensitive chip with a larger pixel size and 8-bit image depth is selected; the lens part uses a 35 focal length lens with the same resolution of 500W; the light source part uses a white ring light source. The ring light source is placed in the center of the lens, and the aperture is adjusted to obtain sufficient depth of field and field of view contrast. In terms of detection algorithm, in order to enhance the gap characteristics and avoid changes in grayscale values caused by changes in environmental factors such as lighting, the filter tool is first added to the visual system to highlight the grayscale difference between the gap and the surrounding area so that the detection items can be stably and accurately identified.
[0110] The control part includes an industrial computer, an image acquisition card and a light source controller to perform real-time image acquisition, storage and analysis according to the current task, and issue control commands to complete the current task.
[0111] The execution part uses two specially designed position robots in this embodiment, and the visual parts are fixed at the ends of the two sets of robots. Considering the light source interference, material interference and roller position, the two robots are placed on the same side, and the detection area and action path are allocated at the same time, so that the camera always maintains a nearly vertical or staggered angle in most cases, so that the light sources of the two cameras do not appear in the field of view of each other.
[0112] In order to complete the detection of automobile parts, a software platform architecture that can adapt to various processing situations is developed based on the project in this embodiment. It is written in C# language, and uses independent threads and blocking queues to control the camera and complete the interaction. In each independent camera, a settable number of independent processes are allocated. Each independent process can load a visual algorithm containing multiple judgment parameters of independent phase conditions and asynchronous operation, and contains an independent yield and result counting system, and can save pictures and data at the same time, and directly transmit the results for communication in the form of event transmission, thereby achieving high-beat detection of dozens of different detection items on the entire automobile part surface through two sets of camera vision systems, and ensuring the high availability and stability of the program. For the outer structure, an inherited expandable subclass mode is adopted, which can be customized and expanded by field engineers through C# language, and multiple interfaces including guidance, deep learning, and digital monitoring management are reserved. The visual processing part can use industrial visual algorithm software, such as VisionPro, halcon, mvtec, etc. In one embodiment of the present application, a visual inspection algorithm generated by the script control of the VisionPro tool plug-in is used, and the tool editing entrance and related interfaces are also opened, so that engineers can easily adjust the algorithm script. The platform program designs the plug-in control process and includes the code module generated by the script. The data part supports the use of SQL Server enterprise database, allowing the call of stored procedures to control data collection and processing. There is an independent thread responsible for the automatic cleaning of images and data to prevent storage overflow. The communication part uses self-encoded Socket communication. The two robots establish two sets of encrypted communications respectively, interact with the program, and establish communication with the PLC to transmit product identification characters to achieve three-party communication.
[0113] In this embodiment, when the automobile parts are inspected, the automobile parts are transported to the inspection station by rollers. At this time, the PLC (Programmable Logic Controller) learns that the automobile parts are in place, and transmits the information of the arrival of the automobile parts to the upper computer vision part and the robot through the signal to the control part. The upper computer vision part obtains the automobile parts identification code, and the robot detects the occurrence of the preset event, and determines the current task by querying the preset inspection items according to the preset event; then, along the preset motion trajectory, the camera is controlled to move to the photographing position corresponding to the target inspection item, and the robot is controlled to send the trigger signal to the visual part through the TCP protocol. The visual part judges the robot and the photographing position and part number of the current signal by interpreting the TCP code, and uses the information as a trigger to press into the independent thread queue of the camera operation, and waits for the queue to be released before calling up the camera and shooting the current feature to collect the original image. At the same time, the recognition method corresponding to the target inspection item number is called, wherein the target inspection item number can be the part number, and then the collected image can be identified by the recognition method to determine whether the target inspection item is qualified. Taking the engine inspection as an example, in one embodiment, when the engine is inspected, the following inspections are included:
[0114] 1. Pin detection: First, the initial positioning based on feature training is used to narrow the processing range, so that the pixel processing points can be reduced as much as possible during the operation to improve the operation speed. During the processing, the opening and closing operations are used and the pixel grayscale value after image processing is subtracted to obtain the pin outline. Then, the pin features are obtained by performing an image operation with the pins eliminated by expansion, and the center coordinates of each pin are obtained by substituting them into the regional analysis. The position relationship is obtained after calculation in the script to detect whether the pins are skewed.
[0115] 2. Thread detection: The initial positioning based on feature training is used to narrow the processing range, so that the pixel processing points can be reduced as much as possible during the operation, thereby improving the operation speed. Contrast enhancement is used during processing, and the threads are stretched into parallel stripes through 180° regional correction, so as to find the parallel line segments of the corresponding threads at the alternating light and dark places, and the number of parallel line segments is judged to meet 80% of the number of thread turns. The thread is recognized as existing and valid. At the same time, in order to prevent scratches and bruises from being mixed in, thread area contrast analysis is added to obtain the black and white pixel variance of the area for judgment.
[0116] 3. Dust cap detection: Dust caps in different positions have different color characteristics. Since color images occupy a high memory space, the color image is first segmented to retain the effective information to improve the image processing speed. Then, the color gamut is calculated for the area. For the red dust cap, the RGB (2, -1, -1) channel filter is used, and for the green dust cap, the RGB (-1, 1.5, -0.5) channel filter is used to obtain a processed image that highlights the detection color. Then, the feature calculation area is captured through the regional tool to determine whether there is any.
[0117] 4. Oil cap detection: The initial positioning based on feature training is adopted to narrow the processing range. At the same time, the new reference coordinate position is obtained as the initial positioning. The surface image of the part is matted using opening and closing operations. The ORC tool is used for character training and recognition. The recognized characters are compared with the character codes transmitted by the PLC to verify that the character codes are consistent. The line finding tool is used to match the feature line segments in the new coordinate system to obtain the oil cap angle, thereby comprehensively judging the detection items including the oil cap angle and model.
[0118] 5. Dipstick detection: Since the dipstick is yellow, use RGB (0.5, 1, -1.5) channel filter for color separation, grab the Blob area of the oil pool, and calculate the pixel size. Since the rotation angle of the dipstick in the axial direction is uncertain, it is necessary to judge based on the length. Set the caliper to extract the edge through the light and dark intersection line at the head and tail, calculate the distance between the edges, and then get the dipstick length, match it with the model in the parameter setting, and confirm the dipstick model.
[0119] 6. Gripper detection: By adjusting the camera angle to 82° slightly perpendicular to the gripper, the reflected light from the black glossy surface is filtered out to obtain a clear photo of the gripper. The interference of the frosted aluminum surface in the background is removed by corroding and dilating the image by 35 pixels. The shape of the gripper is extracted using the Blob tool and the area and grayscale are calculated to determine whether the area is within the preset area range, thereby determining whether the gripper exists.
[0120] 7. QR code recognition: dilate and erode the original image by 55 pixels to obtain a QR code square image in a completely black area. This is used as a training feature to obtain a template matching model for initial positioning. In the initial positioning coordinate system, a search area of 160% of the QR code area is established to narrow the QR code search range and improve efficiency. VisionPro's QR code recognition is used to identify the QR codes of batches of automotive parts. The product QR code is compared with the character code transmitted by the PLC and displayed on the interface. If it matches, the QR code recognition is completed.
[0121] A single model of engine involves a total of 35 visual inspection items, including 8 pins, 12 threads, 8 dust caps, 1 oil cap, 1 dipstick, 3 grippers, and 2 QR code recognitions. Since the inspection items are distributed on the five end faces of the engine except the bottom face, they are assigned to one robot according to the upper end face and the left and left end faces of the front face, and the right side of the upper end face and the right side of the front face and the right end face are assigned to the second robot. I control the direction of the light source on the two robots to always take pictures in a 60-120 degree direction or in a parallel staggered direction, so as to avoid interference from the opposite light source and achieve traversal of all inspection items on the five end faces of the engine. When the camera is triggered to shoot, the camera will transmit the shooting completion signal to the current robot through TCP, and the robot will start to walk to the next point and wait. At this time, the camera will copy the original image into two copies, one of which is stored locally as required, and the other is visually processed. After the processing is completed, the results are fed back to the robot through events to avoid the empty scanning cycle of the thread loop. At the same time, the processed layers and compressed images are superimposed, and a separate thread pool is called to store images and data. After receiving the result feedback, the robot determines whether it has arrived at the next photo-taking point. After arriving at the next point, it determines the position of another robot. If there is no interference, it starts the next photo-taking cycle, so that image processing does not occupy the robot's movement time, achieving the effect of high-speed and beat-saving. When all points are detected, the upper computer system will feedback an end signal to the robot, and the robot will notify the PLC whether to release it. If there is NG, the picture will be sent to the human for secondary confirmation, thereby realizing a full range of quality inspection functions.
[0122] The beneficial effects of the present application are: it can automatically detect whether a preset event occurs; when a preset event occurs, the preset detection item ranking is queried according to the preset event to determine the current task; when the current task is to execute a target detection item, the camera is controlled to move to the photo-taking position corresponding to the target detection item, wherein the target detection item is to detect whether each component of the automotive component is qualified; the camera is controlled to obtain the image at the photo-taking position; the recognition method corresponding to the target detection item number is called; the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automotive component is qualified. Since the method provided by the present application avoids manual visual detection, it not only improves the detection efficiency, but also detects the target detection item based on image recognition and preset recognition methods, reduces the error rate, and improves the detection quality.
[0123] In one embodiment, the above step S101 may be implemented as the following step A1:
[0124] In step A1, when it is detected that the automobile component is transported from the roller conveyor to the inspection station, it is determined that a preset event occurs;
[0125] The above step S102 can be implemented as the following step A2:
[0126] In step A2, the first detection item in the detection item ranking is queried, and it is determined that the current task is to execute the first detection item.
[0127] In one embodiment, the above step S101 may be implemented as the following step B1:
[0128] In step B1, when it is detected that the current detection item is executed, it is determined that a preset event occurs;
[0129] The above step S102 can be implemented as the following steps B2-B3:
[0130] In step B2, query whether the current detection item in the detection item ranking is the last detection item;
[0131] In step B3, when the current detection item is not the last detection item, the current task is determined to be the next detection item to execute the current detection item.
[0132] In one embodiment, the above step S101 may be implemented as the following step C1:
[0133] In step C1, when it is detected that the current detection item is executed, it is determined that a preset event occurs;
[0134] The above step S102 can be implemented as the following steps C2-C3:
[0135] In step C2, query whether the current detection item in the detection item ranking is the last detection item;
[0136] In step C3, when the current inspection item is the last inspection item, it is determined that the current task is to judge whether the automobile component meets the requirements of the offline workstation according to the inspection results of all inspection items.
[0137] In one embodiment, when the target detection item is plug detection, the above step S106 can be implemented as the following steps D1-D4:
[0138] In step D1, the pin outline and the center coordinates of the pin in the plug type detection are determined;
[0139] In step D2, the positional relationship of each pin is calculated according to the center coordinates and contours of the plurality of pins;
[0140] In step D3, whether the pins are skewed is determined based on the positional relationship of the pins;
[0141] In step D4, when the pin is not skewed, it is determined that the target detection item is qualified.
[0142] In this embodiment, the pin outline and the center coordinates of the pins in the plug-type detection are determined. For example, when detecting an engine, each pin is located at a fixed position of the engine, usually 4 pins, and the 4 pins are evenly distributed and arranged in parallel. Therefore, when judging whether the pin is skewed based on the positional relationship of each pin, if the positional relationship of each pin indicates that the center position distances of each pin are different, it means that the pin is skewed. In addition, if the positional relationship of each pin indicates that the pins are not parallel, it also means that the pin is skewed. Therefore, the pin outline is obtained through image recognition, and the center coordinates of each pin are obtained by substituting it into the regional analysis.
[0143] Then, the positional relationship of each pin is calculated based on the center coordinates and contours of the multiple pins. Specifically, a calculation model can be pre-built and stored, and then the positional relationship of each pin is calculated by substituting the center coordinates and contours into the calculation model, and whether the pin is skewed is determined based on the positional relationship of each pin. When the pin is not skewed, it is determined that the target detection item is qualified.
[0144] It is understandable that in order to improve the operating efficiency of the system, before determining the pin outline and the center coordinates of the pin in the plug detection, the initial positioning based on feature training can also be used to narrow the processing range, so that the pixel processing points can be reduced as much as possible during the operation to improve the operation speed. During the processing, the pin outline is obtained by using the opening and closing operation and subtracting the pixel grayscale value after image processing, and then the pin features are obtained by the image operation with the expansion elimination pin, and the center coordinates of each pin are obtained by substituting them into the regional analysis. The position relationship is obtained after calculation in the script to detect whether the pin is skewed.
[0145] In one embodiment, when the target detection item is screw hole detection, the above step S106 can be implemented as the following steps E1-E3:
[0146] In step E1, the image is enhanced and the thread image is stretched into parallel stripes by regional correction;
[0147] In step E2, the number of parallel line segments in the thread image in the screw hole detection is determined;
[0148] In step E3, when the ratio of the number of parallel line segments in the thread image to the actual number of thread turns meets a preset ratio, it is determined that the target detection item is qualified.
[0149] In this embodiment, the initial positioning based on feature training can also be used to narrow the processing range, so that the pixel processing points can be reduced as much as possible during the operation, thereby improving the operation speed.
[0150] The image is enhanced during processing, and the thread image is stretched into parallel stripes through regional correction; specifically, the thread is stretched into parallel stripes through 180° regional correction.
[0151] The number of parallel line segments in the thread image in the screw hole detection is determined; after the thread is stretched into parallel stripes, the parallel line segments corresponding to the thread are found at the alternating light and dark positions.
[0152] When the ratio of the number of parallel line segments in the thread image to the actual number of threads meets the preset ratio, the target detection item is determined to be qualified. For example, if the number of parallel line segments meets 80% of the number of threads, the thread is recognized to exist and is valid. At the same time, in order to prevent scratches and bruises from being mixed in, thread area contrast analysis is added to obtain the black and white pixel variance of the area for judgment.
[0153] In one embodiment, when the target detection item is component detection, the above step S106 can be implemented as the following steps F1-F7:
[0154] In step E1, a target area where the component is located is determined;
[0155] In step E2, the image is region-cut according to the target region, and only the target region is retained;
[0156] In step E3, the target color of the component to be detected is determined by the correspondence between the preset area and the component color;
[0157] In step E4, a corresponding channel filter is selected according to the target color to obtain an image to be processed that highlights the target color;
[0158] In step E5, the area of the target color in the image to be processed is calculated;
[0159] In step E6, when the area of the target color is larger than the preset area, it is determined that the component has passed the inspection;
[0160] In step E7, when all parts have passed the inspection, it is determined that the target inspection item is qualified.
[0161] Figure 2 FIG. 1 is a schematic diagram of a structure of an automobile component detection device in one embodiment of the present application. Figure 2 As shown, the device comprises:
[0162] A detection module 201 is used to detect whether a preset event occurs;
[0163] A first determination module 202 is used to query a preset detection item ranking according to the preset event to determine a current task when a preset event occurs;
[0164] The first control module 203 is used to control the camera to move to a photographing position corresponding to the target detection item when the current task is to execute the target detection item, wherein the target detection item is to detect whether each component of the automobile component is qualified;
[0165] A second control module 204 is used to control the camera to obtain the image at the photographing position;
[0166] A calling module 205 is used to call the recognition method corresponding to the target detection item number;
[0167] The judgment module 206 is used to recognize the image by using the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified.
[0168] In one embodiment, the detection module includes:
[0169] The first determination submodule is used to determine that a preset event occurs when it is detected that the automobile component is transported from the roller conveyor to the inspection station;
[0170] The first determining module includes:
[0171] The first query submodule is used to query the first detection item in the detection item ranking and determine that the current task is to execute the first detection item.
[0172] In one embodiment, the detection module includes:
[0173] A second determination submodule is used to determine the occurrence of a preset event when it is detected that the current detection item has been executed;
[0174] The first determining module includes:
[0175] A second query submodule is used to query whether the current detection item in the detection item ranking is the last detection item;
[0176] The third determination submodule is used to determine that the current task is to execute the next detection item of the current detection item when the current detection item is not the last detection item.
[0177] In one embodiment, the detection module includes:
[0178] A fourth determination submodule is used to determine the occurrence of a preset event when it is detected that the current detection item has been executed;
[0179] The first determining module includes:
[0180] A third query submodule is used to query whether the current detection item in the detection item ranking is the last detection item;
[0181] The fifth determination submodule is used to determine, when the current detection item is the last detection item, whether the current task is to judge whether the automobile component meets the requirements of the offline workstation according to the detection results of all detection items.
[0182] In one embodiment, when the target detection item is plug type detection, the judgment module includes:
[0183] A sixth determination submodule, used to determine the pin outline and the center coordinates of the pin in the plug type detection;
[0184] A first calculation submodule, used for calculating the positional relationship of each pin according to the center coordinates and contours of the plurality of pins;
[0185] A judgment submodule, used to judge whether the pins are skewed according to the position relationship of each pin;
[0186] The seventh determination submodule is used to determine that the target detection item is qualified when the pin is not skewed.
[0187] In one embodiment, when the target detection item is screw hole detection, the judgment module includes:
[0188] A correction submodule is used to enhance the image and stretch the thread image into parallel stripes through regional correction;
[0189] An eighth determination submodule, used to determine the number of parallel line segments in the thread image in the screw hole detection;
[0190] The ninth determination submodule is used to determine that the target detection item is qualified when the ratio of the number of parallel line segments in the thread image to the actual number of thread turns meets a preset ratio.
[0191] In one embodiment, when the target detection item is component detection, the judgment module includes:
[0192] A tenth determination submodule is used to determine the target area where the parts are located;
[0193] A cutting submodule, used for performing region cutting on the image according to the target region, and only retaining the target region;
[0194] An eleventh determination submodule is used to determine the target color of the component to be detected by a correspondence between a preset area and the color of the component;
[0195] A selection submodule, used for selecting a corresponding channel filter according to the target color to obtain an image to be processed that highlights the target color;
[0196] A second calculation submodule, used for calculating the area of the target color in the image to be processed;
[0197] A twelfth determination submodule, configured to determine that the component has passed the inspection when the area of the target color is larger than a preset area;
[0198] The thirteenth determination submodule is used to determine that the target detection item is qualified when all parts have passed the detection.
[0199] Figure 3 FIG. 1 is a schematic diagram of the hardware structure of an automobile component detection system in one embodiment of the present application. Figure 3 As shown, the automobile component detection system comprises:
[0200] at least one processor 320; and,
[0201] A memory 304 in communication with the at least one processor 320; wherein,
[0202] The memory 304 stores instructions that can be executed by the at least one processor 320 , and the instructions are executed by the at least one processor 320 to implement the automobile component detection method described in any of the above embodiments.
[0203] Reference Figure 3 The automobile component detection system 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .
[0204] The processing component 302 generally controls the overall operation of the automotive component detection system 300. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.
[0205] The memory 304 is configured to store various types of data to support the operation of the automobile component detection system 300. Examples of such data include instructions for any application or method operating on the automobile component detection system 300, such as text, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0206] The power component 306 provides power to various components of the automotive component detection system 300. The power component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the automotive component detection system 300.
[0207] The multimedia component 308 includes a screen that provides an output interface between the automotive component detection system 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 may also include a front camera and / or a rear camera. When the automotive component detection system 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0208] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the automobile component detection system 300 is in an operation mode, such as an alarm mode, a recording mode, a voice recognition mode, and a voice output mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.
[0209] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0210] The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the automotive component detection system 300. For example, the sensor assembly 314 may include a sound sensor. In addition, the sensor assembly 314 may detect the open / closed state of the automotive component detection system 300, the relative positioning of the components, such as the display and keypad of the automotive component detection system 300, and the sensor assembly 314 may also detect the operating state of the automotive component detection system 300 or a component of the automotive component detection system 300, the orientation or acceleration / deceleration of the automotive component detection system 300, and the temperature change of the automotive component detection system 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, a material accumulation thickness sensor, or a temperature sensor.
[0211] The communication component 316 is configured to enable the automobile component detection system 300 to provide the ability to communicate with other devices and cloud platforms in a wired or wireless manner. The automobile component detection system 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0212] In an exemplary embodiment, the automobile component detection system 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the automobile component detection method described in any of the above embodiments.
[0213] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the automobile component detection system, the automobile component detection system can implement the automobile component detection method recorded in any of the above embodiments.
[0214] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0215] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0216] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0217] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0218] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for detecting automobile parts, characterized in that: include: Detect whether a preset event occurs; When a preset event occurs, query the preset detection item ranking according to the preset event to determine the current task; When the current task is to execute a target detection item, control the camera to move to a photographing position corresponding to the target detection item; Controlling the camera to acquire an image at the photographing position; Calling the recognition method corresponding to the target detection item number; The image is identified by the identification method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified.
2. The method according to claim 1, characterized in that The detecting whether a preset event occurs includes: When it is detected that the automobile parts are transported from the roller conveyor to the inspection station, it is determined that a preset event occurs; The step of querying a preset detection item sequence according to the preset event to determine the current task includes: The first detection item in the detection item ranking is queried, and a current task is determined to be executing the first detection item.
3. The method according to claim 1, characterized in that The detecting whether a preset event occurs includes: When it is detected that the current detection item has been executed, it is determined that a preset event has occurred; The step of querying a preset detection item sequence according to the preset event to determine the current task includes: Query whether the current detection item in the detection item ranking is the last detection item; When the current detection item is not the last detection item, the current task is determined to be executing the next detection item of the current detection item.
4. The method according to claim 1, characterized in that The detecting whether a preset event occurs includes: When it is detected that the current detection item has been executed, it is determined that a preset event has occurred; The step of querying a preset detection item sequence according to the preset event to determine the current task includes: Query whether the current detection item in the detection item ranking is the last detection item; When the current inspection item is the last inspection item, it is determined that the current task is to judge whether the automobile component meets the requirements of the offline workstation according to the inspection results of all inspection items.
5. The method according to claim 1, characterized in that When the target detection item is a plug detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile component is qualified, including: Determine the pin outline and the center coordinates of the pin in the plug type detection; Calculate the positional relationship of each pin according to the center coordinates and contours of multiple pins; Determine whether the pins are skewed based on the positional relationship of each pin; When the pin is not skewed, it is determined that the target detection item is qualified.
6. The method according to claim 1, characterized in that When the target detection item is a screw hole detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile part is qualified, including: The image is enhanced and the thread image is stretched into parallel stripes through regional correction; Determining the number of parallel line segments in the thread image in the screw hole detection; When the ratio of the number of parallel line segments in the thread image to the actual number of thread turns meets a preset ratio, it is determined that the target detection item is qualified.
7. The method according to claim 1, characterized in that When the target detection item is a component detection, the image is recognized by the recognition method corresponding to the target detection item number to determine whether the target detection item of the automobile component is qualified, including: Determine the target area where the parts are located; Performing region cutting on the image according to the target region, and retaining only the target region; Determine the target color of the component to be inspected by the correspondence between the preset area and the component color; Selecting a corresponding channel filter according to the target color to obtain an image to be processed that highlights the target color; Calculating the area of the target color in the image to be processed; When the area of the target color is larger than the preset area, it is determined that the component has passed the inspection; When all parts have passed the inspection, it is determined that the target inspection item is qualified.
8. An automobile component detection device, characterized in that: include: A detection module, used to detect whether a preset event occurs; A first determination module is used to query a preset detection item ranking according to the preset event to determine a current task when a preset event occurs; A first control module, used for controlling the camera to move to a photographing position corresponding to the target detection item when the current task is to execute the target detection item; A second control module, used for controlling the camera to acquire the image at the photographing position; A calling module, used to call the recognition method corresponding to the target detection item number; The judging module is used to identify the image by using the recognition method corresponding to the target detection item number to judge whether the target detection item of the automobile part is qualified.
9. An automobile component detection system, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the automobile component detection method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor corresponding to the automobile component detection system, the automobile component detection system can implement the automobile component detection method according to any one of claims 1 to 7.
