Defect dynamic annotation methods, systems, storage media and terminals

Abstract

This invention provides a method, system, storage medium, and terminal for dynamic defect annotation, comprising: acquiring motion information and defect information of an object to be annotated; acquiring dynamic defect information of the object to be annotated based on the motion information and defect information; and projecting the dynamic defect information onto the object to be annotated in real time to visualize the annotated defects. This invention, based on a defect detection system to acquire the location and type information of defects, uses a projection device to perform real-time positioning and dynamic projection of defects on the object to be annotated, effectively solving the problem that static defect marking cannot meet the needs of high-speed assembly line production in factories; providing accurate defect annotation information and work guidance for repair stations, reducing the repair area and improving repair efficiency; and having a wide range of applications, not only applicable to the tracking and positioning of defects on vehicle body surfaces and the real-time visualization of annotated defects, but also seamlessly integrating with automated optical defect detection lines and used for assembly guidance of large workpieces.

Description

Technical Field

[0001] This invention relates to the field of defect detection, and in particular to a method, system, storage medium, and terminal for dynamic defect labeling. Background Technology

[0002] Surface defect detection is an essential part of the manufacturing process in industries such as automobile manufacturing, aviation, precision machinery manufacturing, power batteries, 3C products (computers, communications, and consumer electronics), and handicrafts. For large components such as automobiles and aircraft, it is necessary to accurately detect surface defects after paint spraying, mark the location and type of surface defects, and then carry out further grinding, polishing, and quality assessment based on the defects.

[0003] In traditional automotive body paint spraying production lines, surface defects are detected manually by visual inspection and marking. This method is labor-intensive, causes eye fatigue, has a high probability of missed defects, low accuracy, and cannot guarantee the quality of inspection. Furthermore, it lacks the ability to record and statistically analyze defects and guide improvements to the spraying process. Recalls of automobiles due to surface coating defects are common.

[0004] Automated defect detection systems based on industrial robots, machine vision, and deep learning are increasingly being used in automotive paint spraying production lines. The paint defects and their locations on the car body are evaluated, classified, and recorded in real time by the defect analysis system. After the automated defect detection station on the car surface, the detected paint defects need to be manually repaired, such as by sanding and polishing. Using robots to spray different colors onto the car paint surface is a common method. However, during the color selection and marking process, the robot inevitably needs to stop its current inspection sequence and switch to a pen-changing marking process, which inevitably reduces inspection efficiency. Since automotive paint surfaces require 100% inspection, the inspection cycle and efficiency requirements are strict. Furthermore, because automotive paint colors are diverse, the color of the paint surface must be understood during pen-changing marking to select easily observable inkjet colors, further reducing inspection efficiency.

[0005] Therefore, there is a need to propose a defect labeling system that can meet the needs of high-speed production lines in factories and improve defect detection efficiency. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method, system, storage medium and terminal for dynamic defect annotation, so as to solve the problems of complex defect detection procedures and low detection efficiency in the prior art.

[0007] To achieve the above and other related objectives, a first aspect of the present invention provides a method for dynamic defect annotation, comprising: acquiring motion information and defect information of an object to be annotated; acquiring dynamic defect information of the object to be annotated based on the motion information and defect information; and projecting the dynamic defect information onto the object to be annotated in real time to annotate its defects.

[0008] In some embodiments of the first aspect of the present invention, the defect information includes defect range information, defect location information, and defect type information; the method includes: acquiring three-dimensional information of the object to be labeled based on structured light surface defect detection technology; acquiring the defect range information, defect location information, and defect type information based on the three-dimensional information using a defect recognition and classification algorithm combining image processing and deep learning; and projecting the defect location information, defect range information, and defect type information onto the object to be labeled in real time to label its defect location information, defect range information, and defect type information.

[0009] In some embodiments of the first aspect of the present invention, the method is applied to a vehicle paint spraying production line, the production line being provided with a defect detection station and a defect marking station in sequence; the method includes: a structured light phase-shift defect detection module and a laser projection defect marking module are respectively provided on the defect detection station and the defect marking station; when the vehicle moves to the defect detection station, the vehicle's motion information and defect information are acquired; when the vehicle moves to a trigger position, the laser projection defect marking module is triggered to acquire the dynamic defect position information of the vehicle in the coordinate system of the defect marking station in real time; the laser projection defect marking module changes its projection angle in real time based on the dynamic defect position information to project onto the object to be marked and refreshes the projection based on a preset refresh interval to mark its defect position.

[0010] In some embodiments of the first aspect of the present invention, the method includes: acquiring the vehicle's movable distance and vehicle movement information at the defect marking station; acquiring vehicle movement time information based on the vehicle's movable distance and vehicle movement information; and controlling the working state of the laser projection defect marking module based on the vehicle movement time information.

[0011] To achieve the above and other related objectives, a second aspect of the present invention provides a dynamic defect annotation system, comprising: a first information acquisition module for acquiring motion information and defect information of an object to be annotated; a second information acquisition module for acquiring dynamic defect information of the object to be annotated based on the motion information and defect information; and an annotation module for projecting the dynamic defect information onto the object to be annotated in real time to annotate its defects.

[0012] In some embodiments of the second aspect of the present invention, the system includes: a sensor for acquiring the motion information of the object to be labeled; a defect detection system for acquiring the defect information of the object to be labeled; a processor communicatively connected to or integrated thereon with the sensor and the defect detection system for receiving the motion information and the defect information and calculating and acquiring the dynamic defect information; and a defect labeling system communicatively connected to the processor for receiving the dynamic defect information and projecting it onto the object to be labeled in real time to label its defects.

[0013] In some embodiments of the second aspect of the present invention, the defect detection system includes: a structured light surface defect detection system based on an industrial robot, comprising: a light source for projecting structured light stripes onto the surface of an object to be labeled after phase shifting; and an imaging system for receiving the structured light stripes reflected from the surface of the object to be labeled and demodulating them to obtain phase information of the surface being inspected, and obtaining defect information of the object to be labeled based on phase anomaly information in the obtained phase information.

[0014] In some embodiments of the second aspect of the present invention, the defect detection system is applied to the quality inspection of a pipeline bending production line, wherein the defect marking system projects the characteristic pattern of the pipeline onto the actual pipeline to obtain pipeline error information.

[0015] To achieve the above and other related objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic defect annotation method.

[0016] To achieve the above and other related objectives, a fourth aspect of the present invention provides an electronic terminal, comprising: a processor and a memory; the memory for storing a computer program, and the processor for executing the computer program stored in the memory, so that the terminal performs the defect dynamic annotation method.

[0017] As described above, the defect dynamic annotation method, system, storage medium, and terminal proposed in this invention have the following beneficial effects: This invention acquires the location and type information of defects based on a defect detection system, and uses a projection device (such as laser projection) to perform real-time positioning and dynamic projection of defects on the object to be annotated (e.g., surface defects on a moving automotive body paint spraying production line). This provides targets for personnel at defect grinding and polishing workstations, effectively solving the problem that static defect marking cannot meet the needs of high-speed production lines in factories. Furthermore, this invention has a wide range of applications; it can not only annotate defects on automotive paint surfaces but also dynamically annotate, track, and guide the characteristics of large components and moving workstation targets, such as quality inspection in pipe bending production lines. Attached Figure Description

[0018] Figure 1 The diagram shown is a flowchart of a dynamic defect annotation method according to an embodiment of the present invention.

[0019] Figure 2 The diagram shown is a flowchart illustrating a method for marking vehicle surface defects according to an embodiment of the present invention.

[0020] Figure 3 The diagram shown is a structural schematic of a laser dynamic projection annotation system according to an embodiment of the present invention.

[0021] Figure 4A The diagram shown illustrates an application scenario of a laser dynamic projection annotation system according to an embodiment of the present invention.

[0022] Figure 4B The diagram shows an application scenario of another laser dynamic projection annotation system according to an embodiment of the present invention.

[0023] Figure 4C The diagram shows an application scenario of another laser dynamic projection annotation system according to an embodiment of the present invention.

[0024] Figure 5 The diagram shown is a structural schematic of a dynamic defect annotation system according to an embodiment of the present invention.

[0025] Figure 6 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of the present invention. Detailed Implementation

[0026] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0027] It should be noted that in the following description, reference is made to the accompanying drawings, which illustrate several embodiments of the present invention. It should be understood that other embodiments may also be used, and changes in mechanical composition, structure, electrical components, and operation may be made without departing from the spirit (core technology) and scope of the present invention. The following detailed description should not be considered limiting, and the scope of the embodiments of the present invention is defined only by the claims of the published patents. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. Spatially related terms, such as “upper,” “lower,” “left,” “right,” “below,” “below,” “lower part,” “above,” “upper part,” etc., may be used in the text to illustrate the relationship between one element or feature shown in the figures and another element or feature.

[0028] Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, operation, element, component, item, kind, and / or group, but do not preclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition arise only when combinations of elements, functions, or operations are inherently mutually exclusive in some manner.

[0029] This invention provides a method, system, storage medium, and terminal for dynamic defect annotation, which solves the problems of complex defect detection procedures and low detection efficiency in the prior art.

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0031] Example 1

[0032] like Figure 1 The diagram illustrates a flowchart of a dynamic defect annotation method proposed in this embodiment of the invention. The dynamic defect annotation method specifically includes the following steps:

[0033] Step S11. Obtain the motion and defect information of the object to be labeled. Specifically, the object to be labeled can be a vehicle, an aircraft, a power battery, a 3C product, a handicraft, etc. These objects inevitably develop defects during the manufacturing process. On the production line, these objects are generally in motion. Therefore, to accurately obtain and label their defect information, it is necessary to first obtain their motion information, such as speed, acceleration, direction, displacement, and time. The defect information includes defect range information, defect location information, defect type information, etc.

[0034] Step S12. Obtain the dynamic defect information of the object to be labeled based on the motion information and defect information. Specifically, the position coordinates of the defects are dynamically calculated and corrected according to the motion information of the object to be labeled, and the position coordinates of the defects change synchronously with the movement of the object to be labeled.

[0035] Step S13. Based on the dynamic defect information, project the defect onto the object to be labeled in real time to label its defects. Specifically, a projection device is used to project the defect based on the obtained position coordinates of the defect that change synchronously with the object to be labeled. The mark projected onto the object to be labeled can represent the range of the defect through the coverage area of ​​the projection, and the type of defect can be represented by the projected text, color, etc. It can also synchronously project relevant information such as the defect repair method and defect repair personnel corresponding to the defect type.

[0036] In a preferred embodiment of this invention, the method includes: acquiring three-dimensional information of the object to be labeled based on structured light surface defect detection technology; acquiring defect range information, defect location information, and defect type information based on the three-dimensional information using a defect identification and classification algorithm combining image processing and deep learning; and projecting the defect location information, defect range information, and defect type information onto the object to be labeled in real time.

[0037] Specifically, the structured light surface defect detection technology employs a system consisting of a projection device and a camera. The projection device projects specific light information onto the object's surface and background, while the camera collects the changed light signals. Based on the changes in the light signals caused by the object, the system calculates information such as the object's position and depth, thereby obtaining the object's three-dimensional information. Based on this three-dimensional information, a trained deep learning model is used to detect defects in the object, obtaining information such as defect location, defect range, and defect type.

[0038] Optionally, the projection device used for real-time projection on the object to be labeled can be a CRT projector, LCD projector, DLP projector, DLV projector, laser projection device, etc. In a preferred embodiment, a laser projection device is used to project onto the object to be labeled in real time, dynamically annotating its defect information. The laser projection device includes a laser source and a scanning device. The laser source can be a gas laser or an all-solid-state laser, preferably an all-solid-state laser with high electro-optical efficiency and stability. The scanning device includes polyhedral rotating mirror scanning and galvanometer scanning, with galvanometer scanning being preferred, especially suitable for high-speed production lines in factories, and less limited than polyhedral rotating mirror scanning.

[0039] In a preferred embodiment of this invention, the method is applied to a vehicle paint spraying production line, which is provided with a defect detection station and a defect marking station in sequence. The method includes: a structured light phase-shift defect detection module and a laser projection defect marking module are respectively provided at the defect detection station and the defect marking station; when the vehicle moves to the defect detection station, the vehicle's motion information and defect information are acquired; when the vehicle moves to the trigger position, the laser projection defect marking module is triggered to acquire the dynamic defect position information of the vehicle in the coordinate system of the defect marking station in real time; the laser projection defect marking module changes its projection angle in real time based on the dynamic defect position information to project onto the object to be marked and refreshes the projection marking of its defect position based on a preset refresh interval.

[0040] In a preferred embodiment of this invention, the method includes: acquiring the vehicle's movable distance and vehicle movement information at the defect marking station; acquiring vehicle movement time information based on the vehicle's movable distance and vehicle movement information; controlling the working state of the laser projection defect marking module based on the vehicle movement time information. Within the vehicle movement time range, the laser projection defect marking module is in working state, projecting and marking defect information on the vehicle in real time. When the vehicle movement time is exceeded, it is determined that the vehicle has left the defect marking station, and the laser projection defect marking module switches from working state to waiting state.

[0041] like Figure 2As shown in the diagram, this invention proposes a flowchart for a method of marking vehicle surface defects. Specifically: a defect marking workstation coordinate system is established, and the vehicle's movement speed and the maximum allowable movement distance at the workstation are obtained; after the vehicle enters the zero position of the defect marking workstation, the zero-position sensor is triggered, and the laser projection device is started; a vehicle coordinate system is constructed based on the relative relationship between the vehicle coordinate system and the workstation coordinate system, and the defect coordinate values ​​on one side of the vehicle are obtained under the vehicle coordinate system; based on the relative relationship between the vehicle coordinate system and the workstation coordinate system, the defect coordinate values ​​on the other side of the vehicle are obtained under the vehicle coordinate system; laser projection devices installed on both sides of the vehicle simultaneously project the defect coordinate values ​​to mark the defects, updating the coordinates and refreshing the projection as the vehicle moves; the maximum movement time of the vehicle at the workstation is obtained based on the vehicle's movement speed and the maximum allowable movement distance at the workstation; when the vehicle's movement time at the workstation reaches the maximum movement time, i.e., the departure time, the laser projection device stops projecting and enters a waiting state, waiting for the next vehicle to enter; when the vehicle has not reached the maximum movement time, the projection is refreshed based on a preset refresh interval to ensure that the defect marking moves with the vehicle's movement, thereby improving the accuracy of the defect marking.

[0042] In some embodiments, the method can be applied to a controller, such as an ARM (Advanced RISC Machines) controller, an FPGA (Field Programmable Gate Array) controller, a SoC (System on Chip) controller, a DSP (Digital Signal Processing) controller, or an MCU (Microcontroller Unit) controller, etc. In some embodiments, the method can also be applied to a computer including components such as memory, a memory controller, one or more processing units (CPUs), peripheral interfaces, RF circuitry, audio circuitry, speakers, microphones, input / output (I / O) subsystems, displays, other output or control devices, and external ports; the computer includes, but is not limited to, personal computers such as desktop computers, laptops, tablets, smartphones, smart TVs, and personal digital assistants (PDAs). In other embodiments, the method can also be applied to a server, which can be deployed on one or more physical servers based on factors such as function and load, or can be composed of distributed or centralized server clusters.

[0043] Example 2

[0044] like Figure 3 As shown in the diagram, this embodiment of the invention proposes a structural schematic diagram of a laser dynamic projection marking system, which includes: a defect detection module 31 (structured light phase-shift defect detection module) and a laser projection module ( Figure 3 (Taking laser projection device A and laser projection device B as examples). The defect detection module 31 detects and acquires the defect information of the object to be labeled, and sends the defect information to the laser projection module. Based on the received defect information, the position information of the laser projection module, and the current motion status information of the production line, the laser projection module dynamically calculates and updates the real-time position information of the surface defects of the object to be labeled, thereby realizing the dynamic labeling of the target defects.

[0045] The laser dynamic projection marking system is applied to the marking of defects on the vehicle surface. Specifically, when the vehicle is in the automated defect detection station, the defect detection module 31 detects and acquires the vehicle's defect information (target point a1 and target point b1); when the vehicle begins to enter the laser dynamic marking projection station, it first passes through the station zero position and triggers the zero position sensor, causing the laser projection module to enter the working mode; the laser projection module combines the relative relationship between the vehicle coordinate system and the station coordinate system (establishing the station coordinate system by collecting global coordinate target points arranged in the station) to acquire the vehicle's dynamic defect information, and projects it onto target point a2 and target point b2 (i.e., the moved target point a1 and target point b1), marking the location, size, type, etc. of the vehicle defects.

[0046] In some examples, when the vehicle body moves to the zero position of the workstation, the zero-position sensor is triggered, activating the laser projection module to enter the projection program. The module acquires the real-time position of the target point in the workstation coordinate system and triggers the laser projection module to change the projection angle of the two-dimensional galvanometer to project the target point onto the workstation position. Using data from the vehicle body movement direction and speed sensors, the projected target point positions a2 and b2 at time t are obtained. The laser projection is then refreshed, projecting the markings onto positions a2 and b2. Based on the maximum distance the vehicle body moves and its speed at the workstation, the maximum movement time T of the vehicle body at the workstation can be determined. Once the vehicle has reached its maximum movement time T at the workstation, the laser projection stops, and a waiting program begins.

[0047] The laser dynamic projection marking system is applied to an automotive paint spraying production line. Specifically, the vehicle body moves on a conveyor belt in a fixed direction and at a certain speed. First, a structured light surface defect detection system based on industrial robots detects defects on the paint surface and records defect information, such as the size, type, and location of the defects. Then, the defect information and vehicle information are transmitted to the laser projection module. When the vehicle enters the defect repair station (laser dynamic marking projection station), the vehicle body triggers the zero-position sensor of the laser projection module, activating the laser projection module to project the markings. The operator at the repair station then performs grinding / polishing repairs based on the projected markings.

[0048] In a preferred embodiment of this invention, in the structured light phase-shifting defect detection module 31 (i.e., the structured light surface defect detection system based on an industrial robot), structured light is reflected from the highly reflective paint surface of the car being inspected onto a high-resolution industrial CCD via a high-resolution display screen. Surface defects on the paint surface will cause distortion of the imaging stripe image. The structured light stripes are projected sequentially onto the surface being inspected through phase shifting (e.g., 3-step or 4-step phase shifting), and are received sequentially by the industrial CCD imaging system after surface reflection. By demodulating the phase of the phase-shifting stripes, defects on the surface being inspected will cause anomalies in the demodulated phase. Based on the phase anomaly information, defects can be identified. The detected defects are then classified using a deep learning algorithm. Specifically, an industrial robot carrying the structured light phase-shifting defect detection module 31 (structured light stripe projection and receiving system) performs a comprehensive scan of the moving car body to obtain information on the size and type of defects on the car body, and records the location information of the defects based on the robot's motion coordinates.

[0049] In a preferred embodiment of this example, such as Figure 4A , 4B As shown in 4C, the laser dynamic projection annotation system proposed in this embodiment of the invention is applied to the quality inspection of a pipeline bending production line. Specifically, key features on the pipeline design drawings (such as center lines, outlines, arcs, bending positions, welding positions, etc.) are converted into laser projection files; laser projection equipment collects target points arranged on the production line to establish a global coordinate system; the pipeline is straightened by using a hard guide; the laser projection file containing pipeline features is projected onto the pipeline (e.g., ...). Figure 4A , 4B (As shown by the white line in 4C), obtain the error information between the actual pipe and the projected pattern, thereby realizing the quality inspection of the pipe bending production line.

[0050] Large boiler heat exchange pipes (typically over 20 meters in length) need to be transferred to a testing area after bending on the pipe bending production line for testing multiple parameters such as bending angle, arc, and welding position. This process usually involves manually drawing the pipe pattern in an open area and comparing the actual bent pipe with the drawn pattern to check the bending quality and obtain error information. This method requires manual drawing, pipe transfer, and manual comparison, resulting in high time and labor costs, and disrupting the pipeline production, welding, and assembly workflow. The above-described implementation, by applying the laser dynamic marking projection system proposed in this invention to the quality inspection of the pipe bending production line, enables online inspection of pipe bending quality. It eliminates the need to transfer pipes or draw pipe patterns, improving the efficiency of pipe inspection, saving labor costs, and ensuring that the pipeline production, welding, and assembly workflow remains uninterrupted, thus increasing production efficiency.

[0051] In summary, this invention proposes a laser dynamic projection marking system. Based on the defect information of the vehicle body surface to be inspected, the position information of the laser marking projection system, and the current movement status information of the production line, the system dynamically calculates and updates the real-time position information of the defects on the vehicle body surface, achieving dynamic marking of the target defects. The defect markings are converted into the projection angle of the laser projection device, updating the coordinates of the defect markings so that the projected marked defect position on the vehicle body remains unchanged during vehicle movement, improving the efficiency of the defect repair process. Simultaneously, the system projects the location and type information of the defects, improving the efficiency and accuracy of subsequent defect processing steps. By utilizing industrial robots, structured light phase-shifting fringe mirror imaging for 3D scanning and defect detection of automotive paint surfaces, combined with deep learning algorithms, defects are classified, accurately detecting their location and type information.

[0052] Example 3

[0053] like Figure 5 As shown in the figure, an embodiment of the present invention proposes a structural schematic diagram of a dynamic defect annotation system. The dynamic defect annotation system includes: a first information acquisition module 51, used to acquire motion information and defect information of the object to be annotated; a second information acquisition module 52, used to acquire dynamic defect information of the object to be annotated based on the motion information and defect information; and an annotation module 53, used to project the dynamic defect information onto the object to be annotated in real time to annotate its defects.

[0054] In a preferred embodiment of this invention, the system includes: a sensor for collecting motion information of the object to be labeled; a defect detection system for collecting defect information of the object to be labeled; a processor, communicatively connected to or integrated thereon with the sensor and the defect detection system, for receiving the motion information and defect information and calculating and obtaining the dynamic defect information; and a defect labeling system, communicatively connected to the processor, for receiving the dynamic defect information and projecting it onto the object to be labeled in real time to label its defects.

[0055] In a preferred embodiment of this invention, the defect detection system includes: a structured light surface defect detection system based on an industrial robot, comprising: a light source for projecting structured light stripes onto the surface of the object to be labeled after phase shifting; and an imaging system for receiving the structured light stripes reflected from the surface of the object to be labeled and demodulating them to obtain phase information of the surface being inspected, and obtaining defect information of the object to be labeled based on phase anomaly information in the obtained phase information.

[0056] It should be noted that the modules provided in this embodiment are similar to the methods and implementation methods provided above, and therefore will not be repeated. It should also be understood that the division of the various modules in the above device is merely a logical functional division; in actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can all be implemented in software through processing element calls; they can all be implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the first information acquisition module can be a separately established processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its function can be called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0057] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together to form a system-on-a-chip (SOC).

[0058] Example 4

[0059] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned dynamic defect annotation method.

[0060] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented using computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0061] Example 5

[0062] like Figure 6 As shown in the diagram, this embodiment of the invention provides a structural schematic of an electronic terminal. The electronic terminal provided in this embodiment includes: a processor 61, a memory 62, and a communicator 63; the memory 62 is connected to the processor 61 and the communicator 63 via a system bus and completes communication between them; the memory 62 is used to store computer programs; the communicator 63 is used to communicate with other devices; and the processor 61 is used to run the computer program, enabling the electronic terminal to execute the various steps of the above-described dynamic defect annotation method.

[0063] The system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0064] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0065] In summary, this invention provides a method, system, storage medium, and terminal for dynamic defect annotation. It utilizes a projection device to dynamically project information such as the location, size, and type of surface defects on the product to be inspected in real time at the repair station, achieving real-time visualization of the annotated defects and improving the efficiency of the defect repair process. The projection device seamlessly integrates with the defect detection system's defect data, eliminating the inkjet marking process commonly used in defect detection systems, significantly improving the system's cycle time and efficiency. It provides precise defect annotation information and work guidance to the repair station, reducing the repair area and increasing repair efficiency. Its application range is wide, not only applicable to tracking and locating defects on vehicle body surfaces, but also seamlessly integrating with automated optical defect detection lines for PCB boards, OLED curved screens, etc., providing visualized annotation and guidance for subsequent repairs. Furthermore, it can be used for assembly guidance of large workpieces, guiding the adjustment direction of assembly by projecting assembly reference points, lines, and surfaces. Therefore, this invention effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0066] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for dynamic defect annotation, characterized in that, include: Obtain motion and defect information of the object to be labeled; Based on the motion information and defect information, obtain the dynamic defect information of the object to be labeled; Based on the dynamic defect information, the defect is projected onto the object to be labeled in real time to label its defects; The defect information includes defect range information, defect location information, and defect type information; The method includes: acquiring three-dimensional information of the object to be labeled based on structured light surface defect detection technology; acquiring defect range information, defect location information, and defect type information based on the three-dimensional information using a defect recognition and classification algorithm combining image processing and deep learning; and projecting the defect location information, defect range information, and defect type information onto the object to be labeled in real time. The method is applied to a vehicle paint spraying production line, which is equipped with a defect detection station and a defect marking station in sequence. The method includes: a structured light phase-shift defect detection module and a laser projection defect marking module are respectively installed at the defect detection station and the defect marking station; when the vehicle moves to the defect detection station, the vehicle's motion information and defect information are acquired; when the vehicle moves to the trigger position, the laser projection defect marking module is triggered to acquire the dynamic defect position information of the vehicle in the coordinate system of the defect marking station in real time; the laser projection defect marking module changes its projection angle in real time based on the dynamic defect position information to project onto the object to be marked and refreshes the projection based on a preset refresh interval to mark its defect position. The process of obtaining dynamic defect information of the object to be annotated based on the motion information and defect information includes: obtaining the defect coordinate value of the object to be annotated in the defect annotation station coordinate system based on the relative relationship between the vehicle body coordinate system and the defect annotation station coordinate system. Based on the dynamic defect information, the defect is projected onto the object to be labeled in real time to mark its defects, including: laser projection defect labeling modules installed on both sides of the object to be labeled simultaneously projecting the defects to mark them; and updating the position of the projection target point in real time based on the movement direction and speed of the object to be labeled and refreshing the projection based on a preset refresh interval. The method further includes: acquiring the vehicle's movable distance and vehicle movement information at the defect marking station; acquiring vehicle movement time information based on the vehicle's movable distance and vehicle movement information; and controlling the working state of the laser projection defect marking module based on the vehicle movement time information. Within the vehicle's movement time, the laser projection defect marking module is in working state, projecting and marking defect information onto the vehicle in real time. When the vehicle's movement time is exceeded, it is determined that the vehicle has left the defect marking station, and the laser projection defect marking module switches from working state to waiting state.

2. A dynamic defect annotation system, characterized in that, include: The first information acquisition module is used to acquire motion information and defect information of the object to be labeled. The second information acquisition module is used to acquire dynamic defect information of the object to be labeled based on the motion information and defect information. The annotation module is used to project the dynamic defect information onto the object to be annotated in real time to annotate its defects; The defect information includes defect range information, defect location information, and defect type information; The system includes: acquiring three-dimensional information of the object to be labeled based on structured light surface defect detection technology; acquiring defect range information, defect location information, and defect type information based on the three-dimensional information using a defect recognition and classification algorithm combining image processing and deep learning; and projecting the defect location information, defect range information, and defect type information onto the object to be labeled in real time. The system is applied to a vehicle paint spraying production line, which is equipped with a defect detection station and a defect marking station in sequence. The system includes: a structured light phase-shift defect detection module and a laser projection defect marking module respectively installed at the defect detection station and the defect marking station; when the vehicle moves to the defect detection station, the system acquires the vehicle's motion information and defect information; when the vehicle moves to the trigger position, the system triggers the laser projection defect marking module to acquire the dynamic defect position information of the vehicle in the coordinate system of the defect marking station in real time; the laser projection defect marking module changes its projection angle in real time based on the dynamic defect position information to project onto the object to be marked and refreshes the projection based on a preset refresh interval to mark its defect position. The process of obtaining dynamic defect information of the object to be annotated based on the motion information and defect information includes: obtaining the defect coordinate value of the object to be annotated in the defect annotation station coordinate system based on the relative relationship between the vehicle body coordinate system and the defect annotation station coordinate system. Based on the dynamic defect information, the defect is projected onto the object to be labeled in real time to mark its defects, including: laser projection defect labeling modules installed on both sides of the object to be labeled simultaneously projecting the defects to mark them; and updating the position of the projection target point in real time based on the movement direction and speed of the object to be labeled and refreshing the projection based on a preset refresh interval. The system further includes: acquiring the vehicle's movable distance and vehicle movement information at the defect marking station; acquiring vehicle movement time information based on the vehicle's movable distance and vehicle movement information; and controlling the working state of the laser projection defect marking module based on the vehicle movement time information. Within the vehicle's movement time, the laser projection defect marking module is in working state, projecting and marking defect information onto the vehicle in real time. When the vehicle's movement time is exceeded, it is determined that the vehicle has left the defect marking station, and the laser projection defect marking module switches from working state to waiting state.

3. The defect dynamic annotation system according to claim 2, characterized in that, include: Sensors are used to collect the motion information of the object to be labeled; A defect detection system is used to collect the defect information of the object to be labeled; A processor, which is communicatively connected to or integrated thereon with the sensor and the defect detection system, is used to receive the motion information and defect information and calculate and obtain the dynamic defect information; The defect annotation system is communicatively connected to the processor and is used to receive the dynamic defect information and project it onto the object to be annotated in real time to annotate its defects.

4. The defect dynamic annotation system according to claim 3, characterized in that, The defect detection system includes: a structured light surface defect detection system based on an industrial robot, which includes: The light source is used to project structured light stripes onto the surface of the object to be labeled in sequence after phase shifting. An imaging system is used to receive and demodulate structured light fringes reflected from the surface of the object to be labeled to obtain phase information of the surface being inspected, and to obtain defect information of the object to be labeled based on phase anomaly information in the obtained phase information.

5. The defect dynamic annotation system according to claim 3, characterized in that, This system is applied to quality inspection in pipeline bending production lines, wherein the defect marking system projects the characteristic pattern of the pipeline onto the actual pipeline to obtain pipeline error information.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic defect annotation method of claim 1.

7. An electronic terminal, characterized in that, include: Processor and memory; The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal performs the defect dynamic annotation method as described in claim 1.

Images