Quality detection method and device for surface treatment process and electronic equipment

Abstract

The invention provides a surface treatment process quality detection method and device and electronic equipment, and relates to the technical field of process quality detection.The method comprises the steps that an initial shooting path of a to-be-detected workpiece model and an initial shooting path of a shooting equipment model are obtained; determining a visual range of each shooting point relative to the to-be-detected workpiece model in the initial shooting path according to a plurality of triangular plates included in the detected workpiece model, wherein the visual range comprises a plurality of visual points; according to the visual range corresponding to each shooting point, determining incident vectors of triangular plate vertexes corresponding to a plurality of triangular plate indexes included in each visual range to obtain a plurality of incident vector sets, and according to the plurality of incident vector sets, determining an effective detection area on the to-be-detected workpiece model corresponding to the initial shooting path; the quality detection of the surface treatment process is performed on the target detection area of the to-be-detected workpiece according to the effective detection area, so that the quality detection cost of the surface treatment process is saved, and the quality detection efficiency of the surface treatment process is improved.

Description

Technical Field

[0001] This application relates to the field of process quality inspection technology, specifically to a method, apparatus, and electronic equipment for quality inspection of surface treatment processes. Background Technology

[0002] Surface treatment processes, also known as surface modification processes, include spraying, painting, sandblasting, and shot peening. After these processes are completed, quality inspection is usually required to determine their effectiveness and accuracy. For example, in machining and manufacturing, extremely high requirements are placed on the flatness and smoothness of the workpiece surface to meet the requirements of specific processes (specific surface treatment processes).

[0003] Current technologies typically employ vision-based robots for defect detection on workpiece surfaces (i.e., quality inspection of workpiece surfaces after specific processes). The general approach involves manually teaching the robot to generate multiple motion points, with a camera capturing images of the workpiece at each point. These images are then analyzed and compared to provide a description of the defects at each point. However, due to the limited field of view of each camera, multiple on-site adjustments are often required to find suitable points covering the workpiece surface. These repeated on-site adjustments also increase the risk of equipment damage due to human error. This not only demands a certain level of experience from the operator but is also time-consuming, labor-intensive, and increases the cost of surface inspection.

[0004] There is currently no effective technical solution to the aforementioned problems with existing surface treatment process quality inspection methods. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for quality inspection of surface treatment processes, which at least solves the problems of high inspection costs and time-consuming and labor-intensive inspections in existing surface treatment process quality inspection methods.

[0006] According to one aspect of the embodiments of this application, a quality inspection method for surface treatment processes is provided, comprising: obtaining an initial shooting path of a workpiece model to be inspected and a shooting device model, wherein the workpiece model to be inspected includes multiple triangles and the initial shooting path includes multiple shooting points; determining the visible range of each shooting point relative to the workpiece model to be inspected based on the multiple triangles, wherein the visible range includes multiple visible points; determining the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range based on the visible range corresponding to each shooting point, thereby obtaining multiple sets of incident vectors, and determining an effective detection area on the workpiece model to be inspected corresponding to the initial shooting path based on the multiple sets of incident vectors; and performing surface treatment process quality inspection on the target detection area of ​​the workpiece to be inspected corresponding to the workpiece model to be inspected based on the effective detection area.

[0007] According to another aspect of the embodiments of this application, a quality inspection device for surface treatment processes is also provided, comprising: an acquisition unit, configured to acquire an initial shooting path of a workpiece model to be inspected and a shooting device model, the workpiece model to be inspected including multiple triangles, and the initial shooting path including multiple shooting points; a first determination unit, configured to determine the visible range of each shooting point relative to the workpiece model to be inspected based on the multiple triangles, the visible range including multiple visible points; a second determination unit, configured to determine the incident vectors of the vertices of the multiple triangles corresponding to the indexes of each triangle included in each visible range based on the visible range corresponding to each shooting point, to obtain multiple incident vector sets, and to determine the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path based on the multiple incident vector sets; and a quality inspection unit, configured to perform quality inspection of the surface treatment process on the target detection area of ​​the workpiece to be inspected corresponding to the workpiece model to be inspected based on the effective detection area.

[0008] Optionally, the first determining unit includes: an acquisition subunit, used to acquire attribute information corresponding to the camera device, and calculate the camera device bounding box corresponding to the target shooting point based on the attribute information using a bounding box calculation method corresponding to the camera device, wherein the target shooting point is any point in an undetermined set of multiple shooting points; a first filtering subunit, used to perform filtering operations on multiple triangles included in the workpiece model to be inspected based on the camera device bounding box to obtain a first index set, wherein the first index set includes multiple indices of triangles in the camera device bounding box; a filtering subunit, used to perform filtering operations on multiple first indices in the first index set according to a preset filtering rule to obtain a second index set; and a second filtering subunit, used to perform filtering operations on multiple second indices in the second index set based on the view frustum model geometry corresponding to the camera device to obtain the visible range of the camera device corresponding to the target shooting point, and determine the shooting points other than the target shooting point among the multiple shooting points as the target shooting point.

[0009] Optionally, the first filtering subunit includes: a first filtering module, configured to perform a first filtering operation on multiple triangles according to the camera device bounding box to obtain a reference triangle index set, the reference triangle index set including multiple reference triangle indices, and at least one vertex of the reference triangle corresponding to each reference triangle index being within the camera device bounding box; a first determining module, configured to determine multiple reference triangles according to the multiple reference triangle indices, and obtain the reflection vector and normal vector corresponding to each of the multiple reference triangle vertices included in each reference triangle, to obtain a reflection vector set and a normal vector set, the reflection vector being the vector from the reference triangle vertex to the camera device; and a second filtering module, configured to perform a second filtering operation on the reference triangle indices according to the reflection vector set and the normal vector set to obtain a first index set.

[0010] Optionally, the second filtering module includes: a first determining submodule, used to determine any one of the multiple reference triangles as the current triangle, and determine the current reflection vector and the current normal vector corresponding to the current triangle; a second determining submodule, used to determine the vector angle between the current reflection vector and the current normal vector; and an adding submodule, used to add the index corresponding to the current triangle to the first index set when the vector angle is obtuse, and to determine any one of the multiple reference triangles other than the current triangle as the current triangle.

[0011] Optionally, the above-mentioned filtering subunit includes: a second determining module, configured to determine multiple first triangles based on multiple second indices included in the second index set, and calculate the target vector corresponding to each of the multiple first triangles to obtain a target vector set; a third determining module, configured to determine the number of intersection points between each target vector and multiple first triangles other than the first triangle corresponding to the target vector; and a fourth determining module, configured to determine intersecting triangles based on the number of intersection points, and perform filtering operations on the multiple first indices based on the distance from the target vector and the intersecting triangles to the camera device to obtain a second index set.

[0012] Optionally, the second determining unit includes: a first determining subunit, used to determine any shooting point in the initial shooting path as the current shooting point, and to determine the current visible range corresponding to the current shooting point; a second determining subunit, used to determine the visible point reflection vector corresponding to each currently visible point included in the current visible range; a third determining subunit, used to determine the visible point incident vector corresponding to each currently visible point according to the law of reflection of light and multiple visible point reflection vectors, to obtain the set of incident vectors corresponding to the current shooting point; and a fourth determining subunit, used to determine any shooting point in the initial shooting path other than the current shooting point as the current shooting point.

[0013] Optionally, the second determining unit further includes: a fifth determining subunit, used to determine the illumination space range of the light source model, wherein the light source model is integrated with the shooting device and moves with the shooting device model according to the initial shooting path; and a judging subunit, used to determine, after obtaining the set of incident vectors corresponding to the current shooting point, whether the multiple incident vectors included in the set of incident vectors corresponding to the current shooting point intersect with the illumination space range; and in the case where the incident vectors intersect with the illumination space range, determining the current visible point corresponding to the incident vector as a valid detection point, wherein the valid detection area includes multiple valid detection points.

[0014] Optionally, the quality inspection device for the surface treatment process further includes: a path planning unit, used to perform secondary shooting path planning on the area other than the effective detection area corresponding to the initial shooting path on the workpiece model to be inspected when the effective detection area corresponding to the initial shooting path does not meet the preset detection area conditions, to obtain a target shooting path; a third determining unit, used to determine the target shooting path as the initial shooting path and determine the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path; and a quality inspection unit to perform quality inspection of the surface treatment process on the target detection area of ​​the workpiece to be inspected corresponding to the workpiece model to be inspected based on the effective detection area when the effective detection area corresponding to the initial shooting path meets the preset detection area conditions.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores computer instructions for causing a computer to perform a quality inspection method such as the surface treatment process described above.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform a quality inspection method such as the surface treatment process described above.

[0017] Compared with the prior art, the technical solution provided in this application embodiment may include the following beneficial effects:

[0018] The quality inspection method described above for surface treatment processes solves the problems of high inspection costs and time-consuming and labor-intensive inspections in existing surface treatment process quality inspection methods, thereby saving quality inspection costs and improving the efficiency of surface treatment process quality inspection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the hardware environment for a quality inspection method of an optional surface treatment process according to an embodiment of the present invention;

[0021] Figure 2This is a flowchart of a quality inspection method for an optional surface treatment process according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a quality inspection method for an optional surface treatment process according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of a quality inspection device for an optional surface treatment process according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] To address the problems of high testing costs and time-consuming, labor-intensive testing in existing surface treatment process quality inspection methods, this application provides a surface treatment process quality inspection method. As an optional implementation, the above-mentioned surface treatment process quality inspection method can be applied, but is not limited to, to applications such as... Figure 1 The surface treatment process quality inspection system shown consists of terminal device 102 and server 104. For example... Figure 1As shown, terminal device 102 is connected to server 104 via network 110. Network 110 may include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless network includes Bluetooth, Wi-Fi, and other networks that enable wireless communication. Terminal device 102 may include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MIDs (Mobile Internet Devices), tablets, desktop computers, smart TVs, etc.

[0029] The terminal device 102 described above is also equipped with a display 106, a processor 108, and a memory 112. The display 106 can be used to display the effective detection area, the processor 108 can be used to process the workpiece model to be detected, and the memory 112 can be used to store various models and data involved in this application.

[0030] The aforementioned server 104 can be a single server, a server cluster consisting of multiple servers, or a cloud server. The aforementioned server 104 includes a database 114 and a processing engine 116. The database 114 can be used to store various models and data involved in this application, and the processing engine 116 is used to process vertex data and triangles corresponding to the various models.

[0031] According to one aspect of the present invention, the quality inspection system for the surface treatment process described above may further perform the following steps: First, the terminal device 102 executes S102, sending a quality inspection request for the surface treatment process to the server 104 via the network 110; then the server 104 executes S104 to S108: obtaining the initial shooting path of the workpiece model to be inspected and the shooting device model, the workpiece model to be inspected including multiple triangles, the initial shooting path including multiple shooting points; determining the visible range of each shooting point relative to the workpiece model to be inspected based on the multiple triangles, the visible range including multiple visible points; determining the incident vector of the vertex of the triangle corresponding to the multiple triangle indexes included in each visible range based on the visible range corresponding to each shooting point, obtaining multiple incident vector sets, and determining the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path based on the multiple incident vector sets; performing quality inspection of the surface treatment process on the target detection area of ​​the workpiece to be inspected corresponding to the workpiece model to be inspected based on the effective detection area.

[0032] In the above embodiments of the present invention, the quality inspection method of the above surface treatment process solves the problems of high inspection cost and time-consuming and labor-intensive inspection of existing surface treatment process quality inspection methods, saves the quality inspection cost of surface treatment process, and improves the quality inspection efficiency of surface treatment process.

[0033] The above is merely an example, and no limitations are made in this embodiment.

[0034] As an alternative implementation method, please refer to Figure 2 This document illustrates a flowchart of a quality inspection method for a surface treatment process provided in an embodiment of this application. The execution entity for each step of this method can be the terminal device and server described above. In the following method embodiments, for ease of description, the execution entity for each step will only be described as a "computer device." This method may include at least one of the following steps (S202 to S208):

[0035] S202, Obtain the initial shooting path of the workpiece model to be inspected and the shooting device model. The workpiece model to be inspected includes multiple triangular pieces, and the initial shooting path includes multiple shooting points.

[0036] S204, determine the visible range of each shooting point relative to the workpiece model to be inspected based on multiple triangular pieces, and the visible range includes multiple visible points;

[0037] S206. Based on the visible range corresponding to each shooting point, determine the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range, obtain multiple sets of incident vectors, and determine the effective detection area on the workpiece model to be detected corresponding to the initial shooting path based on the multiple sets of incident vectors.

[0038] S208, perform quality inspection of surface treatment process on the target detection area of ​​the workpiece to be inspected corresponding to the model of the workpiece to be inspected, based on the effective detection area.

[0039] It should be noted that surface treatment processes include processes such as spraying, sandblasting, shot peening, grinding, and polishing to treat the surface of a workpiece. The workpiece model to be inspected in S202 is a model corresponding to the workpiece to be inspected. When obtaining the workpiece model, the initial model of the workpiece to be inspected is obtained first. The initial model can be a solid model or a triangular model. The initial model can be obtained from other software, such as CAD software. However, the representation method of the initial model (model accuracy, model format, etc.) may be inconsistent with the representation method of the workpiece model required in the quality inspection method of the surface treatment process in this application. Therefore, this application, upon obtaining the initial model... After modeling, the initial model is discretized into a workpiece model to be inspected that meets a preset precision (the preset precision corresponds to the attribute information of the imaging device corresponding to the imaging device model; that is, the preset precision is the discretization precision pre-set based on the attribute information of the imaging device). Specifically, a discretization algorithm (e.g., uniform discretization) can be used to discretize the initial model into a workpiece model to be inspected consisting of multiple triangular pieces. The side lengths of the multiple triangular pieces are equal, or the difference in side lengths of the multiple triangular pieces is less than a preset difference (a pre-set difference in the side lengths of the triangular pieces). The discretization process of the initial model is equivalent to discretizing the initial model into a layer to be inspected (i.e., the aforementioned workpiece model to be inspected), and the layer to be inspected consists of a set of triangular pieces. The specific discretization precision of the initial model is related to the effective detection area determined in S206. The aforementioned initial imaging path is the pre-planned initial imaging path corresponding to the imaging device.

[0040] Each shooting point in S204 corresponds to a visible range, which includes multiple visible points. The operation in S206 can be understood, but is not limited to, determining the effective area corresponding to each shooting point based on its visible range, ultimately obtaining the effective detection area corresponding to the initial shooting path. Then, in S208, the surface treatment process quality inspection is performed on the workpiece to be inspected based on the effective detection area.

[0041] Through the above embodiments of this application, the quality inspection method of the above surface treatment process solves the problems of high inspection cost and time-consuming and labor-intensive inspection in the existing surface treatment process quality inspection methods, saves the quality inspection cost of the surface treatment process, and improves the quality inspection efficiency of the surface treatment process.

[0042] As an optional implementation, the above method of determining the visible range of each shooting point relative to the workpiece model to be inspected based on multiple triangular pieces includes: S1, obtaining attribute information corresponding to the camera device, and calculating the camera device bounding box corresponding to the camera device at the target shooting point using the bounding box calculation method corresponding to the camera device based on the attribute information, wherein the target shooting point is any point in the set of multiple shooting points with an undetermined visible range; S2, performing a filtering operation on the multiple triangular pieces included in the workpiece model to be inspected based on the camera device bounding box to obtain a first index set, wherein the first index set includes the indices of multiple triangular pieces in the camera device bounding box; S3, performing a filtering operation on multiple first indices in the first index set according to a preset filtering rule to obtain a second index set; S4, performing a filtering operation on multiple second indices in the second index set based on the view frustum model geometry corresponding to the camera device to obtain the visible range of the camera device corresponding to the target shooting point, and determining the shooting points other than the target shooting point among the multiple shooting points as the target shooting point.

[0043] It should be noted that the camera equipment in S1 mentioned above includes orthographic cameras and perspective cameras, and can also be other shooting devices. The attribute information corresponding to the camera equipment includes the basic hardware attributes of the camera equipment (camera equipment type and size, resolution, pixels, lens parameters, etc.), shooting function attributes (video shooting capabilities (maximum resolution and frame rate, encoding format, etc.), array and buffer, low light performance, etc.), and physical and design attributes (size and weight, protection performance, etc.). Different camera equipment has different bounding box calculation methods. Therefore, it is necessary to combine the attribute information of the camera equipment and adopt the bounding box calculation method corresponding to the camera equipment to ensure the accuracy of the bounding box calculation. The bounding box of the camera equipment can be understood as, but is not limited to, the bounding box of the camera equipment's view frustum. The view frustum position of the camera equipment is different at different shooting points, so the bounding box of the camera equipment corresponding to different shooting points is also different.

[0044] The operation in S2 above can be understood, but is not limited to, as a selection of triangles based on whether the vertices of the triangles are within the bounding box of the camera device. Specifically, the operation in S2 includes:

[0045] S2-1, Perform a first filtering operation on multiple triangles based on the camera device bounding box to obtain a reference triangle index set. The reference triangle index set includes multiple reference triangle indices, and each reference triangle index has at least one triangle vertex in the camera device bounding box.

[0046] S2-2, determine multiple reference triangles based on multiple reference triangle indices, and obtain the reflection vector and normal vector corresponding to each of the multiple reference triangle vertices included in each reference triangle, to obtain the reflection vector set and normal vector set. The reflection vector is the vector from the reference triangle vertex to the camera device.

[0047] S2-3, perform a second filtering operation on the reference triangle index based on the reflection vector set and the normal vector set to obtain the first index set.

[0048] The operation in S2-1 above can be understood, but is not limited to, as first filtering out triangles from multiple triangles whose vertices are within the camera's bounding box. In this case, the first filtering operation is equivalent to a filtering operation, and what is filtered out are triangles whose vertices are not within the camera's bounding box. The reference triangles included in the obtained reference triangle index set have at least one vertex within the camera's bounding box. The process of performing the first filtering operation can be to determine whether the position information corresponding to each vertex of each triangle is within the camera's bounding box based on the position information corresponding to each vertex, thereby realizing the first filtering operation.

[0049] The operation in S2-3 above can be understood, but is not limited to, as the process of determining the visible range of the camera device at the target shooting point based on the angle between the reflection vector and the normal vector. Specifically, the operation in S2-3 includes: S2-3-1, determining any one of the multiple reference triangles as the current triangle, and determining the current reflection vector and current normal vector corresponding to the current triangle; S2-3-2, determining the vector angle between the current reflection vector and the current normal vector; S2-3-3, if the vector angle is obtuse, adding the index corresponding to the current triangle to the first index set, and determining any one of the multiple reference triangles other than the current triangle as the current triangle.

[0050] In S2-3-1 above, the current reflection vector is the reflection vector corresponding to the current triangle, and the current normal vector is the normal vector corresponding to the current triangle. The current reflection vector is the vector pointing from the center of the current triangle to the camera device, and the current normal vector is the normal vector corresponding to the current triangle and the current reflection vector.

[0051] The operation in S3 above can be understood, but is not limited to, removing triangles that may be occluded. The operation in S3 specifically includes: S3-1, determining multiple first triangles based on the multiple second indices included in the second index set, and calculating the target vectors corresponding to each of the multiple first triangles to obtain a target vector set; S3-2, determining the number of intersection points between each target vector and multiple first triangles other than the first triangle corresponding to the target vector; S3-3, determining the intersecting triangles based on the number of intersection points, and performing a filtering operation on the multiple first indices based on the distance from the target vector and the intersecting triangles to the camera device to obtain a second index set.

[0052] The number of intersection points in S3-2 above includes 0 (no intersection), 1 (one intersection point), and multiple (multiple intersection points).

[0053] The view frustum model geometry in S4 above can be understood, but is not limited to, as the view frustum corresponding to the target shooting point of the camera device, and the operation in S4 can be understood, but is not limited to, as determining whether the vertex in the triangle is in the view frustum model geometry.

[0054] Assuming the imaging device is a camera, in order to automatically confirm the surface range of the workpiece in the surface inspection process in the robot offline programming software, several steps as described in this application are required, which are combined below. Figure 3 (like Figure 3 As shown, camera 302 and light source 304 are integrated hardware devices that move together according to the initial shooting path. Camera shooting area 306 is the area on the workpiece model to be inspected that camera 302 can capture at the target shooting point. Camera effective area 308 is the effective detection area corresponding to the target shooting point determined according to the operation steps in this application. The operations in S1 to S4 above are illustrated with examples:

[0055] 1. Workpiece model discretization. A uniform discretization method can be used to discretize the workpiece model (i.e., the initial model of the workpiece to be inspected) to generate a layer to be inspected (i.e., the aforementioned workpiece model to be inspected). The layer to be inspected consists of a set of triangular pieces.

[0056] 2. At a shooting point of the robot (i.e., the target shooting point determined from multiple shooting points included in the initial shooting path), calculate the camera (e.g., ... Figure 3 The visible range of the camera 302 shown. The effective set of triangles is denoted as T. valid ={t i}, where t i This refers to the triangular patch index. This application uses a geometric method to determine whether each vertex (triangular patch vertex) on the layer to be detected is within the camera's visible area (i.e., the visible area of ​​the aforementioned camera device). The specific steps are as follows:

[0057] (1) Calculate the camera's bounding box (i.e., the camera device bounding box mentioned above) (i.e., the operation in S1 above), traverse each triangle in the layer to be detected, and determine the vertex v of each triangle. i (For each vertex in each triangle, determine and process accordingly) whether it is within the camera bounding box (i.e., the operation in S2 above). The specific determination and processing process is as follows:

[0058] If vertex v i If it's not in the camera bounding box, then traverse the next vertex.

[0059] If vertex v i Within the camera bounding box, it is necessary to determine vertex v. i Whether it faces the camera. Specifically: [Is the vertex v] facing the camera? i The vector to the camera position is denoted as VC. i If VC i With vertex v i The normal vector n i If the included angle is obtuse, then iterate to the next vertex; otherwise, add the index of that triangle to T. valid ={t i}middle.

[0060] It should be noted that if the index of a triangle is added to T based on any vertex of the triangle... valid ={t i After this step, the other vertices in the triangle no longer need to undergo the aforementioned judgment (i.e., determining whether a vertex is within the camera bounding box). After these operations, a set consisting of multiple triangle indices, including those where at least one vertex is within the camera bounding box, is obtained; this is the aforementioned first index set.

[0061] (2) Remove T valid The possible occlusion triangles in T (i.e., the operation in S3 above). Specifically, iterate through T. valid For each triangle, obtain the vector CC from the center of the triangle to the camera position. i Calculate vector CC i With T valid Find all intersections of the other triangles and perform a response operation based on the number of intersections:

[0062] If vector CC i With T valid Since none of the other triangles in T intersect, we continue iterating through T. valid The next triangle in the middle.

[0063] If vector CC i With T valid If the other triangles in T intersect, and there is only one intersection point, then continue traversing T. validIf the next triangle in the image has multiple intersection points, calculate the distance d from each intersection point to the camera position. m If the distance from the center of the triangle to the camera position is greater than the minimum of all distances, i.e., ||CC i ||>min{d m}, then from T valid Delete the triangular index; otherwise, continue iterating through T. valid The other triangular pieces in the process are then removed to obtain the final result. valid The set of triangular indices of the occluding triangular pieces that exist in the array, namely the second index set mentioned above.

[0064] (3) Traverse T valid For each vertex of each triangle, check vertex v. i Whether in the camera model geometry (i.e., the view frustum model geometry mentioned above), if vertex v i In the camera model geometry, then vertex v i Add to camera view V valid In this way, the visible range of the camera at the target shooting point can be obtained.

[0065] By sequentially identifying the shooting points in the initial shooting path as target shooting points and performing the above operations on each of them, the visible range corresponding to each shooting point in the initial shooting path can be obtained.

[0066] Through the above-described embodiments of this application, the visible range corresponding to each shooting point in the initial shooting path can be accurately determined, thereby improving the accuracy of subsequently determining the effective detection area corresponding to the initial shooting path, further improving the accuracy of subsequent quality inspection of the surface treatment process of the effective detection area corresponding to the workpiece to be inspected, and further saving inspection time and improving inspection efficiency.

[0067] As an optional implementation, based on the visible range corresponding to each shooting point, the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range are determined, resulting in multiple sets of incident vectors, including: S1, determining any shooting point in the initial shooting path as the current shooting point, and determining the current visible range corresponding to the current shooting point; S2, determining the visible point reflection vector corresponding to each currently visible point included in the current visible range; S3, determining the visible point incident vector corresponding to each currently visible point according to the law of light reflection and multiple visible point reflection vectors, resulting in a set of incident vectors corresponding to the current shooting point; S4, determining any shooting point in the initial shooting path other than the current shooting point as the current shooting point.

[0068] The operations S1 to S4 above can be understood, but are not limited to, as the process of determining the set of incident vectors corresponding to each shooting point based on the visible range corresponding to each shooting point. The operations S1 to S4 above involve calculating the set of light sources (i.e., the aforementioned visible range) V. valid Each vertex v i incident direction LV i (i.e., the incident vector corresponding to the vertex), that is, the vector from the light source to the vertex of the triangle. According to the law of reflection of light: Angle (VC i ,n i =Angle(n) i ,VC i ), and the incident direction LV can be calculated. i .

[0069] Then, based on the set of incident vectors corresponding to each shooting point, the effective detection area can be determined. Specifically, this includes: determining the illumination space range of the light source model, which is integrated with the shooting device and moves along the initial shooting path with the shooting device model; after obtaining the set of incident vectors corresponding to the current shooting point, determining whether the multiple incident vectors included in the set of incident vectors corresponding to the current shooting point intersect with the illumination space range; if the incident vectors intersect with the illumination space range, determining the current visible point corresponding to the incident vector as the effective detection point, and the effective detection area includes multiple effective detection points.

[0070] It should be noted that after determining the current viewpoint as a valid detection point, the process also includes marking the current viewpoint with a valid color. The color marked for the current viewpoint is different from any of the colors in the workpiece model, the camera device model, and the light source (also known as the light source model), so that users can more intuitively see the valid detection area on the workpiece model that corresponds to the initial shooting path.

[0071] The above method determines the effective detection area on the workpiece model corresponding to the initial shooting path based on multiple sets of incident vectors. For example, the illumination range of the light source is approximated as a rectangle (i.e., the illumination space range mentioned above), and it is calculated whether the straight line of the incident direction of the light source at the vertex intersects the rectangle. If they intersect, then the point is a detectable point (i.e., an effective detection point); otherwise, the point cannot be effectively detected, and the point is removed from V. valid Delete it. Iterate through V. valid For each point in the target shooting path (i.e., one of the shooting points in the initial shooting path), the above operation is performed to obtain the effective detection area corresponding to the target shooting point. Then, each shooting point is traversed, and the above operation is performed on each shooting point as the target shooting point in turn to obtain the effective detection area of ​​all shooting points of the camera.

[0072] Through the above-described embodiments of this application, the areas where surface treatment processes can be accurately and quickly identified, thereby improving the accuracy and efficiency of surface treatment process quality inspection.

[0073] As an optional implementation, after determining the effective detection area on the workpiece model corresponding to the initial shooting path based on multiple sets of incident vectors, the method further includes:

[0074] S1. If the effective detection area corresponding to the initial shooting path does not meet the preset detection area conditions, a secondary shooting path is planned for the area outside the effective detection area corresponding to the initial shooting path on the workpiece model to be inspected, to obtain the target shooting path.

[0075] S2, determine the target shooting path as the initial shooting path, and determine the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path;

[0076] S3, if the effective detection area corresponding to the initial shooting path meets the preset detection area conditions, perform surface treatment process quality inspection on the target detection area of ​​the workpiece to be inspected corresponding to the workpiece model to be inspected based on the effective detection area.

[0077] It should be noted that the effective detection area in S1 above refers to the area corresponding to the workpiece model to be inspected and the initial shooting path, which can effectively detect the quality inspection of the surface treatment process. The preset detection area conditions mentioned above are pre-set detection area conditions. The preset detection area conditions may include, for example, the area of ​​the effective detection area, the location of the effective detection area, and the preset detection area conditions corresponding to the quality inspection requirements of the surface treatment process (the inspection requirements carried in the quality inspection request of the surface treatment process). When the preset detection area condition is an area, for example, the area of ​​the effective detection area (i.e., the area of ​​the area in the workpiece model to be inspected that has been effectively color-marked) can be used to determine whether the conditions are met.

[0078] The operation in S2 above can be understood, but is not limited to, planning the shooting path for the model area of ​​the workpiece model other than the effective detection area corresponding to the initial shooting path (i.e., the secondary shooting path planning mentioned above). Then, the planned target shooting path can be determined as the initial shooting path, and the same operations in S202 to S208 above are performed to determine the effective detection area corresponding to the target shooting path (i.e., the new initial shooting path). This process of determining the effective detection area is iteratively repeated until the determined effective detection area meets the preset detection area conditions. The effective detection areas determined each time are then integrated to obtain the final effective detection area. Finally, the workpiece is subjected to quality inspection of the surface treatment process based on the final effective detection area.

[0079] Through the above-described embodiments of this application, a rapid and accurate quality inspection of the surface treatment process on the workpiece can be achieved, improving the efficiency and accuracy of surface treatment process quality inspection. Furthermore, the surface treatment process quality inspection method in this application realizes automated detection of the effective area of ​​the camera in a virtual scene. Users only need to build the scene and generate the robot's shooting path, requiring no additional operations. In addition, considering the potential model occlusion of complex workpieces, a camera visibility range detection method based on a geometric model is proposed. Moreover, by simulating the optical characteristics of the camera, the effective detection area of ​​the camera is accurately calculated.

[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0081] According to another aspect of the present invention, a quality inspection apparatus for a surface treatment process for implementing the above-described surface treatment process quality inspection method is also provided, such as... Figure 4 As shown, the device includes:

[0082] The acquisition unit 402 is used to acquire the initial shooting path of the workpiece model to be inspected and the shooting device model. The workpiece model to be inspected includes multiple triangular pieces, and the initial shooting path includes multiple shooting points.

[0083] The first determining unit 404 is used to determine the visible range of each shooting point relative to the workpiece model to be inspected based on multiple triangular pieces, and the visible range includes multiple visible points.

[0084] The second determining unit 406 is used to determine the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range according to the visible range corresponding to each shooting point, to obtain multiple incident vector sets, and to determine the effective detection area on the workpiece model to be detected corresponding to the initial shooting path according to the multiple incident vector sets.

[0085] The quality inspection unit 408 is used to perform quality inspection of the surface treatment process on the target inspection area of ​​the workpiece to be inspected, corresponding to the model of the workpiece to be inspected, based on the effective inspection area.

[0086] The specific methods of execution of each unit in the above device embodiments have been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0087] According to another aspect of the present invention, an electronic device for implementing the quality inspection method of the above-described surface treatment process is also provided. This electronic device may be as follows: Figure 5 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 5 As shown, the electronic device includes: at least one processor 504; and a memory 502 communicatively connected to at least one processor 504; wherein the memory 502 stores a computer program that can be executed by at least one processor 504, and the computer program is executed by at least one processor 504 to cause at least one processor 504 to perform the steps in any of the above-described embodiments of the quality inspection method for surface treatment processes.

[0088] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0089] Optionally, in this embodiment, the processor can be configured to execute the various steps in the quality inspection method of the surface treatment process via a computer program.

[0090] Alternatively, as those skilled in the art will understand, Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 5 The different configurations shown.

[0091] The memory 502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the surface treatment process quality inspection method and apparatus in this embodiment of the invention. The processor 504 executes various functional applications and data processing by running the software programs and modules stored in the memory 502, thereby realizing the aforementioned surface treatment process quality inspection method. The memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 502 may further include memory remotely located relative to the processor 504, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 502 may be used, but is not limited to, to store various models and data involved in this application. As an example, such as Figure 5 As shown, the memory 502 may include, but is not limited to, the acquisition unit 402, the first determination unit 404, the second determination unit 406, and the quality detection unit 408 from the quality detection device for the surface treatment process described above. Furthermore, it may include, but is not limited to, other module units from the quality detection device for the surface treatment process described above, which will not be elaborated upon in this example.

[0092] Optionally, the transmission device 506 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 506 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 506 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0093] In addition, the above-mentioned electronic device also includes a display 508 and a connection bus 510 for connecting the various module components in the above-mentioned electronic device.

[0094] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0095] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0096] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0097] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and executes the computer instructions to cause the computer device to perform the quality inspection method of the surface treatment process described above.

[0098] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the quality inspection method of the surface treatment process described above.

[0099] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes as described in the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory (FM), hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0100] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0101] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0102] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A quality inspection method for a surface treatment process, characterized in that, include: Obtain the initial shooting path of the workpiece model to be inspected and the shooting device model. The workpiece model to be inspected includes multiple triangular pieces, and the initial shooting path includes multiple shooting points. The visible range of each shooting point relative to the workpiece model to be inspected is determined based on multiple triangular pieces, and the visible range includes multiple visible points; Based on the visible range corresponding to each shooting point, the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range are determined to obtain multiple sets of incident vectors, and the effective detection area on the workpiece model to be detected corresponding to the initial shooting path is determined based on the multiple sets of incident vectors. The surface treatment process quality inspection is performed on the target detection area of ​​the workpiece model to be inspected based on the effective detection area.

2. The method according to claim 1, characterized in that, Determining the visible range of each shooting point relative to the workpiece model to be inspected based on multiple triangular pieces includes: Obtain the attribute information corresponding to the camera device, and calculate the camera device bounding box corresponding to the camera device at the target shooting point using the bounding box calculation method corresponding to the camera device based on the attribute information. The target shooting point is any point in the set of undetermined visible ranges among multiple shooting points. The camera device bounding box is used to filter multiple triangular pieces included in the workpiece model to be inspected to obtain a first index set, which includes multiple indices of the triangular pieces in the camera device bounding box. A second index set is obtained by filtering multiple first indices in the first index set according to preset filtering rules. Based on the viewing cone model geometry corresponding to the camera device, a filtering operation is performed on multiple second indices in the second index set to obtain the visible range of the camera device at the target shooting point, and the shooting points other than the target shooting point among the multiple shooting points are determined as the target shooting point.

3. The method according to claim 2, characterized in that, The camera device bounding box is used to filter multiple triangular pieces included in the workpiece model to be inspected, resulting in a first index set, including: A first filtering operation is performed on the multiple triangles according to the camera device bounding box to obtain a reference triangle index set. The reference triangle index set includes multiple reference triangle indices, and at least one vertex of the reference triangle corresponding to each reference triangle index exists in the camera device bounding box. Multiple reference triangles are determined based on multiple reference triangle indices, and the reflection vector and normal vector corresponding to each of the multiple reference triangle vertices included in each reference triangle are obtained to obtain a set of reflection vectors and a set of normal vectors. The reflection vector is the vector from the vertex of the reference triangle to the camera device. A second filtering operation is performed on the reference triangle index based on the set of reflection vectors and the set of normal vectors to obtain the first index set.

4. The method according to claim 3, characterized in that, A second filtering operation is performed on the reference triangle index based on the reflection vector set and the normal vector set to obtain the first index set, including: Any one of the multiple reference triangles is determined as the current triangle, and the current reflection vector and the current normal vector corresponding to the current triangle are determined. Determine the vector angle between the current reflection vector and the current normal vector; When the included angle of the vectors is obtuse, the index corresponding to the current triangle is added to the first index set, and any one of the multiple reference triangles other than the current triangle is determined as the current triangle.

5. The method according to claim 3, characterized in that, According to preset filtering rules, multiple first indices in the first index set are filtered to obtain a second index set, including: Multiple first triangles are determined based on multiple second indices included in the second index set, and the target vectors corresponding to each of the multiple first triangles are calculated to obtain a target vector set; Determine the number of intersection points between each target vector and a plurality of first triangles other than the first triangle corresponding to the target vector; The number of intersection points determines the intersecting triangles, and the first indexes are filtered based on the target vector and the distance from the intersecting triangles to the camera device to obtain the second index set.

6. The method according to claim 2, characterized in that, Based on the visible range corresponding to each shooting point, the incident vectors of the vertices of the triangles corresponding to the multiple triangle indices included in each visible range are determined, resulting in multiple sets of incident vectors, including: Any shooting point in the initial shooting path is determined as the current shooting point, and the current visible range corresponding to the current shooting point is determined. Determine the viewpoint reflection vector corresponding to each currently visible point included in the current visible range; Based on the law of reflection of light and the reflection vectors of multiple visible points, the incident vector of each current visible point is determined, thus obtaining the set of incident vectors corresponding to the current shooting point. The current shooting point is determined as any shooting point in the initial shooting path other than the current shooting point.

7. The method according to claim 6, characterized in that, The effective detection area on the workpiece model to be inspected corresponding to the initial shooting path is determined based on multiple sets of incident vectors, including: The illumination space range of the light source model is determined. The light source model is integrated with the shooting device and moves with the shooting device model according to the initial shooting path. After obtaining the set of incident vectors corresponding to the current shooting point, it is determined whether the multiple incident vectors included in the set of incident vectors corresponding to the current shooting point intersect with the illumination space range. When the incident vector intersects with the illumination space range, the current visible point corresponding to the incident vector is determined as a valid detection point, and the valid detection area includes multiple valid detection points.

8. The method according to claim 7, characterized in that, After determining the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path based on multiple sets of incident vectors, the method further includes: If the effective detection area corresponding to the initial shooting path does not meet the preset detection area conditions, a secondary shooting path is planned for the area on the workpiece model to be inspected other than the effective detection area corresponding to the initial shooting path to obtain the target shooting path. The target shooting path is determined as the initial shooting path, and the effective detection area on the workpiece model to be inspected corresponding to the initial shooting path is determined; If the effective detection area corresponding to the initial shooting path meets the preset detection area conditions, the surface treatment process quality inspection is performed on the target detection area of ​​the workpiece model to be inspected based on the effective detection area.

9. A quality inspection device for a surface treatment process, characterized in that, include: The acquisition unit is used to acquire the initial shooting path of the workpiece model to be inspected and the shooting device model. The workpiece model to be inspected includes multiple triangular pieces, and the initial shooting path includes multiple shooting points. The first determining unit is configured to determine the visible range of each shooting point relative to the workpiece model to be inspected based on the plurality of triangular pieces, wherein the visible range includes a plurality of visible points; The second determining unit is used to determine the incident vector of the vertex of the triangle corresponding to the multiple triangle indexes included in each visible range according to the visible range corresponding to each shooting point, to obtain multiple incident vector sets, and to determine the effective detection area on the workpiece model to be detected corresponding to the initial shooting path according to the multiple incident vector sets. The quality inspection unit is used to perform quality inspection of the surface treatment process on the target inspection area of ​​the workpiece model to be inspected, based on the effective inspection area.

10. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the quality inspection method of the surface treatment process according to any one of claims 1-8.

Images