Product process detection method, device, computer equipment and medium

Abstract

The embodiment of the present invention discloses a product process detection method, device, computer equipment and medium, wherein the method includes: calculating the process detection path according to a preset scanning index set, the line scanning laser data and the process processing trajectory; performing image acquisition on the target product according to the process detection path and the line scanning laser data to obtain a single-frame image; performing feature point data set calculation on the single-frame image according to the feature point configuration corresponding to the target product; performing process detection feature data calculation according to the feature point data set; judging whether the process is qualified according to the process detection feature data according to the standard feature data set corresponding to the target product, and obtaining a single-frame process detection result. Thereby improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a product process detection method, device, computer equipment and medium. Background Art

[0002] With the development of artificial intelligence, more and more studies are being conducted on the application of artificial intelligence to process inspection. At present, process inspection based on artificial intelligence is carried out on regular flat products, such as mobile phone parts. Process inspection based on artificial intelligence performs three-dimensional imaging, image processing and feature data calculation on the products to determine whether the inspection process is qualified. However, the imaging of irregular three-dimensional space curved surface products is prone to produce image blind spots, resulting in low accuracy of image processing and feature data calculation. Summary of the invention

[0003] Based on this, it is necessary to propose a product process detection method, device, computer equipment and medium to address the above problems.

[0004] A product process detection method, the method comprising:

[0005] Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0006] According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image;

[0007] Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product;

[0008] Calculating process detection feature data according to the feature point data set;

[0009] According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

[0010] Further, the scanning index set includes: a first scanning index, a second scanning index and a third scanning index;

[0011] The first scanning index is used to make the process surface of the target product be located at the center position of the laser line array corresponding to the line scanning laser data, wherein the center position is the center point position of the X-axis of the three-dimensional line scanning camera corresponding to the line scanning laser data;

[0012] The second scanning index is used to make the process surface perpendicular to the emission direction of the laser line array;

[0013] The third scanning index is used to make the arrangement direction of the laser line array perpendicular to the moving direction of the process inspection object on the process surface, and the arrangement direction of the laser line array parallel to the tangent direction of the process inspection object.

[0014] Furthermore, the step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product to obtain a single-frame process detection result includes:

[0015] Acquire each standard feature data corresponding to the process detection feature data from the standard feature data set as a target feature data set;

[0016] Determining whether the process detection feature data is within a feature range corresponding to the target feature data set;

[0017] If yes, the single-frame process inspection result is determined to be qualified;

[0018] If not, the single-frame process inspection result is determined to be unqualified.

[0019] Furthermore, after the step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product and obtaining a single-frame process detection result, the step further includes:

[0020] Constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single-frame images to form an initial three-dimensional point cloud image;

[0021] The initial three-dimensional point cloud image is marked according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product.

[0022] Furthermore, the step of constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single frame images to form an initial three-dimensional point cloud image includes:

[0023] Using the process detection path as a splicing path;

[0024] Taking any one of the single-frame images as the image to be stitched;

[0025] Calculating the stitching position of the images to be stitched according to the process detection path;

[0026] Calculating a stitching angle of the image to be stitched according to the scanning index set, the stitching position and the process detection path;

[0027] Inserting the image to be stitched into the stitching path according to the stitching angle and the stitching position;

[0028] Repeat the step of using any one of the single-frame images as the image to be stitched until all the single-frame images are inserted into the stitching path;

[0029] The initial three-dimensional point cloud image is determined according to each of the single-frame images on the stitching path.

[0030] Furthermore, the step of performing image acquisition on the target product according to the process detection path and the line scan laser data to obtain a single frame image includes:

[0031] According to the target scanning mode corresponding to the target product, the process detection path and the line scanning laser data, the target product is imaged to obtain the single frame image;

[0032] The step of calculating the stitching position of the images to be stitched according to the process detection path includes:

[0033] If the target scanning mode is scanning at a preset time interval, the shooting time of the image to be stitched is determined according to the time interval and each of the single-frame images, and the stitching position is calculated according to the shooting time, the process inspection path and the corresponding moving speed of the target product;

[0034] If the target scanning mode is a trigger scanning, the stitching position calculation is performed on the image to be stitched according to the preset shooting position data and the process detection path.

[0035] Furthermore, the step of marking the initial three-dimensional point cloud image according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product includes:

[0036] Taking each of the single-frame images with unqualified single-frame process detection results as an image to be marked;

[0037] Using a preset marking format to mark each of the to-be-marked images in the initial three-dimensional point cloud image;

[0038] The marked initial three-dimensional point cloud image is used as the target three-dimensional point cloud image.

[0039] A product process detection device, comprising:

[0040] A process detection path determination module is used to obtain the process processing trajectory and line scanning laser data of the target product, and calculate the process detection path according to a preset scanning indicator set, the line scanning laser data and the process processing trajectory;

[0041] A single-frame image determination module, used to collect images of the target product according to the process detection path and the line scanning laser data to obtain a single-frame image;

[0042] A feature point data set determination module, used to calculate a feature point data set for the single frame image according to the feature point configuration corresponding to the target product;

[0043] A process detection feature data determination module, used for calculating process detection feature data according to the feature point data set;

[0044] The single-frame process detection result determination module is used to determine whether the process is qualified based on the standard feature data set corresponding to the target product, and obtain a single-frame process detection result.

[0045] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0046] Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0047] According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image;

[0048] Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product;

[0049] Calculating process detection feature data according to the feature point data set;

[0050] According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0052] Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0053] According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image;

[0054] Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product;

[0055] Calculating process detection feature data according to the feature point data set;

[0056] According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

[0057] The product process detection method of the present application calculates the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory, so that the process detection path meets the requirements of the scanning index set, which is beneficial to improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection results; by performing image acquisition on the target product according to the process detection path and the line scan laser data, image acquisition is achieved by using laser line scanning. Compared with the non-laser line scanning method, the laser line scanning method is beneficial to improving the imaging accuracy of irregular products and further improving the accuracy of the single-frame process detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] in:

[0060] Figure 1 A flowchart of a product process detection method in one embodiment;

[0061] Figure 2 A schematic diagram of an application scenario of this application;

[0062] Figure 3 is a structural block diagram of a product process detection device in one embodiment;

[0063] Figure 4 FIG. 4 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] like Figure 1 As shown, in one embodiment, a product process detection method is provided. The method can be applied to both a terminal and a server. This embodiment is illustrated by applying to a terminal. The product process detection method specifically includes the following steps:

[0066] S102: Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0067] Among them, the execution equipment is controlled according to the process trajectory to realize the generation of products according to the process trajectory.

[0068] The target product is the product that needs to be inspected. The target product can be a regular flat product or an irregular three-dimensional curved surface product. Products include but are not limited to: 3D (three-dimensional) glasses.

[0069] In the production and manufacturing of products, it is necessary to first formulate the process processing trajectory of the product, and operate the corresponding execution equipment to grasp the product and execute it along the process processing trajectory, so as to process the product to form a process object; in order to determine whether the process object is qualified, the process object needs to be inspected, so the process object is used as the process inspection object. The execution equipment includes but is not limited to: six-axis robot. The value range of the process includes but is not limited to: dispensing, assembly and welding. The value range of the process inspection object includes but is not limited to: glue path, assembly gap and weld scar.

[0070] Among them, the operation execution equipment grasps the product and executes along the process inspection path to perform process inspection on the process inspection object of the product.

[0071] If the target product is 3D glasses, the spatial position relationship between the execution device and the camera is calculated; the 3D glasses are scanned and a corresponding standard model is generated; the standard model is divided into multiple areas, and feature points are captured in each area, wherein the range of each area is 1mm to 5mm; the feature points in each area are connected to form the process track of the 3D glasses. Thus, by dividing multiple areas on the generated standard model and controlling the areas within a certain range, the product can form a nearly planar structure in the divided areas, so that normal teaching operations can be performed in the areas, and the defective products caused by the incompatibility of the products with teaching can be reduced.

[0072] Line scan laser data is the characteristic data of the laser line array of the 3D line scan camera. Line scan laser data includes but is not limited to: emission position, emission direction and arrangement direction. It can be understood that the 3D line scan camera will emit multiple laser lines, all of which are arranged in a row to form a laser line array. The emission position is the starting position of the laser line array of the 3D line scan camera. The emission direction is the transmission direction of the laser emitted by the 3D line scan camera. The arrangement direction is a direction perpendicular to the center line of the laser line array and parallel to the laser line array.

[0073] Specifically, the process processing trajectory and line scan laser data of the target product input by the user can be obtained, the process processing trajectory and line scan laser data of the target product can be obtained from the database, and the process processing trajectory and line scan laser data of the target product can also be obtained from a third-party application.

[0074] The scanning index set includes one or more scanning indexes. The scanning index is the positional relationship between the laser line array and the target product.

[0075] Specifically, a process detection path is calculated based on the line scan laser data and the process processing trajectory, wherein the positional relationship between the laser line array corresponding to the line scan laser data and the preset target product meets the requirements of the scanning index set.

[0076] S104: performing image acquisition on the target product according to the process detection path and the line scan laser data to obtain a single frame image;

[0077] Specifically, the execution device grabs the target product using a preset grabbing method, and then executes according to the process detection path to move the target product to the bottom of the three-dimensional line scan camera corresponding to the line scan laser data; the three-dimensional line scan camera uses the laser line array corresponding to the line scan laser data to capture the image of the target product, and uses the captured two-dimensional image as a single frame image. In other words, the three-dimensional line scan camera is fixed, and the execution device grabs the target product to move the target product according to the process detection path.

[0078] The target product is imaged using a laser line array, that is, scanned using a contour mode.

[0079] It can be understood that a single-frame image is an XZ two-dimensional image, that is, the coordinate system of the single-frame image includes an X-axis and a Z-axis, where the X-axis is the arrangement direction and the Z-axis is the coordinate of the emission direction. The coordinate system of the single-frame image is the coordinate system of the three-dimensional line scan camera corresponding to the line scan laser data.

[0080] The single-frame image is a two-dimensional image composed of points. Each point in the single-frame image corresponds to a laser line in the laser line column corresponding to the line scanning laser data.

[0081] It can be understood that steps S106 to S110 can be executed every time a single-frame image of the target product is captured to determine the single-frame process inspection result, or after all single-frame images of the target product are captured, steps S106 to S110 can be executed for each single-frame image to determine the single-frame process inspection result.

[0082] S106: Calculating a feature point data set for the single frame image according to the feature point configuration corresponding to the target product;

[0083] The feature point configuration includes one or more feature point calculation formulas, which are used to calculate the parameters of the feature points in the process inspection object.

[0084] For example, when the present application is applied to process inspection of 3D glasses, the value range of the feature points includes: glue breakage, the position of the glue path, the shape of the glue path, the size of the assembly gap, and the size of the weld scar.

[0085] Specifically, the feature point configuration corresponding to the target product input by the user can be obtained, the feature point configuration corresponding to the target product can be obtained from a database, or the feature point configuration corresponding to the target product can be obtained from a third-party application; for each feature point calculation formula in the feature point configuration corresponding to the target product, parameter data of the feature points of the single frame image are calculated; and the calculated parameter data are used as the feature point data set.

[0086] The feature point data set can be calculated by analyzing the contour formed by the points in the single frame image.

[0087] S108: Calculating process inspection feature data according to the feature point data set;

[0088] Specifically, feature data of feature points are calculated according to the feature point data set, and each feature data obtained by calculation is used as the process detection feature data.

[0089] For example, when the present application is applied to process inspection of 3D glasses, the process inspection feature data includes but is not limited to: glue line width, glue line height, glue line outer margin, glue line inner margin, glue break size, glue line volume, assembly gap size, weld scar size, burr size and flash size.

[0090] S110: judging whether the process is qualified on the process detection feature data according to the standard feature data set corresponding to the target product, and obtaining a single-frame process detection result.

[0091] Specifically, according to the product identification corresponding to the target product, a feature data set is obtained from a preset standard feature library, and the obtained feature data set is used as the standard feature data set corresponding to the target product; it is determined whether each feature data in the process inspection feature data is within the data range required by the standard feature data set, and if so, the single-frame process inspection result is determined to be qualified; if any feature data in the process inspection feature data is outside the data range required by the standard feature data set, the single-frame process inspection result is determined to be unqualified.

[0092] For example, the required range of the glue path width is 1mm to 3mm. If the glue path width in the process detection feature data (that is, the feature data) is 3.5mm, and 3.5mm is outside the data range corresponding to 1mm to 3mm, then the single-frame process detection result will be determined as unqualified.

[0093] The standard feature library includes: product identification and feature data set. The product identification can be data that uniquely identifies a product, such as product name, product ID, etc. The feature data set includes one or more standard feature data.

[0094] This embodiment calculates the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory, so that the process detection path meets the requirements of the scanning index set, which is beneficial to improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection results; by performing image acquisition on the target product according to the process detection path and the line scan laser data, image acquisition is achieved by using laser line scanning. Compared with the non-laser line scanning method, the laser line scanning method is beneficial to improving the imaging accuracy of irregular products and further improving the accuracy of the single-frame process detection results.

[0095] In one embodiment, the scanning index set includes: a first scanning index, a second scanning index, and a third scanning index;

[0096] The first scanning index is used to make the process surface of the target product be located at the center position of the laser line array corresponding to the line scanning laser data, wherein the center position is the center point position of the X-axis of the three-dimensional line scanning camera corresponding to the line scanning laser data;

[0097] The second scanning index is used to make the process surface perpendicular to the emission direction of the laser line array;

[0098] The third scanning index is used to make the arrangement direction of the laser line array perpendicular to the moving direction of the process inspection object on the process surface, and the arrangement direction of the laser line array parallel to the tangent direction of the process inspection object.

[0099] The process surface is the surface of the process operation, which can be a regular plane or an irregular three-dimensional space surface.

[0100] like Figure 2 As shown, the surface corresponding to ABCD is the process surface, XL is the laser line array, DX is the process detection object (that is, the glue path), OF is the emission direction of the laser line array, OY is the moving direction of the process detection object on the process surface, and OP is the arrangement direction of the laser line array.

[0101] This embodiment clearly determines the relationship between the laser line array, the process surface and the process inspection object through the first scanning index, the second scanning index and the third scanning index, so that the process inspection path determined according to the preset scanning index set is unique; by making the process surface of the target product located at the center of the laser line array corresponding to the line scanning laser data, the process inspection object is located at the center of the field of view of the three-dimensional line scanning camera, thereby minimizing the image blind area and improving the accuracy of the single-frame image; by making the process surface perpendicular to the emission direction of the laser line array, the arrangement direction of the laser line array perpendicular to the moving direction of the process inspection object of the process surface, and the arrangement direction of the laser line array parallel to the tangent direction of the process inspection object, an accurate section of the process inspection object is obtained from the imaging principle and geometric characteristics of the three-dimensional line scanning camera, thereby ensuring the accuracy of the process inspection feature data and facilitating improving the accuracy of the single-frame process inspection result.

[0102] In one embodiment, the step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product to obtain a single-frame process detection result includes:

[0103] S302: Acquire each standard feature data corresponding to the process detection feature data from the standard feature data set as a target feature data set;

[0104] Specifically, each standard feature data corresponding to the process detection feature data is acquired from the standard feature data set, and each extracted feature data is used as a target feature data set, thereby determining all standard feature data that the process detection feature data needs to satisfy.

[0105] S304: Determine whether the process detection feature data is within the feature range corresponding to the target feature data set;

[0106] Specifically, any feature data in the process detection feature data is used as the data to be analyzed; it is determined whether the data to be analyzed is located in the feature range corresponding to the standard feature data corresponding to the data to be analyzed in the target feature data set; if so, it is determined that the data to be analyzed is located within the feature range; otherwise, it is determined that the data to be analyzed is located outside the feature range.

[0107] When all feature data in the process detection feature data are qualified, the single-frame process detection result is determined to be qualified; otherwise, the single-frame process detection result is determined to be unqualified.

[0108] S306: If yes, determining the single-frame process inspection result as qualified;

[0109] Specifically, if yes, that is, all feature data in the process detection feature data are within the feature range, therefore, the single-frame process detection result is determined to be qualified.

[0110] S308: If not, determining the single-frame process inspection result as unqualified.

[0111] Specifically, if not, that is, part of the data or all of the data in the process detection feature data is outside the feature range, therefore, the single-frame process detection result is determined to be unqualified.

[0112] In one embodiment, after the step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product to obtain a single-frame process detection result, the step further includes:

[0113] S402: constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single-frame images to form an initial three-dimensional point cloud image;

[0114] Specifically, according to the scanning indicator set, each of the single frame images is inserted into the process inspection path, each point corresponding to each of the single frame images on the process inspection path is used as a three-dimensional point cloud image, and the three-dimensional point cloud image is used as an initial three-dimensional point cloud image.

[0115] That is, the initial 3D point cloud image is a 3D image composed of point clouds.

[0116] S404: marking the initial three-dimensional point cloud image according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product.

[0117] Specifically, the initial three-dimensional point cloud image is marked according to each of the single-frame process inspection results using a preset marking specification, and the marked initial three-dimensional point cloud image is used as the target three-dimensional point cloud image corresponding to the target product.

[0118] Optionally, the marking specification is to use the single-frame image whose process inspection result is unqualified as the image to be marked, and mark the image to be marked as a first preset color in the initial three-dimensional point cloud image.

[0119] Optionally, the marking specification is to use the single-frame image with a qualified single-frame process inspection result as the image to be marked, and mark the image to be marked as a second preset color in the initial three-dimensional point cloud image.

[0120] This embodiment can intuitively describe the product shape by constructing a three-dimensional point cloud image, and mark the initial three-dimensional point cloud image according to each of the single-frame process inspection results, thereby integrating the single-frame process inspection results into the three-dimensional point cloud image, which is conducive to intuitively displaying the results of process inspection.

[0121] In one embodiment, the step of constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single frame images to form an initial three-dimensional point cloud image includes:

[0122] S502: Using the process detection path as a splicing path;

[0123] Specifically, the process detection path is used as a splicing path, thereby determining a path basis for inserting a single frame image.

[0124] S504: taking any one of the single-frame images as an image to be stitched;

[0125] Specifically, any one of the single-frame images is used as the image to be stitched, that is, one single-frame image is inserted into the stitching path each time.

[0126] S506: Calculating the stitching position of the image to be stitched according to the process detection path;

[0127] Specifically, according to the shooting time of the image to be spliced, the shooting position of the image to be spliced ​​corresponding to the process inspection path is determined, and the shooting position is used as the splicing position.

[0128] The shooting position is the intersection position of the laser line array corresponding to the line scanning laser data and the process detection path.

[0129] S508: Calculating a stitching angle for the image to be stitched according to the scanning index set, the stitching position, and the process detection path;

[0130] Specifically, the shape data of the process detection path at the stitching position is first determined, and then based on the constraints of the scanning index set, the stitching angle of the image to be stitched is calculated according to the shape data.

[0131] That is to say, the stitching angle of the to-be-stitched images needs to meet the requirements of each scanning index in the scanning index set.

[0132] S510: inserting the image to be stitched into the stitching path according to the stitching angle and the stitching position;

[0133] Specifically, the image to be stitched is inserted into the stitching position of the stitching path at the stitching angle.

[0134] S512: Repeat the step of using any one of the single-frame images as the image to be stitched, until all the single-frame images are inserted into the stitching path;

[0135] Specifically, the step of taking any one of the single-frame images as the image to be stitched is repeated, that is, steps S504 to S512 are repeated until all the single-frame images are inserted into the stitching path; when all the single-frame images are inserted into the stitching path, steps S504 to S512 are stopped and step S514 is started.

[0136] S514: Determine the initial three-dimensional point cloud image according to each of the single-frame images on the stitching path.

[0137] Specifically, a point cloud image composed of points corresponding to each of the single-frame images on the stitching path is used as the initial three-dimensional point cloud image.

[0138] This embodiment inserts the image to be stitched into the stitching path according to the stitching angle and the stitching position, thereby improving the accuracy of constructing the three-dimensional point cloud image; and calculates the stitching angle according to the scanning indicator set and the process detection path, so that the stitching angle of the image to be stitched conforms to the angle of image acquisition, thereby improving the accuracy of insertion; by performing stitching angle calculation, stitching position calculation, and path for inserting the image to be stitched based on the process detection path, the three-dimensional point cloud image of the target product is accurately restored, thereby improving the accuracy of the target three-dimensional point cloud image.

[0139] In one embodiment, the step of performing image acquisition on the target product according to the process detection path and the line scan laser data to obtain a single frame image includes:

[0140] S602: performing image acquisition on the target product according to the target scanning mode corresponding to the target product, the process detection path and the line scanning laser data to obtain the single frame image;

[0141] The target scanning mode includes: scanning at a preset time interval or trigger scanning. Scanning at a preset time interval is to emit a laser line array for scanning at a preset time interval, and each time an image is scanned as the single frame image. Trigger scanning is to emit a laser line array for scanning when a scanning signal is received.

[0142] Specifically, the execution device adopts a preset grasping method to grasp the target product, and then executes according to the process detection path to move the target product to the bottom of the three-dimensional line scan camera corresponding to the line scan laser data; the three-dimensional line scan camera emits laser lines according to the target scanning mode corresponding to the target product and the line scan laser data to realize image acquisition of the target product, and the acquired two-dimensional image is used as a single-frame image.

[0143] The step of calculating the stitching position of the images to be stitched according to the process detection path includes:

[0144] S604: If the target scanning mode is scanning at a preset time interval, determining the shooting time of the image to be stitched according to the time interval and each of the single-frame images, and calculating the stitching position according to the shooting time, the process inspection path, and the corresponding moving speed of the target product;

[0145] Specifically, if the target scanning mode is to scan at a preset time interval, it means that the interval between the shooting times of two adjacent single-frame images is the same. Therefore, the sorting number of the image to be spliced ​​in each of the single-frame images can be calculated, and the shooting time of the image to be spliced ​​can be calculated according to the sorting number and the time interval; then, the moving distance of the target product is determined according to the shooting time and the moving speed corresponding to the target product, and the end position corresponding to the moving distance is determined on the process inspection path, and the end position is used as the splicing position.

[0146] S606: If the target scanning mode is a trigger scanning, the stitching position calculation is performed on the image to be stitched according to the preset shooting position data and the process detection path.

[0147] Specifically, if the target scanning mode is a triggered scanning, it means that each single frame image is captured only after receiving a scanning signal. Therefore, the position data corresponding to the image to be stitched is determined from the preset shooting position data, and the position of the position data corresponding to the image to be stitched on the process inspection path is used as the stitching position.

[0148] This embodiment adopts the target scanning mode corresponding to the target product to perform image acquisition, and calculates the stitching position of the image to be stitched based on the stitching position calculation method corresponding to the target scanning mode corresponding to the target product, thereby realizing automatic image acquisition and three-dimensional point cloud image construction according to different target scanning modes, thereby improving the degree of automation of this application.

[0149] In one embodiment, the step of marking the initial three-dimensional point cloud image according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product includes:

[0150] S702: taking each of the single-frame images with unqualified single-frame process inspection results as an image to be marked;

[0151] Specifically, each of the single-frame images whose single-frame process detection results are unqualified is used as an image to be marked, thereby providing a basis for marking the position where the process detection fails.

[0152] S704: Marking each of the to-be-marked images in the initial three-dimensional point cloud image using a preset marking format;

[0153] Specifically, the color of each point in the initial three-dimensional point cloud image corresponding to each of the images to be marked is marked as a color corresponding to a preset marking format.

[0154] Optionally, the preset mark format is to use a preset color.

[0155] Optionally, the single-frame process inspection result includes: inspection conclusion and unqualified feature point set; any image to be marked is used as the image to be processed; and each point in the initial three-dimensional point cloud image corresponding to the image to be processed is marked using the marking format corresponding to the unqualified feature point set corresponding to the image to be processed. This enables different marking formats to be used for different unqualified feature point sets, and different unqualified feature points to be marked by different marking formats, further improving the speed of locating and solving problems.

[0156] The unqualified feature point set is a set of unqualified feature points.

[0157] S706: Using the marked initial three-dimensional point cloud image as the target three-dimensional point cloud image.

[0158] Specifically, the marked initial three-dimensional point cloud image is used as the target three-dimensional point cloud image, thereby obtaining a target three-dimensional point cloud image with unqualified point clouds marked.

[0159] This embodiment uses a preset marking format to mark the single-frame image in the initial three-dimensional point cloud image whose process inspection result is unqualified, thereby intuitively displaying the unqualified single-frame image on the target three-dimensional point cloud image, thereby improving the speed of locating and solving problems.

[0160] like Figure 3 As shown, a product process detection device, the device comprises:

[0161] A process detection path determination module 802 is used to obtain a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0162] A single-frame image determination module 804 is used to collect images of the target product according to the process detection path and the line scanning laser data to obtain a single-frame image;

[0163] A feature point data set determination module 806 is used to calculate a feature point data set for the single frame image according to the feature point configuration corresponding to the target product;

[0164] A process detection feature data determination module 808 is used to calculate process detection feature data according to the feature point data set;

[0165] The single-frame process detection result determination module 810 is used to determine whether the process is qualified based on the standard feature data set corresponding to the target product, and obtain a single-frame process detection result.

[0166] This embodiment calculates the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory, so that the process detection path meets the requirements of the scanning index set, which is beneficial to improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection results; by performing image acquisition on the target product according to the process detection path and the line scan laser data, image acquisition is achieved by using laser line scanning. Compared with the non-laser line scanning method, the laser line scanning method is beneficial to improving the imaging accuracy of irregular products and further improving the accuracy of the single-frame process detection results.

[0167] Figure 4 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement the product process detection method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement the product process detection method. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0168] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0169] Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0170] According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image;

[0171] Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product;

[0172] Calculating process detection feature data according to the feature point data set;

[0173] According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

[0174] This embodiment calculates the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory, so that the process detection path meets the requirements of the scanning index set, which is beneficial to improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection results; by performing image acquisition on the target product according to the process detection path and the line scan laser data, image acquisition is achieved by using laser line scanning. Compared with the non-laser line scanning method, the laser line scanning method is beneficial to improving the imaging accuracy of irregular products and further improving the accuracy of the single-frame process detection results.

[0175] In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0176] Acquire a process processing trajectory and line scan laser data of a target product, and calculate a process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory;

[0177] According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image;

[0178] Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product;

[0179] Calculating process detection feature data according to the feature point data set;

[0180] According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

[0181] This embodiment calculates the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory, so that the process detection path meets the requirements of the scanning index set, which is beneficial to improving the accuracy of the single-frame image obtained by image acquisition and improving the accuracy of the single-frame process detection results; by performing image acquisition on the target product according to the process detection path and the line scan laser data, image acquisition is achieved by using laser line scanning. Compared with the non-laser line scanning method, the laser line scanning method is beneficial to improving the imaging accuracy of irregular products and further improving the accuracy of the single-frame process detection results.

[0182] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0184] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims. Please enter the specific implementation content part.

Claims

1. A product process detection method, the method comprising: Acquire the process processing trajectory and line scan laser data of the target product, and calculate the process detection path according to the preset scanning index set, the line scan laser data and the process processing trajectory; wherein the scanning index set includes: a first scanning index, a second scanning index and a third scanning index; the first scanning index is used to make the process surface of the target product located at the center position of the laser line array corresponding to the line scan laser data, wherein the center position is the center point position of the X-axis of the three-dimensional line scan camera corresponding to the line scan laser data; the second scanning index is used to make the process surface perpendicular to the emission direction of the laser line array; the third scanning index is used to make the arrangement direction of the laser line array perpendicular to the moving direction of the process detection object of the process surface, and the arrangement direction of the laser line array is parallel to the tangent direction of the process detection object; According to the process detection path and the line scanning laser data, an image of the target product is acquired to obtain a single frame image; Calculating a feature point data set for the single-frame image according to the feature point configuration corresponding to the target product; Calculating process detection feature data according to the feature point data set; According to the standard feature data set corresponding to the target product, the process detection feature data is judged whether the process is qualified to obtain a single-frame process detection result.

2. The product process detection method according to claim 1, characterized in that: The step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product to obtain a single-frame process detection result includes: Acquire each standard feature data corresponding to the process detection feature data from the standard feature data set as a target feature data set; Determining whether the process detection feature data is within a feature range corresponding to the target feature data set; If yes, the single-frame process inspection result is determined to be qualified; If not, the single-frame process inspection result is determined to be unqualified.

3. The product process detection method according to claim 1, characterized in that: After the step of determining whether the process is qualified based on the process detection feature data according to the standard feature data set corresponding to the target product to obtain a single-frame process detection result, the method further includes: Constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single-frame images to form an initial three-dimensional point cloud image; The initial three-dimensional point cloud image is marked according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product.

4. The product process detection method according to claim 3, characterized in that: The step of constructing a three-dimensional point cloud image according to the scanning index set, the process detection path and each of the single frame images to form an initial three-dimensional point cloud image includes: Using the process detection path as a splicing path; Taking any one of the single-frame images as the image to be stitched; Calculating the stitching position of the images to be stitched according to the process detection path; Calculating a stitching angle of the image to be stitched according to the scanning index set, the stitching position and the process detection path; Inserting the image to be stitched into the stitching path according to the stitching angle and the stitching position; Repeat the step of using any one of the single-frame images as the image to be stitched until all the single-frame images are inserted into the stitching path; The initial three-dimensional point cloud image is determined according to each of the single-frame images on the stitching path.

5. The product process detection method according to claim 4, characterized in that: The step of performing image acquisition on the target product according to the process detection path and the line scan laser data to obtain a single frame image includes: According to the target scanning mode corresponding to the target product, the process detection path and the line scanning laser data, the target product is imaged to obtain the single frame image; The step of calculating the stitching position of the images to be stitched according to the process detection path includes: If the target scanning mode is scanning at a preset time interval, the shooting time of the image to be stitched is determined according to the time interval and each of the single-frame images, and the stitching position is calculated according to the shooting time, the process inspection path and the corresponding moving speed of the target product; If the target scanning mode is a trigger scanning, the stitching position calculation is performed on the image to be stitched according to the preset shooting position data and the process detection path.

6. The product process detection method according to claim 3, characterized in that: The step of marking the initial three-dimensional point cloud image according to each of the single-frame process detection results to obtain a target three-dimensional point cloud image corresponding to the target product includes: Taking each of the single-frame images with unqualified single-frame process detection results as an image to be marked; Using a preset marking format to mark each of the to-be-marked images in the initial three-dimensional point cloud image; The marked initial three-dimensional point cloud image is used as the target three-dimensional point cloud image.

7. A product process detection device, characterized in that: The device comprises: A process detection path determination module is used to obtain the process processing trajectory and line scan laser data of the target product, and calculate the process detection path according to a preset scanning index set, the line scan laser data and the process processing trajectory; wherein the scanning index set includes: a first scanning index, a second scanning index and a third scanning index; the first scanning index is used to make the process surface of the target product located at the center position of the laser line array corresponding to the line scan laser data, wherein the center position is the center point position of the X-axis of the three-dimensional line scan camera corresponding to the line scan laser data; the second scanning index is used to make the process surface perpendicular to the emission direction of the laser line array; the third scanning index is used to make the arrangement direction of the laser line array perpendicular to the moving direction of the process detection object of the process surface, and the arrangement direction of the laser line array is parallel to the tangent direction of the process detection object; A single-frame image determination module, used to collect images of the target product according to the process detection path and the line scanning laser data to obtain a single-frame image; A feature point data set determination module, used to calculate a feature point data set for the single frame image according to the feature point configuration corresponding to the target product; A process detection feature data determination module, used for calculating process detection feature data according to the feature point data set; The single-frame process detection result determination module is used to determine whether the process is qualified based on the standard feature data set corresponding to the target product, and obtain a single-frame process detection result.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

Images