Data acquisition method and device oriented to skills of quality inspection workers, terminal and medium

Abstract

The invention provides a quality inspection worker skill-oriented data acquisition method and device, a terminal and a medium, and relates to the technical field of data acquisition, the method comprises the following steps: in a current workpiece defect detection scene, acquiring depth information in a process of performing defect detection on a workpiece by a worker, which is acquired by depth information acquisition equipment in real time; and based on the depth information, generating point cloud data used for representing quality inspection skills when the worker performs defect detection on the workpiece. According to the invention, the depth information acquisition device acquires the depth information of the worker in the process of detecting the workpiece defects in real time, and based on the depth information, the point cloud data which can be used for representing the quality inspection skills executed when the worker detects the defects of the workpiece is generated.

Description

Technical Field

[0001] The present invention relates to the technical field of data collection, and in particular to a method, device, terminal and medium for collecting data on the skills of quality inspection workers. Background Art

[0002] At present, when skilled quality inspectors in factories detect workpiece defects, they mostly rely on their long-term work experience to complete the work of workpiece quality inspection. Their quality inspection skills are not quantified through data, and their workpiece quality inspection skills cannot be checked through specific data records.

[0003] Therefore, how to collect relevant data for characterizing the quality inspection skills performed by workers when performing defect inspection on workpieces and quantify the workers' quality inspection skills with data is a technical problem to be solved by technical personnel in this field. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in response to the above-mentioned defects of the prior art, a data collection method, device, terminal and medium for quality inspection workers' skills are provided, which can collect data representing the quality inspection skills performed by workers when performing defect detection on workpieces, and enable data quantification of the workers' quality inspection skills.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] A data collection method for quality inspection worker skills, wherein the method comprises:

[0007] In the current workpiece defect detection scenario, the depth information collected by the depth information acquisition device in real time during the defect detection process of the worker on the workpiece is obtained;

[0008] Point cloud data is generated based on the depth information to characterize the quality inspection skills of workers when performing defect inspection on workpieces.

[0009] In one implementation, the process of acquiring the depth information collected in real time by the depth information collection device during the defect detection process of the worker on the workpiece includes:

[0010] A depth image and an RGB image captured in real time by a depth camera during a worker's defect detection process on a workpiece are obtained; wherein the depth camera is installed on a helmet worn by the worker.

[0011] In one implementation, the method for collecting data on the skills of quality inspection workers further includes:

[0012] The depth camera is calibrated using a calibration plate to determine camera internal parameters and distortion parameters.

[0013] In one implementation, generating point cloud data for characterizing the quality inspection skills of a worker when performing defect inspection on a workpiece based on the depth information includes:

[0014] According to the depth value of each pixel in the depth image and the internal parameters of the camera, each pixel in the depth image is converted into point coordinates in three-dimensional space to obtain point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces.

[0015] In one implementation, before converting each pixel in the depth image into point coordinates in a three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece, the method further includes:

[0016] Performing noise removal processing on the depth image to obtain a processed depth image;

[0017] The processed depth image is geometrically corrected according to the camera internal parameters and the distortion parameters to obtain a corrected depth image.

[0018] A coordinate alignment process is performed on the corrected depth image and the RGB image.

[0019] In one implementation, after converting each pixel in the depth image into point coordinates in a three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece, the method further includes:

[0020] Using the Canny edge detection algorithm to identify the edges of objects in the depth image to extract corresponding contour information;

[0021] A feature extraction algorithm is used to extract key points and feature descriptors from the depth image and the RGB image.

[0022] The point cloud data is optimized using the contour information, the key points and the feature descriptors to obtain optimized point cloud data.

[0023] In one implementation, before the depth camera is used to collect the depth image of the worker during the defect detection process on the workpiece in real time, the method further includes:

[0024] Using the depth camera to calibrate the light source position of the light source in the current workpiece defect detection scene and the workpiece position in the worker's hand;

[0025] Based on the position of the light source and the workpiece, the relative positions among the depth camera, the light source and the workpiece, and the relative position of the depth camera in the world coordinate system are determined in real time by a six-axis gyroscope installed on the helmet worn by the worker.

[0026] The present invention also discloses a data collection device for quality inspection worker skills, wherein the device comprises:

[0027] The depth information acquisition module is used to obtain the depth information of the worker performing defect detection on the workpiece collected in real time by the depth information acquisition device in the current workpiece defect detection scenario;

[0028] The point cloud data generation module is used to generate point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces based on the depth information.

[0029] The present invention also discloses a terminal, which includes: a memory, a processor, and a data collection program for quality inspection workers' skills stored in the memory and runnable on the processor. When the data collection program for quality inspection workers' skills is executed by the processor, the steps of the data collection method for quality inspection workers' skills as described above are implemented.

[0030] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the data collection method for quality inspection worker skills as described above.

[0031] The present invention provides a data collection method, device, terminal and medium for the skills of quality inspection workers. The data collection method for the skills of quality inspection workers includes: in the current workpiece defect detection scenario, obtaining the depth information of the worker in the process of defect detection on the workpiece collected in real time by the depth information collection device; based on the depth information, generating point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece. It can be seen that the present invention collects the depth information of the worker in the process of defect detection on the workpiece in real time through the depth information collection device, and then generates point cloud data characterizing the quality inspection skills of the worker when performing defect detection on the workpiece based on the depth information, that is, the technical solution of the present invention can collect point cloud data characterizing the quality inspection skills performed by the worker when performing defect detection on the workpiece, and realize data quantification of the worker's quality inspection skills. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of a preferred embodiment of the data collection method for quality inspection worker skills in the present invention;

[0033] Figure 2It is a flow chart of a specific data collection method for quality inspection worker skills disclosed in the present invention;

[0034] Figure 3 It is a schematic diagram of a helmet equipped with a depth camera and a six-axis gyroscope disclosed in the present invention;

[0035] Figure 4 It is a flow chart of a specific data collection method for quality inspection worker skills disclosed in the present invention;

[0036] Figure 5 This is a functional principle block diagram of a preferred embodiment of a data collection device for quality inspection of worker skills in the present invention;

[0037] Figure 6 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] See also Figure 1 , Figure 1 Flowchart of the data collection method for quality inspection worker skills in the present invention. Figure 1 As shown, the data collection method for quality inspection worker skills according to the embodiment of the present invention includes:

[0040] Step S11: In the current workpiece defect detection scenario, the depth information collected in real time by the depth information acquisition device during the worker's defect detection process on the workpiece is obtained.

[0041] In this embodiment, the depth information of the scene is collected in real time by the depth information collection device, that is, in the current workpiece defect detection scene, the depth information of the worker performing defect detection on the workpiece collected in real time by the depth information collection device is obtained. It can be understood that the depth information of the worker detecting the workpiece defect is collected in real time by the depth information collection device.

[0042] It should be noted that the depth information acquisition device can be a laser radar, a depth camera, or a depth information acquisition device that combines multiple sensors. For example, for the detection of large-scale workpiece defects, a laser radar can be used to perform high-precision three-dimensional scanning of the workpiece to obtain detailed depth information. For workpieces that workers can hold for inspection, a depth camera can be used to scan and obtain the corresponding depth information. In addition, by using different types of sensors, such as RGB cameras, depth cameras, IMUs, etc., the accuracy and robustness of detection can be improved.

[0043] Step S12: generating point cloud data for characterizing the quality inspection skills of workers when performing defect inspection on workpieces based on the depth information.

[0044] In this embodiment, after the depth information of the worker in the process of detecting workpiece defects is collected in real time by the depth information collection device, the depth information is processed to obtain data that can characterize the quality inspection skills performed by the worker when performing defect detection on the workpiece, that is, point cloud data that can characterize the quality inspection skills performed by the worker when performing defect detection on the workpiece is generated based on the depth information, thereby realizing the data quantification of the worker's quality inspection skills and realizing the data collection of the worker's skills in detecting workpiece defects. In this embodiment, after the point cloud data that can quantify the worker's skills in detecting workpiece defects is generated based on the depth information, the point cloud data can be processed using a Poisson reconstruction algorithm or a triangulation algorithm to generate a smooth three-dimensional surface model, which can be used for visualization of a three-dimensional mesh.

[0045] It should be pointed out that the execution subject of the present application can be a computer capable of processing depth information, that is, the depth information acquisition device uploads the collected depth information to the computer for processing, and then executes the depth information acquisition and processing operations corresponding to the above steps S11 and S12 through the computer, and finally uploads the point cloud data to the cloud server to provide data quantification related to the worker's quality inspection skills for application scenarios in other fields, thereby completing the data collection task. Specifically, the point cloud data is sent to the cloud server through an HTTP POST request, and a suitable timeout and retry mechanism can be set in the cloud server, and a data processing program can also be set in the cloud server. Moreover, before the depth information acquisition device uploads the collected depth information to the computer for processing, it can also perform corresponding preprocessing on the collected depth information through an architecture that supports real-time data stream processing to ensure the stability and speed of transmission, wherein the architecture that supports real-time data stream processing can be a message queue or stream processing framework, such as MQTT, Kafka, WebSocket, etc. For example, the depth image and color image are compressed through the architecture to reduce the amount of transmitted data, and the data is converted into a byte stream to ensure that the data format is suitable for transmission.

[0046] It can be seen that in the embodiment of the present application, the depth information of the workers in the process of defect detection on the workpiece is collected in real time by the depth information collection equipment, and then point cloud data representing the quality inspection skills of the workers when performing defect detection on the workpiece is generated based on the depth information. That is, the technical solution of the present invention can collect point cloud data representing the quality inspection skills performed by the workers when performing defect detection on the workpiece, and realize data quantification of the workers' quality inspection skills.

[0047] For example, robot skill learning is one of the most popular and innovative directions at present. Many companies have made significant progress in home automation. For example, smart cleaning robots can effectively complete basic tasks such as floor cleaning and clothing sorting. Although these systems perform well in efficiency, most of them still rely on preset, simple operating procedures and lack the ability to adapt to complex tasks. In other words, the level of robot learning skills remains at the level of simple mechanical repetition of simple human actions. One of the major limitations of robot skill learning is that it is difficult to learn more humane and intelligent skills, which makes it difficult for robots to handle tasks flexibly according to the scene in an intelligent and humane manner. Therefore, in the specific application scenario of robot learning of workpiece defect detection skills, the current level of robot learning skills can only simply and mechanically repeat simple human actions, but cannot be intelligent and humane. In order to solve the problem of flexibly handling tasks according to the scenario, the point cloud data related to the workers' quality inspection skills that can be quantified can be used to train the robot, so that the robot can learn skills based on these point cloud data, and thus can learn more humanized and intelligent workpiece defect skills, and then can flexibly handle tasks according to the scenario. That is, the depth information of the workers in the process of defect detection on the workpiece is collected in real time by the depth information acquisition equipment, and then point cloud data representing the quality inspection skills of the workers when performing defect detection on the workpiece is generated based on the depth information, and the point cloud data is uploaded to the cloud server to provide more training data for training the robot to learn workpiece detection skills. In other words, using the point cloud data to train the robot to learn skills about workpiece defect detection can enable the robot to learn more humanized workpiece defect detection skills, so that the trained robot can have a higher degree of intelligence.

[0048] See also Figure 2 As shown, the embodiment of the present invention discloses a specific data collection method for quality inspection workers' skills. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0049] Step S21, in the current workpiece defect detection scenario, obtain a depth image and an RGB image captured in real time by a depth camera during a worker's defect detection process on the workpiece; wherein the depth camera is installed on a helmet worn by the worker.

[0050] In this embodiment, a depth camera installed on a helmet worn by a worker is used to collect in real time depth images and RGB images (color images) of the worker while he is performing defect inspection on a workpiece. The installation position of the depth camera on the helmet must ensure that the depth camera can capture the situation of the worker holding the workpiece when the worker is performing defect inspection on the workpiece. That is to say, the worker wears a helmet equipped with a depth camera during the process of inspecting defects in the workpiece, and then the depth camera on the helmet is used to collect in real time depth images and RGB images of the worker holding the workpiece when the worker is performing defect inspection on the workpiece.

[0051] In this embodiment, before the depth image of the worker performing defect detection on the workpiece is collected in real time by the depth camera, it can also include: using the depth camera to calibrate the light source position of the light source in the current workpiece defect detection scene and the workpiece position of the workpiece in the worker's hand; based on the light source position and the workpiece position, and through the six-axis gyroscope installed on the helmet worn by the worker, the relative position between the depth camera, the light source and the workpiece, and the relative position of the depth camera in the world coordinate system are determined in real time. It can be understood that the RGB-D camera (depth camera) is used to perform a preliminary calibration of the workshop where the worker is located, and the main calibration targets are the workpiece position of the workpiece in the worker's hand and the light source position in the workshop where the worker is located, and then the six-axis gyroscope installed on the helmet worn by the worker determines the relative positions of the camera, the workpiece, and the light source in real time, so as to ensure that the worker can perform workpiece defect detection normally.

[0052] It should be noted that the six-axis gyroscope and depth camera are installed on the helmets worn by workers. For their specific placement and angle on the helmets, see Figure 3 As shown, in the data collection process, at the beginning, the worker wearing the helmet only needs to look around the current workpiece defect detection scene once, which does not affect the worker's normal work, making data collection more convenient. The data collected in this way is more intelligent and can better simulate the work of people performing workpiece defect detection based on experience.

[0053] Among them, the built-in infrared laser transmitter of the RGB-D camera periodically emits pulsed infrared light to cover the entire observation scene of the current workpiece defect detection. The wavelength of the infrared light is generally 940nm. The emitted infrared light propagates in the scene and encounters different object surfaces. Objects of different materials will reflect these lights in different ways, and then the infrared light reflected from the surface of the object is received by the depth sensor. The depth sensor is usually a CMOS image sensor, which can record the moment of emitting pulsed infrared light and the moment of receiving reflected light. Then, by calculating the time difference between the two moments, the depth camera can accurately measure the total time required for light to be emitted to received, and then the depth value of each pixel can be calculated according to the speed of light. Finally, the depth values ​​of all pixels are integrated to generate a depth image. That is, the value of each pixel in the depth image corresponds to the distance of the corresponding position in the scene. Therefore, the depth camera can quickly and accurately obtain the depth information of the three-dimensional space through the above principle, and can calibrate the workpiece and the light source position.

[0054] Among them, the six-axis gyroscope is composed of a three-axis gyroscope and a three-axis accelerometer, which can simultaneously measure the angular velocity and linear acceleration of an object. It is particularly important in posture detection. For example, by measuring the angular velocity of the object in three dimensions, such as pitch, roll, and yaw, the rotation state of the object can be monitored in real time. Therefore, in this embodiment, the six-axis gyroscope can be used to ensure that after the preliminary calibration of the workpiece position and the light source position, the workers can perform the defect detection action normally.

[0055] Step S22: calibrate the depth camera using a calibration plate to determine camera internal parameters and distortion parameters.

[0056] It is understandable that the camera internal parameters and distortion parameters are determined by camera calibration, wherein the camera calibration process utilizes a calibration plate, i.e., a chessboard calibration plate, such as by taking photos of the calibration plate at different positions and angles, and using the known feature points in these photos, the camera internal parameters and distortion parameters can be calculated, and the camera external parameters can also be calculated. For example, the distortion parameters can be determined by calibrating calibration points on a workbench and determining the distance between two calibration points. Image distortion generally occurs at the edge. By measuring the distance between the two calibration points to compare with the actual distance, the distortion is corrected, thereby determining the distortion coefficient of the camera lens.

[0057] It should be noted that the camera internal parameters may include focal length, principal point coordinates, pixel size, etc., while the distortion parameters may include radial distortion and tangential distortion coefficients.

[0058] Step S23, according to the depth value of each pixel in the depth image and the internal parameters of the camera, each pixel in the depth image is converted into point coordinates in three-dimensional space to obtain point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces.

[0059] In this embodiment, 3D reconstruction is performed based on the depth image, that is, each pixel in the depth image is converted into point coordinates (x, y, z) in three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera, so as to obtain point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces.

[0060] In this embodiment, before performing 3D reconstruction based on the depth image, the depth image may be preprocessed, the depth image may be subjected to noise removal to obtain a processed depth image; the processed depth image may be subjected to geometric correction according to the camera internal parameters and distortion parameters to obtain a corrected depth image; the corrected depth image and the RGB image may be subjected to coordinate alignment, and then 3D reconstruction may be performed based on the preprocessed depth image. It is understandable that the depth image may be subjected to geometric correction to correct image distortion, and then image alignment may be performed by coordinate transformation to ensure that the depth image and the RGB image with different pixels can be aligned in the same coordinate system.

[0061] In this embodiment, the depth image may also be mapped into a pseudo-color image for analysis and visualization. Generally, a larger depth value corresponds to a colder color, such as blue, and a smaller depth value corresponds to a warmer color, such as red.

[0062] It should be pointed out that noise removal processing can include multiple filtering methods such as median filtering, Gaussian filtering, and depth filling. Among them, median filtering replaces the central pixel by taking the median of the neighboring pixels, which is particularly effective in removing salt and pepper noise; Gaussian filtering uses a Gaussian kernel to smooth the depth image to reduce the impact of high-frequency noise on subsequent processing; depth filling can use interpolation methods such as bilinear interpolation to fill in missing or invalid depth values ​​in the depth image to ensure data integrity.

[0063] It can be seen that in the embodiment of the present invention, the depth image and RGB image of the worker in the process of defect detection on the workpiece are acquired by the depth camera, and then each pixel in the depth image is converted into a point coordinate in the three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera, so as to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece. That is, the technical solution of the embodiment of the present invention can collect point cloud data for characterizing the quality inspection skills performed by the worker when performing defect detection on the workpiece, and realize data quantification of the worker's quality inspection skills.

[0064] See also Figure 4 As shown, the embodiment of the present invention discloses a specific data collection method for quality inspection workers' skills. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0065] Step S31, in the current workpiece defect detection scenario, obtain a depth image and an RGB image captured in real time by a depth camera during a worker's defect detection process on the workpiece; wherein the depth camera is installed on a helmet worn by the worker.

[0066] Step S32: calibrate the depth camera using a calibration plate to determine camera internal parameters and distortion parameters.

[0067] Step S33: convert each pixel in the depth image into point coordinates in three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera, so as to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece.

[0068] Step S34: using the Canny edge detection algorithm to identify the edges of objects in the depth image to extract corresponding contour information, and using a feature extraction algorithm to extract key points and feature descriptors from the depth image and the RGB image.

[0069] In this embodiment, after generating point cloud data based on the depth image, the features of the depth image and the color image can also be extracted. Specifically, feature extraction is performed using edge detection and feature description methods, that is, the Canny edge detection algorithm is used to identify the edges of objects in the depth image to extract the corresponding contour information, and the feature extraction algorithm is used to extract key points and feature descriptors from the depth image and the RGB image.

[0070] Step S35: Optimize the point cloud data using the contour information, the key points and the feature descriptors to obtain optimized point cloud data.

[0071] In this embodiment, after feature extraction is performed using edge detection and feature description methods to obtain contour information, key points and feature descriptors, the point cloud data is optimized using the contour information, key points and feature descriptors to obtain optimized point cloud data.

[0072] It should be pointed out that combining point cloud registration with two-dimensional image feature point matching can improve the registration accuracy. Finally, using color values ​​for texture mapping can make the reconstructed model have color attributes, thereby increasing the realism and restoration of the reconstruction.

[0073] For the specific contents of the above-mentioned step S31 and step S33, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be repeated here.

[0074] It can be seen that in the embodiment of the present invention, the depth image and RGB image of the worker in the process of defect detection on the workpiece are acquired by the depth camera, and then each pixel in the depth image is converted into a point coordinate in the three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera, so as to obtain point cloud data for characterizing the quality inspection skills of the worker in defect detection on the workpiece, and then optimize the point cloud data based on the object contour information, key points and feature descriptors extracted from the depth image and RGB image, so as to collect point cloud data characterizing the quality inspection skills performed by the worker in defect detection on the workpiece, and realize data quantification of the worker's quality inspection skills.

[0075] In one embodiment, if Figure 5 As shown, based on the above-mentioned data collection method for quality inspection worker skills, the present invention also provides a data collection device for quality inspection worker skills, including:

[0076] The depth information acquisition module 11 is used to acquire the depth information of the worker performing defect detection on the workpiece collected in real time by the depth information acquisition device in the current workpiece defect detection scenario;

[0077] The point cloud data generating module 12 is used to generate point cloud data for characterizing the quality inspection skills of workers when performing defect inspection on workpieces based on the depth information.

[0078] Figure 6 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0079] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0080] When the processor 502 executes the program, the data collection method for quality inspection worker skills provided in the above embodiment is implemented.

[0081] Furthermore, the terminal further includes:

[0082] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0083] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0084] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0085] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0086] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0087] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0088] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the above-mentioned data collection method for quality inspection worker skills is implemented.

[0089] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0090] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0091] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, in which the order shown or discussed may not be followed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0092] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can read instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary and then storing it in a computer memory.

[0093] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0094] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0096] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0097] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A data collection method for quality inspection worker skills, characterized in that: The method comprises: In the current workpiece defect detection scenario, the depth information collected by the depth information acquisition device in real time during the defect detection process of the worker on the workpiece is obtained; Point cloud data is generated based on the depth information to characterize the quality inspection skills of workers when performing defect inspection on workpieces.

2. The data collection method for quality inspection worker skills according to claim 1 is characterized in that: The depth information acquired by the depth information acquisition device in real time during the defect detection process of the workpiece by the worker includes: A depth image and an RGB image captured in real time by a depth camera during a worker's defect detection process on a workpiece are obtained; wherein the depth camera is installed on a helmet worn by the worker.

3. The data collection method for quality inspection worker skills according to claim 2 is characterized in that: Also includes: The depth camera is calibrated using a calibration plate to determine camera internal parameters and distortion parameters.

4. The data collection method for quality inspection worker skills according to claim 3 is characterized in that: The step of generating point cloud data based on the depth information for characterizing the quality inspection skills of workers when performing defect inspection on workpieces includes: According to the depth value of each pixel in the depth image and the internal parameters of the camera, each pixel in the depth image is converted into point coordinates in three-dimensional space to obtain point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces.

5. The method for collecting data on the skills of quality inspection workers according to claim 4, characterized in that: Before converting each pixel in the depth image into point coordinates in a three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece, the method further includes: Performing noise removal processing on the depth image to obtain a processed depth image; Performing geometric correction processing on the processed depth image according to the camera internal parameters and the distortion parameters to obtain a corrected depth image; A coordinate alignment process is performed on the corrected depth image and the RGB image.

6. The method for collecting data on the skills of quality inspectors according to any one of claims 2 to 5, characterized in that: After converting each pixel in the depth image into point coordinates in a three-dimensional space according to the depth value of each pixel in the depth image and the internal parameters of the camera to obtain point cloud data for characterizing the quality inspection skills of the worker when performing defect detection on the workpiece, the method further includes: Using the Canny edge detection algorithm to identify the edges of objects in the depth image to extract corresponding contour information; Extracting key points and feature descriptors from the depth image and the RGB image using a feature extraction algorithm; The point cloud data is optimized using the contour information, the key points and the feature descriptors to obtain optimized point cloud data.

7. The method for collecting data on the skills of quality inspection workers according to claim 6, characterized in that: Before collecting the depth image of the worker performing defect detection on the workpiece in real time through the depth camera, the method further includes: Using the depth camera to calibrate the light source position of the light source in the current workpiece defect detection scene and the workpiece position in the worker's hand; Based on the position of the light source and the workpiece, the relative positions among the depth camera, the light source and the workpiece, and the relative position of the depth camera in the world coordinate system are determined in real time by a six-axis gyroscope installed on the helmet worn by the worker.

8. A data collection device for quality inspection workers' skills, characterized in that: The device comprises: The depth information acquisition module is used to obtain the depth information of the worker performing defect detection on the workpiece collected in real time by the depth information acquisition device in the current workpiece defect detection scenario; The point cloud data generation module is used to generate point cloud data for characterizing the quality inspection skills of workers when performing defect detection on workpieces based on the depth information.

9. A terminal, characterized in that: include: A memory, a processor, and a data collection program for quality inspection worker skills stored in the memory and executable on the processor, wherein the data collection program for quality inspection worker skills, when executed by the processor, implements the steps of the data collection method for quality inspection worker skills as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the method for collecting data on the skills of quality inspection workers as described in any one of claims 1 to 7.

Images