Workpiece data acquisition device and data acquisition method

Abstract

The invention provides a data acquisition device and a data acquisition method for a workpiece. The device comprises a shell, a plurality of projection devices, a plurality of visual sensors and a control device, the control method comprises the following steps: determining a target acquisition mode in a plurality of acquisition modes according to an acquisition instruction; and controlling a visual sensor and a projection device corresponding to the target acquisition mode to perform data acquisition on the workpiece to obtain a workpiece point cloud. According to the invention, convenient and economical data acquisition for workpieces under different working conditions is realized.

Description

Technical Field

[0001] The technical field of the present application is the field of data acquisition, and specifically relates to a data acquisition device and a data acquisition method for a workpiece. Background Art

[0002] In the field of workpiece welding, welding robots are often used in conjunction with visual sensors for intelligent welding. However, welding conditions are relatively harsh, the environment is complex, the workpiece models are complex, and there are many types of workpieces. The key to solving the weld positioning problem lies in accurate data collection of the workpiece. However, existing data acquisition devices require different devices and different data acquisition methods for workpieces in different working conditions, which is more troublesome and requires the purchase of different data acquisition devices, increasing the measurement cost.

[0003] Therefore, how to conduct convenient and economical data collection for workpieces under different working conditions is an issue that needs to be solved urgently. Summary of the invention

[0004] The present application aims to solve the technical problem of how to conveniently and economically collect data for workpieces in different working conditions.

[0005] According to one aspect of an embodiment of the present application, the present application provides a data acquisition device for a workpiece, the device comprising:

[0006] a housing forming an outer shell of the device;

[0007] A plurality of projection devices, disposed in the housing, for projecting a measurement pattern onto the workpiece;

[0008] A plurality of visual sensors are disposed in the housing and are used to photograph the measurement pattern displayed on the workpiece to obtain measurement photos;

[0009] The control device is arranged in the shell and is used to select projection devices and visual sensors located at different positions to collect data on the workpiece according to the collection instructions. Different collection instructions instruct different projection devices and visual sensors to collect data on the workpiece to obtain the workpiece point cloud.

[0010] According to one aspect of an embodiment of the present application, the data acquisition device includes: a first visual sensor, a second visual sensor, and a first projection device;

[0011] Wherein, the first projection device is used to project a measurement pattern onto the workpiece;

[0012] The first visual sensor, the second visual sensor, and the first projection device are arranged on the same side of the housing, and the first visual sensor and the second visual sensor are arranged on both sides of the first projection device, respectively.

[0013] According to one aspect of the embodiment of the present application, the data acquisition device further includes: a first visual sensor, a second visual sensor, a second projection device, and a third projection device;

[0014] Wherein, the second projection device and the third projection device are used to emit laser to the workpiece to form a light spot and / or light as the measurement pattern on the surface of the workpiece;

[0015] The first visual sensor, the second visual sensor, the second projection device, and the third projection device are arranged on the same side of the shell, the first visual sensor is arranged on the left side of the second projection device, and the second visual sensor is arranged on the right side of the third projection device.

[0016] According to one aspect of the embodiment of the present application, the data acquisition device further includes a fourth projection device and a third visual sensor;

[0017] Among them, the fourth projection device is used to emit laser to the workpiece to form a light spot and / or light on the surface of the workpiece as the measurement pattern, the third visual sensor is used to photograph the measurement pattern displayed on the workpiece, and the fourth projection device and the third visual sensor are arranged on the same side of the shell.

[0018] According to one aspect of an embodiment of the present application, a method for collecting data of a workpiece is provided, wherein the device as described in any one of the above items is used to collect data of the workpiece, and the method includes:

[0019] Determine the target acquisition mode;

[0020] Controlling the visual sensor and projection corresponding to the target acquisition mode Device Data is collected on the workpiece to obtain a workpiece point cloud.

[0021] According to one aspect of an embodiment of the present application, determining a target acquisition mode further includes:

[0022] A target acquisition mode is determined among a plurality of acquisition modes according to the recognized workpiece type.

[0023] According to one aspect of an embodiment of the present application, the acquisition mode includes a first acquisition mode, the first acquisition mode is a target acquisition mode, and controlling a visual sensor and a projection device corresponding to the target acquisition mode to collect data from the workpiece includes:

[0024] Controlling the first projection device to project Gray code fringes onto the surface of the workpiece as a measurement pattern;

[0025] Controlling the first visual sensor and the second visual sensor to photograph the workpiece surface covered by the measurement pattern to obtain a first measurement photo and a second measurement photo;

[0026] According to the deformation of the Gray code stripes in the first measurement photo and the second measurement photo, coordinate information of the area covered by the Gray code stripes on the workpiece surface is calculated to obtain a workpiece point cloud.

[0027] According to one aspect of an embodiment of the present application, the acquisition mode includes a second acquisition mode, the second acquisition mode is a target acquisition mode, and controlling a visual sensor and a projection device corresponding to the target acquisition mode to collect data from the workpiece includes:

[0028] Controlling the second projection device and the third projection device to emit laser light to the surface of the workpiece to form light on the surface of the workpiece as a measurement pattern;

[0029] Controlling the first visual sensor and the second visual sensor to respectively photograph the workpiece surface covered by the measurement pattern to obtain a third measurement photo and a fourth measurement photo, wherein the second projection device, the third projection device, the first visual sensor and the second visual sensor are located in the same coordinate system;

[0030] According to the pixel positions of the light in the third measurement photo and the fourth measurement photo, and the coordinate information of the second projection device, the third projection device, the first visual sensor, and the second visual sensor, a triangulation method is used to calculate the coordinate information of the area covered by the light on the surface of the workpiece, and a left-view point cloud and a right-view point cloud are obtained respectively;

[0031] The left-view point cloud and the right-view point cloud are combined to obtain a line point cloud of the workpiece, wherein the workpiece point cloud includes at least one line point cloud.

[0032] According to one aspect of an embodiment of the present application, the acquisition mode includes a third acquisition mode, which is a target acquisition mode, and controlling a visual sensor and a projection device corresponding to the target acquisition mode to collect data from the workpiece includes:

[0033] Controlling the fourth projection device to emit laser light toward the workpiece to form light on the surface of the workpiece as a measurement pattern;

[0034] Controlling the third visual sensor to photograph the surface of the workpiece showing the light to obtain a fifth measurement photo, wherein the fourth projection device and the third visual sensor are located in the same coordinate system;

[0035] According to the pixel position of the light in the fifth measurement photo, and the coordinate information of the fourth projection device and the third visual sensor, the coordinate information of the area covered by the light on the surface of the workpiece is calculated using a triangulation method to obtain a line point cloud of the workpiece, wherein the workpiece point cloud includes at least one line point cloud.

[0036] According to one aspect of an embodiment of the present application, after acquiring the line point cloud of the workpiece, the method further includes:

[0037] Controlling the light to move on the surface of the workpiece to obtain a plurality of line workpiece point clouds;

[0038] A plurality of the line workpiece point clouds are combined to obtain a workpiece point cloud.

[0039] According to one aspect of the embodiment of the present application, the method further includes:

[0040] The moving speed of the light and / or the acquisition frequency of the line workpiece point cloud are controlled to adjust the data acquisition accuracy and data acquisition speed of the data acquisition device.

[0041] In the present application, by setting a plurality of projection devices and visual sensors on the data acquisition device, different projection devices and visual sensors in a single data acquisition device can be used directly according to the acquisition instruction to perform different methods of data acquisition on the workpiece. And because each projection device and visual sensor in the data acquisition device belongs to the same data acquisition device, the coordinate system can be unified more conveniently, and different methods can be used in sequence to collect data on the workpiece and combined to obtain more accurate workpiece data. While being able to execute multiple data acquisition methods and using multiple data acquisition devices in different ways, the economic cost required for data acquisition on the workpiece is also established.

[0042] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0045] Figure 1 A schematic diagram of a data acquisition device according to an embodiment of the present application is shown.

[0046] Figure 2 A flow chart of a method for collecting data of a workpiece according to an implementation of the present application is shown.

[0047] Figure 3 A flow chart of collecting data on a workpiece using a first component according to an embodiment of the present application is shown.

[0048] Figure 4 A flow chart of using a second component to collect data on a workpiece according to an embodiment of the present application is shown.

[0049] Figure 5 A flow chart of using a third component to collect data on a workpiece according to an embodiment of the present application is shown.

[0050] Figure 6 A flow chart of obtaining a workpiece point cloud according to an embodiment of the present application is shown.

[0051] Figure 7 A flow chart of automatically determining a target acquisition mode according to an embodiment of the present application is shown.

[0052] Figure 8 The hardware structure diagram of the data acquisition method according to one embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present application will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The accompanying drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0054] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present application. However, those skilled in the art will appreciate that the technical solution of the present application may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring the present application and making the various aspects of the present application obscure.

[0055] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0056] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a data acquisition device according to an embodiment of the present application. The data acquisition device may include a housing, wherein the material and shape of the housing are not limited in any way. The shape of the housing may be as follows: Figure 1 The rectangular block shown.

[0057] The data acquisition device is also provided with a plurality of projection devices to project measurement patterns onto the workpiece to be measured. The projection devices can project different measurement patterns onto the workpiece, and different measurement patterns refer to the measurement patterns being different in at least one of shape and size. The measurement pattern can be a complex pattern, such as a grating or Gray code stripe. It can also be a light spot or a light line.

[0058] The data acquisition device is also equipped with multiple visual sensors, which are used to capture the measurement pattern displayed on the workpiece to obtain a measurement picture. It should be further explained that filters of the same wavelength are attached to both sides of the lens of the visual sensor to avoid excessive light noise caused by reflection of the workpiece, which makes it impossible to obtain clear measurement photos.

[0059] The data acquisition device is also provided with a control device, which is used to select projection devices and visual sensors located at different positions to collect data from the workpiece according to the acquisition instructions. Different acquisition instructions instruct different projection devices and visual sensors to collect data from the workpiece. The control device calculates the coordinate information of the area covered by the measurement pattern on the workpiece according to the deformation condition of the measurement pattern in the measurement photo obtained by shooting, so as to realize the data acquisition of the workpiece. According to different workpiece conditions, visual sensors and projection devices with different parameters can be used to collect data from the workpiece.

[0060] In one embodiment of the present application, the data acquisition device includes a first component, and the first component includes a first visual sensor, a second visual sensor, and a first projection device. The first projection device is used to project a measurement pattern onto a workpiece. The first visual sensor, the second visual sensor, and the first projection device are arranged on the same side of the housing, and the first visual sensor and the second visual sensor are respectively arranged on both sides of the first projection device, for photographing the measurement pattern projected on the workpiece at different angles. It should be further explained that the measurement pattern includes but is not limited to a grating and a Gray code stripe.

[0061] In some embodiments, the same side may refer to the same side surface.

[0062] In an embodiment of the present application, different fringe patterns can be projected onto the workpiece through the first projection device, and then the planar array structured light can be used to achieve accurate measurement of the workpiece under complex working conditions. In one embodiment of the present application, the working distance of the first component is 500-1000mm, and the working distance refers to the distance between the plane where the projection device is located and the position to be projected on the workpiece. Secondly, based on the same band filters attached to both sides of the lens of the visual sensor to reduce the influence of environmental noise, the acquisition accuracy is improved, and the reflective workpiece has an anti-glare effect. Therefore, the first component can collect data for complex and bright workpiece surfaces.

[0063] In one embodiment of the present application, the data acquisition device includes a second component, which includes a first visual sensor, a second visual sensor, a second projection device, and a third projection device. The second projection device and the third projection device are used to emit laser light to the workpiece to form a light spot and / or light on the surface of the workpiece as a measurement pattern;

[0064] The first visual sensor, the second visual sensor, the second projection device, and the third projection device are arranged on the same side of the data acquisition device. The first visual sensor is arranged on the left side of the second projection device, and is mainly used to photograph the measurement pattern projected by the second projection device on the workpiece to obtain a measurement photo. The second visual sensor is arranged on the right side of the third projection device, and is mainly used to photograph the measurement pattern projected by the third projection device on the workpiece to obtain a measurement photo. By photographing the measurement pattern on the workpiece at different angles, and then completing the measurement of the measurement area of ​​the measurement pattern on the workpiece surface based on the obtained measurement photos, multi-angle measurement of the workpiece surface is achieved, and more accurate workpiece data can be obtained.

[0065] By obtaining accurate workpiece data, the weld on the workpiece can be more accurately located, or a workpiece point cloud can be generated. Various processing operations on the workpiece can be done with half the effort with accurate workpiece point clouds, greatly improving the user's processing efficiency for the workpiece.

[0066] In the embodiment of the present application, the second projection device and the third projection device can project measurement patterns onto the workpiece from different angles, so better data collection can be performed on more obvious raised portions or recessed portions of the workpiece, such as groove welds.

[0067] In one embodiment of the present application, the working distance of the second component is 500-1000mm, and the line length direction of the laser can be positioned to 200@500mm and 500@1000mm, with higher precision and a smaller data collection range.

[0068] In one embodiment of the present application, the data acquisition device includes a third component, and the third component includes a fourth projection device and a third visual sensor. The fourth projection device is used to emit laser light to the workpiece to form a light spot and / or light on the surface of the workpiece as a measurement pattern, and the third visual sensor is used to photograph the measurement pattern displayed on the workpiece to obtain a measurement photo, and the fourth projection device and the third visual sensor are arranged on the same side of the data acquisition device.

[0069] The line length direction of the laser projected by the fourth projection device can be positioned to 1200@900mm and 1500@1500mm, and data collection can be completed quickly on a workpiece surface with a larger area.

[0070] In another embodiment of the present application, there are corresponding data collection methods for the first component, the second component, and the third component to collect data on the workpiece. It should be clear that the data collection methods corresponding to the first component, the second component, and the third component can all use the structured light measurement method to collect data on the workpiece to obtain the workpiece point cloud of the workpiece.

[0071] See also Figure 2 , Figure 2 A flowchart of a data collection method for a workpiece according to an implementation of the present application is shown. The present application embodiment provides steps of the data collection method, including:

[0072] Step S210, determining a target acquisition mode from among multiple acquisition modes according to an acquisition instruction;

[0073] Step S220, controlling the visual sensor and the projection device corresponding to the target acquisition mode to collect data on the workpiece to obtain a workpiece point cloud.

[0074] The above two steps are described in detail below.

[0075] In step S210, according to the acquisition instruction, a target acquisition mode is determined from multiple acquisition modes. First of all, it should be clear that the acquisition instruction can be preset by the user. It can also be generated based on the user's operation behavior. For example, the user generates different acquisition instructions by pressing a button set on the data acquisition device.

[0076] It should be further clarified that, in some embodiments, the user's operation behavior may refer to the manipulation of a terminal device connected to the data acquisition device, such as a mobile phone, a watch, a computer, and other devices with wired or wireless communication.

[0077] In some embodiments, the user's operation behavior may be to make a voice. The data acquisition device receives the voice information made by the user, analyzes it, and determines the acquisition mode that the user wants to trigger based on the voice information.

[0078] In some embodiments, the user's operation behavior may be a body organ movement, including but not limited to gestures, facial expressions, facial movements, eye movements, font movements, and other dynamic changes of any part of the human body. The data acquisition device analyzes the user's body organ movement and determines the acquisition mode that the user wants to trigger.

[0079] It should be made clear that the user operation behavior can be directly acquired through relevant sensors set in the data acquisition device, or it can be indirectly acquired through wireless or wired transmission to obtain relevant audio or video information.

[0080] Each acquisition instruction corresponds to an acquisition mode. When an acquisition instruction is received, the acquisition mode corresponding to the acquisition instruction is the target acquisition mode. Each acquisition mode corresponds to using a component to execute a corresponding data acquisition method to collect data from the workpiece. The components at least include a first component, a second component, and a third component. Each component has a corresponding visual sensor and a projection device to execute a corresponding data acquisition method. The visual sensor and projection device corresponding to the acquisition instruction are the components used by the acquisition instruction.

[0081] In another embodiment of the present application, a target acquisition mode is determined from multiple acquisition modes according to the identified workpiece type. Each workpiece type has a corresponding acquisition mode as its target acquisition mode.

[0082] In some embodiments, the type of the workpiece can be identified by a visual sensor provided in the data acquisition device. In some embodiments, the workpiece can also be identified by other sensors provided, such as an identification piece corresponding to the workpiece type provided on the workpiece, and the data acquisition device determines the workpiece type by sensing the workpiece information recorded in the identification piece. In some embodiments, the workpiece type can also be obtained through information input by the user. For example, if the user types the workpiece type, the data acquisition device obtains the workpiece type input by the user.

[0083] In some embodiments, the workpiece types include: large workpieces, complex workpieces (with right-angle welds, circular welds), and workpieces with groove welds. The large workpiece corresponds to the third component, the workpiece with groove welds corresponds to the second component, and the complex workpiece corresponds to the first component.

[0084] In step S220, according to the target acquisition instruction, the component corresponding to the target acquisition instruction is determined. According to the data acquisition method corresponding to the component, the projection device and the visual sensor included in the component are controlled to collect data on the workpiece. It should be clear that data acquisition on the workpiece refers to collecting coordinate information on the surface of the workpiece to determine the working condition of the workpiece surface, or to locate the welding position and weld on the surface of the workpiece.

[0085] As for the explanation of the data acquisition principle, since each component is on the same side of the acquisition device, the measurement coordinate system is constructed with the side on which it is located as the reference plane. If the workpiece is a flat plane, the size or position of the measurement pattern in the measurement photo will change accordingly according to the distance of the workpiece relative to the reference plane. The distance of the workpiece surface relative to the reference plane is calculated based on this change, and finally the coordinate information of the workpiece surface in the measurement coordinate system is obtained. If the workpiece is an uneven plane, the measurement pattern will appear in different sizes in areas at different heights in the measurement photo, causing the measurement pattern to deform as a whole. According to the degree of deformation of the measurement pattern in different parts of the workpiece surface, the distance of each part of the measurement pattern in the surface coverage area of ​​the workpiece relative to the reference plane is determined, and then the coordinate information of the workpiece surface is obtained.

[0086] In some embodiments, each component may use a corresponding structured light measurement method to obtain coordinate information of a coverage area of ​​a measurement pattern on the surface of a workpiece, thereby completing data collection on the workpiece.

[0087] See also Figure 3 , Figure 3 A flow chart of using a first component to collect data from a workpiece according to an embodiment of the present application is shown. The first acquisition mode is a target acquisition mode, and the component corresponding to the first acquisition mode is the first component. The embodiment of the present application provides steps for using the first component to collect data from a workpiece, including:

[0088] Step S221a, controlling the first projection device to project Gray code fringes onto the workpiece surface as a measurement pattern;

[0089] Step S222a, controlling the first visual sensor and the second visual sensor to photograph the workpiece surface covered by the measurement pattern to obtain a first measurement photo and a second measurement photo;

[0090] Step S223a, calculating the coordinate information of the area covered by the Gray code stripes on the workpiece surface according to the deformation of the Gray code stripes in the first measurement photo and the second measurement photo, and obtaining the workpiece point cloud.

[0091] The above three steps are described in detail below.

[0092] In step S221a, the first projection device is controlled to project Gray code stripes onto the workpiece surface as a measurement pattern, and the measurement pattern is displayed on the workpiece surface. It should be noted that the measurement pattern can be completely located on the workpiece surface or partially located on the workpiece surface.

[0093] In another embodiment of the present application, the first projection device can also project other stripes onto the surface of the workpiece.

[0094] In step S222a, the first visual sensor and the second visual sensor are controlled to photograph the workpiece surface covered by the measurement pattern to obtain a first measurement photo and a second measurement photo.

[0095] In step S223a, the coordinate information of the area covered by the measurement pattern on the workpiece is determined according to the deformation of the measurement pattern in the first measurement photo and the second measurement photo.

[0096] In one embodiment of the present application, if the first component cannot complete data collection on the workpiece surface at one time, the measurement of different positions on the workpiece surface can be achieved by changing the position of the measurement pattern or moving the data acquisition device to obtain multiple regional point clouds of the workpiece, and finally the multiple regional point clouds are combined to obtain the workpiece point cloud.

[0097] In one embodiment of the present application, the first projection device is a DLP optical machine, and the precise phase shift of Gray code stripes or other stripes can be realized by programming without moving the first projection device to realize data collection of the workpiece. In this way, the data of the workpiece can be collected by combining Gray code and phase shift.

[0098] As a flexible encoding method, the technology combining Gray code and phase shift method is one of the most practical and advanced methods for structured light 3D reconstruction. The traditional Gray code combined with phase shift method is not always clear in the black and white transition boundaries due to the interference caused by the uneven reflectivity of the scanned object, background intensity and defocus, which may introduce incorrect decoding phase order and cause phase errors. In addition, during actual scanning, there is also the possibility of local decoding errors for the simulated code obtained by the phase shift method.

[0099] Therefore, in the embodiment of the present application, according to the mathematical characteristics of Gray code and phase shift method, positive and negative Gray code grayscale stripe patterns are designed, and the area with the same coding is found as an initial coding cycle. The four-step phase shift method or the three-step phase shift method is used to achieve the surface information coverage of the coding pattern and improve the coding accuracy, and to a certain extent eliminate some decoding errors caused by harmonic signals. By decoding the coding pattern, the coordinate information of the area covered by the Gray code stripes on the workpiece is obtained.

[0100] See also Figure 4 , Figure 4 A flow chart of using a second component to collect data from a workpiece according to an embodiment of the present application is shown. The second acquisition mode is a target acquisition mode, and the component corresponding to the second acquisition mode is a second component. The embodiment of the present application provides steps for using the second component to collect data from a workpiece, including:

[0101] Step S221b, controlling the second projection device and the third projection device to emit laser light to the surface of the workpiece, forming light on the surface of the workpiece as a measurement pattern;

[0102] Step S222b, controlling the first visual sensor and the second visual sensor to respectively photograph the workpiece surface covered by the measurement pattern to obtain a third measurement photo and a fourth measurement photo, wherein the second projection device, the third projection device, the first visual sensor and the second visual sensor are located in the same coordinate system;

[0103] Step S223b, according to the pixel positions of the light in the third measurement photo and the fourth measurement photo, and the coordinate information of the second projection device, the third projection device, the first visual sensor, and the second visual sensor, a triangulation method is used to calculate the coordinate information of the light coverage area on the workpiece surface, and a left view point cloud and a right view point cloud are obtained respectively;

[0104] Step S224b, combining the left view point cloud and the right view point cloud to obtain a line point cloud of the workpiece, wherein the workpiece point cloud includes at least one line point cloud.

[0105] The above four steps are described in detail below.

[0106] In step S221b, the second projection device and the third projection device are controlled to emit laser light to the surface of the workpiece to form light on the surface of the workpiece as a measurement pattern.

[0107] In step S222b, the first visual sensor is controlled to shoot the light projected by the second projection device on the workpiece on the left side of the second projection device to obtain a third measurement photo. The second visual sensor is controlled to shoot the light projected by the third projection device on the workpiece on the right side of the third projection device to obtain a fourth measurement photo. That is, for the same area on the surface of the workpiece, the measurement pattern is shot using two different viewing angles to obtain the measurement photo. Among them, the second projection device, the third projection device, the first visual sensor and the second visual sensor are located in the same coordinate system. That is, for the same point on the surface of the workpiece, the coordinate information determined by different measurement photos is the same.

[0108] In some embodiments, the first visual sensor can simultaneously capture all or part of the light formed by the second projection device and the third projection device on the workpiece surface to obtain a third measurement photo. The second visual sensor can also simultaneously capture all or part of the light formed by the second projection device and the third projection device on the workpiece surface to obtain a fourth measurement photo.

[0109] In step S223b, according to the pixel position of the light in the third measurement photo, that is, the deformation of the light in the third measurement photo is obtained, and the coordinate information of the area covered by the light on the workpiece surface is calculated according to the deformation of the light to form a left-view point cloud. According to the pixel position of the light in the fourth measurement photo, the coordinate information of the area covered by the light on the workpiece surface is calculated to form a right-view point cloud.

[0110] In some embodiments, the coordinate information of the area covered by the light on the surface of the workpiece can be obtained by using triangulation method on the third measurement photo and the fourth measurement photo respectively.

[0111] In step S224b, since the left-view point cloud and the right-view point cloud are based on the same coordinate system, the left-view point cloud and the right-view point cloud can be directly combined to obtain a line point cloud, that is, the coordinate information of the area covered by the light projected by the second projection device and the third projection device on the workpiece surface.

[0112] In some embodiments, if the first visual sensor can simultaneously capture all or part of the light formed by the second projection device and the third projection device on the surface of the workpiece, a third measurement photo is obtained. The second visual sensor can also simultaneously capture all or part of the light formed by the second projection device and the third projection device on the surface of the workpiece to obtain a fourth measurement photo. When acquiring the line point cloud, the data points with the same coordinate information in the left-view point cloud and the right-view point cloud can be combined to further ensure the accuracy of the combination of the left-view point cloud and the right-view point cloud.

[0113] In one embodiment of the present application, by moving the light projected by the second projection device and the third projection device on the workpiece surface, rapid measurement of the workpiece surface is achieved to obtain multiple line point clouds.

[0114] In one embodiment of the present application, the light projected by the second projection device and the third projection device is moved on the workpiece surface until the light traverses the surface to be measured on the workpiece at a set interval, thereby obtaining a plurality of third measurement photos and fourth measurement photos. Then, a plurality of left view point clouds and right view point clouds are calculated based on the third measurement photos and the fourth measurement photos, and then all the left view point clouds and the right view point clouds are directly combined to obtain the workpiece point cloud.

[0115] In this embodiment, the data points having the same coordinate information in the left view point cloud and the right view point cloud may also be combined.

[0116] In one embodiment of the present application, there is no restriction on the order in which the second projection device and the third projection device project light onto the workpiece surface. For example, the data acquisition device can sequentially control the second projection device and the third projection device to alternately project light onto the workpiece surface, and sequentially and alternately obtain the third measurement photo and the fourth measurement photo. It is also possible to simultaneously project light onto the workpiece surface and obtain the third measurement photo and the fourth measurement photo.

[0117] See also Figure 5 , Figure 5 A flow chart of using a third component to collect data from a workpiece according to an embodiment of the present application is shown. The third acquisition mode is a target acquisition mode, and the component corresponding to the third acquisition mode is a third component. The embodiment of the present application provides steps for using the third component to collect data from a workpiece, including:

[0118] Step S221c, controlling the fourth projection device to emit laser light toward the workpiece, forming light on the surface of the workpiece as a measurement pattern;

[0119] Step S222c, controlling the third visual sensor to photograph the workpiece surface showing light, to obtain a fifth measurement photo, and the fourth projection device and the third visual sensor are located in the same coordinate system;

[0120] Step S223c, according to the pixel position of the light in the fifth measurement photo, and the coordinate information of the fourth projection device and the third visual sensor, use the triangulation method to calculate the coordinate information of the light coverage area on the workpiece surface to obtain the line point cloud of the workpiece, and the workpiece point cloud includes at least one line point cloud.

[0121] The above three steps are described in detail below.

[0122] In step S221c, the fourth projection device is controlled to emit laser light to the workpiece, forming light on the surface of the workpiece as a measurement pattern.

[0123] In step S222c, the third visual sensor is controlled to photograph the surface of the workpiece displaying the light to obtain a fifth measurement photo, and the fourth projection device and the third visual sensor are located in the same coordinate system.

[0124] In step S223c, according to the pixel position of the light in the fifth measurement photo, and the coordinate information of the fourth projection device and the third visual sensor, the coordinate information of each point in the light coverage area on the workpiece surface is calculated using a triangulation method to obtain a line point cloud of the workpiece. In another embodiment of the present application, according to the pixel position of the light in the fifth measurement photo, that is, the deformation of the light in the fifth measurement photo is obtained, and the coordinate information of the light in the coverage area on the workpiece surface is calculated according to the deformation of the light.

[0125] See also Figure 6 , Figure 6 A flowchart of obtaining a workpiece point cloud according to an embodiment of the present application is shown. The present application embodiment provides steps for obtaining a workpiece point cloud, including:

[0126] Step S301, controlling the light to move on the workpiece surface to obtain a plurality of line workpiece point clouds;

[0127] Step S302, combining multiple line workpiece point clouds to obtain a workpiece point cloud.

[0128] The above two steps are described in detail below.

[0129] In step S301, each time the second component and the third component are measured, only line point clouds are obtained. When the surface of the workpiece is large, line point clouds can be obtained multiple times by moving the light to obtain multiple line point clouds.

[0130] In some embodiments, the light can be moved based on the user's operation behavior.

[0131] In some embodiments, the light can be moved at set intervals based on the user's operating behavior.

[0132] In some embodiments, the light may be moved randomly or at intervals of a set distance based on pre-set parameters in the second acquisition mode or the third acquisition mode.

[0133] In step S302, since the line point clouds obtained each time are in the same coordinate system, the line point clouds can be directly fused to obtain the workpiece point cloud.

[0134] In some embodiments, at least two of the first acquisition mode, the second acquisition mode, and the third acquisition mode can be arbitrarily selected for the workpiece, and data acquisition is performed on the workpiece to obtain multiple workpiece point clouds. Then the multiple workpiece point clouds are combined to eliminate the error factors that affect the accuracy of the workpiece point clouds brought by various acquisition modes. On the other hand, since the first component, the second component, and the third component are all arranged on the same side of the data acquisition device, located in the same coordinate system, and the side where the first component, the second component, and the third component are located is used as a reference plane, the obtained workpiece point clouds are all located in the same coordinate system, so the multiple workpiece point clouds can be directly combined to avoid the errors caused by the coordinate system conversion. In this way, the workpiece surfaces of various working conditions can be dealt with, and the data acquisition of the workpiece can be accurately completed. In specific applications, various types of welds on the workpiece surface can be accurately identified.

[0135] In one embodiment of the present application, when acquiring a line workpiece point cloud, the moving speed of the light and / or the acquisition frequency of the line workpiece point cloud can be controlled to adjust the data acquisition accuracy and data acquisition speed of the data acquisition device. Wherein, when the acquisition frequency of the line workpiece point cloud is constant, the smaller the moving speed of the light, the higher the data acquisition accuracy of the data acquisition device, and the slower the data acquisition speed. When the moving speed of the light is constant, the smaller the acquisition frequency of the line workpiece point cloud, the lower the data acquisition accuracy of the data acquisition device.

[0136] In some embodiments, the moving speed of the light and the acquisition frequency of the line workpiece point cloud are determined according to the target acquisition accuracy and target acquisition speed preset in the data acquisition device. Specifically, the moving speed of the light is determined according to the target acquisition speed. The target acquisition speed of the line workpiece point cloud is determined according to the target acquisition accuracy.

[0137] In some embodiments, by controlling the trigger frequency of the encoder and the interval between the light rays, the data collection accuracy and light movement speed can be improved, large workpieces can be better positioned, and the problem of difficulty in robot positioning of large workpieces can be solved.

[0138] See also Figure 7 , Figure 7 A flow chart of automatically determining a target acquisition mode according to an embodiment of the present application is shown. The present application embodiment provides steps of automatically determining a target acquisition mode, including:

[0139] Step S401, determining the target exposure of the visual sensor when photographing the area to be measured of the workpiece according to the brightness of the environment in which the workpiece is located, wherein the brightness of the environment is negatively correlated with the target exposure;

[0140] Step S402, driving any visual sensor to photograph the workpiece surface at a target exposure to obtain a workpiece photo;

[0141] Step S403, determining an acquisition instruction according to the size of the area to be measured displayed in the workpiece photo and the pixel value of each pixel point in the workpiece photo.

[0142] The above three steps are described in detail below.

[0143] In step S401, the target exposure of the visual sensor when photographing the area to be measured of the workpiece is determined according to the brightness of the environment in which the workpiece is located, and the brightness of the environment is negatively correlated with the target exposure. In this way, the workpiece with high reflectivity is prevented from affecting the shooting.

[0144] In step S402, any one visual sensor is driven to photograph the workpiece surface at the target exposure to obtain a workpiece photo.

[0145] In step S403, the size of the workpiece is determined in the workpiece photo. The larger the workpiece, the larger the surface area of ​​the workpiece, that is, the larger the area to be measured. The pixel value of each pixel point in the workpiece photo is determined. If the pixel value distribution is uniform, it means that the surface of the workpiece is relatively flat.

[0146] If the pixel value distribution is more disorderly, it means that the surface of the workpiece is more complex. Exemplarily, the area representing the measurement area in the workpiece photo is divided into a set number of measurement areas, and the pixel mean of each measurement area is calculated. If the fluctuation range between the pixel means of each measurement area is less than or equal to the set fluctuation threshold, it means that the surface of the workpiece is relatively flat. If the fluctuation range between the pixel means of each measurement area is greater than the set fluctuation threshold, it means that the surface of the workpiece is more complex. The fluctuation range between the pixel means of each measurement area can be calculated by calculating the difference between the maximum and minimum values ​​of the pixel means of each measurement area.

[0147] If the average pixel value in the workpiece photo is larger or smaller, it means that the workpiece surface is highly reflective. For example, if the average pixel value is greater than the first set threshold or less than the second set threshold, it means that the workpiece surface is highly reflective, otherwise it means that the workpiece surface is less reflective.

[0148] Different components are used for data collection in the above three situations. Since the first component is characterized by more accurate data collection when facing reflective workpieces compared to the second and third components, when the mean pixel value in the workpiece photo is greater than the first set threshold or less than the second set threshold, the first collection mode is used as the target collection mode.

[0149] Since the second component and the third component can both collect data on the surface of the workpiece with complex working conditions relative to the first component, if the fluctuation range between the mean values ​​of the pixel points in each metering area in the workpiece photo is greater than the set fluctuation threshold, and the mean value of the pixel value in the workpiece photo is less than or equal to the first set threshold and greater than or equal to the second set threshold, then any one of the first acquisition mode and the second acquisition mode is used as the target acquisition mode.

[0150] In the embodiment of the present application, in view of this situation, the first acquisition mode and the second acquisition mode may be prioritized to determine which acquisition mode is used as the target acquisition mode. The user may also be prompted to instruct which acquisition mode is used as the target acquisition mode.

[0151] In view of the above situation, in some embodiments, the second acquisition mode is used as the target acquisition mode, because the second acquisition mode can complete data acquisition of the workpiece with less data processing volume than the first acquisition mode, which can save the computing resources of the data acquisition device and increase the acquisition efficiency of the data acquisition device.

[0152] Since the third component can measure with a larger line length than the first component and the second component, that is, it can quickly measure a larger area of ​​the workpiece surface, when the area to be measured is larger than the set area threshold, the mean pixel value in the workpiece photo is less than or equal to the first set threshold, greater than or equal to the second set threshold, and the fluctuation range of the mean pixel point value in each measurement area is less than or equal to the set fluctuation threshold, the third acquisition mode is used as the target acquisition mode.

[0153] The data collection method of the embodiment of the present application can be Figure 8 The hardware device shown is implemented. Figure 8 To describe the hardware device according to the embodiment of the present application. Figure 8 The hardware device shown is only an example and should not bring any limitation to the functions and scope of application of the embodiments of the present application.

[0154] like Figure 8 As shown, the hardware device may be in the form of a general computing device. The components of the hardware device may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0155] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the description of the exemplary method described above in this specification. For example, the processing unit 810 can perform the following steps: Figure 3 The steps shown in .

[0156] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0157] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0158] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0159] The hardware device may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the hardware device, and / or may communicate with any device that enables the hardware device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 850. Furthermore, the hardware device may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 860. As shown, the network adapter 860 communicates with other modules of the hydrogen refueling device through a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the hardware device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0160] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0161] In an exemplary embodiment of the present application, a computer program medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer is enabled to execute the method described in the above method embodiment.

[0162] According to one embodiment of the present application, a program product for implementing the method in the above method embodiment is also provided, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0163] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0164] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0165] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0166] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0167] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0168] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0169] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0170] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions applied for herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that the present application does not apply for. The specification and embodiments are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

Claims

1. A workpiece data acquisition device, characterized in that: The device comprises: a housing forming an outer shell of the device; A plurality of projection devices, disposed in the housing, for projecting a measurement pattern onto the workpiece; A plurality of visual sensors, disposed in the housing, for photographing a measurement pattern displayed on the workpiece to obtain a measurement photograph; The control device is arranged in the shell and is used to select projection devices and visual sensors located at different positions to collect data on the workpiece according to the collection instructions. Different collection instructions instruct different projection devices and visual sensors to collect data on the workpiece to obtain the workpiece point cloud.

2. The device according to claim 1, characterized in that: The data acquisition device comprises: a first visual sensor, a second visual sensor, and a first projection device; Wherein, the first projection device is used to project a measurement pattern onto the workpiece; The first visual sensor, the second visual sensor, and the first projection device are arranged on the same side of the housing, and the first visual sensor and the second visual sensor are arranged on both sides of the first projection device respectively.

3. The device according to claim 1, characterized in that The data acquisition device also includes: a first visual sensor, a second visual sensor, a second projection device and a third projection device; Wherein, the second projection device and the third projection device are used to emit laser to the workpiece to form a light spot and / or light as the measurement pattern on the surface of the workpiece; The first visual sensor, the second visual sensor, the second projection device, and the third projection device are arranged on the same side of the shell, the first visual sensor is arranged on the left side of the second projection device, and the second visual sensor is arranged on the right side of the third projection device.

4. The device according to claim 1, characterized in that The data acquisition device also includes a fourth projection device and a third visual sensor; Among them, the fourth projection device is used to emit laser to the workpiece to form a light spot and / or light on the surface of the workpiece as the measurement pattern, the third visual sensor is used to photograph the measurement pattern displayed on the workpiece, and the fourth projection device and the third visual sensor are arranged on the same side of the shell.

5. A method for collecting data of a workpiece, using the device as claimed in any one of claims 1 to 4 to collect data of the workpiece, characterized in that: The method comprises: Determine the target acquisition mode; The visual sensor and the projection device corresponding to the target acquisition mode are controlled to collect data from the workpiece to obtain a workpiece point cloud.

6. The method according to claim 5, characterized in that Determine the target acquisition mode, including: A target acquisition mode is determined among a plurality of acquisition modes according to the recognized workpiece type.

7. The method according to claim 5, characterized in that The acquisition mode includes a first acquisition mode, which is a target acquisition mode, and controls a visual sensor and a projection device corresponding to the target acquisition mode to acquire data from the workpiece, including: Controlling the first projection device to project Gray code fringes onto the surface of the workpiece as a measurement pattern; Controlling the first visual sensor and the second visual sensor to photograph the workpiece surface covered by the measurement pattern to obtain a first measurement photo and a second measurement photo; According to the deformation of the Gray code stripes in the first measurement photo and the second measurement photo, coordinate information of the area covered by the Gray code stripes on the workpiece surface is calculated to obtain a workpiece point cloud.

8. The method according to claim 5, characterized in that The acquisition mode includes a second acquisition mode, which is a target acquisition mode, and controls a visual sensor and a projection device corresponding to the target acquisition mode to acquire data from the workpiece, including: Controlling the second projection device and the third projection device to emit laser light to the surface of the workpiece to form light on the surface of the workpiece as a measurement pattern; Controlling the first visual sensor and the second visual sensor to respectively photograph the workpiece surface covered by the measurement pattern to obtain a third measurement photo and a fourth measurement photo, wherein the second projection device, the third projection device, the first visual sensor and the second visual sensor are located in the same coordinate system; According to the pixel positions of the light in the third measurement photo and the fourth measurement photo, and the coordinate information of the second projection device, the third projection device, the first visual sensor, and the second visual sensor, a triangulation method is used to calculate the coordinate information of the area covered by the light on the surface of the workpiece, and a left-view point cloud and a right-view point cloud are obtained respectively; The left-view point cloud and the right-view point cloud are combined to obtain a line point cloud of the workpiece, wherein the workpiece point cloud includes at least one line point cloud.

9. The method according to claim 5, characterized in that The acquisition mode includes a third acquisition mode, which is a target acquisition mode, and controls the visual sensor and the projection device corresponding to the target acquisition mode to collect data from the workpiece, including: Controlling the fourth projection device to emit laser light toward the workpiece to form light on the surface of the workpiece as a measurement pattern; Controlling the third visual sensor to photograph the surface of the workpiece showing the light to obtain a fifth measurement photo, wherein the fourth projection device and the third visual sensor are located in the same coordinate system; According to the pixel position of the light in the fifth measurement photo, and the coordinate information of the fourth projection device and the third visual sensor, the coordinate information of the area covered by the light on the surface of the workpiece is calculated using a triangulation method to obtain a line point cloud of the workpiece, wherein the workpiece point cloud includes at least one line point cloud.

10. The method according to claim 7 or 8, characterized in that: After obtaining the line point cloud of the workpiece, the method further includes: Controlling the light to move on the surface of the workpiece to obtain a plurality of line workpiece point clouds; A plurality of the line workpiece point clouds are combined to obtain a workpiece point cloud.

11. The method according to claim 10, characterized in that The method further comprises: The moving speed of the light and / or the acquisition frequency of the line workpiece point cloud are controlled to adjust the data acquisition accuracy and data acquisition speed of the data acquisition device.

Images