Automatic winding method and system based on 3D vision and storage medium

Abstract

The application relates to the field of automatic control vision technology and provides an automatic winding solution based on 3D vision, including a winding detection method, an automatic winding control method, a winding detection device, an automatic winding system and a computer program storage medium. The automatic winding method, the system and the storage medium based on 3D vision can realize accurate detection of the current winding state by using a 3D image sensor, for example, the wire entry angle, the wire spacing, the height difference and other parameter information of the winding; and the transverse feeding speed of the feeding wire and the rotating speed of the winding drum in the winding process can be controlled according to the current winding state, so as to form a closed-loop control system, effectively increase the winding precision, improve the winding quality and the winding efficiency, reduce the winding error rate and the labor cost; and the 3D image sensor can detect the edge information of the winding, so as to replace the edge sensor to accurately control the winding direction.

Description

Technical Field

[0001] This application relates to the field of automated control vision technology, and proposes an automatic winding solution based on 3D vision, including a winding detection method, an automatic winding control method, a winding detection device, an automatic winding system, and its computer program storage medium. Background Technology

[0002] A winding machine is a device that winds a thread-like object onto a specific workpiece. Currently, the domestic cable industry still suffers from uneven winding in the automated winding of large cables. Many factors, such as the precision of the machine, winding speed, and thread size, affect the winding quality. Traditional winding solutions are mostly based on manual control, which significantly reduces winding speed and increases labor costs. Some automated winding solutions only roughly control the winding by setting the rotation speed of the winding drum and the winding and unwinding speed, and by using edge sensors to control the winding direction. However, they hardly intervene in the control of the winding process itself. This makes such automated winding solutions prone to miswinding, reducing the winding yield and thus increasing production costs. Summary of the Invention

[0003] To overcome at least one problem or deficiency in the prior art, this application proposes an automatic winding solution based on 3D vision.

[0004] I. Wire Wrapping Detection Method Based on 3D Vision

[0005] One objective of this application is to provide a 3D vision-based wire winding detection method, which is applied to a wire winding detection device and mainly includes the following steps: acquiring 2D and 3D images collected by an image sensor during the winding process; extracting the 3D coordinates of each winding line from the highest point of the image sensor based on the 2D and 3D images; dividing the 3D coordinates of each winding line from the highest point of the image sensor into upper and lower layers using a height clustering method, and fitting the upper winding line and / or the lower winding line; identifying the feed line and determining the coordinates of the current point; calculating the feed line angle θ between the feed line and the upper or lower winding line; and calculating the feed line spacing Δl based on the horizontal distance between the current point and the highest point of the upper or lower winding line. This application can accurately detect the current winding status through a 3D image sensor, acquire 2D and 3D images of the winding process, and automatically identify the current winding, namely the upper winding, lower winding and feed line, using 3D vision technology. It can extract parameter information such as the feed angle θ and the line spacing Δl, so that the winding system can perform winding control and / or winding quality analysis, and provide data support for it.

[0006] Based on the above winding detection method, the following steps may be further included: calculating the height difference Δh of the feed line based on the vertical distance between the current point and the lower winding line.

[0007] Based on the above winding detection method, the following steps may be further included: calculating the position information of the inner walls on both sides of the winding drum; and calculating the edge distance L of the feed line based on the horizontal distance between the current point and the inner wall of the winding drum in the feed direction.

[0008] Based on the above winding detection method, the following steps may be further included: when fitting the upper winding straight line and the lower winding straight line, noise information is filtered by using a parallel line fitting method.

[0009] Based on the above winding detection method, the following steps may be further included: the step of extracting the coordinates of the distance from the highest point of the image sensor on each winding includes the following steps: performing binarization processing on the 2D image, extracting the 2D centroid coordinates of each brightest point in the axial direction of the winding drum in the target winding area; and filtering out the 3D coordinates of the brightest point of each winding according to the matching relationship between the 2D image and the 3D image, based on the height and spacing range of each brightest point.

[0010] II. Winding Inspection Equipment

[0011] One of the objectives of this application is to provide a winding detection device, which mainly includes an image sensor, a processor, and a memory; wherein, the image sensor is used to acquire 2D and 3D images during the winding process; the memory is used to store a computer program; when the computer program is executed by the processor, the winding detection device implements the operation corresponding to the 3D vision-based winding detection method described in Part 1.

[0012] Based on the above-mentioned winding detection equipment, further: the image sensor is mainly composed of a linear array image sensor, or mainly composed of a linear array image sensor and a 2D image sensor.

[0013] III. Automatic winding control method based on 3D vision

[0014] One objective of this application is to provide an automatic winding control method based on 3D vision. The automatic winding control method is applied to an automatic winding system and mainly includes the following steps: acquiring 2D and 3D images collected by an image sensor during the winding process; extracting the 3D coordinates of each winding loop from the highest point of the image sensor based on the 2D and 3D images; dividing the 3D coordinates of each winding loop from the highest point of the image sensor into upper and lower layers using a height clustering method, and fitting the upper winding straight line and / or the lower winding straight line; identifying the feed line and determining the current point coordinates; and determining the coordinates of the current point based on the relationship between the feed line and the upper winding straight line. The feed line angle θ is calculated using the upper or lower winding line. The line spacing Δl is calculated based on the horizontal distance between the current point and the highest point of either the upper or lower winding line. Based on the feed line angle θ and line spacing Δl, the approximate ratio of the current feed line's lateral feed speed and drum speed is determined. If the feed line angle θ and / or line spacing Δl are detected to be outside the normal range, the lateral feed speed and / or drum speed are adjusted accordingly based on the deviation. This application can accurately detect the current winding state using a 3D image sensor, acquiring 2D and 3D images of the winding process. 3D vision technology is used to automatically identify the current winding (i.e., upper winding, lower winding, and feed line), extracting parameters such as the feed line angle θ and line spacing Δl. This allows the winding system to precisely control the drum speed and the feed line's lateral feed speed, achieving precise control of the winding quality.

[0015] Based on the above-mentioned automatic winding control method, the following steps may be further included: calculating the height difference Δh of the feed line based on the vertical distance between the current point and the lower winding line; determining whether the ratio of the lateral feed speed of the current feed line and the drum speed is appropriate based on the feed line's feed angle θ, line spacing Δl, and height difference Δh; if the feed line's feed angle θ, line spacing Δh, and / or height difference Δh are detected to be outside the normal range, adjusting the lateral feed speed and / or drum speed accordingly based on the deviation value.

[0016] Based on the above-mentioned automatic winding control method, the following steps may be further included: calculating the position information of the inner walls on both sides of the winding drum; calculating the edge distance L of the feed line based on the horizontal distance between the current point and the inner wall of the winding drum in the feed direction; and controlling the turning operation of the feed line based on the edge distance L of the feed line.

[0017] Based on the above-mentioned automatic winding control method, the following steps may be further included: if it is detected that the feed line angle θ, line spacing Δl and / or height difference Δh are not within the normal range during the current or historical winding process, the feed line is controlled to be retrieved to the error point, and the transverse feed speed and / or drum speed are adjusted accordingly based on the deviation value to rewind the wire.

[0018] Based on the above-mentioned automatic winding control method, the following steps may be further included: analyzing the winding quality according to the winding process status information to obtain winding quality evaluation information, wherein the winding process status information includes the feed line angle θ, the line spacing Δl and / or the height difference Δh.

[0019] IV. Automatic winding system

[0020] One objective of this application is to provide an automatic winding system, which mainly includes an image sensor, a control system, a winding mechanism, and a feeding mechanism. The image sensor is used to acquire 2D and 3D images during the winding process. The control system includes a controller for controlling the actions of the winding mechanism and the feeding mechanism, and a memory for storing a computer program. The winding mechanism is used to perform the winding action. The feeding mechanism is used to perform the feeding action. When the computer program is executed by the controller, the automatic winding system implements the operation corresponding to the automatic winding control method described in Part Three.

[0021] V. Computer program storage media

[0022] One objective of this application is to provide a computer-readable storage medium. The storage medium is used to store a computer program, and when a computer reads the computer program from the storage medium, the computer executes the operation corresponding to the 3D vision-based wire winding detection method described in Part One.

[0023] One of the objectives of this application is to provide a computer-readable storage medium. The storage medium is used to store a computer program, and when a computer reads the computer program from the storage medium, the computer performs the operations corresponding to the automatic winding control method described in Part Three.

[0024] In summary, due to the adoption of the above technical solutions, the beneficial effects of this invention include: accurately detecting the current winding state using a 3D image sensor, such as the winding feed angle θ, line spacing Δl, height difference Δh, and other parameter information; and controlling the lateral feed speed of the feed line and the rotation speed of the winding drum during the winding process based on the current winding state, forming a closed-loop control system, effectively increasing winding accuracy, improving winding quality and efficiency, and reducing winding error rate and labor costs. Furthermore, the 3D image sensor can detect the edge information of the winding, replacing the edge sensor to accurately control the winding direction.

[0025] It should be noted that different embodiments of this application may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or more combinations mentioned in this application, or any other beneficial effects that may be obtained that are not exhaustively described. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. For those skilled in the art, without creative effort, the solutions shown in these drawings can be replaced, adjusted, combined, etc., to create different technical solutions; and this application can also be applied to other similar scenarios based on these drawings to obtain application solutions for other scenarios.

[0027] in:

[0028] Figure 1 A schematic diagram illustrating the working principle of winding detection according to some embodiments of this application;

[0029] Figure 2 Experimental photographs shown according to some embodiments of this application;

[0030] Figure 3 Exemplary flowcharts shown according to some embodiments of the winding detection method of this application;

[0031] Figure 4 Exemplary flowcharts shown according to some embodiments of the automatic winding control method of this application;

[0032] Figure 5 Exemplary system block diagrams shown according to some embodiments of the automatic winding system of this application.

[0033] In the picture:

[0034] 101-Drum, 102-Feed line, 103-Upper winding, 104-Lower winding, 105-Upper winding straight line, 106-Lower winding straight line, 107-Current point, 108-Left inner wall of drum, 109-Right inner wall of drum, 110-Image sensor.

[0035] It should be noted that identical reference numerals in the figures represent the same structure or operation. Similar reference numerals and letters in the figures of this application indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Because this application has numerous figures and reference numerals, if there are discrepancies between the descriptions of the figures and the illustrations in the specification, those skilled in the art should understand them based on the logical coherence of the technical principles described in this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and it is not possible to exhaustively describe all embodiments.

[0037] I. Wire Wrapping Detection Method Based on 3D Vision

[0038] like Figure 3 As shown, a 3D vision-based wire winding detection method is applied to a wire winding detection device and mainly includes the following steps:

[0039] S100: Acquire 2D and 3D images captured by image sensor 110 during the winding process.

[0040] For example, a 2D image sensor can be used to acquire 2D images, and a 3D image sensor can be used to acquire 3D images. Alternatively, a 3D image sensor can be used to acquire both 2D and 3D images simultaneously. The 3D image sensor can be a stereo camera, a structured light camera, a Time-of-Flight (TOF) depth camera, or a linear or area scan camera. In some embodiments, a linear laser camera can be used to acquire image information during the winding process, obtaining 2D grayscale images and 3D depth maps. If some linear laser cameras do not support both 2D and 3D modes simultaneously, they can be used in conjunction with a regular 2D camera.

[0041] S200: Based on the 2D and 3D images, extract the 3D coordinates of the distance from the highest point of each winding loop to the image sensor. Since the drum 101 and the winding itself are cylindrical, we can find the coordinates of the highest point of each winding loop to determine the position, height, spacing, and other information of each winding loop. Those skilled in the art can select different stereoscopic vision technologies and implement this operation in different ways according to actual needs. For example, the centroid coordinates of the brightest points in the axial direction of the winding drum in the target winding area can be obtained using a grayscale image. Then, based on the matching relationship between the 2D and 3D images, and according to the height and spacing range of each brightest point, the 3D coordinates of the brightest point of each winding loop can be selected. Since the winding itself has varying thicknesses, the height and spacing range can be manually set, automatically calculated, or a combination of both during the selection process. Those skilled in the art can choose according to accuracy requirements, the actual situation of the winding itself, and other factors.

[0042] In some implementations, the grayscale image can be binarized first based on a brightness threshold, then connected component extraction can be performed on the image to obtain discrete bright spots, and the centroid coordinates of these bright spots can be extracted. Further, the extracted centroid coordinates of the bright spots are sorted according to their X-axis values, the distance between any two adjacent bright spots is calculated, the average distance between bright spots is obtained, and the average Y-axis coordinate of the centroid coordinates of the bright spots is also calculated. Bright spots with large Y-axis offsets of their centroids are then deleted, thus completing the height-based filtering process. The offset can be manually set according to actual conditions.

[0043] In some embodiments, when sequentially traversing bright spots, the distance between the current bright spot and the previous bright spot may be too large. For example, if the distance between two adjacent bright spots is greater than twice the average distance, it indicates that the grayscale value of the bright spot is low when binarizing the grayscale image according to the brightness value threshold, below the set brightness value threshold, and the bright spot is filtered out. Therefore, to overcome this problem, the winding detection method may further include an effective bright spot inspection step: traversing and statistically analyzing the distance between each pair of adjacent bright spots and their average distance, calculating the deviation value between the distance of the current bright spot and the previous bright spot and the average distance, and if the distance deviation value of the current bright spot is greater than the upper limit of the preset distance threshold, then the bright spot area is re-binarized to recover the lost bright spot. It is understood that this is a dynamic real-time inspection process. If, after recovering the bright spot and processing it again, the distance deviation value between the bright spot and the previous bright spot is still greater than the upper limit of the preset distance threshold, then there may be a winding error, and the winding speed is not accurately controlled, resulting in the winding coil being too loose. Of course, in some embodiments, the distance between the current bright spot and the next bright spot can also be calculated. This method may be slightly slower than the method described above, but it is still a feasible approach. Furthermore, the distance can be compared simultaneously with the distance between the previous and next bright spots to determine whether the filtered bright spot is positioned earlier or later.

[0044] Correspondingly, in some embodiments, when sequentially traversing bright spots, the distance between the current bright spot and the previous / next bright spot may be too close. For example, if the distance between two adjacent bright spots is less than 0.5 times the average distance, it indicates the presence of noise at that point. Therefore, to overcome this problem, the winding detection method may further include an effective bright spot inspection step: traversing and statistically analyzing the distance between each pair of adjacent bright spots and their average distance, calculating the deviation value between the distance between the current bright spot and the previous / next bright spot and the average distance. If the distance deviation value of the current bright spot is less than a preset lower limit of the distance threshold, then calculating the distance between the second bright spot before the current bright spot and the current bright spot and the bright spot before the current bright spot, respectively, and retaining the two points whose distance is closer to the average distance. If the distance between the current bright spot and the two bright spots before it is closer to the average distance value, then deleting the bright spot before the current bright spot; otherwise, deleting the current bright spot. The calculation method and judgment logic for the current bright spot and the next bright spot are similar and will not be repeated here.

[0045] S300, through high-level clustering, divides the 3D coordinates of each winding loop from the highest point of the image sensor into upper and lower layers, and fits the upper winding line 105 and / or the lower winding line 106, as shown. Figure 1 As shown.

[0046] In some embodiments, only one of the upper winding line or the lower winding line can be fitted, and the current point 107 can be directly analyzed and processed with one of the upper winding line or the lower winding line. However, in order to improve accuracy, it is preferable to fit both upper and lower lines at the same time.

[0047] In some embodiments, the specific process of high-level clustering may include: calculating the average height value of the Y-axis coordinate of the bright spot centroid coordinates based on the filtered bright spot centroid coordinates; the Y-axis coordinates may be classified according to clustering methods such as K-means. Since the target of this application is wire winding, which generally only has two layers, the number of clusters K can be set to K=2. Initial values ​​for the upper and lower Y-coordinates are set, typically based on the average height value of the Y-axis coordinate of the bright spot centroid coordinates, such as average height value ± m. This parameter m can be an empirical value or a value set based on the thickness of the wire winding.

[0048] Then, find the m points whose Y-axis coordinate values ​​of the bright spot centroid are closest to the initial Y-coordinate value of the upper layer, and take them as the first type of points. Find the m points whose Y-axis coordinate values ​​of the bright spot centroid are closest to the initial Y-coordinate value of the lower layer, and take them as the second type of points. Then update the initial Y-coordinate value of the upper layer to the average Y-coordinate value of the first type of points, and update the initial Y-coordinate value of the lower layer to the average Y-coordinate value of the second type of points. Finally, traverse all bright spots and divide all bright spots into two categories, namely, upper winding line 103 and lower winding line 104, and fit the upper winding line and / or the lower winding line.

[0049] In some embodiments, the upper and lower winding lines can be fitted using a parallel line fitting method, which is beneficial for noise reduction and filtering of noise information. For example, when performing parallel line fitting based on points that have been clustered into two layers, upper winding line 103 and lower winding line 104, the slopes of the lines containing the points on the upper and lower winding lines should be the same due to their parallel relationship. We can assume the equation of the upper winding line 105 is y1 = ax1 + b1, and the equation of the lower winding line 106 is y2 = ax2 + b2. We select two points on the upper winding line to calculate the equation y1 = ax1 + b1, obtaining the slope 'a'. Then, we fit the lower winding line based on the slope 'a' and calculate the value of 'b2'. Next, we calculate the distance from each point on the upper winding line to the equation y1. If this distance is greater than a preset height threshold, the point is considered invalid. Similarly, we determine whether each point on the lower winding line is invalid. We count the number of invalid points on both the upper and lower winding lines. If the number of invalid points is within a certain range, the line fitting is considered successful; otherwise, we refit the line.

[0050] S400, identify the feed line 102 and determine the coordinates of the current point 107. In some embodiments, the current point can be determined first and then the feed line can be determined in the 2D image. In some embodiments, the feed line can be determined first and then the current point can be determined based on the relationship between the feed line and the upper or lower winding line.

[0051] The S500 automatically extracts state parameter information during the winding process based on the fitted upper winding line, lower winding line, feed line, current point, etc.

[0052] For example, in S501, the feed angle θ between the feed line and the upper winding line or the lower winding line is calculated. Figure 2 As shown. The feed line angle θ can be used to determine whether the lateral movement speed of the feed mechanism meets the requirements. If the feed line angle θ is too large, it may cause the winding to be too loose, and vice versa, it may be too tight. The feed line angle θ can be an acute angle between the feed line and the upper and lower winding lines, or it can be an acute angle between the feed line and the perpendicular lines of the upper and lower winding lines.

[0053] For example, in S502, the line spacing Δl of the feed line is calculated based on the horizontal distance between the current point and the highest point of either the upper or lower winding line. The line spacing Δl can be used to determine the tightness of the winding, thereby further controlling the ratio between the lateral movement speed of the feed mechanism and the drum rotation speed.

[0054] In some embodiments, the height difference Δh between the feed line and the lower winding can be further collected. For example, in S503, the height difference Δh of the feed line is calculated based on the vertical distance between the current point and the lower winding line. This height difference Δh is used to determine whether the winding is flat, and it can also be used for verification during the parallel line fitting process.

[0055] In some embodiments, the distance L between the feed line and the inner wall of the drum can be further acquired to control the winding direction at the edge. Specifically, as in S504, the position information of the inner walls on both sides of the winding drum can be calculated, that is, the longitudinal straight lines of the left inner wall 108 and the right inner wall 109 of the drum can be fitted. These longitudinal straight lines are generally on the same plane as the upper and lower winding lines, and are generally oblique lines in the field of view of the image sensor 110, such as... Figure 2 As shown; when calculating the distance L to the edge of the feed line, the coordinates of the points at the same horizontal height as the inclined line of the inner wall of the winding drum in the feed direction can be determined according to the height of the current point, and the distance L to the edge is the horizontal distance between the two points.

[0056] Furthermore, at the initial stage of winding, there is only one layer of winding, and there is no distinction between upper and lower layers. Therefore, in some embodiments, an initial detection step can be set to address this issue: when binarizing the grayscale image according to a brightness threshold, the unwound portion of the roll will form a bright line, while only the wound portion will show bright spots. Therefore, in this initial detection case, it can be directly determined that this is the initial winding, i.e., the case with only one layer of winding. Connected component extraction can also be combined with this judgment. Since only the wound portion has bright spots, the number of connected components extracted in this case is also small. When a bright line is detected at the highest point from the image sensor during the initial detection process, and the number of connected components is less than a preset threshold, it is determined that this is the initial winding. In the case of initial winding, the winding quality detection procedure can be simplified, such as omitting certain parameters like height differences, or it can remain unchanged and still be executed according to the corresponding embodiment.

[0057] It should be noted that the above steps S100-S504 are not intended to limit the order of the processing flow, but are merely used to illustrate several aspects of the winding detection process. Some of these steps do have a logical order, but some processes can be processed sequentially or simultaneously. This is something that those skilled in the art can understand or adjust according to actual needs.

[0058] II. Winding Inspection Equipment

[0059] The winding detection device proposed in this application mainly includes an image sensor, a processor, and a memory. The image sensor, memory, processor, and other related components are directly or indirectly connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0060] The image sensor can be used to acquire 2D and 3D images during the winding process. In some embodiments, to better achieve the winding detection effect, the image sensor may mainly consist of a linear array image sensor, or mainly consist of a linear array image sensor and a 2D image sensor.

[0061] The memory can be used to store computer programs; the memory can be, but is not limited to: Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0062] The processor is used to execute computer programs stored in the memory. The processor can be an integrated circuit chip with signal processing capabilities. It can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0063] like Figure 5 As shown, the processor may include Figure 5 The image analysis unit shown may mainly consist of a coordinate extraction module, a height clustering analysis module, a parallel line fitting module, and an automatic parameter extraction module. It can implement or execute the various methods, steps, and functions disclosed in the embodiments of Part One of this application. When the computer program is executed by the processor, the winding detection device implements the operation corresponding to the 3D vision-based winding detection method described in Part One. It can automatically identify the current winding (i.e., upper winding, lower winding, and feed line) using 3D vision technology, and extract parameter information such as the feed angle θ, line spacing Δl, and height difference Δh, so that the winding system can perform winding control and / or winding quality analysis, providing data support for it.

[0064] In some embodiments, the coordinate extraction module can perform the relevant functions of step S200 in any embodiment of the winding detection method, and can extract the 3D coordinates of the distance from the highest point of the image sensor on each winding line based on the 2D image and the 3D image.

[0065] In some embodiments, the high-level clustering analysis module can complete the upper and lower layer correlation processing involved in step S300 in any embodiment of the winding detection method. It can divide the 3D coordinates of each winding from the highest point of the image sensor into upper and lower layers through high-level clustering.

[0066] In some embodiments, the parallel line fitting module can complete the parallel line fitting related processing involved in step S300 in any embodiment of the winding detection method. It can fit the upper winding line and the lower winding line through the parallel line fitting method, which is beneficial for noise resistance and filtering noise information.

[0067] In some embodiments, the automatic parameter extraction module can perform the related functions of steps S400 and S500 in any embodiment of the winding detection method. It can be used to identify the feed line, determine the current point coordinates, and automatically extract state parameter information during the winding process based on the fitted upper winding line, lower winding line, feed line, current point, etc.

[0068] It should be noted again that the winding detection equipment described in this section corresponds to the winding detection method described in the first part of this application. It has corresponding software function modules and computer programs, and can complete the operations corresponding to all embodiments included in the winding detection method. There are also various corresponding embodiments, which will not be described in detail in this section.

[0069] III. Automatic winding control method based on 3D vision

[0070] The automatic winding control method proposed in this application is applied to an automatic winding system and mainly includes the winding detection process and winding control process described in Part I of this application.

[0071] In some embodiments, the winding detection process includes the following steps: S100, acquiring 2D and 3D images collected by an image sensor during the winding process; S200, extracting the 3D coordinates of each winding line from the highest point of the image sensor based on the 2D and 3D images; S300, dividing the 3D coordinates of each winding line from the highest point of the image sensor into upper and lower layers using a height clustering method, and fitting the upper winding line and / or the lower winding line; S400, identifying the feed line and determining the coordinates of the current point; S500, automatically extracting state parameter information during the winding process based on the fitted upper winding line, lower winding line, feed line, and current point; for example, calculating the feed line angle θ between the feed line and the upper winding line or the lower winding line; calculating the feed line spacing Δl based on the horizontal distance between the current point and the highest point of the upper or lower winding line. As disclosed in Part I, further information such as the height difference Δh of the feed line and the distance L from the feed line to the edge can be extracted to determine the winding state.

[0072] In some embodiments, the winding control process may include the following steps: S600, controlling the winding according to the automatically extracted state parameter information during the winding process. This application can accurately detect the current winding state using a 3D image sensor, acquire 2D and 3D images of the winding process, and automatically identify the current winding (i.e., upper winding, lower winding, and feed line) using 3D vision technology. It can extract parameter information such as the feed angle θ and the line spacing Δl, so that the winding system can accurately control the drum speed and the lateral feed speed of the feed line, achieving precise control of the winding quality.

[0073] This may include step S601, which determines whether the ratio of the current feed line's transverse feed speed and the drum speed is appropriate based on the feed line's feed angle θ and line spacing Δl. If the feed line's feed angle θ and / or line spacing Δl are detected to be outside the normal range, the transverse feed speed and / or drum speed are adjusted accordingly based on the deviation value.

[0074] This may include S602, which calculates the height difference Δh of the feed line based on the vertical distance between the current point and the lower winding line, and uses this to determine whether the winding is flat; further, it may determine whether the ratio of the current feed line's lateral feed speed and the drum speed is appropriate based on the feed line's feed angle θ, line spacing Δl, and height difference Δh. If the feed line's feed angle θ, line spacing Δl, and / or height difference Δh are detected to be outside the normal range, the lateral feed speed and / or drum speed are adjusted accordingly based on the deviation value.

[0075] In some embodiments, the method may further include S700, controlling the winding direction. As disclosed in S504 of the winding detection method, the position information of the inner walls on both sides of the winding drum is calculated. Based on the horizontal distance between the current point and the inner wall of the winding drum in the feed direction, the edge-to-edge distance L of the feed line is calculated. Then, the turning operation of the feed line is controlled according to the edge-to-edge distance L. When the edge-to-edge distance L is less than a preset edge-to-edge threshold, the feed line is controlled to move in the opposite direction. This solution can be used to replace the edge-to-edge sensor. Of course, in other embodiments without this winding control process, the edge-to-edge sensor can continue to be used for turning control, and even both solutions can be used simultaneously.

[0076] In some embodiments, the following steps can be used to resolve the problem of non-compliance in real time: if it is detected that the feed angle θ, the line spacing Δl, and / or the height difference Δh of the feed line are not within the normal range during the current or historical winding process, the feed mechanism and / or the drum mechanism are controlled to perform a recovery action to recover the feed line to the error point position, and the transverse feed speed and / or the drum speed are adjusted accordingly based on the deviation value to rewind the line.

[0077] In some embodiments, the winding quality evaluation can also be achieved through the following steps: S800, the winding quality is analyzed based on the winding process status information to obtain winding quality evaluation information, wherein the winding process status information includes the feed line angle θ, the line spacing Δl and / or the height difference Δh.

[0078] In some embodiments, an automatic stop winding process function can also be set, which can be implemented through the following steps: S900, calculate the number of winding layers that have been completed, and control the winding to stop when the number of layers equals the budgeted layer value. Specifically, the number of winding layers that have been completed can be determined by the 3D coordinates of the current point, or the number of edge turning control signals can be used to determine whether the specified number of turns has been completed.

[0079] IV. Automatic winding system

[0080] The automatic winding system proposed in this application mainly includes an image sensor, a control system, a winding mechanism, and a feeding mechanism. The image sensor is used to acquire 2D and 3D images during the winding process. The control system includes a controller for controlling the actions of the winding mechanism and the feeding mechanism, and a memory for storing computer programs. The winding mechanism is used to perform winding actions. The feeding mechanism is used to perform wire feeding actions. When the computer program is executed by the controller, the automatic winding system implements the operations corresponding to the automatic winding control method disclosed in Part Three.

[0081] In some embodiments, the control system can implement or execute the various methods, steps, and functions disclosed in the embodiments of Part One of this application. When the computer program is executed by the processor, the winding detection system implements the operation corresponding to the 3D vision-based winding detection method described in Part One. It can automatically identify the current winding (i.e., upper winding, lower winding, and feed line) using 3D vision technology, and extract parameter information such as the feed angle θ, line spacing Δl, and height difference Δh, so that the winding system can perform winding control and / or winding quality analysis, providing data support for it. Figure 5 As shown, the control system may include an image acquisition unit, an image analysis unit, and a winding control unit. The image acquisition unit mainly consists of an image acquisition module, which can receive data via wired communication or wireless communication. The image processing unit may mainly consist of a coordinate extraction module, a height clustering analysis module, a parallel line fitting module, and an automatic parameter extraction module. The winding control unit may include a winding state analysis module and a PID control module; in some embodiments, it may also include a winding quality evaluation module.

[0082] In some embodiments, the image acquisition module is used to acquire 2D and 3D images collected by the image sensor. The coordinate extraction module, height clustering analysis module, parallel line fitting module, automatic parameter extraction module, and other modules can achieve the same functions as the winding detection device described in Part Two, and can all implement the relevant operations disclosed in the various embodiments of the winding detection method described in Part One.

[0083] In some embodiments, the winding state analysis module can implement the relevant operations of each embodiment in steps S600, S700, and / or S900 of the automatic winding control method disclosed in Part Three, determine the winding quality based on the automatically extracted state parameter information during the winding process, and issue control commands. The PID control module can execute the relevant operations of each embodiment in steps S600, S700, and / or S900 according to the control commands issued by the winding state analysis module, control the drum mechanism and the feed mechanism to perform corresponding actions, and complete the automatic winding.

[0084] In some embodiments, the automatic winding system may also be provided with a winding quality evaluation module, which is used to analyze the winding quality based on the winding process status information to obtain winding quality evaluation information.

[0085] In some embodiments, the automatic winding system can be a distributed system or an integrated device. The automatic winding system includes the winding detection device disclosed in Part Two, wherein the image sensor, processor, and memory included in the winding detection device are part of the automatic winding system, and the winding detection device can be a camera module integrating the image sensor, processor, and memory; the processor and memory in the winding detection device can also be integrated into the controller of the automatic winding system; or they can be a processing module independent of the image sensor and controller. It should be understood that the control system can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic and circuit structures; the software portion can be stored in memory and executed by an appropriate instruction system.

[0086] It should be noted that the above description of the system and its modules is for convenience only and should not limit this application to the scope of the embodiments described. It is understood that those skilled in the art, after understanding the principle of the system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from this principle. The terms "system," "device," "equipment," and / or "module" used in this application are a way of distinguishing different components, elements, parts, sections, units, modules, or assemblies at different levels. However, if other terms can achieve the same purpose, they can be replaced by other expressions. Generally, the components, elements, parts, sections, units, modules, assemblies, processes, and procedures described in the embodiments of this application and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0087] Flowcharts are used in this application to illustrate the operations performed by a "system," "apparatus," "device," and / or "module" according to embodiments of this application. It should be understood that preceding or subsequent operations are not necessarily performed in exact order. Instead, certain steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from them.

[0088] V. Computer program storage media

[0089] If the functions described in this application are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0090] This application proposes a computer-readable storage medium. The storage medium stores a computer program, and when a computer reads the computer program from the storage medium, the computer executes the operation corresponding to the 3D vision-based wire winding detection method described in Part One.

[0091] This application also proposes a computer-readable storage medium. The storage medium is used to store a computer program, and when a computer reads the computer program from the storage medium, the computer performs operations corresponding to the automatic winding control method described in Part Three.

[0092] Computer-readable storage media may contain a propagated data signal encoded with a computer program, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, or suitable combinations thereof. The storage medium may be, but is not limited to: floppy disks, optical disks, hard disks, USB flash drives, TF cards (T-Flash Card, also known as MicroSD cards), SD cards (Secure Digital Memory Cards), MMC cards (Multi Media Cards), SM cards (Smart Media Cards), Memory Sticks, XD cards, CF cards (Compact Flash Cards), etc.

[0093] This medium can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. Program code located on a computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0094] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby, and Groovy, or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0095] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0096] Furthermore, this application uses specific terms to describe its embodiments. For example, "one embodiment" or "some embodiments" refers to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "some embodiments" or "one embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0097] As indicated in this application and claims, unless the context clearly indicates otherwise, the terms "a," "first," "second," and / or "the" are not specifically singular in quantity, but rather descriptive terms used for distinction and classification. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0098] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A wire winding detection method based on 3D vision, characterized in that: Includes the following steps: Acquire 2D and 3D images captured by an image sensor during the winding process; Based on the 2D and 3D images, extract the 3D coordinates of the distance from the highest point of the image sensor on each loop of the winding line; By using a high-level clustering method, the 3D coordinates of each loop of winding wire from the highest point of the image sensor are divided into upper and lower layers, and the upper layer winding line and / or the lower layer winding line are fitted. Identify the feed line and determine the coordinates of the current point; The feed angle θ between the feed line and the upper winding line or the lower winding line is calculated. The feed line spacing Δl is calculated based on the horizontal distance between the current point and the highest point of either the upper or lower winding line.

2. The winding detection method according to claim 1, characterized in that: The winding detection method further includes the following steps: calculating the height difference Δh of the feed line based on the vertical distance between the current point and the lower winding line.

3. The winding detection method according to claim 1, characterized in that: The winding detection method further includes the following steps: Calculate the position information of the inner walls on both sides of the winding drum; The distance L from the edge of the feed line is calculated based on the horizontal distance between the current point and the inner wall of the winding drum in the feed direction.

4. The winding detection method according to claim 1, characterized in that: When fitting the upper and lower layer winding lines, noise information is filtered out by using parallel line fitting.

5. The winding detection method according to claim 1, characterized in that: The process of extracting the coordinates of the distance from the highest point of the image sensor on each loop of the winding line includes the following steps: The 2D image is binarized to extract the 2D centroid coordinates of the brightest points in the axial direction of the winding drum in the target winding region. Based on the matching relationship between 2D and 3D images, and according to the height and spacing range of each brightest point, the 3D coordinates of the brightest point in each loop are selected.

6. A winding inspection device, characterized in that: This includes an image sensor, a processor, and memory; The image sensor is used to acquire 2D and 3D images during the winding process; The memory is used to store computer programs; When the computer program is executed by the processor, the winding detection device performs the operation corresponding to the winding detection method as described in any one of claims 1-5.

7. The winding inspection device according to claim 6, characterized in that: The image sensor is mainly composed of a linear array image sensor, or mainly composed of a linear array image sensor and a 2D image sensor.

8. An automatic winding control method, characterized in that: Includes the following steps: Acquire 2D and 3D images captured by an image sensor during the winding process; Based on the 2D and 3D images, extract the 3D coordinates of the distance from the highest point of the image sensor on each loop of the winding line; By using a high-level clustering method, the 3D coordinates of each loop of winding wire from the highest point of the image sensor are divided into upper and lower layers, and the upper layer winding line and / or the lower layer winding line are fitted. Identify the feed line and determine the coordinates of the current point; The feed angle θ between the feed line and the upper winding line or the lower winding line is calculated. The feed line spacing Δl is calculated based on the horizontal distance between the current point and the highest point of either the upper or lower winding line. Based on the feed line's feed angle θ and line spacing Δl, determine whether the ratio of the current feed line's transverse feed speed and drum speed is appropriate. If the feed line's feed angle θ and / or line spacing Δl are detected to be outside the normal range, adjust the transverse feed speed and / or drum speed accordingly based on the deviation value.

9. The automatic winding control method according to claim 8, characterized in that: It also includes the following steps: The height difference Δh of the feed line is calculated based on the vertical distance between the current point and the lower winding line. Based on the feed line's feed angle θ, line spacing Δl, and height difference Δh, determine whether the ratio of the current feed line's transverse feed speed and drum speed is appropriate. If the feed line's feed angle θ, line spacing Δl, and / or height difference Δh are detected to be outside the normal range, then adjust the transverse feed speed and / or drum speed accordingly based on the deviation value.

10. The automatic winding control method according to claim 8, characterized in that: It also includes the following steps: Calculate the position information of the inner walls on both sides of the winding drum; The distance L from the edge of the feed line is calculated based on the horizontal distance between the current point and the inner wall of the winding drum in the feed direction. The direction of the feed line is controlled based on the distance L from the edge of the feed line.

11. The automatic winding control method according to claim 8 or 9, characterized in that: If the feed line angle, spacing, and / or height difference are detected to be outside the normal range during the current or historical winding process, the feed line is controlled to be retrieved to the error point, and the transverse feed speed and / or drum speed are adjusted accordingly based on the deviation value to rewind the wire.

12. The automatic winding control method according to claim 8 or 9, characterized in that: The winding quality is analyzed based on the winding process status information to obtain winding quality evaluation information. The winding process status information includes the feed line angle θ, the line spacing Δl, and / or the height difference Δh.

13. An automatic winding system, characterized in that: Includes image sensor, control system, roll mechanism and feed mechanism; The image sensor is used to acquire 2D and 3D images during the winding process; The control system includes a controller for controlling the movement of the drum mechanism and the feed mechanism, and a memory for storing computer programs. The winding mechanism is used to perform the winding action; The feeding mechanism is used to perform the wire feeding action; When the computer program is executed by the controller, the automatic winding system performs the operation corresponding to the automatic winding control method as described in any one of claims 8-12.

14. A computer program storage medium, characterized in that: The storage medium is used to store a computer program. When the computer reads the computer program in the storage medium, the computer runs the winding detection method as described in any one of claims 1-5.

15. A computer program storage medium, characterized in that: The storage medium is used to store a computer program. When the computer reads the computer program in the storage medium, the computer runs the automatic winding control method as described in any one of claims 8-12.

Images