Automatic wire arranging method and device based on real-time 2D visual feedback

Abstract

The application discloses an automatic wire arranging method and device based on 2D visual real-time feedback, and belongs to the technical field of computer vision and automatic production. The technical problem to be solved by the application is how to realize abnormality and edge detection in the wire arranging process, so that corresponding action operation is performed to achieve the effect of precise wire arranging. The technical scheme is as follows: a current wire collecting scene image is acquired through a 2D visual acquisition module; left and right alternating imaging data are acquired based on the 2D visual acquisition module; a scene image is identified in real time by using a deep visual network V1, and an image inference result is output in real time, including normal, wire stacking, flash gap, edge and reset; middle imaging data are acquired based on the 2D visual acquisition module, and a target point positioning result is output in real time by a deep visual network V2, so as to calculate a wire collecting interval; when the wire collecting interval is greater than half of a wire diameter, it is regarded as a flash gap; a wire arranging device receives a wire stacking signal to perform same-direction wire arranging, and receives an edge signal to perform reverse wire arranging.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and automated production technology, specifically to an automatic wiring method and apparatus based on real-time feedback from 2D vision. Background Technology

[0002] Machine vision is a rapidly developing branch of artificial intelligence. Simply put, machine vision uses machines to replace human eyes for measurement and judgment. A machine vision system uses machine vision products (i.e., image acquisition devices, which can be CMOS or CCD) to convert the captured target into image signals, which are then transmitted to a dedicated image processing system. This system obtains the target's shape information and, based on pixel distribution, brightness, color, and other information, converts it into digital signals. The image system performs various calculations on these signals to extract the target's features, and then controls the on-site equipment based on the judgment results.

[0003] Machine vision is a comprehensive technology that includes image processing, mechanical engineering, control, electric lighting, optical imaging, sensors, analog and digital video technology, and computer hardware and software technology (image enhancement and analysis algorithms, image cards, I / O cards, etc.).

[0004] Currently, automatic cable routing methods based on machine vision typically use traditional image processing techniques to extract image parameters from the cable reel, then compare them with a database to determine the current working status of the cable reel. This method has low adaptability to environmental changes, insufficient detection accuracy, and real-time performance is difficult to guarantee due to database comparison.

[0005] Therefore, how to detect anomalies and edge detection during the wiring process and execute corresponding actions to achieve precise wiring is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] The technical objective of this invention is to provide an automatic cabling method and apparatus based on 2D vision real-time feedback to solve the problem of how to detect anomalies and edges that occur during the cabling process, thereby executing corresponding actions to achieve precise cabling.

[0007] The technical objective of this invention is achieved as follows: an automatic cable arrangement method based on real-time 2D visual feedback, the specific method of which is as follows:

[0008] The scene image of the current line collection is acquired through the 2D vision acquisition module;

[0009] The system acquires alternating left and right imaging data based on a 2D vision acquisition module, uses a deep vision network V1 to perform real-time recognition of scene images, and outputs image inference results in real time, including normal, overlapping lines, flash seams, edge-to-edge, and reset; among them, normal and overlapping lines are conjugate, and edge-to-edge and reset are conjugate.

[0010] The 2D vision acquisition module acquires intermediate imaging data, and the depth vision network V2 outputs the target point localization results in real time to calculate the line spacing.

[0011] When the distance between the take-up threads is greater than half the thread diameter, it is considered a flash seam;

[0012] When the cable laying device receives a cable stacking signal, it lays the cable in the same direction. When it receives a signal indicating that the cable has reached the edge, it lays the cable in the opposite direction. At the same time, upon feedback from the signal indicating that the cable has reached the edge, the take-up device performs an automatic reversing function. The cable laying direction is relative to the movement direction of the take-up device.

[0013] The cable runner performs the cable run operation until it receives a reset signal (conjugate to edge signal) or a normal signal (conjugate to overlap signal) or a reset signal, and then returns to the zero position.

[0014] After receiving the flash gap signal, the take-up device moves in the opposite direction to reduce the take-up gap until it is controlled within half the wire diameter.

[0015] Preferably, the take-up device is used for winding the wire. The take-up device includes a take-up bracket, which is I-shaped. A linear motor is provided at the bottom of the I-shaped take-up bracket to realize the left and right movement of the take-up device. A rotary motor is provided in the middle of the I-shaped take-up bracket to realize the rotational movement of the take-up device.

[0016] More preferably, the cable guide is used to achieve two degrees of freedom of movement during the cable guiding operation; the cable guide includes a cable guide bracket, which is an inverted T-shape composed of a cable base and two parallel cable support parts. An X-axis servo motor is provided at the upper center of the cable base, which is located between the two parallel cable support parts. The X-axis servo motor is used for the left and right movement of the cable guide bracket; a Z-axis servo motor is provided at the lower center of any cable support part, which is used to realize the up and down movement of the 2D vision acquisition module and the cable.

[0017] More preferably, the 2D vision acquisition module is used to acquire image data of the current line-receiving state; the 2D vision acquisition module includes a left RGB camera, a middle RGB camera and a right RGB camera, with the middle RGB camera located between the left RGB camera and the right RGB camera;

[0018] The left, middle, and right RGB cameras were all set to low aperture and automatic gain mode to adapt to changes in the external environment and obtain images with balanced brightness in the current scene.

[0019] More preferably, the deep vision network V1 includes the PPLiteSeg segmentation model and the desnet121 classification model, as follows:

[0020] The PPLiteSeg segmentation model performs background removal on the image to obtain the connected regions of the feature map after background removal, reducing the interference of complex background on the region of interest (coil area + cable area).

[0021] After performing post-image segmentation, the influence of invalid data is further reduced by retaining the largest bounding rectangle of the connected regions of the feature maps after background removal, thus obtaining images of different sizes.

[0022] To ensure image consistency, images of varying sizes are scaled proportionally to their major and minor axes, with the minor axis filled with 0s or 1s, ultimately resulting in a fixed-size image (e.g., 512x512).

[0023] A fixed-size image is input into the DesNet121 classification model for four-class classification inference. The DesNet121 classification model uses dropblocks to randomly discard a continuous block on the feature map, thereby improving the model's generalization ability.

[0024] The Deep Vision Network V2 calculates the line spacing as follows:

[0025] The V2 deep vision network segments the target line and the reference line to obtain the segmentation result.

[0026] Perform circle fitting on the segmentation results to obtain the coordinates of the two circle centers (O1, O2) and the radius (r1, r2). Calculate the distance between the converging lines:

[0027] When the spacing is greater than the set threshold, it is considered a flash gap; where the threshold = (r1 + r2) / 2;

[0028] The Deep Vision Network V2 uses the PPLiteSeg segmentation model.

[0029] Better still, offline data augmentation is performed on the four types of datasets obtained: normal, overlapping lines, edge-to-edge, and reset. The offline data augmentation processing methods include random cropping (50%-100%), random rotation (-10°-10°), and color jitter. After manual screening, images of the coil area and cable area (where there is a region of interest, ROI) are retained.

[0030] An automatic cable laying device based on 2D vision real-time feedback, the device being used to implement the automatic cable laying method based on 2D vision real-time feedback as described in any one of claims 1-7; the device comprises:

[0031] The 2D vision acquisition module is used to acquire the scene image of the current line collection. For alternating left and right imaging data, the deep vision network V1 is used to recognize the scene image in real time and output the image inference results in real time, including normal, overlapping line, flash gap, edge reach, and reset. Among them, normal and overlapping line are conjugate, and edge reach and reset are conjugate. For the imaging data in the middle, the deep vision network V2 outputs the target point positioning results in real time and calculates the line collection distance: when the line collection distance is greater than half the line diameter, it is regarded as a flash gap.

[0032] The cable runner is used to run cables in the same direction when it receives a cable overlap signal and in the opposite direction when it receives a cable arrival signal. At the same time, the take-up device performs an automatic reversing function upon feedback from the cable arrival signal. The cable runner is relative to the movement direction of the take-up device. The cable runner performs the cable run operation until it receives a reset signal (conjugate to the cable arrival signal) or a normal signal (conjugate to the cable overlap signal) or a reset signal, and then returns to the zero position.

[0033] The take-up mechanism is used to reverse its movement after receiving a flash gap signal, reducing the take-up gap until it is controlled within half the wire diameter.

[0034] Preferably, the take-up device is used for winding the wire. The take-up device includes a take-up bracket, which is I-shaped. A linear motor is installed at the bottom of the I-shaped take-up bracket to realize the left and right movement of the take-up device. A rotary motor is installed in the middle of the I-shaped take-up bracket to realize the rotational movement of the take-up device.

[0035] The cable guide is used to achieve two degrees of freedom of movement during cable laying operations. The cable guide includes a cable support bracket, which is an inverted T-shape composed of a cable base and two parallel cable support parts. An X-axis servo motor is located at the upper center of the cable base, between the two parallel cable support parts, and is used for the left and right movement of the cable support bracket. A Z-axis servo motor is located at the lower center of either cable support part, and is used to realize the up and down movement of the 2D vision acquisition module and the cable.

[0036] The 2D vision acquisition module is used to acquire image data of the current line take-up status; the 2D vision acquisition module includes a left RGB camera, a middle RGB camera and a right RGB camera, with the middle RGB camera located between the left RGB camera and the right RGB camera;

[0037] The left, middle, and right RGB cameras were all set to low aperture and automatic gain mode to adapt to changes in the external environment and obtain images with balanced brightness in the current scene.

[0038] An electronic device includes: a memory and at least one processor;

[0039] The memory contains computer programs;

[0040] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the automatic wiring method based on real-time 2D visual feedback as described above.

[0041] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the automatic wiring method based on real-time 2D visual feedback as described above.

[0042] The automatic cable laying method and apparatus based on 2D vision real-time feedback of the present invention have the following advantages:

[0043] (i) Compared with traditional cable laying operations, this invention can greatly reduce the occurrence of random dents or protrusions caused by insufficient control precision of the cable winding equipment itself, thereby reducing the abnormalities in the winding process and the operation of laying the cable at the edge, and saving labor costs.

[0044] (ii) This invention can adapt to changes in different wire diameters and background environments, solve abnormalities such as depressions and protrusions in the cable laying process in real time, and realize automatic edge reversal function, improve laying accuracy, eliminate manual intervention, have a high degree of automation, and greatly save labor costs.

[0045] (III) Based on 2D visual reasoning technology, this invention can adapt to changes in different wire diameters and external environment, realize real-time precise cable routing, save labor costs, and has a high degree of automation.

[0046] (iv) This invention is based on a wire laying device consisting of a 2D vision acquisition module, a wire puller, and a wire take-up device. By using deep vision network inference, it performs anomaly (overlapping wires, flash seams) and edge detection. Combined with the operation of the wire laying and take-up modules, it gets rid of the influence of different wire diameters and changes in the external environment, performs real-time high-precision wire laying, eliminates manual intervention, has a high degree of automation, and greatly saves labor costs.

[0047] (V) The cable laying device of the present invention receives a stacked cable signal and lays the cable in the same direction, and receives an edge signal and lays the cable in the opposite direction. Under the feedback of the edge signal, the take-up module performs an automatic reversing function. The cable laying device performs the cable laying operation until a normal signal or a reset signal is received, and returns to the zero position. When the take-up receives a gap signal, the linear motor moves in the opposite direction to reduce the take-up spacing until the spacing is controlled within the set threshold. It can realize the detection of abnormalities and edge arrival during the cable laying process, and thus perform corresponding actions to achieve the effect of precise cable laying. Attached Figure Description

[0048] The invention will be further described below with reference to the accompanying drawings.

[0049] Appendix Figure 1 This is a flowchart of an automatic cable arrangement method based on real-time 2D visual feedback.

[0050] Appendix Figure 2 This is a schematic diagram of an automatic cable laying device based on real-time 2D visual feedback.

[0051] Appendix Figure 3 This is a diagram illustrating the closing state of the line;

[0052] Appendix Figure 4 A schematic diagram illustrating specific application scenarios.

[0053] Figure 2 In the middle, S1, take-up receiver, S 1-1 Linear motor, S 1-2 Rotary motor, S 1-3 , cable take-up bracket, S 2-1 Z-axis servo motor, S 2-2 X-axis servo motor, S 2-3 , ribbon cable bracket, S 2-3-1 , ribbon cable base, S 2-3-2 Cable support section, S3, 2D vision acquisition module, S 3-1 RGB camera on the left, S 3-2 RGB camera on the right, S 3-3 1. Middle RGB camera.

[0054] Figure 3 In the middle: a) Normal state; b) Overlapping line abnormality; c) Flash seam abnormality; d) Edge state; e) Reset state. Detailed Implementation

[0055] The automatic cable arrangement method and apparatus based on real-time 2D visual feedback of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] In the description of this invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0057] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0058] Example 1:

[0059] As attached Figure 1 As shown in the figure, this embodiment provides an automatic cable arrangement method based on real-time 2D visual feedback, which is as follows:

[0060] S1. Acquire the scene image of the current line collection using the 2D vision acquisition module;

[0061] S2. Based on the 2D vision acquisition module, the alternating left and right imaging data is acquired. The deep vision network V1 is used to recognize the scene image in real time and output the image inference results in real time, including normal, overlapping line, flash seam, edge to, and reset. Among them, normal and overlapping line are conjugate, and edge to and reset are conjugate.

[0062] As attached Figure 3 As shown, (a) normal state refers to the target line and reference line being on the same horizontal plane, closely arranged, and the number of windings in the same layer > 2; (b) overlapping abnormal state refers to the distance between the target line and reference line being less than one wire diameter, and the number of windings in the same layer > 2; (c) flash gap abnormal state refers to the distance between the target line and reference line being greater than half a wire diameter, and the number of windings in the same layer > 1; (d) edge state refers to the complete overlapping state when the number of windings in the same layer = 2; (e) reset state refers to the normal state when the number of windings in the same layer = 2.

[0063] S3. Based on the 2D vision acquisition module, intermediate imaging data is acquired, and the depth vision network V2 outputs the target point localization results in real time, calculating the line spacing:

[0064] When the distance between the take-up threads is greater than half the thread diameter, it is considered a flash seam;

[0065] S4. When the cable laying device receives the overlapping signal, it lays the cable in the same direction. When it receives the edge signal, it lays the cable in the opposite direction. At the same time, under the feedback of the edge signal, the take-up device performs the automatic reversing function. The laying direction is relative to the movement direction of the take-up device.

[0066] S5. The cable runner performs the cable run operation until it receives a reset signal (conjugate to edge signal) or a normal signal (conjugate to overlap signal) or a reset signal, and then returns to the zero position.

[0067] S6. After receiving the flash gap signal, the take-up device moves in the opposite direction to reduce the take-up gap until the take-up gap is controlled within half the wire diameter.

[0068] As attached Figure 2 As shown, the take-up device in this embodiment is used for winding the cable. The take-up device S1 includes a take-up bracket S. 1-3 Cable take-up bracket S 1-3 It is H-shaped, H-shaped cable take-up bracket S 1-3 A linear motor S is installed at the bottom. 1-1 Linear motor S 1-1 Used to enable the left and right movement of the take-up device S1; I-shaped take-up bracket S 1-3 A rotary motor S is installed in the middle. 1-2 Rotary motor S 1-2 Used to achieve the rotational movement of the take-up coil S1.

[0069] In this embodiment, the cable guide S2 achieves two degrees of freedom of movement during the cable routing operation; the cable guide S2 includes a cable support S. 2-3 ribbon cable bracket S 2-3 It consists of the ribbon cable base part S 2-3-1 and two parallel cable support sections S 2-3-2 The inverted T-shape is formed by the S-shaped base of the ribbon cable. 2-3-1 The upper part of the center is equipped with an X-axis servo motor S 2-2 X-axis servo motor S 2-2 Located on two parallel cable support sections S 2-3-2 Between, X-axis servo motor S 2-2 For use in cable bracket S 2-3 The left and right movement; any cable support part S 2-3-2 The Z-axis servo motor S is installed in the lower middle part. 2-1 Z-axis servo motor S 2-1 Used to move the 2D vision acquisition module S3 and its cabling up and down.

[0070] In this embodiment, the 2D vision acquisition module S3 is used to acquire image data of the current line take-up state; the 2D vision acquisition module S3 includes a left-side RGB camera S... 3-1 , middle RGB camera S 3-3 And the RGB camera S on the right 3-2 RGB camera S in the middle 3-3 RGB camera S located on the left 3-1 and the RGB camera on the right 3-2 between;

[0071] Left RGB camera S 3-1 , middle RGB camera S 3-3 And the RGB camera S on the right3-2 During acquisition, all images were set to low aperture and automatic gain mode to adapt to changes in the external environment and obtain images with balanced brightness in the current scene.

[0072] The deep vision network V1 in this embodiment includes the PPLiteSeg segmentation model and the desnet121 classification model, as detailed below:

[0073] (1) The PPLiteSeg segmentation model performs background removal on the image to obtain the connected regions of the feature map after background removal, thereby reducing the interference of complex background on the region of interest (coil area + cable area).

[0074] (2) After performing post-image segmentation, further reduce the impact of invalid data, retain the largest bounding rectangle of the connected region of the feature map after background removal, and obtain images of different sizes;

[0075] (3) In order to solve the problem of image consistency, images of different sizes are scaled proportionally to the major and minor axes of the image, and 0 or 1 are filled in the minor axis direction to finally form an image of a fixed size (such as 512x512).

[0076] (4) Input a fixed-size image into the DesNet121 classification model and perform four-class classification inference. In the DesNet121 classification model, dropblock is used to randomly discard a continuous block on the feature map, thereby improving the generalization ability of the model.

[0077] In this embodiment, the depth vision network V2 calculates the line spacing as follows:

[0078] (1) The V2 deep vision network is used to segment the target line and the reference line to obtain the segmentation result;

[0079] (2) Perform circle fitting on the segmentation results to obtain the coordinates of the center of two circles (O1, O2) and the radius (r1, r2), and calculate the line spacing:

[0080] (3) When the spacing is greater than the set threshold, it is considered a flash gap; where the threshold = (r1 + r2) / 2;

[0081] (4) The deep vision network V2 adopts the PPLiteSeg segmentation model.

[0082] In this embodiment, step S2 involves offline data augmentation of the four types of datasets obtained: normal, overlapping lines, edge-to-edge, and reset. The offline data augmentation process includes random cropping (50%-100%), random rotation (-10°-10°), and color jitter. After manual screening, images of the coil area and cable area (where there is a region of interest, ROI) are retained.

[0083] Example 2:

[0084] This embodiment provides an automatic cable laying device based on 2D visual real-time feedback. This device is used to implement the automatic cable laying method based on 2D visual real-time feedback in this embodiment. The device includes:

[0085] The 2D vision acquisition module S3 is used to acquire the scene image of the current line collection. For the alternating left and right imaging data, the deep vision network V1 is used to recognize the scene image in real time and output the image inference results in real time, including normal, overlapping line, flash gap, edge reach, and reset. Among them, normal and overlapping line are conjugate, and edge reach and reset are conjugate. For the imaging data in the middle, the deep vision network V2 outputs the target point positioning results in real time and calculates the line collection distance: when the line collection distance is greater than half the line diameter, it is regarded as a flash gap.

[0086] The cable runner S2 is used to run cables in the same direction when it receives a cable overlap signal and to run cables in the opposite direction when it receives a cable arrival signal. At the same time, the take-up device performs an automatic reversing function under the feedback of the cable arrival signal. The cable runner is relative to the movement direction of the take-up device. The cable runner performs the cable run operation until it receives a reset signal (conjugate to the cable arrival signal) or a normal signal (conjugate to the cable overlap signal) or a reset signal, and then returns to the zero position.

[0087] The take-up reel S1 is used to reverse its movement after receiving the flash gap signal, reducing the take-up gap until it is controlled within half the wire diameter.

[0088] As attached Figure 2 As shown, the take-up device in this embodiment is used for winding the cable. The take-up device S1 includes a take-up bracket S. 1-3 Cable take-up bracket S 1-3 It is H-shaped, H-shaped cable take-up bracket S 1-3 A linear motor S is installed at the bottom. 1-1 Linear motor S 1-1 Used to enable the left and right movement of the take-up device S1; I-shaped take-up bracket S 1-3 A rotary motor S is installed in the middle. 1-2 Rotary motor S 1-2 Used to achieve the rotational movement of the take-up coil S1.

[0089] In this embodiment, the cable guide S2 achieves two degrees of freedom of movement during the cable routing operation; the cable guide S2 includes a cable support S. 2-3 ribbon cable bracket S 2-3 It consists of the ribbon cable base part S 2-3-1 and two parallel cable support sections S 2-3-2 The inverted T-shape is formed by the S-shaped base of the ribbon cable. 2-3-1 The upper part of the center is equipped with an X-axis servo motor S 2-2 X-axis servo motor S 2-2Located on two parallel cable support sections S 2-3-2 Between, X-axis servo motor S 2-2 For use in cable bracket S 2-3 The left and right movement; any cable support part S 2-3-2 The Z-axis servo motor S is installed in the lower middle part. 2-1 Z-axis servo motor S 2-1 Used to move the 2D vision acquisition module S3 and its cabling up and down.

[0090] In this embodiment, the 2D vision acquisition module S3 is used to acquire image data of the current line take-up state; the 2D vision acquisition module S3 includes a left-side RGB camera S... 3-1 , middle RGB camera S 3-3 And the RGB camera S on the right 3-2 RGB camera S in the middle 3-3 RGB camera S located on the left 3-1 and the RGB camera on the right 3-2 between;

[0091] Left RGB camera S 3-1 , middle RGB camera S 3-3 And the RGB camera S on the right 3-2 During acquisition, all images were set to low aperture and automatic gain mode to adapt to changes in the external environment and obtain images with balanced brightness in the current scene.

[0092] As attached Figure 1 As shown, the working process of this device is as follows:

[0093] Step 1, 2D RGB sensor (left camera S) 3-1 Right camera S 3-2 and intermediate camera S 3-3 The image shows the current scene of the line being retrieved. There are five retrieving states: normal, overlapping lines, flashing seams, reaching the edge, and reset. (See attached image.) Figure 3 As shown, (a) normal state refers to the target line and reference line being on the same horizontal plane, closely arranged, and the number of windings in the same layer > 2; (b) overlapping abnormal state refers to the distance between the target line and reference line being less than one wire diameter, and the number of windings in the same layer > 2; (c) flash gap abnormal state refers to the distance between the target line and reference line being greater than half a wire diameter, and the number of windings in the same layer > 1; (d) edge state refers to the complete overlapping state when the number of windings in the same layer = 2; (e) reset state refers to the normal state when the number of windings in the same layer = 2.

[0094] Step 2, based on the left camera S 3-1 and right camera S 3-2Alternating imaging data are used, and a deep vision network V1 performs real-time inference to derive classification results, including four categories: normal, overlapping lines, to the edge, and reset. Normal and overlapping lines are conjugates, as are to the edge and reset. Based on the intermediate camera S... 3-3 The imaging data and the depth vision network V2 output the coordinates of the target positioning point in real time to calculate the wire spacing. When the spacing is greater than a set threshold (half a wire diameter), it is considered a flash gap.

[0095] As attached Figure 4 As shown, the Deep Vision Network V1 consists of a segmentation model and a classification model. Segmentation uses the PPLiteSeg model, and classification uses the DesNet121 model, but the use of these two models is not limited to this. The specific inference process is as follows: First, the PPLiteSeg model performs background removal on the image to reduce the interference of complex backgrounds on the Region of Interest (ROI) (coil area + cable area). After image segmentation, the influence of invalid data is further reduced. The maximum bounding rectangle of the connected regions in the background-removed feature maps is retained, resulting in images of varying sizes. To ensure consistency in the dimensions of the segmented images, they are scaled proportionally to their major and minor axes, and filled with 0s or 1s along the minor axis, ultimately forming a fixed-size image (e.g., 512x512). This image is then fed into the DesNet121 model to obtain the classification result, which is finally converted into different types of signals for output. Due to the limited amount of edge data, offline data augmentation was performed to maintain a suitable proportion of the classification dataset. This included random cropping (50%-100%), random rotation (-10°-10°), and color jitter. Subsequently, manual screening was conducted to retain only images containing regions of interest (coil area + cable area).

[0096] As attached Figure 4 As shown, the main body of the deep vision network V2 is composed of a segmentation model, which also uses the PPLiteSeg model for segmentation. The specific calculation process is as follows: First, the PPLiteSeg model segments and locates the target line and the reference line. Then, it performs circle fitting on the segmented region to obtain the coordinates of the two circle centers (O1, O2) and the radius (r1, r2). The distance between the closing lines is calculated. When the distance is greater than the set threshold ((r1+r2) / 2), it is considered a flash gap anomaly.

[0097] Step 3: The cable laying device S2 receives the overlapping signal and lays the cable in the same direction. When it receives the edge signal, it lays the cable in the opposite direction. Upon feedback from the edge signal, the take-up device S1 performs the automatic reversing function.

[0098] Step 4: After the cable puller S2 completes the cable pulling operation, it returns to its initial position until it receives a normal signal or a reset signal.

[0099] Step 5: When the take-up device S1 receives the flash gap signal, it moves in the opposite direction to reduce the take-up gap until the gap is controlled within the set threshold.

[0100] Example 3:

[0101] This embodiment also provides an electronic device, including: a memory and a processor;

[0102] The memory stores the instructions executed by the computer.

[0103] The processor executes computer execution instructions stored in the memory, causing the processor to execute the automatic wiring method based on real-time 2D visual feedback in any embodiment of the present invention.

[0104] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0105] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory devices, or other volatile solid-state storage devices.

[0106] Example 4:

[0107] This embodiment also provides a computer-readable storage medium storing multiple instructions, which are loaded by a processor to cause the processor to execute the automatic wiring method based on 2D vision real-time feedback in any embodiment of the present invention. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0108] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0109] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0110] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0111] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic wire arranging method based on real-time feedback of 2D vision, characterized in that, The method is as follows: The current winding scene image is acquired by a 2D vision acquisition module; Left and right alternating imaging data are acquired based on the 2D vision acquisition module, a deep vision network V1 is used to perform real-time identification on the scene image, and an image inference result is output in real time, including normal, overlapping line, flash gap, edge and reset; wherein normal and overlapping line are conjugate, and edge and reset are conjugate; The middle imaging data are acquired based on the 2D vision acquisition module, and a deep vision network V2 outputs a target point positioning result in real time, and calculates the winding interval: When the winding interval is greater than half the wire diameter, it is considered as a flash gap; The line arranging device receives the overlapping line signal to arrange the line in the same direction, receives the edge signal to arrange the line in the opposite direction, and simultaneously feeds back the edge signal to the winding device to perform automatic reversing function; wherein the line arranging direction is relative to the moving direction of the winding device; The line arranging device performs the line arranging operation until the reset signal, the normal signal or the reset signal is received, and the position is restored to zero; After the winding device receives the flash gap signal, the winding device moves in the opposite direction to reduce the winding interval until the winding interval is controlled within half the wire diameter; The deep vision network V1 includes a PPLiteSeg segmentation model and a desnet121 classification model, and the details are as follows: The PPLiteSeg segmentation model performs background removal operation on the image to obtain a feature map connected region after background removal; The maximum circumscribed rectangle of the feature map connected region after background removal is reserved to obtain images of different sizes; The images of different sizes are scaled to the length and short axis of the image at a constant ratio, and 0 or 1 is filled in the short axis direction, and finally a fixed size image is formed; The fixed size image is input into the desnet121 classification model to perform four classification inference, wherein the dropblock block is used in the desnet121 classification model to randomly discard a continuous block on the feature map; The deep vision network V2 calculates the winding interval as follows: The deep vision network V2 segments the target line and the reference line to obtain a segmentation result; The segmentation result is subjected to circle fitting processing to obtain two circle center coordinates (O1, O2) and radii (r1, r2), and the winding interval is calculated: When the interval is greater than the set threshold, it is considered as a flash gap; wherein the threshold is (r1+r2) / 2; The deep vision network V2 uses the PPLiteSeg segmentation model.

2. The automatic wire arranging method based on real-time feedback of 2D vision according to claim 1, characterized in that, The winding device is used for winding operation, and the winding device includes a winding support, the winding support is in an I shape, a linear motor is arranged at the bottom of the I-shaped winding support, the linear motor is used to realize the left and right movement of the winding device; a rotating motor is arranged at the middle of the I-shaped winding support, and the rotating motor is used to realize the rotating movement of the winding device.

3. The automatic wire arranging method based on real-time feedback of 2D vision according to claim 1 or 2, characterized in that, The winding device is used for double-degree-of-freedom movement realized in a winding operation; the winding device comprises a winding support composed of a winding base part and two parallel winding support parts in the shape of an inverted T, the center upper part of the winding base part is provided with an X-axis servo motor, the X-axis servo motor is located between the two parallel winding support parts, and the X-axis servo motor is used for left-right movement of the winding support; a Z-axis servo motor is arranged at a lower middle position of any winding support part, and the Z-axis servo motor is used for up-down movement of a 2D vision acquisition module and winding.

4. The automatic wire arranging method based on real-time feedback of 2D vision according to claim 3, characterized in that, The 2D vision acquisition module is used for acquiring image data of a current winding state; the 2D vision acquisition module comprises a left RGB camera, a middle RGB camera and a right RGB camera, and the middle RGB camera is located between the left RGB camera and the right RGB camera; When the left RGB camera, the middle RGB camera and the right RGB camera are collecting, they are all set to a low aperture and an automatic gain mode, so as to adapt to changes of an external environment and acquire images with balanced brightness of a current winding scene.

5. The automatic wire arranging method based on real-time feedback of 2D vision according to claim 4, characterized in that, The acquired four kinds of classified data sets of normal winding, overlapping winding, edge reaching and resetting are subjected to offline data augmentation, the processing mode of offline data augmentation includes random cutting, random rotation and color jittering, and after manual screening, images of a reel area and a cable area are reserved.

6. An automatic wire arranging device based on real-time feedback of 2D vision, characterized in that, The device is used for realizing the automatic winding method based on 2D vision real-time feedback in any one of claims 1 to 5; the device comprises: A 2D vision acquisition module is used for acquiring a scene image of current winding; for left-right alternating imaging data, a deep vision network V1 is used for real-time identification of the scene image and real-time output of image inference results of normal, overlapping winding, flash gap, edge reaching and resetting; wherein normal and overlapping winding are conjugate, and edge reaching and resetting are conjugate; for middle imaging data, a deep vision network V2 outputs a target point positioning result in real time, and calculates a winding interval: when the winding interval is greater than half a wire diameter, it is considered as a flash gap; A winding device is used for receiving an overlapping winding signal to perform same-direction winding, receiving an edge reaching signal to perform reverse winding, and simultaneously performing an automatic reversing function of the winding device under feedback of the edge reaching signal; wherein the winding direction is relative to the movement direction of the winding device; the winding device performs a winding operation until a resetting signal or a normal signal or a resetting signal is received, and returns to a zero position; A winding device is used for reverse movement of the winding device after receiving a flash gap signal, so as to reduce the winding interval until the winding interval is controlled within half a wire diameter.

7. The automatic wire arranging device based on real-time feedback of 2D vision according to claim 6, characterized in that, The winding device is used for winding operation, and the winding device comprises a winding support in the shape of an I-beam, the bottom of the I-beam-shaped winding support is provided with a linear motor, the linear motor is used for left-right movement of the winding device; the middle part of the I-beam-shaped winding support is provided with a rotating motor, and the rotating motor is used for rotating movement of the winding device; The winding device is used for double-degree-of-freedom movement realized in the winding operation; the winding device comprises a winding support which is inverted T-shaped and consists of a winding base part and two parallel arranged winding support parts, the X-axis servo motor is arranged on the upper center of the winding base part, the X-axis servo motor is located between the two parallel arranged winding support parts, and the X-axis servo motor is used for left and right movement of the winding support; the Z-axis servo motor is arranged at the lower middle position of any winding support part, and the Z-axis servo motor is used for up and down movement of the 2D vision acquisition module and the winding; The 2D vision acquisition module is used for acquiring image data of the current winding state; the 2D vision acquisition module comprises a left RGB camera, a middle RGB camera and a right RGB camera, and the middle RGB camera is located between the left RGB camera and the right RGB camera; The left RGB camera, the middle RGB camera and the right RGB camera are all set as low aperture and automatic gain mode when collecting, so as to adapt to the change of external environment and acquire images with balanced brightness of the current winding scene.

8. An electronic device, comprising: Comprise: a memory and at least one processor; wherein the memory has stored thereon a computer program; the at least one processor executes the computer program stored in the memory, so that the at least one processor executes the automatic winding method based on 2D vision real-time feedback as claimed in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored therein a computer program, and the computer program can be executed by the processor to realize the automatic winding method based on 2D vision real-time feedback as claimed in any one of claims 1 to 5.

Images