Bobbin positioning method and device based on binocular image matching and storage medium

Abstract

The invention relates to the technical field of computer vision positioning based on image data processing, and discloses a bobbin positioning method and device based on binocular image matching and a storage medium, the method comprises the following steps: using a binocular camera to obtain left and right view angle images, each image comprising a plurality of bobbins; and pairing each bobbin in the left visual angle image with each bobbin in the right visual angle image to form a plurality of candidate pairing combinations. Aiming at each combination, determining that the two cameras respectively point to the intersection point of the sight lines of the bobbins in the respective images, and calculating the spatial distance between the intersection point and the reference point of the binocular camera; and searching a positioning point of the main bobbin based on spatial distance distribution characteristics and geometric constraints between the intersection point and the origin of the coordinate system of the left and right cameras. According to the process, a statistical law and physical geometric constraint are utilized for screening, and under the condition that multiple targets coexist, view fields are overlapped and features are similar, a main body bobbin is accurately recognized and positioned according to reasonability and consistency of a three-dimensional space.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision positioning technology based on image data processing, and for example to a yarn tube positioning method, apparatus and storage medium based on binocular image matching. Background Technology

[0002] Binocular vision triangulation, a key technology in computer vision, calculates the coordinates of a yarn tube in three-dimensional space using images captured from two different perspectives and based on a stereo matching algorithm. It is widely used in autonomous driving, robot navigation, and industrial inspection. This technology utilizes binocular cameras to simulate the principle of human binocular vision, calculating the three-dimensional coordinates of the yarn tube by matching identical yarn tubes in images captured by the left and right cameras.

[0003] However, in some complex scenarios, the main yarn tube that needs to be located is often densely distributed with other non-main yarn tubes within the same field of view, resulting in both the left and right cameras capturing images containing multiple yarn tubes that look similar and are spatially adjacent. For example, in a spinning scenario, multiple yarn tubes are usually arranged in rows. When a yarn tube breaks, it is necessary to accurately locate the broken yarn tube (i.e., the main yarn tube). However, since all yarn tubes are clearly visible in the left and right view images and their geometric features are highly similar, traditional binocular matching methods have difficulty distinguishing between the main yarn tube and non-main yarn tubes, easily leading to mismatches and making it impossible to accurately and reliably locate the main yarn tube.

[0004] Therefore, how to effectively select the correct main yarn tube matching combination and achieve accurate three-dimensional positioning of the main yarn tube in a complex environment with multiple yarn tubes coexisting, overlapping fields of view and similar characteristics has become an urgent technical problem to be solved. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This disclosure provides a method, apparatus, and storage medium for yarn tube positioning based on binocular image matching, which can accurately identify the main yarn tube and obtain its positioning point in complex environments where multiple yarn tubes coexist, fields of view overlap, and features are similar.

[0007] According to a first aspect of this disclosure, a yarn tube localization method based on binocular image matching is provided, comprising: The left-view image captured by the left camera of the binocular camera and the right-view image captured by the right camera are acquired. Both the left-view image and the right-view image contain multiple yarn tubes that are photographed. The multiple yarn tubes that are photographed are evenly spaced along the preset direction of the spinning equipment. Each yarn tube in the left-view image is paired with each yarn tube in the right-view image to obtain multiple candidate pairing combinations; For each candidate pairing, determine the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculate the spatial distance between the intersection point and the reference point of the binocular camera. Based on the spatial distance distribution characteristics of each candidate pairing combination and the geometric constraint relationship between the intersection point of each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, the positioning point of the main yarn tube corresponding to the binocular camera is searched.

[0008] In some embodiments, based on the distribution characteristics of the spatial distance corresponding to each candidate pairing combination and the geometric constraint relationship between the intersection point corresponding to each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, the positioning point of the main yarn tube corresponding to the binocular camera is searched, including: The positioning point search process involves performing a positioning point search procedure to find the positioning point of the main yarn tube corresponding to the stereo camera. The positioning point search procedure includes the following steps: Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings. The validity of the candidate intersection points is verified based on the geometric constraints between the candidate intersection points and the origins of the coordinate systems of the left and right cameras. If a candidate intersection point is invalid, delete the spatial distance of the candidate pairing combination corresponding to the candidate intersection point, and execute the location point search process again; When a candidate intersection point is valid, the candidate intersection point is determined as the positioning point of the main yarn tube corresponding to the binocular camera, and the positioning point search process ends.

[0009] In some embodiments, determining candidate intersection points from the intersection points corresponding to all candidate pairings based on the distribution characteristics of the spatial distances corresponding to all candidate pairings includes: Sort the spatial distances of all candidate pairings; Select the median spatial distance from the sorted spatial distance results, and determine the intersection point corresponding to the median spatial distance as the candidate intersection point.

[0010] In some embodiments, the spatial distance includes a first distance between the intersection point and the reference point of the binocular camera in a first direction, and a first distance between the intersection point and the reference point of the binocular camera in a second direction, wherein the first direction is parallel to the rehearsal direction of the plurality of photographed yarn tubes, and the second direction is perpendicular to the rehearsal direction of the plurality of photographed yarn tubes. Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings, including: Sort the first distances for all candidate pairings and sort the second distances for all candidate pairings; Select the median of the first distance from the sorted results of the first distance, and select the median of the second distance from the sorted results of the second distance; The intersection point corresponding to the median of the first distance and the intersection point corresponding to the median of the second distance are both determined as candidate intersection points.

[0011] In some embodiments, the validity of the candidate intersection point is verified based on the geometric constraints between the candidate intersection point and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, including: Calculate the distance difference from the candidate intersection point to the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, and determine whether the distance difference is less than the preset distance difference threshold. If the distance difference is not less than the distance difference threshold, the candidate intersection point is invalid. Candidate intersection points are considered valid if the distance difference is less than the distance difference threshold.

[0012] In some embodiments, calculating the distance difference from the candidate intersection point to the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera includes: Calculate the first distance from the candidate intersection point to the origin of the coordinate system of the left camera, and the second distance from the candidate intersection point to the origin of the coordinate system of the right camera; Calculate the distance difference between the first distance and the second distance.

[0013] In some embodiments, the distance difference threshold is determined based on the distance between the origin of the left camera's coordinate system and the origin of the right camera's coordinate system, as well as the focal lengths of the left and right cameras.

[0014] In some embodiments, the reference point of the binocular camera is the midpoint of the line connecting the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

[0015] In some embodiments, each yarn tube in the left-view image is paired with each yarn tube in the right-view image to obtain multiple candidate pairing combinations, including: The overlapping regions of the fields of view were identified in the left-view and right-view images, respectively. The overlapping regions of the fields of view are the areas in the images that overlap with each other in the fields of view of the left and right cameras. Each yarn tube in the overlapping field of view of the left-view image is paired with each yarn tube in the overlapping field of view of the right-view image to obtain multiple candidate pairing combinations.

[0016] According to a second aspect of this disclosure, a yarn tube positioning device based on binocular image matching is provided, including a processor and a memory storing program instructions, wherein the processor executes the yarn tube positioning method based on binocular image matching provided according to a first aspect of this disclosure.

[0017] According to a third aspect of this disclosure, a storage medium is provided that stores computer program instructions, which, when executed by a processor, perform the yarn tube positioning method based on binocular image matching provided according to a first aspect of this disclosure.

[0018] The yarn tube positioning method, apparatus, and storage medium based on binocular image matching provided in this disclosure can achieve the following technical effects: The yarn tube localization method based on binocular image matching provided in this disclosure performs fully connected pairing of each yarn tube in the left-view image with each yarn tube in the right-view image, generating all possible candidate pairing combinations to ensure that no correct matching pairs are missed. Next, for each candidate pairing combination, the spatial distance between its corresponding intersection point and the binocular camera reference point is calculated, thereby transforming the two-dimensional matching problem into a three-dimensional spatial analysis problem. Based on the distribution characteristics of spatial distance and the geometric constraints between the intersection point and the origin of the coordinate systems of the left and right cameras, the method cleverly utilizes the prior knowledge that the main target is located near the center of the baseline field-of-view overlap region to accurately search for the location point of the main yarn tube. This search process combines statistical regularity and physical geometric constraints in its selection strategy, relying not on the appearance characteristics of the yarn tube, but on its rationality and consistency in three-dimensional space. Therefore, it can accurately identify the main yarn tube and obtain its location point in complex environments with multiple yarn tubes coexisting, overlapping fields of view, and similar features.

[0019] The above general description and the description below are exemplary and illustrative only and are not intended to limit this disclosure. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic diagram of a hardware architecture for implementing a yarn tube positioning method based on binocular image matching, provided in an embodiment of this disclosure. Figure 2 This is a schematic diagram of a yarn tube positioning method based on binocular image matching provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of another yarn tube positioning method based on binocular image matching provided in this embodiment of the disclosure; Figure 4This is a schematic diagram of another yarn tube positioning method based on binocular image matching provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram of another yarn tube positioning method based on binocular image matching provided in this embodiment of the disclosure; Figure 6 This is an application example of a yarn tube positioning method based on binocular image matching provided in this disclosure; Figure 7 This is a schematic diagram of a yarn tube positioning device based on binocular image matching provided in an embodiment of this disclosure. Detailed Implementation

[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar yarn tubes and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0023] Unless otherwise stated, the term "multiple" means two or more.

[0024] In this embodiment of the disclosure, the character " / " indicates that the front and rear yarn tubes are in an "or" relationship. For example, A / B means: A or B.

[0025] The term "and / or" describes a relationship in yarn tubes, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0026] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0027] This disclosure provides a yarn tube positioning method and device based on binocular image matching. Figure 1A schematic diagram of the hardware architecture for implementing a yarn tube positioning method based on binocular image matching is shown. In this application scenario, the yarn tube positioning device based on binocular image matching (hereinafter referred to as the positioning device) can communicate with a binocular camera. A binocular camera is an imaging system that mimics human binocular vision. A binocular camera consists of two cameras with fixed spatial positions and parallel or approximately parallel optical axes, referred to as the left camera and the right camera, respectively. For ease of understanding and description, the image captured by the left camera is defined as the left-view image, and the image captured by the right camera is defined as the right-view image.

[0028] This embodiment of the disclosure is applied to a spinning scenario. Both the left-view image captured by the left camera and the right-view image captured by the right camera contain multiple yarn tubes, which are evenly spaced along a preset direction of the spinning equipment. Furthermore, due to the reasonable spatial field-of-view design of the binocular cameras, there is a significant overlap between the fields of view of the left and right cameras, ensuring that at least some yarn tubes are visible in both the left-view and right-view images. These yarn tubes include a main yarn tube to be located and other non-main yarn tubes. The main yarn tube is typically located near the center of the baseline field-of-view overlap area and is visible in both the left-view and right-view images. The main yarn tube is the core focus of the current localization task; for example, in a broken yarn tube identification task in a spinning scenario, the main yarn tube could be the yarn tube where the break occurred.

[0029] In conjunction with the positioning device provided in the embodiments of this disclosure, the embodiments of this disclosure provide a yarn tube positioning method based on binocular image matching, such as... Figure 2 As shown, the yarn tube localization method based on binocular image matching includes the following steps: S201, the positioning device acquires the left-view image captured by the left camera and the right-view image captured by the right camera of the binocular camera.

[0030] S202, the positioning device pairs each yarn tube in the left-view image with each yarn tube in the right-view image to obtain multiple candidate pairing combinations.

[0031] S203, for each candidate pairing, the positioning device determines the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculates the spatial distance between the intersection point and the reference point of the binocular camera.

[0032] In some embodiments, the reference point of the binocular camera can be the midpoint of the line connecting the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

[0033] S204, the positioning device searches for the positioning point of the main yarn tube corresponding to the binocular camera based on the distribution characteristics of the spatial distance corresponding to each candidate pairing combination and the geometric constraint relationship between the intersection point corresponding to each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

[0034] The yarn tube localization method based on binocular image matching provided in this disclosure performs fully connected pairing of each yarn tube in the left-view image with each yarn tube in the right-view image, generating all possible candidate pairing combinations to ensure that no correct matching pairs are missed. Next, for each candidate pairing combination, the spatial distance between its corresponding intersection point and the binocular camera reference point is calculated, thereby transforming the two-dimensional matching problem into a three-dimensional spatial analysis problem. Based on the distribution characteristics of spatial distance and the geometric constraints between the intersection point and the origin of the coordinate systems of the left and right cameras, the method cleverly utilizes the prior knowledge that the main target is located near the center of the baseline field-of-view overlap region to accurately search for the location point of the main yarn tube. This search process combines statistical regularity and physical geometric constraints in its selection strategy, relying not on the appearance characteristics of the yarn tube, but on its rationality and consistency in three-dimensional space. Therefore, it can accurately identify the main yarn tube and obtain its location point in complex environments with multiple yarn tubes coexisting, overlapping fields of view, and similar features.

[0035] In some embodiments, the location point of the main yarn tube corresponding to the binocular camera is searched based on the distribution characteristics of the spatial distance corresponding to each candidate pairing combination and the geometric constraint relationship between the intersection point corresponding to each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera. This includes: searching for the location point of the main yarn tube corresponding to the binocular camera by executing a location point search process, wherein the location point search process includes the following steps: determining candidate intersection points from the intersection points corresponding to all candidate pairing combinations based on the distribution characteristics of the spatial distance corresponding to all candidate pairing combinations; verifying the validity of the candidate intersection points based on the geometric constraint relationship between the candidate intersection points and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera; deleting the spatial distance of the candidate pairing combination corresponding to the candidate intersection point when the candidate intersection point is invalid, and executing the location point search process again when the candidate intersection point is valid; determining the candidate intersection point as the location point of the main yarn tube corresponding to the binocular camera when the candidate intersection point is valid, and ending the location point search process.

[0036] The localization point search process first determines candidate intersection points based on the spatial distance distribution characteristics of candidate pairings, fully utilizing the prior knowledge that the main yarn tube is usually located in the center of the field of view, which improves the reliability of the finally selected intersection points. Subsequently, the validity is verified based on the geometric constraints between the candidate intersection points and the origins of the left and right camera coordinate systems, ensuring that the finally selected intersection points meet the stereo consistency conditions of the binocular system and effectively eliminating spatial anomalies caused by incorrect pairings. When a candidate intersection point does not meet the geometric constraints, the spatial distance corresponding to the candidate intersection point is deleted iteratively, and the localization point search process is re-executed, enhancing the algorithm's robustness and fault tolerance. The localization point search process does not rely on the target's appearance features, but only on the rationality and consistency of three-dimensional space. In complex scenarios with multiple coexisting targets and similar features, it can significantly reduce the false matching rate, improve localization accuracy and stability, and ultimately achieve reliable identification and precise localization of the main yarn tube.

[0037] like Figure 3 As shown, this disclosure provides another yarn tube positioning method based on binocular image matching. The yarn tube positioning method based on binocular image matching includes the following steps: S301, the positioning device acquires the left-view image captured by the left camera and the right-view image captured by the right camera of the binocular camera.

[0038] S302, the positioning device pairs each yarn tube in the left-view image with each yarn tube in the right-view image to obtain multiple candidate pairing combinations.

[0039] S303, for each candidate pairing, the positioning device determines the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculates the spatial distance between the intersection point and the reference point of the binocular camera.

[0040] Following S303, the positioning device searches for the positioning point of the main yarn tube corresponding to the binocular camera by executing a positioning point search procedure. This positioning point search procedure includes steps S304 to S307.

[0041] S304, the positioning device determines the candidate intersection point from the intersection points corresponding to all candidate pairing combinations based on the distribution characteristics of the spatial distances corresponding to all candidate pairing combinations.

[0042] S305, the positioning device verifies the validity of the candidate intersection point based on the geometric constraint relationship between the candidate intersection point and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

[0043] S306, when the candidate intersection point is invalid, the positioning device deletes the spatial distance of the candidate pairing combination corresponding to the candidate intersection point and executes the positioning point search process again.

[0044] S307, when the candidate intersection point is valid, the positioning device determines the candidate intersection point as the positioning point of the main yarn tube corresponding to the binocular camera and ends the positioning point search process.

[0045] In some embodiments, determining candidate intersection points from the intersection points corresponding to all candidate pairings based on the distribution characteristics of the spatial distances corresponding to all candidate pairings includes: sorting the spatial distances corresponding to all candidate pairings; selecting the median of the spatial distances from the sorting results; and determining the intersection point corresponding to the median of the spatial distances as a candidate intersection point.

[0046] Since the main yarn tube is usually located near the center of the overlapping area of ​​the binocular camera's baseline field of view, the spatial distance between the candidate pairings formed by the main yarn tube in the left-view image and the corresponding main yarn tube in the right-view image is likely to be in the middle range among all spatial distances. Therefore, determining the intersection point corresponding to the median spatial distance as the candidate intersection point can effectively utilize this prior knowledge of spatial distribution and more quickly filter out the candidate intersection points corresponding to the candidate pairings of two main yarn tubes. This strategy helps to efficiently and accurately locate the candidate pairings of two main yarn tubes in complex multi-object scenes, thereby obtaining the location point of the main yarn tube more reliably.

[0047] In some embodiments, the spatial distance includes a first distance between the intersection point and the reference point of the binocular camera in a first direction, and a first distance between the intersection point and the reference point of the binocular camera in a second direction, wherein the first direction is parallel to the rehearsal direction of the plurality of photographed yarn tubes, and the second direction is perpendicular to the rehearsal direction of the plurality of photographed yarn tubes.

[0048] Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings. This includes: sorting the first distances corresponding to all candidate pairings and sorting the second distances corresponding to all candidate pairings; selecting the median of the first distance from the sorted first distances and the median of the second distance from the sorted second distances; and determining the intersection points corresponding to the medians of the first and second distances as candidate intersection points. Here, by sorting the distances in the first and second directions separately and selecting the intersection points corresponding to their respective medians as candidate intersection points, it is possible to avoid omitting candidate intersection points that meet the requirements.

[0049] In some embodiments, the validity of a candidate intersection point is verified based on the geometric constraints between the candidate intersection point and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera. This includes: calculating the distance difference between the candidate intersection point and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, and determining whether the distance difference is less than a preset distance difference threshold; if the distance difference is not less than the distance difference threshold, the candidate intersection point is determined to be invalid; if the distance difference is less than the distance difference threshold, the candidate intersection point is determined to be valid.

[0050] Since the main yarn tube is usually located near the center of the overlapping area of ​​the binocular camera's baseline field of view, the intersection point formed by the line of sight from the left camera to the main yarn tube in the left-view image and the line of sight from the right camera to the main yarn tube in the right-view image is also highly likely to be near the center of the overlapping area. In this case, the distance difference from this intersection point to the origin of the coordinate system of the left and right cameras is extremely small, even approaching zero under an ideal symmetrical configuration. Based on this geometric characteristic, by setting a reasonable distance difference threshold, candidate intersection points are determined to be valid when the distance difference is less than the threshold. That is, the candidate intersection point is determined to be the intersection point of the line of sight from the left camera to the main yarn tube in the left-view image and the line of sight from the right camera to the main yarn tube in the right-view image. This verification method makes full use of the prior spatial distribution of the main target and the physical imaging law, significantly improving the accuracy and reliability of intersection point selection.

[0051] In some embodiments, the distance difference threshold is determined based on the distance between the origin of the left camera's coordinate system and the origin of the right camera's coordinate system, as well as the focal lengths of the left and right cameras.

[0052] like Figure 4 As shown, this disclosure provides another yarn tube positioning method based on binocular image matching. The yarn tube positioning method based on binocular image matching includes the following steps: S401, the positioning device acquires the left-view image captured by the left camera and the right-view image captured by the right camera of the binocular camera.

[0053] S402, the positioning device pairs each yarn tube in the left-view image with each yarn tube in the right-view image to obtain multiple candidate pairing combinations.

[0054] S403, for each candidate pairing, the positioning device determines the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculates the spatial distance between the intersection point and the reference point of the binocular camera.

[0055] Following S403, the positioning device searches for the positioning point of the main yarn tube corresponding to the binocular camera by executing a positioning point search procedure. This positioning point search procedure includes steps S404 to S408.

[0056] S404, The positioning device sorts the spatial distances corresponding to all candidate pairing combinations.

[0057] S405, the positioning device selects the median of spatial distance from the sorting results of spatial distances, and determines the intersection point corresponding to the median of spatial distance as a candidate intersection point.

[0058] S406, the positioning device calculates the distance difference from the candidate intersection point to the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, and determines whether the distance difference is less than the preset distance difference threshold.

[0059] Here, we can calculate the first distance from the candidate intersection point to the origin of the left camera's coordinate system, and the second distance from the candidate intersection point to the origin of the right camera's coordinate system. Then, we can calculate the distance difference between the first and second distances to obtain the distance difference between the candidate intersection point and the left and right cameras.

[0060] S407, if the distance difference is not less than the distance difference threshold, the positioning device determines that the candidate intersection point is invalid, deletes the current median spatial distance, and executes the positioning point search process again.

[0061] S408, if the distance difference is less than the distance difference threshold, the positioning device determines that the candidate intersection point is valid, identifies the candidate intersection point as the positioning point of the main yarn tube corresponding to the binocular camera, and ends the positioning point search process.

[0062] In some embodiments, each yarn tube in the left-view image is paired with each yarn tube in the right-view image to obtain multiple candidate pairing combinations, including: identifying the field-of-view overlap region in the left-view image and the right-view image respectively, wherein the field-of-view overlap region is the region in the image that overlaps with the field of view of the left camera and the right camera; and pairing each yarn tube in the field-of-view overlap region of the left-view image with each yarn tube in the field-of-view overlap region of the right-view image to obtain multiple candidate pairing combinations.

[0063] The embodiments disclosed herein only pair yarn tubes in the overlapping areas of the left and right viewpoint images, which can effectively limit the matching search range, exclude invalid yarn tubes located in a single viewpoint, significantly reduce the number of pairing combinations, reduce computational redundancy, and improve matching efficiency.

[0064] like Figure 5 As shown, this disclosure provides another yarn tube positioning method based on binocular image matching. The yarn tube positioning method based on binocular image matching includes the following steps: S501, the positioning device acquires the left-view image captured by the left camera and the right-view image captured by the right camera of the binocular camera.

[0065] S502, the positioning device identifies the overlapping area of ​​the field of view in the left-view image and the right-view image respectively.

[0066] Among them, the field of view overlap region is the region in the image that overlaps with the field of view of the left camera and the right camera; S503, the positioning device pairs each yarn tube in the overlapping area of ​​the field of view of the left-view image with each yarn tube in the overlapping area of ​​the field of view of the right-view image to obtain multiple candidate pairing combinations.

[0067] S504, for each candidate pairing, the positioning device determines the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculates the spatial distance between the intersection point and the reference point of the binocular camera.

[0068] S505, the positioning device searches for the positioning point of the main yarn tube corresponding to the binocular camera based on the distribution characteristics of the spatial distance corresponding to each candidate pairing combination and the geometric constraint relationship between the intersection point corresponding to each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

[0069] like Figure 5 As shown, a designated area includes yarn tubes a, b, c, d, and e. The image P is a left-view image of the designated area captured by the left camera. L The image includes yarn tubes a, b, c, and d, and is a right-view image P captured by the right camera of the designated area. R The image includes yarn tubes b, c, d, and e. The overlapping fields of view in both the left and right view images include yarn tubes b, c, and d. Therefore, it is necessary to pair yarn tubes b, c, and d in the left view image with those in the right view image, resulting in nine candidate pairings.

[0070] O L O represents the origin of the coordinate system for the left camera. R O represents the origin of the coordinate system for the right camera, and M represents the reference point for the stereo camera. From the origin O... L The ray drawn out is the line of sight of the corresponding yarn tube of the left camera; originating from the coordinate system O. R The ray drawn out is the line of sight of the corresponding yarn tube of the right camera. From the origin O of the coordinate system... L The ray drawn out from the origin O of the coordinate system RThe focal point of the derived ray is the intersection of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image. For example, point Q is the intersection of the line of sight from the left camera to the yarn tube b in the left-view image and the line of sight from the right camera to the yarn tube b in the right-view image; point P is the intersection of the line of sight from the left camera to the yarn tube c in the left-view image and the line of sight from the right camera to the yarn tube c in the right-view image; and point G is the intersection of the line of sight from the left camera to the yarn tube b in the left-view image and the line of sight from the right camera to the yarn tube d in the right-view image. The line connecting the reference point and the corresponding intersection point represents the spatial distance between the intersection point and the reference point of the binocular cameras. Here, yarn tube c is the main target pair, so the intersection point of the candidate pairing of two yarn tubes c will ultimately be determined as the positioning point of the main yarn tube corresponding to the binocular cameras.

[0071] Combination Figure 7 As shown, this embodiment of the present disclosure provides a yarn tube positioning device 700 based on binocular image matching, including a processor 701 and a memory 702. Optionally, the yarn tube positioning device 700 based on binocular image matching may further include a communication interface 703 and a bus 704. The processor 701, communication interface 703, and memory 702 can communicate with each other via the bus 704. The communication interface 703 can be used for information transmission. The processor 701 can call logical instructions in the memory 702 to execute the yarn tube positioning method based on binocular image matching described in the above embodiment.

[0072] Furthermore, the logic instructions in the aforementioned memory 702 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0073] The memory 702, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 701 executes functional applications and data processing by running the program instructions / modules stored in the memory 702, that is, it implements the yarn tube positioning method based on binocular image matching in the above embodiments.

[0074] The memory 702 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 702 may include high-speed random access memory and may also include non-volatile memory.

[0075] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described yarn tube positioning method based on binocular image matching.

[0076] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code. The above description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Furthermore, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed any and all possible combinations. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0077] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0078] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A yarn tube localization method based on binocular image matching, characterized in that, include: The left-view image captured by the left camera of the binocular camera and the right-view image captured by the right camera are acquired. Both the left-view image and the right-view image contain multiple yarn tubes that are photographed. The multiple yarn tubes that are photographed are evenly spaced along the preset direction of the spinning equipment. Each yarn tube in the left-view image is paired with each yarn tube in the right-view image to obtain multiple candidate pairing combinations; For each candidate pairing, determine the intersection point of the line of sight from the left camera to the yarn tube in the left-view image and the line of sight from the right camera to the yarn tube in the right-view image, and calculate the spatial distance between the intersection point and the reference point of the binocular camera. Based on the spatial distance distribution characteristics of each candidate pairing combination and the geometric constraint relationship between the intersection point of each candidate pairing combination and the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, the positioning point of the main yarn tube corresponding to the binocular camera is searched.

2. The yarn tube positioning method based on binocular image matching according to claim 1, characterized in that, Based on the spatial distance distribution characteristics of each candidate pairing combination and the geometric constraints between the intersection point of each candidate pairing combination and the origins of the coordinate systems of the left and right cameras, the positioning points of the main yarn tube corresponding to the binocular camera are searched, including: The positioning point search process involves performing a positioning point search procedure to find the positioning point of the main yarn tube corresponding to the stereo camera. The positioning point search procedure includes the following steps: Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings. The validity of the candidate intersection points is verified based on the geometric constraints between the candidate intersection points and the origins of the coordinate systems of the left and right cameras. If a candidate intersection point is invalid, delete the spatial distance of the candidate pairing combination corresponding to the candidate intersection point, and execute the location point search process again; When a candidate intersection point is valid, the candidate intersection point is determined as the positioning point of the main yarn tube corresponding to the binocular camera, and the positioning point search process ends.

3. The yarn tube positioning method based on binocular image matching according to claim 2, characterized in that, Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings, including: Sort the spatial distances of all candidate pairings; Select the median spatial distance from the sorted spatial distance results, and determine the intersection point corresponding to the median spatial distance as the candidate intersection point.

4. The method for determining the symmetry axis of a yarn tube based on a binocular camera according to claim 2, characterized in that, The spatial distance includes a first distance between the intersection point and the reference point of the binocular camera in a first direction, and a first distance between the intersection point and the reference point of the binocular camera in a second direction, wherein the first direction is parallel to the rehearsal direction of the multiple yarn tubes being photographed, and the second direction is perpendicular to the rehearsal direction of the multiple yarn tubes being photographed. Based on the distribution characteristics of the spatial distances corresponding to all candidate pairings, candidate intersection points are determined from the intersection points corresponding to all candidate pairings, including: Sort the first distances corresponding to all candidate pairings and sort the second distances corresponding to all candidate pairings; Select the median of the first distance from the sorted results of the first distance, and select the median of the second distance from the sorted results of the second distance; The intersection point corresponding to the median of the first distance and the intersection point corresponding to the median of the second distance are both determined as candidate intersection points.

5. The yarn tube positioning method based on binocular image matching according to claim 2, characterized in that, The validity of the candidate intersection points is verified based on the geometric constraints between the candidate intersection points and the origins of the coordinate systems of the left and right cameras, including: Calculate the distance difference from the candidate intersection point to the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, and determine whether the distance difference is less than the preset distance difference threshold. If the distance difference is not less than the distance difference threshold, the candidate intersection point is invalid. Candidate intersection points are considered valid if the distance difference is less than the distance difference threshold.

6. The yarn tube positioning method based on binocular image matching according to claim 5, characterized in that, Calculate the distance difference from the candidate intersection point to the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera, including: Calculate the first distance from the candidate intersection point to the origin of the coordinate system of the left camera, and the second distance from the candidate intersection point to the origin of the coordinate system of the right camera; Calculate the distance difference between the first distance and the second distance.

7. The yarn tube positioning method based on binocular image matching according to claim 5, characterized in that, The distance difference threshold is determined based on the distance between the origin of the left camera's coordinate system and the origin of the right camera's coordinate system, as well as the focal lengths of the left and right cameras.

8. The yarn tube positioning method based on binocular image matching according to claim 1, characterized in that, The reference point of a binocular camera is the midpoint of the line connecting the origin of the coordinate system of the left camera and the origin of the coordinate system of the right camera.

9. The yarn tube positioning method based on binocular image matching according to claim 1, characterized in that, Each yarn tube in the left-view image is paired with each yarn tube in the right-view image to obtain multiple candidate pairing combinations, including: The overlapping regions of the fields of view are identified in the left-view and right-view images, respectively. The overlapping regions of the fields of view are the areas in the images that overlap with each other in the fields of view of the left and right cameras. Each yarn tube in the overlapping field of view of the left-view image is paired with each yarn tube in the overlapping field of view of the right-view image to obtain multiple candidate pairing combinations.

10. A yarn tube positioning device based on binocular image matching, comprising a processor and a memory storing program instructions, characterized in that, The processor executes the yarn tube positioning method based on binocular image matching as described in any one of claims 1 to 9.

11. A storage medium, characterized in that, The storage medium stores computer program instructions, which, when executed by a processor, perform the yarn tube positioning method based on binocular image matching as described in any one of claims 1 to 9.

Images