Cosine similarity-based multi-criteria visual recognition and three-dimensional positioning method for container locking pins

Abstract

The application discloses a kind of container locking pin multi-standard visual identification and three-dimensional positioning method based on cosine similarity, belong to computer vision and industrial detection field.Method includes: constructing the locking pin feature library covering multiple international standards, each locking pin type corresponds a high-dimensional feature vector and standard identification;Get the visual image of the lock pin to be identified, extract high-dimensional feature vector by convolutional neural network;Calculate the cosine similarity between the extracted feature vector and the feature vector of each locking pin type in the feature library;All cosine similarities are sorted by size, and the top K candidate locking pin types with the highest similarity are output as the identification result;Three-dimensional space position calculation is carried out on the identified lock pin, and its three-dimensional coordinates in the space coordinate system are output.The feature library of the application is organized according to international standards, locking pin categories, variant versions three-dimensional index, taking into account identification coverage and retrieval efficiency, three-dimensional positioning and identification process decoupling, supporting independent upgrade.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and industrial inspection, specifically relating to a method and system for multi-standard visual recognition and three-dimensional positioning of container lock pins based on cosine similarity. Background Technology

[0002] Container locking pins are key mechanical components connecting containers to vehicle chassis, and different countries and regions follow different international standards for their specifications. There are many types of locking pins used in container shipping, including standards such as ISO 1161, JIS Z1615, and GB / T1835. Each standard has different categories such as standard locking pins, semi-automatic locking pins, and fully automatic locking pins, as well as various variant versions. In automated unloading and transfer scenarios, robots need to accurately identify the type of locking pin and determine its three-dimensional spatial position. Existing technical solution 1: Lock pin recognition method based on template matching. This method pre-stores standard template images for each type of lock pin and performs template matching with the image to be detected through a sliding window. Its drawbacks are: template matching is sensitive to changes in lighting, shooting angle deviations, and local occlusion. When the lock pin surface is corroded, oily, or the lighting is uneven, the matching score drops significantly, leading to misidentification; moreover, the number of templates increases linearly with the number of lock pin types, and when dozens of lock pin types are covered, the matching time increases significantly, which is not conducive to real-time applications. Existing technical solution two: Single-standard lock pin identification based on classification networks. This method trains a classification neural network model to distinguish lock pin types, but it is usually trained only on a single standard (such as ISO 1161). Its drawbacks are: when encountering lock pin types belonging to other standards, the model cannot accurately identify them, requiring re-collection of data and retraining of the entire model; the model lacks cross-standard generalization ability, the recognition coverage is limited by the training data, and the cost of adding new lock pin types is high. Existing technical solution three: a serial method of identification followed by positioning. This method first identifies the lock type, and then switches to an independent positioning process to calculate the 3D position. Its drawbacks are: there is a strict data dependency between the two processes; if the identification process fails or times out, the positioning process will be completely unable to execute, and the system lacks fault tolerance; moreover, the two processes cannot be upgraded and iterated independently. The main technical problems existing in the prior art are: when the types of locking pins span multiple international standards and are numerous, how to achieve efficient retrieval and matching while maintaining high recognition coverage; and the problem that the tight coupling between visual recognition and three-dimensional positioning processes can lead to a failure in any link affecting the whole. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-standard visual recognition and 3D positioning method for container lock pins based on cosine similarity, which solves the problems of existing technologies in achieving both coverage and efficiency in multi-standard lock pin recognition scenarios, as well as the problem of overly tight coupling between visual recognition and 3D positioning processes. The technical solution adopted in this invention is as follows: A high-dimensional feature vector library is constructed, organized according to international standards, lock pin categories, and variant versions using a three-dimensional index. A convolutional neural network is used to extract the high-dimensional feature vector of the lock pin to be identified. The cosine similarity between this vector and all vectors in the feature library is calculated and ranked, and the top K candidate results are selected. For the top-ranked candidate type, its three-dimensional spatial position is independently calculated using triangulation, combined with laser ranging data. This solution decouples the feature library query process from the three-dimensional positioning process, ensuring that the positioning calculation does not depend on the intermediate states of the feature matching process. The beneficial effects of this invention include: First, the feature library adopts a three-dimensional index structure, and the retrieval time increases logarithmically rather than linearly with the number of lock types, maintaining high retrieval efficiency even when covering more than nine categories of three international standards; Second, cosine similarity is insensitive to differences in the magnitude of feature vectors. When image acquisition conditions change (such as illumination fluctuations or occlusion), the magnitude of feature vectors may change, but the direction remains relatively stable, making the recognition results more robust than Euclidean distance matching; Third, the three-dimensional positioning and visual recognition processes are decoupled. When the confidence of the recognition result is insufficient or the laser data is unavailable, both processes can run independently and output their respective usable data, improving the system's fault tolerance and maintainability. Attached Figure Description Figure 1 This is a schematic diagram of the complete identification and positioning pipeline process in an embodiment of the present invention.

[0004] Figure 2 This is a schematic diagram of the three-dimensional index architecture of the feature library in an embodiment of the present invention.

[0005] Figure 3 This is a schematic diagram of vision and laser fusion positioning in an embodiment of the present invention. The meanings of the markings in the diagram are as follows: 100 series ( Figure 1 ): 101- Construct a lock pin feature library, 102- Obtain visual images of lock pins to be identified, 103- Extract high-dimensional feature vectors using a convolutional neural network, 104- Calculate the cosine similarity with all vectors in the feature library, 105- Sort by similarity from largest to smallest, 106- Similarity greater than a threshold, 107- Output the top K candidate lock pin types, 108- Combine with laser ranging data, 109- Calculate the three-dimensional spatial position using triangulation, 110- Output the recognition result and three-dimensional coordinates, 111- Generate an attention heatmap, 112- Return the unknown lock pin type; 200 series ( Figure 2): 201 to 218 - Feature library storage units, each number corresponds to a lock-type feature vector and standard identifier entry, 206 - Cosine similarity calculation, 207 - Sort and output the top K candidate results; 300 series ( Figure 3 ): 301-Industrial camera and imaging geometry, 302-Laser rangefinder, 303-Triangulation module, 304-3D spatial coordinate output, 305-Feature point and depth point registration, 306-Imaging geometry model, 307-Depth constraint model, 308-Container lock pin image, 309-Laser ranging data. Figure 2 The meanings of the markings are as follows: 1-ISO 1161, 2-JIS Z1615, 3-GB / T 1835, 1-Standard locking pin, 2-Semi-automatic locking pin, 3-Fully automatic locking pin, 1-Variant A, 2-Variant B, 201-Feature vector + standard identifier, 202-Feature vector + standard identifier, 203-Feature vector + standard identifier, 204-Feature vector + standard identifier, 205-Feature vector + standard identifier, 206-Cosine similarity calculation, 207-Sorting and outputting the top K candidates, 208-Feature vector to be identified. Figure 3 The meanings of the markings are as follows: 301-301-Industrial camera imaging geometry, 302-302-Laser rangefinder TOF depth value d, 303-303-Triangulation method back projection calculation, 304-304-Three-dimensional spatial coordinates (Xc,Yc,Zc), 305-305-Feature point and depth point registration, 306-306-Imaging geometry model, 307-307-Depth constraint model, 308-308-Container lock pin image, 309-309-Laser rangefinder data. Detailed Implementation To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This invention provides a multi-standard visual recognition and 3D positioning method for container locking pins based on cosine similarity, enabling a complete recognition and positioning pipeline such as... Figure 1 As shown. like Figure 1 As shown in marker 101, the lock pin feature library is first constructed. The three-dimensional index architecture of the feature library is as follows: Figure 2 As shown, the feature library is organized according to three dimensions: international standards, lock pin categories, and variant versions. Figure 2The labels 201 to 218 correspond to storage units for each type of latch in the feature library. Each storage unit contains a high-dimensional feature vector and corresponding standard identification information. The international standards covered by the feature library include ISO 1161, JIS Z1615, and GB / T 1835. Each standard is further subdivided into three categories: standard latches, semi-automatic latches, and fully automatic latches. Each category contains multiple variant versions, and each variant version corresponds to a storage unit numbered from 201 to 218. like Figure 1 As shown in label 102, images of container latches are captured from an industrial camera. The intrinsic parameter matrix and distortion coefficients of the industrial camera have been pre-calibrated. Figure 3 As shown in label 301, the industrial camera acquires a visual image of the area to be identified, and its imaging geometry (including focal length, principal point coordinates and distortion model) has been pre-calibrated and stored in the system configuration. like Figure 1 As shown in label 103, the acquired image is input into a convolutional neural network model, and the high-dimensional feature vector of the lock to be identified is extracted through the output layer of the model. In a specific embodiment, the feature vector has a dimension of 512, and the value of each component is normalized to between -1 and 1. like Figure 1 As shown in marker 104, the cosine similarity is calculated one by one between the high-dimensional feature vector extracted in step 103 and the high-dimensional feature vector corresponding to each storage unit in the feature library. For example... Figure 2 As shown in the figure 206, the cosine similarity is the dot product of two vectors divided by the product of their magnitudes. The mathematical expression is cos(θ)=(A·B) / (|A|×|B|). The calculation result is between -1 and 1. The closer it is to 1, the more consistent the directions of the two feature vectors are. like Figure 1 As shown in marks 105 and 106, the cosine similarity scores for all lock pin types are sorted from largest to smallest, and it is determined whether the highest similarity score is greater than a preset threshold. The preset threshold can be adjusted according to the actual deployment scenario, with a typical value of 0.7. If the highest similarity score is greater than a preset threshold, such as Figure 1 As shown in marker 107, the top K lock pin types with the highest similarity are output as multi-candidate recognition results, where K is a positive integer. Meanwhile, as... Figure 1 As shown in Figure 111, a heatmap of the convolutional neural network's attention is generated. The heatmap uses a Gaussian kernel function to produce a gradually changing pixel-level floating-point matrix within the lock detection region, used to visualize the image region of interest to the model, facilitating manual review by operators. If all similarities do not meet the threshold condition, such as... Figure 1 As indicated by mark 112, an unknown lock pin type is returned, and the operator is notified to confirm manually. like Figure 1 As shown in marks 108 and 109, for the lock pin type ranked first among multiple candidate recognition results, its three-dimensional spatial position in the camera coordinate system is independently calculated using triangulation, combined with laser ranging data. Figure 3 As shown, the industrial camera marked 301 provides the imaging geometry, including the pixel coordinates of the feature points on the locking pin surface in the image; the laser rangefinder marked 302 provides the depth value from the camera to the locking pin surface using the time-of-flight method; the data from both are merged into the triangulation module marked 303. Mark 305 describes the registration process for identifying feature points and depth points, ensuring that the visual feature points and laser rangefinder points point to the same physical location on the locking pin surface. Marks 306 and 307 describe the imaging geometry model and the depth constraint model, respectively. The triangulation method uses the intersection of the back-projection ray from the camera's optical center to the pixel and the depth constraint plane provided by the laser rangefinder to determine the three-dimensional spatial position of the locking pin center in the camera coordinate system, ultimately outputting as shown. Figure 3 The three-dimensional coordinates are indicated by the mark 304. like Figure 1 As shown in the middle label 110, the system finally outputs a complete identification result including the top-ranked lock pin type identifier, confidence level, and three-dimensional coordinates. It also outputs a complete list of K candidate types for the downstream task scheduling module to make a decision. It should be noted that the calculation of 3D spatial position is independent of the feature matching process. When the lock pin identification result is unavailable, the positioning module can still independently output 3D coordinates by combining the coarse lock pin area information; when the 3D positioning data is unavailable, the identification module can still independently output the lock pin type identification result. This decoupled design between the two makes the system more fault-tolerant. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for multi-standard visual recognition and 3D localization of container locking pins based on cosine similarity, characterized in that, include: Construct a lock pin feature library, which is organized according to three dimensions: the international standard to which the lock pin belongs, the lock pin category, and the variant version. Each lock pin type is mapped to a high-dimensional feature vector. Extract the lock pin region to be identified from the container lock pin images captured by industrial cameras, and obtain the high-dimensional feature vector of the lock pin to be identified through the output layer of the convolutional neural network model; The high-dimensional feature vector of the lock pin to be identified is compared with the high-dimensional feature vector of each lock pin type in the feature library. The cosine similarity is calculated by dividing the dot product of the two vectors by the product of their magnitudes. The cosine similarity of all lock pin types is sorted from largest to smallest, and the top K lock pin types with the highest similarity are selected as the multi-candidate recognition results, where K is a positive integer; For the lock pin type ranked first among the multiple candidate recognition results, its three-dimensional spatial position in the camera coordinate system is independently calculated by triangulation method in combination with laser ranging data. The calculation of the three-dimensional spatial position is independent of the feature matching process. The output includes the recognition result containing the top-ranked lock type, confidence score, and 3D coordinates, as well as a complete list of K candidate types; The method is executed by a computing device.

2. The method according to claim 1, characterized in that, The feature library covers international standards including ISO1161, JIS Z1615 and GB / T 1835. Each standard is further subdivided into standard locking pins, semi-automatic locking pins and fully automatic locking pins, and each category contains multiple variant versions.

3. The method according to claim 1, characterized in that, The high-dimensional feature vector has a dimension of 512, and the value range of each component of the feature vector is normalized to between -1 and 1.

4. The method according to claim 1, characterized in that, The calculation of the three-dimensional spatial position is independent of the feature matching process. It uses the imaging geometry inside the camera and the laser ranging value to calculate the three-dimensional coordinates of the locking pin center in space through triangulation.

5. The method according to claim 1, characterized in that, While outputting the recognition results, a convolutional neural network attention heatmap is generated. The heatmap generates a gradually changing pixel-level floating-point matrix in the lock detection region using a Gaussian kernel function.

6. A multi-standard visual recognition and three-dimensional positioning system for container locking pins based on cosine similarity, characterized in that, include: The feature library management module is used to build and maintain a high-dimensional feature vector library indexed by international standards, lock categories, and variant versions in three dimensions. The feature extraction module is used to extract high-dimensional feature vectors of the lock pins to be identified from camera images using a convolutional neural network. The similarity matching module is used to calculate the cosine similarity between the feature vector to be identified and all vectors in the feature library, and outputs the top K candidate results sorted from high to low similarity. The three-dimensional positioning module is used to calculate the three-dimensional spatial coordinates of the locking pin by combining laser ranging data; The results output module is used to assemble and output a complete recognition data package containing the preferred recognition results, candidate list, confidence score, heatmap data, and three-dimensional coordinates.

7. The system according to claim 6, characterized in that, The feature library management module supports adding new lock types at runtime without retraining the convolutional neural network model. Adding a new lock only requires extracting its feature vector and registering the standard, category, and variant version information.

8. The system according to claim 6, characterized in that, The three-dimensional positioning module still outputs three-dimensional coordinates independently when the lock pin recognition result is unavailable, and the feature extraction module still outputs recognition results independently when the three-dimensional positioning data is unavailable.

9. A computer-readable storage medium comprising executable instructions that, when executed by a processor, implement the method of any one of claims 1 to 5.

Images