Method for identifying and positioning a rack based on 3D vision

Abstract

The present application relates to the field of visual recognition, especially to a rack identification and positioning method based on 3D vision, which realizes accurate identification and positioning of the rack by utilizing 3D point cloud data, can accurately detect the existence, absence, position or attitude anomaly of the material, and the integrity of the rack structure, through steps such as spatial region division, spatial mutation feature recognition, and identification cycle analysis, can quickly locate the potential abnormal area and the material state, improve the efficiency and accuracy of rack management, at the same time, through multiple identification and review scanning, the reliability of the identification result is ensured, and misjudgment is avoided, in addition, through automatic triggering of system calibration process, the situation of point cloud registration anomaly or rack model misalignment is coped with, the robustness and stability are further improved, the automation level of rack management is effectively improved, the artificial intervention cost is reduced, and the production efficiency and quality control ability are improved.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition, and in particular to a material rack recognition and positioning method based on 3D vision. Background Art

[0002] In the fields of industrial production, warehousing and logistics, the management of racks and the precise positioning of materials are key links in improving production efficiency and quality control. With the development of 3D vision technology, the use of 3D point cloud data for rack identification and positioning has become a new solution. However, existing 3D vision technology still has some shortcomings in rack identification, such as low accuracy in rack identification in complex environments, difficulty in quickly locating abnormal materials and partitions, and limited ability to detect the structural integrity of racks. Therefore, the development of a rack identification and positioning method based on 3D vision that can accurately identify the status of racks and materials while possessing efficient, reliable and adaptive calibration capabilities is of great significance for improving the level of automation in rack management.

[0003] Chinese Patent Authorization Announcement No. CN117745806B discloses a multi-camera visual recognition and positioning method, which includes the following steps: setting a turret positioning camera to measure and record the position deviation of all nozzles on the turret mechanism; setting a pin recognition camera to calibrate the coordinates of the pins to facilitate the nozzles on the turret mechanism to pick up the chips placed on the pins; setting a chip recognition camera to detect whether there is a chip on the nozzle of the turret mechanism and calculate the deviation of the chip center point; setting a gantry correction camera to calculate the offset of the chip center point on the gantry mechanism; setting a bonding camera to obtain bonding position information for chip mounting. This invention sets corresponding recognition cameras at each key chip transfer link to monitor and record the position of the chip, thereby providing real-time feedback and control, improving the success rate and stability of chip transfer and mounting.

[0004] Chinese patent application publication number CN111310726A discloses a method for visually identifying and locating ground wires within data cables. The method includes steps S1: rotating the wire bundle within an aluminum foil layer counter-rotatingly along the winding direction and pressing it into a lantern shape; S2: photographing the wire bundle to obtain a data line image and extracting H, S, and V channel images from the image; S3: extracting the significant result image of the blue signal line from the H channel image and performing threshold segmentation to obtain an H channel binary image; S4: using the H channel binary image to extract the blue signal line image from the V channel image and finding the coordinate point P with the maximum amplitude; S5: calculating the left and right positions of the effective region in the blue signal line image to obtain a V channel ROI image; S6: obtaining a recognition result image based on the V channel ROI image; and S7: finding the center positions of the nearest connected components on both sides of point P in the recognition result image as the positions of the ground wires on both sides of the blue signal line. This invention utilizes machine vision recognition instead of human visual recognition, enabling accurate ground line identification.

[0005] However, the above methods have the following problems: they mainly rely on two-dimensional vision technology, which is inefficient, inaccurate, and difficult to detect the posture of the target object. Summary of the Invention

[0006] To this end, the present invention provides a material rack identification and positioning method based on 3D vision to overcome the problems of low efficiency, insufficient accuracy and difficulty in detecting the posture of the target object in the existing technology which mainly relies on two-dimensional vision technology.

[0007] To achieve the above objectives, the present invention provides a rack identification and positioning method based on 3D vision, comprising:

[0008] Obtain 3D point cloud data of the target rack;

[0009] Divide the 3D point cloud data into spatial regions to determine a spatial positioning interval and a plurality of material partitions of the target rack, wherein the spatial positioning interval refers to the position and posture range of the target rack in the spatial coordinate system, and the material partition refers to a sub-region on the target rack for placing independent materials or parts;

[0010] Identify the spatial mutation feature of the 3D point cloud data and determine the spatial phase and recognition period of the spatial mutation feature, wherein the spatial mutation feature is a significant change feature that characterizes the presence, absence, or abnormal posture of a material, the spatial phase refers to the azimuth or position information of the spatial mutation feature in the spatial coordinate system, and the recognition period refers to the spatial interval at which the spatial mutation feature recurs in continuous recognition;

[0011] In response to the spatial phase, determining potential missing materials or potential abnormal materials in the target rack;

[0012] In response to the identification cycle, determining a potential abnormal partition in the target rack, wherein the potential abnormal partition refers to an area on the rack where material placement errors or structural damage may occur;

[0013] Based on the potential missing material or the potential abnormal material and the potential abnormal partition, the specific material status information and the rack status information of the target rack are determined, wherein the specific material status information refers to the existence, missing, position or posture information of a specific material, and the rack status information refers to the occupancy status or structural integrity of the rack partition.

[0014] Furthermore, obtaining 3D point cloud data of the target rack includes:

[0015] Using a 3D vision sensor array to collect point cloud frames of the target rack at different viewing angles;

[0016] The point cloud frames are spatially registered and fused to generate 3D point cloud data of the target rack.

[0017] Further, the spatial region division of the 3D point cloud data comprises:

[0018] According to the preset rack geometry model, a spatial positioning interval of the target rack is set;

[0019] According to the material placement position defined in the rack geometry model, a plurality of material partitions are set;

[0020] A rack coordinate system is established with the spatial positioning interval as the reference.

[0021] Further, the identification of the spatial mutation feature in the 3D point cloud data comprises:

[0022] Detecting a plurality of 3D point cloud data in the 3D point cloud data that exceeds a preset spatial smooth range as a spatial mutation feature, wherein the preset spatial smooth range refers to the change range of the 3D point cloud data of the rack in a normal non-interference state;

[0023] Calculate the spatial orientation of the spatial mutation feature in the rack coordinate system as the spatial phase of the spatial mutation feature.

[0024] Further, the step of determining the identification period comprises:

[0025] According to the spatial phase, determine the equivalent spatial mutation feature appearing in each identification;

[0026] Determine the spatial interval at which the equivalent spatial mutation feature repeatedly appears in continuous identification, and record it as the identification period of the spatial phase.

[0027] Further, the step of determining the potential abnormal partition comprises:

[0028] Compare the identification period with a preset rack identification reference period to generate a period difference map;

[0029] According to the period difference map, determine the corresponding potential abnormal partition on the rack geometry model.

[0030] Further, the step of determining the potential missing material or potential abnormal material comprises:

[0031] Map the identification period to the material distribution period of the rack geometry model to form a corresponding material state mapping relationship;

[0032] According to the material state mapping relationship, record the target material corresponding to the material partition in the identification period of the spatial mutation feature as a potential missing material or a potential abnormal material.

[0033] Furthermore, in response to the spatial positional association between the potential missing material or the potential abnormal material and the potential abnormal partition being consistent in multiple identifications, the state information of the corresponding target material is determined as the final material state;

[0034] In response to the potential missing material or the potential abnormal material having no spatial correlation with the potential abnormal partition in any identification, determining that the potential abnormal partition has a structural abnormality;

[0035] Wherein, a single complete identification process includes at least five consecutive identification cycle scans of the target material rack.

[0036] Furthermore, if the recognition period does not show a mappable correlation with the rack recognition reference period within any recognition period, it is determined that the point cloud registration is abnormal or the rack model is misaligned, and the system calibration process is triggered.

[0037] Furthermore, when determining the final material status information of the target material, a material status review is performed, and a review scan of at least three identification cycles is performed;

[0038] In each review scanning cycle, the material partition corresponding to the target material is focused on, and the status information is verified based on the review result of the point cloud feature of the partition.

[0039] Compared with the existing technology, the present invention realizes the precise identification and positioning of material racks by utilizing 3D point cloud data, and can accurately detect the existence, absence, position or posture abnormality of materials, as well as the integrity of the material rack structure. Through steps such as spatial area division, spatial mutation feature recognition, and recognition cycle analysis, it can quickly locate potential abnormal areas and material status, thereby improving the efficiency and accuracy of material rack management. At the same time, through multiple identification and review scans, the reliability of the recognition results is ensured and misjudgment is avoided. In addition, by automatically triggering the system calibration process, it can deal with point cloud registration abnormalities or material rack model misalignment, further improving the robustness and stability, effectively improving the automation level of material rack management, reducing the cost of manual intervention, and improving production efficiency and quality control capabilities.

[0040] Furthermore, by utilizing a 3D vision sensor array to capture point cloud frames of the target rack from different perspectives and performing spatial registration and fusion, complete and accurate 3D point cloud data can be generated. This effectively resolves occlusion or incomplete data issues that may arise from a single perspective, thereby improving the accuracy and reliability of rack identification and positioning. At the same time, the fusion of multi-perspective data can more comprehensively reflect the three-dimensional structural information of the rack and its materials, providing a higher-quality data foundation for subsequent spatial area division, spatial mutation feature recognition, and accurate judgment of material and rack status, further improving the performance and efficiency of the entire rack identification and positioning system.

[0041] Furthermore, by partitioning the 3D point cloud data into spatial regions based on a preset rack geometry model, the spatial positioning interval and multiple material partitions of the target rack can be efficiently and accurately determined. A rack coordinate system based on the spatial positioning interval is then established, making the three-dimensional spatial information of the rack and its materials more structured and organized, facilitating subsequent feature extraction, state detection, and anomaly analysis. At the same time, the geometric model-based partitioning method reduces manual intervention, improves automation, and can quickly adapt to racks of varying specifications and layouts, enhancing versatility and flexibility and providing a solid foundation for efficient and accurate rack identification and positioning.

[0042] Furthermore, by detecting spatial mutation features in 3D point cloud data and calculating their spatial phase, it is possible to accurately identify abnormal conditions of racks and their materials, avoiding misjudgments. At the same time, calculating the spatial phase can accurately locate the orientation of abnormal features, providing key information for subsequent material status analysis and abnormal area location. This improves the sensitivity and accuracy of rack identification and positioning, enabling rapid response to the presence, absence, or abnormal posture of materials, thereby enhancing the automation and reliability of rack management.

[0043] Furthermore, by determining the recognition period of spatial mutation characteristics, it is possible to further analyze the dynamic change patterns of the target material rack and its materials, effectively identify periodic abnormal characteristics, and thus determine whether the materials are placed according to the expected rules or whether there are periodic interference factors. This provides important time dimension information for subsequent potential abnormality zoning and material status analysis, helping to more accurately locate the source of the problem.

[0044] Furthermore, by comparing the recognition cycle with the preset rack recognition reference cycle and generating a cycle difference map, the potential abnormal partitions on the rack can be accurately located. Not only spatial information is considered, but also the analysis of the time dimension is introduced. It can effectively distinguish between normal material placement patterns and abnormal situations, and intuitively identify areas on the target rack where material placement errors or structural damage may occur, thereby providing clear guidance for subsequent material status analysis and rack maintenance, and improving the efficiency and reliability of rack management.

[0045] Furthermore, by mapping the identification cycle to the material distribution cycle of the rack geometry model and forming a material status mapping relationship, it is possible to accurately identify potential missing materials or potential abnormal materials, quickly locate problem materials, and provide a basis for further review and processing.

[0046] Furthermore, determining the final material state through the consistency of spatial position associations across multiple recognitions can effectively improve recognition accuracy and reliability, reduce misjudgments, and ensure that the target material's state is determined as the final state only when an anomaly is shown across multiple recognitions. Furthermore, if a potentially missing material or a potentially abnormal material is not spatially associated with a potentially abnormal partition in any recognition, a structural anomaly in the potentially abnormal partition can be determined, allowing for timely identification of structural issues with the target rack. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a rack identification and positioning method based on 3D vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0050] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0051] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0052] See also Figure 1As shown, it is a flow chart of a rack identification and positioning method based on 3D vision according to an embodiment of the present invention, including:

[0053] Step S1, obtaining 3D point cloud data of the target rack;

[0054] Step S2: Divide the 3D point cloud data into spatial regions to determine the spatial positioning interval of the target rack and multiple material partitions, where the spatial positioning interval refers to the position and posture range of the target rack in the spatial coordinate system, and the material partition refers to the sub-region on the target rack for placing independent materials or parts;

[0055] Step S3: Identify the spatial mutation features of the 3D point cloud data and determine the spatial phase and recognition period of the spatial mutation features. The spatial mutation features are significant changes that characterize the presence, absence, or abnormal posture of materials. The spatial phase refers to the azimuth or position information of the spatial mutation features in the spatial coordinate system. The recognition period refers to the spatial interval at which the spatial mutation features recur in continuous recognition.

[0056] Step S4, determining potential missing materials or potential abnormal materials in the target rack in response to the spatial phase;

[0057] Step S5, in response to the identification cycle, determining a potential abnormal partition in the target rack, wherein the potential abnormal partition refers to an area on the rack where material placement errors or structural damage may occur;

[0058] Step S6, determining the specific material status information and rack status information of the target rack based on the potential missing materials or potential abnormal materials and the potential abnormal partitions, wherein the specific material status information refers to the existence, missing, position or posture information of a specific material, and the rack status information refers to the occupancy status or structural integrity of the rack partition.

[0059] Specifically, obtaining the 3D point cloud data of the target rack includes:

[0060] Use a 3D vision sensor array to collect point cloud frames of the target rack at different viewing angles;

[0061] The point cloud frames are spatially registered and fused to generate 3D point cloud data of the target rack.

[0062] In specific implementation, multiple 3D vision sensors are reasonably arranged around the target material rack to ensure full coverage of the target material rack from different angles.

[0063] The 3D vision sensor array is activated to collect point cloud frames of the target rack from different perspectives. Each point cloud frame collected by the sensor contains the three-dimensional spatial information of the rack and its materials from its perspective.

[0064] Because point cloud frames collected by different sensors may have inconsistent coordinate systems, the Iterative Closest Point (ICP) algorithm is used to accurately align the point clouds. The aligned point cloud frames are fused to generate a complete 3D point cloud. A weighted averaging method is used during the fusion process to eliminate duplicate point cloud data and fill any data gaps.

[0065] By utilizing a 3D vision sensor array to capture point cloud frames of the target rack from different perspectives and performing spatial registration and fusion, complete and accurate 3D point cloud data can be generated. This effectively resolves occlusion or incomplete data issues that may arise from a single perspective, thereby improving the accuracy and reliability of rack identification and positioning. Furthermore, the fusion of multi-perspective data can more comprehensively reflect the three-dimensional structural information of the rack and its materials, providing a higher-quality data foundation for subsequent spatial area division, spatial mutation feature identification, and accurate judgment of material and rack status, further enhancing the performance and efficiency of the entire rack identification and positioning system.

[0066] Specifically, the spatial region division of 3D point cloud data includes:

[0067] According to the preset rack geometry model, set the spatial positioning range of the target rack;

[0068] Set multiple material partitions according to the material placement positions defined in the rack geometry model;

[0069] Establish a rack coordinate system based on the spatial positioning interval.

[0070] By partitioning 3D point cloud data into spatial regions based on a preset rack geometry model, the spatial positioning interval and multiple material partitions of the target rack can be efficiently and accurately determined. A rack coordinate system based on the spatial positioning interval is then established, making the three-dimensional spatial information of the rack and its materials more structured and organized, facilitating subsequent feature extraction, state detection, and anomaly analysis. Furthermore, this geometric model-based partitioning method reduces manual intervention, improves automation, and can quickly adapt to racks of varying specifications and layouts, enhancing versatility and flexibility and providing a solid foundation for efficient and accurate rack identification and positioning.

[0071] Specifically, identifying spatial mutation features in 3D point cloud data includes:

[0072] Detecting a number of 3D point cloud data that exceed a preset spatial stability range in the 3D point cloud data as spatial mutation features, wherein the preset spatial stability range refers to the range of change of the 3D point cloud data of the rack under a normal and interference-free state;

[0073] The spatial orientation of the spatial mutation feature in the rack coordinate system is calculated as the spatial phase of the spatial mutation feature.

[0074] In a specific implementation, it is assumed that the normal height range of a certain material partition on the target rack is 10-15 cm, and the preset spatial smooth range is a height change of no more than 2 cm. If it is detected that the height of a certain point cloud of the partition is 20 cm, which is obviously beyond the smooth range, it is marked as a spatial mutation feature.

[0075] By detecting the spatial mutation features in the 3D point cloud data and calculating the spatial phase thereof, the abnormal state of the rack and the materials thereof can be accurately identified, and misjudgment can be avoided. At the same time, the calculation of the spatial phase can accurately locate the orientation of the abnormal features, and provide key information for subsequent material state analysis and abnormal area positioning, thereby improving the sensitivity and accuracy of rack identification and positioning, and quickly responding to the existence, absence or posture abnormality of the materials, so as to improve the automation level and reliability of rack management.

[0076] Specifically, the step of determining the identification period includes:

[0077] According to the spatial phase, the equivalent spatial mutation features appearing in each identification are determined;

[0078] The spatial interval at which the equivalent spatial mutation features repeatedly appear in continuous identification is determined, and is recorded as the identification period of the spatial phase.

[0079] In a specific implementation, it is assumed that when the target rack is continuously scanned, it is found that a certain material partition appears spatial mutation features in the 1st, 4th and 7th scans, while the region is normal in other scans. By analyzing the spatial phase of the spatial mutation features (assuming that they are all 45 degrees), it is determined that this is an equivalent spatial mutation feature. The scanning interval of adjacent two spatial mutation features is 3 (from the 1st to the 4th, and from the 4th to the 7th), and therefore the identification period of the equivalent spatial mutation feature is 3.

[0080] By determining the identification period of the spatial mutation features, the dynamic change rule of the target rack and the materials thereof can be further analyzed, and the periodically appearing abnormal features can be effectively identified, so as to judge whether the materials are placed according to the expected rule or whether there is a periodic interference factor, thereby providing important time dimension information for subsequent potential abnormal partition and material state analysis, and helping to more accurately locate the problem source.

[0081] Specifically, the step of determining the potential abnormal partition includes:

[0082] The identification period is compared with a preset rack identification reference period to generate a period difference mapping;

[0083] According to the period difference mapping, the corresponding potential abnormal partition on the rack geometric model is determined.

[0084] In a specific implementation, a rack identification reference period is set according to the normal operation process of the target rack (such as the material replenishment period, the production rhythm, etc.). This rack identification reference period reflects the dynamic change rule of the target rack in the normal state. For example, if the material on the target rack is replenished every 3 scans, the rack identification reference period is 3. If the identification period of a certain space phase is 5, and the rack identification reference period is 3, then the period difference of the space phase is 2.

[0085] By comparing the identification period with the preset rack identification reference period and generating a period difference mapping, the potential abnormal partition on the rack can be accurately located. Not only the spatial information is considered, but also the time dimension analysis is introduced, which can effectively distinguish the normal material placement rule from the abnormal situation, intuitively identify the area on the target rack where the material placement error or structure damage may occur, thereby providing clear guidance for subsequent material state analysis and rack maintenance, and improving the efficiency and reliability of rack management.

[0086] Specifically, the step of determining the potential missing material or the potential abnormal material includes:

[0087] Mapping the identification period to the material distribution period of the rack geometric model to form a corresponding material state mapping relationship;

[0088] According to the material state mapping relationship, the target material corresponding to the material partition in the identification period with the spatial mutation feature is recorded as the potential missing material or the potential abnormal material.

[0089] By mapping the identification period to the material distribution period of the rack geometric model and forming the material state mapping relationship, the potential missing material or the potential abnormal material can be accurately identified, and the problem material can be quickly located, providing a basis for further review and processing.

[0090] Specifically, in response to the spatial position of the potential missing material or the potential abnormal material and the potential abnormal partition being consistent in multiple recognitions, the state information of the corresponding target material is determined as the final material state;

[0091] In response to the spatial position of the potential missing material or the potential abnormal material and the potential abnormal partition being irrelevant in any identification, it is determined that the potential abnormal partition has a structural abnormality;

[0092] Among them, a single complete identification process includes at least 5 continuous identification period scans of the target rack.

[0093] In a specific implementation, the target rack is scanned at least 5 times continuously. The selection of this number is based on the analysis of the rack state change rule and the evaluation of the recognition ability, and 5 scans can provide sufficient time dimension data to ensure the stability and reliability of the identification result.

[0094] Determining the final material state through the consistency of spatial position associations across multiple recognitions effectively improves recognition accuracy and reliability, reduces misjudgments, and ensures that the target material's state is determined as the final state only when anomalies are shown across multiple recognitions. Furthermore, if a potentially missing or abnormal material is not spatially associated with a potentially abnormal partition in any recognition, a structural anomaly can be determined within the potentially abnormal partition, allowing for timely identification of structural issues with the target rack.

[0095] Specifically, if the recognition cycle does not show a mappable correlation with the rack recognition reference cycle within any recognition cycle, it is determined that the point cloud registration is abnormal or the rack model is inaccurate, and the system calibration process is triggered.

[0096] Specifically, when determining the final material status information of the target material, a material status review is performed, and a review scan of at least three identification cycles is performed;

[0097] Among them, in each review scanning cycle, focus is placed on the material partition corresponding to the target material, and status information verification is performed based on the review results of the partition point cloud features.

[0098] In practice, three recheck scans provide a baseline sample size for determining statistical significance of the target material's state. By performing at least three recheck scans during the final determination of the target material's state and focusing on the material partitions corresponding to the target material, misjudgments due to accidental errors or interference are reduced.

[0099] In practical applications, the above-mentioned 3D vision-based rack identification and positioning method still has certain limitations in dynamic scenarios (such as micro-deformation of the rack due to load-bearing, posture changes caused by high-frequency storage and retrieval of materials, and sensor vibration). To further improve the robustness and adaptability of the method, it can be expanded and optimized as follows.

[0100] When acquiring the 3D point cloud data of the target rack and performing spatial registration, it is necessary to consider that the rack may produce non-rigid deformations (such as beam bending and slight deformation of the shelves) due to factors such as load bearing and vibration. It is difficult to achieve accurate registration only through rigid transformation. In this case, a non-rigid registration model is needed to estimate the rigid motion parameters (rotation matrix and translation vectors ) and calculate the non-rigid deformation displacement of each point Specifically, let Frame point cloud is , the reference point cloud is , non-rigid registration is achieved by minimizing the objective function:

[0101]

[0102] Among them, the first item is the data item, which is used to ensure the accuracy of point cloud registration; the second item is the regularization item, which constrains the smoothness of deformation based on the principle of elastic mechanics to avoid excessive distortion. is the regularization weight (range 0.2-0.5, determined by cross-validation), Indicates the The non-rigid deformation displacement obtained by solving the objective function is , which can correct the point cloud offset caused by rack deformation and provide more accurate basic data for subsequent spatial area division.

[0103] Furthermore, in order to deal with the impact of rack deformation on material partitioning in advance, the LSTM model can be used to predict the deformation trend. As input, the temporal features of deformation are learned through LSTM cell state updates and hidden state transmission. The LSTM cell state update formula is:

[0104]

[0105] in, are the outputs of the forget gate and input gate respectively, are model training parameters, is the hidden state of the previous moment. Based on the current hidden state , which can predict the future Deformation increment at a moment ( is the output layer parameter), and the spatial range of the material partition is adjusted in advance accordingly: the partition boundary Original Boundary ,in is a safety factor (valued at 1.2-1.5) to avoid partition shift due to deformation.

[0106] When identifying spatial mutation features, in order to make up for the shortcomings of single 3D point cloud features that are easily blocked and interfered by noise in dynamic scenes, the dynamic features of 3D point cloud and 2D RGB image can be fused. For 3D point cloud modality, the motion vector of each point is extracted. (displacement of the same point in adjacent frames) and local curvature ( is the curvature radius of the point); for 2D RGB mode, the pixel motion vector is calculated by the optical flow algorithm , and extract the color gradient ( is pixel brightness).

[0107] In order to achieve the effective fusion of multimodal features, a cross-modal attention mechanism is introduced to dynamically allocate the weights of the two modalities. Suppose the cosine similarity of point cloud and RGB features is and (Cosine similarity ), then the attention weight of the point cloud modality is:

[0108]

[0109] The weight of RGB mode is The fused feature vector is are high-dimensional features of point cloud and RGB respectively), the mutation probability can be calculated based on the fusion features: ,in is the sigmoid function, is the classification parameter, when mutation , it is marked as a high-confidence mutation region.

[0110] In addition, in order to quantify the uncertainty in the recognition process and improve the reliability of decision making, the Bayesian inference framework is used to model the key parameters. and non-rigid deformation Both are modeled as Gaussian distributions: , ,in is the parameter estimate, is the covariance matrix (the residual of point cloud matching calculated).

[0111] For the prediction of mutation features, the Bayesian LSTM model is used to output probability distribution rather than single point value, that is, , where the mean ,variance ( is a model parameter). Based on the above uncertainty modeling, the registration confidence can be calculated (tr is the matrix trace) and prediction confidence , and dynamically adjust the judgment threshold: .when When the error is detected, it is judged as abnormal, and the verification cycle can be shortened from 5 scans to 3 in high-confidence scenarios and extended to 7 in low-confidence scenarios, so as to improve response efficiency while ensuring accuracy.

[0112] Through the above-mentioned expansion and optimization, the 3D vision-based rack identification and positioning method can better adapt to dynamic scenes, effectively deal with problems such as non-rigid deformation of racks, insufficient feature robustness and recognition uncertainty, and further improve the automation level and reliability of rack management.

[0113] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0114] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A rack identification and positioning method based on 3D vision, characterized in that: include: Obtain 3D point cloud data of the target rack; Divide the 3D point cloud data into spatial regions to determine a spatial positioning interval and a plurality of material partitions of the target rack, wherein the spatial positioning interval refers to the position and posture range of the target rack in the spatial coordinate system, and the material partition refers to a sub-region on the target rack for placing independent materials or parts; Identify the spatial mutation feature of the 3D point cloud data and determine the spatial phase and recognition period of the spatial mutation feature, wherein the spatial mutation feature is a significant change feature that characterizes the presence, absence, or abnormal posture of a material, the spatial phase refers to the azimuth or position information of the spatial mutation feature in the spatial coordinate system, and the recognition period refers to the spatial interval at which the spatial mutation feature recurs in continuous recognition; In response to the spatial phase, determining potential missing materials or potential abnormal materials in the target rack; In response to the identification cycle, determining a potential abnormal partition in the target rack, wherein the potential abnormal partition refers to an area on the rack where material placement errors or structural damage may occur; Based on the potential missing material or the potential abnormal material and the potential abnormal partition, the specific material status information and the rack status information of the target rack are determined, wherein the specific material status information refers to the existence, missing, position or posture information of a specific material, and the rack status information refers to the occupancy status or structural integrity of the rack partition.

2. The rack identification and positioning method based on 3D vision according to claim 1 is characterized in that: Obtaining 3D point cloud data of the target rack includes: Using a 3D vision sensor array to collect point cloud frames of the target rack at different viewing angles; The point cloud frames are spatially registered and fused to generate 3D point cloud data of the target rack.

3. The rack identification and positioning method based on 3D vision according to claim 2, characterized in that: Spatial area division of 3D point cloud data includes: According to a preset rack geometry model, the spatial positioning interval of the target rack is set; Setting the plurality of material partitions according to the material placement positions defined in the rack geometric model; A material rack coordinate system is established based on the spatial positioning interval.

4. The rack identification and positioning method based on 3D vision according to claim 3, characterized in that: Identifying spatial mutation features in 3D point cloud data includes: Detecting a number of 3D point cloud data exceeding a preset spatial stability range in the 3D point cloud data as spatial mutation features, wherein the preset spatial stability range refers to a range of changes in the 3D point cloud data of the target rack under a normal, interference-free state; The spatial orientation of the spatial mutation feature in the rack coordinate system is calculated as the spatial phase of the spatial mutation feature.

5. The rack identification and positioning method based on 3D vision according to claim 4 is characterized in that: The steps to determine the identification cycle include: Determining, based on the spatial phase, equivalent spatial mutation features occurring in each recognition; The spatial interval at which the equivalent spatial mutation feature repeatedly appears in continuous recognition is determined and recorded as the recognition period of the spatial phase.

6. The rack identification and positioning method based on 3D vision according to claim 5, characterized in that: The steps to identify potentially abnormal partitions include: Comparing the identification period with a preset rack identification reference period to generate a period difference map; According to the period difference map, corresponding potential abnormal partitions on the rack geometric model are determined.

7. The rack identification and positioning method based on 3D vision according to claim 6, characterized in that: The steps to identify potentially missing or unusual material include: Mapping the identification period to the material distribution period of the rack geometric model to form a corresponding material state mapping relationship; According to the material state mapping relationship, the target material corresponding to the material partition within the identification period in which the spatial mutation feature occurs is recorded as a potential missing material or a potential abnormal material.

8. The rack identification and positioning method based on 3D vision according to claim 7, characterized in that: In response to the spatial positional association between the potential missing material or the potential abnormal material and the potential abnormal partition being consistent in multiple identifications, determining the corresponding state information of the target material as the final material state; In response to the potential missing material or the potential abnormal material having no spatial correlation with the potential abnormal partition in any identification, determining that the potential abnormal partition has a structural abnormality; Wherein, a single complete identification process includes at least five consecutive identification cycle scans of the target material rack.

9. The rack identification and positioning method based on 3D vision according to claim 8, characterized in that: If the recognition cycle does not show a mappable correlation with the rack recognition reference cycle within any recognition cycle, it is determined that the point cloud registration is abnormal or the rack model is misaligned, and the system calibration process is triggered.

10. The rack identification and positioning method based on 3D vision according to claim 9, characterized in that: Also includes: When determining the final material status information of the target material, perform a material status review and perform a review scan of at least three identification cycles; In each review scanning cycle, the material partition corresponding to the target material is focused on, and the status information is verified according to the review result.

Images