Indoor target positioning method and system based on machine vision and deep learning

Abstract

The present invention relates to the field of target positioning technology, and specifically to a method and system for indoor target positioning based on machine vision and deep learning. The method selects a suitable image preprocessing algorithm by analyzing the characteristics of obstructions in real-time video; detects and spatially locates obscured products in combination with Bluetooth beacon positioning data; constructs a three-dimensional representation of the product and tracks its motion trajectory; and triggers a target recapture mechanism based on Bluetooth positioning when tracking fails due to occlusion. The present invention solves the problem of obstructions interfering with product positioning and tracking in a workshop environment, and achieves high-precision positioning and continuous tracking of products under complex conditions through multi-source data fusion and intelligent algorithms, providing reliable real-time location information for workshop production management and effectively improving the automation and intelligence level of the production process.

Description

Technical Field

[0001] The present invention relates to the field of target positioning technology, and in particular to an indoor target positioning method and system based on machine vision and deep learning. Background Art

[0002] Production workshops within factories, particularly those in electronics and chemical plants, are often equipped with high-position snapshot cameras. These cameras are primarily used to monitor the production process, ensure product quality and safety, and locate and track target products in real time. Furthermore, based on the video images captured by the cameras, products in the production workshop can be captured, enabling intelligent, digital, and information-based tracking and positioning of the production workshop, ensuring timely understanding of product progress and quality. However, lighting conditions within the workshop are complex and variable, and these frequent changes in light pose significant interference to machine vision-based target positioning systems. Furthermore, the numerous production equipment within the workshop often obstructs target products due to worker movement and work. This obstruction not only hinders visual acquisition of the target product but also introduces significant background noise, making target positioning extremely difficult. Because equipment and workers on the production line are in variable positions, the obstruction of the target product is dynamic, with constantly changing size, shape, and position. Traditional target detection algorithms struggle to adapt to such complex application scenarios. At the same time, the target products move quickly on the production line, placing extremely high demands on real-time performance. Frequent occlusions can cause target tracking failures, and quickly recapture the target after an occlusion occurs is a major technical challenge. In summary, achieving accurate positioning and real-time tracking of product targets in the complex lighting and occlusion environments of the workshop is a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The purpose of the present invention is to provide an indoor target positioning method and system based on machine vision and deep learning to achieve accurate positioning and real-time tracking of product targets.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides an indoor target positioning method based on machine vision and deep learning, the method comprising the following steps:

[0006] S100: Acquire real-time video images of the workshop, analyze the movement trajectory and size of obstructions in the images, determine the frequency and duration of the obstructions, and determine the degree of impact of the obstructions on the target product;

[0007] S200 , preprocessing the video image based on the degree of impact of the occlusion on the target product, analyzing the transparency and refractive index properties of the pixels in the occluded area, inferring the material type of the occluded object, and selecting a occluded area processing method for that material type to obtain a preprocessed video image;

[0008] S300: Obtain an estimated distance and azimuth between the target product and the Bluetooth beacon, perform position detection on the obscured target product based on the preprocessed video image, spatially align the target product's position detection result with the Bluetooth beacon data, obtain an estimated distance and azimuth of the Bluetooth beacon relative to the target product based on Bluetooth positioning, and detect and locate the target product by constructing a spatial topological relationship between the product and the beacon, obtaining the target product's position coordinates and bounding box information.

[0009] S400: Tracking and predicting the motion state of the target product based on the location coordinates and bounding box information of the target product in combination with Bluetooth positioning data to obtain the motion trajectory and speed information of the target product;

[0010] S500, during the target product tracking process, if an obstruction blocks the target product, causing tracking failure, a local search is performed in the area where the target product may appear based on the target product's historical motion trajectory and the current Bluetooth positioning estimated position until the target product's motion trajectory and speed information are recaptured;

[0011] S600, based on the target product's motion trajectory and speed information, obtains the target product's depth information through a depth camera, combines it with the distance and azimuth estimated by Bluetooth positioning, constructs a three-dimensional representation of the target product, generates the position and tracking motion information of the indoor target product, and transmits it to the indoor workshop production management system to achieve target positioning of indoor products that are obscured during the production process.

[0012] Optionally, in S100, acquiring a real-time video image of the workshop, analyzing the movement trajectory and size of the obstruction in the image, determining the frequency and duration of the obstruction, and determining the degree of impact of the obstruction on the target product, includes:

[0013] S110, acquiring real-time video streams captured by multiple cameras, and performing image preprocessing on the video streams, wherein the image preprocessing includes median filtering, histogram equalization, and Sobel operator edge sharpening;

[0014] S120, constructing a scene background using a Gaussian mixture model, performing a difference operation between the current frame and the scene background to obtain an initial outline of the occluder;

[0015] S130, matching and associating the initial outline of the occluder between consecutive frames to obtain a motion trajectory of the occluder on the image plane;

[0016] S140, performing a time series analysis on the movement trajectory of the obstruction, calculating the number of times the obstruction appears in a specific area and the length of time it stays there, to obtain statistical data on the frequency and duration of the obstruction;

[0017] S150, based on the spatial position and time distribution of the obstruction, find the affected target products and analyze the impact of the obstruction on each target product; set differentiated impact weights for different production stages to determine the degree of impact of the obstruction on product quality and production efficiency.

[0018] Optionally, in S200, preprocessing the video image based on the degree of impact of the occlusion on the target product, analyzing the transparency and refractive index properties of pixels in the occluded area, inferring the material type of the occluded object, and selecting an occluded area processing method for the material type to obtain the preprocessed video image includes:

[0019] S210, acquiring a real-time video image captured by a camera, locating the outline of the obstruction, and obtaining a pixel set of the obstruction area;

[0020] S220, calculating brightness distribution characteristics and gradient information based on the pixel set of the occluded area, analyzing the spatial relationship between the occluded area and the target product in combination with the location information of the target product, and determining a preliminary impact range;

[0021] S230, extracting a material feature vector based on the texture features and color histogram of the occluded area, and mapping the feature vector to a predefined material category using a support vector machine classifier; if the material category is glass, processing the occluded area using an alpha blending algorithm; if the material category is translucent plastic, processing the occluded area using a transparency-based image enhancement algorithm; if the material category is opaque metal, processing the occluded area using an edge-preserving filtering algorithm;

[0022] S240 , optimizing parameters of the occluded area processing method, traversing the parameter space using a grid search method, evaluating the quality of the processing result using a structural similarity index, and applying the optimized method to the occluded area of ​​the original image to obtain a preprocessed image.

[0023] Optionally, in S300, the step of obtaining an estimated distance and azimuth between the target product and the Bluetooth beacon, performing position detection on the obscured target product based on the preprocessed video image, spatially registering the target product position detection result with the Bluetooth beacon data, obtaining an estimated distance and azimuth of the Bluetooth beacon relative to the target product based on Bluetooth positioning, and detecting and locating the target product by constructing a spatial topological relationship between the product and the beacon to obtain the position coordinates and bounding box information of the target product includes:

[0024] S310, obtaining signal strength data transmitted by Bluetooth beacons, selecting a signal strength attenuation model based on the signal strength data; and calculating an estimated distance between the target product and each Bluetooth beacon based on the signal strength attenuation model;

[0025] S320, analyzing the preprocessed image to obtain position and size information of the target product in the image, and determining an initial position of the target product by combining the estimated distance and the position and size information of the target product in the image;

[0026] S330, calculating the distance and azimuth of the target product relative to each beacon, and using an adaptive weighting method based on signal strength to perform a weighted average of the distance and azimuth of the target product relative to each beacon to obtain a positioning result for the target product; wherein the weight in the adaptive weighting method is positively correlated with the signal strength and is mapped to a range of 0 to 1 using a sigmoid function;

[0027] S340: Establishing a spatial topological relationship between the target product and the Bluetooth beacon based on the initial position and positioning result of the target product. Based on the spatial topological relationship and in combination with the initial position and positioning result of the target product, employing an extended Kalman filter algorithm to estimate and track the position of the target product in real time; wherein the state vector includes position, velocity, and acceleration, and the observation vector includes the image detection position and the Bluetooth positioning result;

[0028] S350 uses an extended Kalman filter algorithm to estimate the initial position of the target product in real time to optimize the target product's position coordinates. Furthermore, a particle filter algorithm is introduced to address occlusion situations. This algorithm maintains multiple hypothetical particles to handle multimodal distributions and improve tracking stability under occlusion. The state vector of the extended Kalman filter algorithm includes position, velocity, and acceleration, and the observation vector includes image detection position and Bluetooth positioning results.

[0029] S360 maps the optimized position coordinates back to the image plane and combines them with the bounding box information of the target detection to obtain the precise position and size of the target product in the image.

[0030] Optionally, in S400, tracking and predicting the motion state of the target product based on the position coordinates and bounding box information of the target product in combination with Bluetooth positioning data to obtain the motion trajectory and speed information of the target product includes:

[0031] S410, obtaining the location coordinates, bounding box information, and Bluetooth positioning data of the target product;

[0032] S420, using a Kalman filter algorithm to fuse the position coordinates, the bounding box information, and the Bluetooth positioning data to obtain fused position data; wherein the state vector of the Kalman filter algorithm includes a three-dimensional position and velocity, and the observation vector includes a two-dimensional coordinate and a three-dimensional coordinate;

[0033] S430, fitting the motion trajectory of the target product using the least squares method based on the fused position data; constructing a motion state model of the target product based on the motion trajectory and the fused position data; wherein the motion state model includes a linear regression model and a long short-term memory network model;

[0034] S440, predicting the future trajectory of the target product based on the motion state model; wherein the linear regression model is used to predict the short-term future position, and the long short-term memory network model is used to capture the long-term motion pattern; and optimizing the future trajectory using a particle filter algorithm to obtain an optimized future trajectory.

[0035] Optionally, in S500, during the target product tracking process, if an obstruction blocks the target product, resulting in tracking failure, a local search is performed in an area where the target product may appear based on the target product's historical motion trajectory and the current Bluetooth positioning estimated position until the target product's motion trajectory and speed information are recaptured, including:

[0036] S510, receiving consecutive frame images and target detection confidence information, and determining whether occlusion has caused tracking failure based on the consecutive frame images and target detection confidence information. If the target detection confidence is lower than a preset threshold or the number of consecutive undetected frames exceeds a preset number of frames, a target recapture mechanism is triggered;

[0037] S520, obtaining the location and time information of the last successful tracking of the target product, and using the Kalman filter algorithm to predict the current location of the target product;

[0038] S530, receiving target product position estimation information, fusing the Kalman filter prediction result and the Bluetooth positioning result through a covariance-based adaptive weight allocation method to obtain a fused estimate of the target product position;

[0039] S540, determining a local search area based on the fused estimate of the target product position, and employing an adaptive grid search algorithm within the local search area to recapture the target; if the target is detected within a certain grid, performing a refined search on the grid; receiving continuous multi-frame tracking results, and confirming successful target capture if the number of continuous tracking frames exceeds a preset frame number threshold;

[0040] S550 , obtaining historical trajectory information, Bluetooth positioning data, and new visual tracking results of the target product, back-estimating the motion state of the target product during the occlusion period, and obtaining the motion trajectory and speed information of the target product.

[0041] Optionally, in S600, based on the target product's motion trajectory and speed information, depth information of the target product is obtained through a depth camera, and combined with the distance and azimuth estimated by Bluetooth positioning, a three-dimensional representation of the target product is constructed, and the position and tracking motion information of the indoor target product is generated and transmitted to the indoor workshop production management system to achieve target positioning of indoor products obscured during the production process, including:

[0042] S610, obtaining motion trajectory and speed information of the target product, and smoothing the information using a Kalman filter algorithm to obtain filtered motion data;

[0043] S620, collecting a depth image of the target product through a depth camera and extracting a depth profile of the target product;

[0044] S630, performing illumination unevenness correction on the depth image using a three-scale Retinex algorithm, and improving detail representation of the depth image using a CLAHE algorithm;

[0045] S640, constructing a three-dimensional point cloud representation of the target product based on the processed depth information and Bluetooth positioning data;

[0046] S650, registering the three-dimensional point cloud representation with a pre-established CAD model of the target product to determine the precise pose of the target product; if the mean absolute error of the precise pose exceeds a preset threshold, triggering anomaly detection;

[0047] S660: Pack the precise position and filtered motion data and transmit them to an indoor workshop production management system via industrial Ethernet.

[0048] In a second aspect, an embodiment of the present invention provides an indoor target positioning system based on machine vision and deep learning, the system comprising:

[0049] at least one processor;

[0050] at least one memory for storing at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of the first aspects.

[0052] The beneficial effects of the present invention are as follows: the present invention discloses an indoor target positioning method and system based on machine vision and deep learning. By analyzing the characteristics of obstructions in real-time video, an appropriate image preprocessing algorithm is selected; in combination with Bluetooth beacon positioning data, obscured products are detected and spatially located; a three-dimensional representation of the product is constructed and its motion trajectory is tracked; and when occlusion causes tracking failure, a target recapture mechanism based on Bluetooth positioning is triggered. The present invention solves the problem of obstructions interfering with product positioning and tracking in workshop environments. Through multi-source data fusion and intelligent algorithms, high-precision positioning and continuous tracking of products under complex conditions are achieved, providing reliable real-time location information for workshop production management and effectively improving the automation and intelligence level of the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 1 is a flow chart of an indoor target positioning method based on machine vision and deep learning in an embodiment of the present invention;

[0055] Figure 2 This is a structural diagram of an indoor target positioning system based on machine vision and deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other unless there is a conflict.

[0057] In factory production workshops, Bluetooth target positioning based on machine vision and deep learning technologies encounters challenges: unstable lighting conditions, with light levels fluctuating. Furthermore, moving equipment and workers on the production line frequently obstruct target objects, interfering with product detection. To address these technical issues, the present invention provides a method and system for indoor target positioning based on machine vision and deep learning.

[0058] See Figure 1 , Figure 1 The present invention provides an indoor target positioning method based on machine vision and deep learning, which includes the following steps:

[0059] S110, acquiring real-time video images of the workshop, analyzing the movement trajectory and size of obstructions in the images, determining the frequency and duration of the obstructions, and determining the extent of the obstruction's impact on the target product;

[0060] S120 , pre-processing the video image based on the degree of impact of the occlusion on the target product, analyzing the transparency and refractive index properties of the pixels in the occluded area, inferring the material type of the occluded object, and selecting a occluded area processing method for that material type to obtain a pre-processed video image;

[0061] S130: Obtain an estimated distance and azimuth between the target product and the Bluetooth beacon, perform position detection on the obscured target product based on the preprocessed video image, spatially align the target product's position detection result with the Bluetooth beacon data, obtain an estimated distance and azimuth of the Bluetooth beacon relative to the target product based on Bluetooth positioning, detect and locate the target product by constructing a spatial topological relationship between the product and the beacon, and obtain the target product's position coordinates and bounding box information;

[0062] S140 , tracking and predicting the motion state of the target product based on the location coordinates and bounding box information of the target product in combination with the Bluetooth positioning data, to obtain the motion trajectory and speed information of the target product;

[0063] S150, during the target product tracking process, if an obstruction blocks the target product, causing tracking failure, a local search is performed in the area where the target product may appear based on the target product's historical motion trajectory and the current Bluetooth positioning estimated position until the target product's motion trajectory and speed information are recaptured;

[0064] S160, based on the target product's motion trajectory and speed information, obtains the target product's depth information through a depth camera, combines it with the distance and azimuth estimated by Bluetooth positioning, constructs a three-dimensional representation of the target product, generates the position and tracking motion information of the indoor target product, and transmits it to the indoor workshop production management system to achieve target positioning of indoor products that are obscured during the production process.

[0065] In the embodiments provided by the present invention, a suitable image preprocessing algorithm is selected by analyzing the features of obstructions in real-time video; the obstructed product is detected and spatially located in combination with Bluetooth beacon positioning data; a three-dimensional representation of the product is constructed and its motion trajectory is tracked; when the occlusion causes tracking failure, a target recapture mechanism based on Bluetooth positioning is triggered; the present invention solves the problem of interference of obstructions on product positioning and tracking in a workshop environment, and through multi-source data fusion and intelligent algorithms, high-precision positioning and continuous tracking of products under complex conditions are achieved, providing reliable real-time location information for workshop production management, and effectively improving the automation and intelligence level of the production process.

[0066] In some embodiments, in S100, acquiring a real-time video image of the workshop, analyzing the movement trajectory and size of obstructions in the image, determining the frequency and duration of the obstructions, and determining the degree of impact of the obstructions on the target product, includes:

[0067] S110, acquiring real-time video streams captured by multiple cameras, and performing image preprocessing on the video streams, wherein the image preprocessing includes median filtering, histogram equalization, and Sobel operator edge sharpening;

[0068] S120, constructing a scene background using a Gaussian mixture model, performing a difference operation between the current frame and the scene background to obtain an initial outline of the occluder;

[0069] S130, matching and associating the initial outline of the occluder between consecutive frames to obtain a motion trajectory of the occluder on the image plane;

[0070] S140, performing a time series analysis on the movement trajectory of the obstruction, calculating the number of times the obstruction appears in a specific area and the length of time it stays there, and obtaining statistical data on the frequency and duration of the obstruction;

[0071] S150, based on the spatial position and time distribution of the obstruction, find the affected target products and analyze the impact of the obstruction on each target product; set differentiated impact weights for different production stages to determine the degree of impact of the obstruction on product quality and production efficiency.

[0072] For example, real-time video streams captured by multiple high-definition cameras in the workshop are obtained, and image preprocessing is performed on the video frames, including using median filtering for denoising, histogram equalization for contrast enhancement, and Sobel operator for edge sharpening to improve image quality. A Gaussian mixture model is used to construct the scene background, and the current frame is differentially calculated from the background to extract the foreground target and obtain the initial outline of the occluder. For the extracted outline of the occluder, the KCF algorithm is used to match and associate it between consecutive frames to obtain the motion trajectory of the occluder on the image plane. Based on the calibration parameters of multiple cameras, the three-dimensional position and actual size of the occluder are calculated using the triangulation principle. The trajectory is smoothed using the Kalman filter to eliminate the influence of noise.

[0073] The motion trajectory of obstructions is analyzed in time series, with the number of occurrences and duration of obstructions within a specific area calculated. This yields statistical data on the frequency and duration of obstructions. Frequency and duration thresholds are set. If the frequency of an obstruction exceeds the threshold or the duration of a single obstruction exceeds the threshold, it is considered an abnormal obstruction. Based on the motion characteristics of the obstruction, the type of obstruction caused by human movement and equipment movement is distinguished, and the impact range of the abnormal obstruction is preliminarily determined. Based on the production line layout information, a mapping between product location and camera field of view is constructed. Based on the spatial location and temporal distribution of abnormal obstructions, the affected target products are identified. An impact assessment model is developed, taking into account factors such as obstruction size, duration, production line process flow, and critical control points, to quantify the impact score of an obstruction on each target product. Differentiated impact weights are assigned to different production stages to comprehensively assess the impact of an obstruction on product quality and production efficiency. Multiple high-definition network cameras with a resolution of 1920x1080 pixels and a frame rate of 30 fps are installed in the workshop. The captured video streams are denoised using a median filter with a filter window size of 3x3 pixels. Histogram equalization is then used to enhance image contrast, redistributing pixel intensity values ​​to the range of 0–255. The Sobel operator is used for edge sharpening and object outlines. A Gaussian mixture model with five Gaussian components and a learning rate of 0.01 is used to construct the scene background. Background subtraction is performed to obtain foreground objects, and morphological opening is applied to remove small noise. The structural element size is 5x5 pixels. The KCF algorithm uses the HOG feature descriptor with a feature cell size of 4x4 pixels and a block size of 2x2 cells. The bandwidth parameter σ of the Gaussian kernel function is set to 0.5. Filter weights are updated by minimizing the objective function with a learning rate of 0.02. Multi-camera calibration is performed using the Zhang Zhengyou calibration method, with a checkerboard grid size of 9x6, and each grid cell size of 25mmx25mm. For triangulation, the baseline length is 1 meter, the focal length is 8mm, and the disparity accuracy is 0.1 pixel. The process noise covariance matrix Q and the measurement noise covariance matrix R of the Kalman filter are initially set to the identity matrix and adjusted through iterative optimization. The occlusion trajectory analysis uses a sliding time window with a window size of 10 minutes and a step size of 1 minute. The number of occlusions and their cumulative duration are counted in each window. The frequency threshold is set to 5 times per 10 minutes, and the duration threshold is set to 2 minutes cumulatively. Exceeding any threshold is considered an abnormal occlusion. Based on the average speed and acceleration characteristics of the occlusion, a threshold is set to distinguish between people walking, with a speed of <1.5m / s and an acceleration of <0.5m / s. 2and equipment movement. The initial impact range is determined by the projected area and movement range of the obstruction. The production line layout is represented in a grid with a grid size of 0.5mx0.5m. The position of each product in the production process is tracked in real time via Bluetooth. The impact assessment model takes into account the size of the obstruction, which accounts for 20% of the area, the duration of the obstruction, which accounts for 30%, the process stage of the product, which accounts for 30%, and the position of the obstruction, which accounts for 20%. The weights of different production stages are set according to the importance of the process, such as the weight of key quality inspection points is 1.5, and the weight of general processing points is 1.0. The final impact score is 0-100. A score greater than 80 is considered a serious impact and requires immediate attention.

[0074] In some embodiments, in S200, preprocessing the video image based on the degree of impact of the occlusion on the target product, analyzing the transparency and refractive index properties of pixels in the occluded area, inferring the material type of the occluded object, and selecting an occluded area processing method for that material type to obtain the preprocessed video image includes:

[0075] S210, acquiring a real-time video image captured by a camera, locating the outline of the obstruction, and obtaining a pixel set of the obstruction area;

[0076] S220, calculating brightness distribution characteristics and gradient information based on the pixel set of the occluded area, analyzing the spatial relationship between the occluded area and the target product in combination with the location information of the target product, and determining a preliminary impact range;

[0077] S230, extracting a material feature vector based on the texture features and color histogram of the occluded area, and mapping the feature vector to a predefined material category using a support vector machine classifier; if the material category is glass, processing the occluded area using an alpha blending algorithm; if the material category is translucent plastic, processing the occluded area using a transparency-based image enhancement algorithm; if the material category is opaque metal, processing the occluded area using an edge-preserving filtering algorithm;

[0078] S240 , optimizing parameters of the occluded area processing method, traversing the parameter space using a grid search method, evaluating the quality of the processing result using a structural similarity index, and applying the optimized method to the occluded area of ​​the original image to obtain a preprocessed image.

[0079] For example, real-time video footage captured by a high-definition camera in the workshop is acquired. The GrabCut algorithm is used to precisely locate the outline of the obstruction, generating a pixel set within the obstruction area. The brightness distribution characteristics and gradient information of the pixels within the obstruction area are calculated. Combined with the known location information of the target product, the spatial relationship between the obstruction area and the target product is analyzed to determine the initial impact range. Material feature vectors are extracted based on the texture features and color histogram of the obstruction area. A pre-trained support vector machine classifier is used to map the feature vectors to predefined material categories, such as glass, plastic, and metal. The approximate transparency and reflectivity of the obstruction are estimated based on the ambient lighting conditions and the surface reflectivity of the obstruction. Based on the inferred material type, an appropriate processing method is selected from a library of pre-set algorithms for processing the obstruction area. For glass, an alpha blending algorithm is used; for translucent plastic, a transparency-based image enhancement algorithm is used; and for opaque metal, an edge-preserving filter is applied. Initial algorithm parameters, such as the alpha value, enhancement strength, and filter kernel size, are set based on the material characteristics and the degree of occlusion. The parameters of the selected occlusion region processing algorithm were optimized using a grid search method to traverse the parameter space. The quality of the processing results was evaluated using a structural similarity index after each iteration. The optimization process was terminated when the evaluation index reached a preset threshold or the number of iterations exceeded the upper limit. The optimized algorithm was applied to the occluded region of the original image to obtain a preprocessed image. By comparing the visibility change of the target product before and after processing, an occlusion impact score was calculated for subsequent product quality control and production scheduling decisions. A high-definition camera with a resolution of 1920x1080 pixels was installed in the workshop, capturing a real-time video stream at a frame rate of 30 fps. The GrabCut algorithm was used to segment the occluded objects in each image frame, with the number of iterations set to 5 and a Gaussian mixture model using 5 components each for the foreground and background. The brightness histogram of the pixels within the occluded region was calculated, and the 256 grayscale values ​​were divided into 16 bins to obtain the brightness distribution feature vector. The Sobel operator was used to calculate gradient information, with the gradient amplitude threshold set to 50. The image coordinate system is converted to the world coordinate system through the pre-calibrated camera parameters. Combined with the known three-dimensional position (x, y, z) of the target product, the minimum Euclidean distance between the occluded area and the product is calculated. If the distance is less than 0.5 meters, it is judged as a direct impact. The grayscale co-occurrence matrix of the occluded area is extracted as the texture feature, and four statistics: energy, contrast, homogeneity and entropy are calculated. The RGB color histogram of the 32-bin interval is used to represent the color feature. The texture and color features are spliced ​​into a 64-dimensional vector and input into the pre-trained SVM classifier, which uses the RBF kernel function, C=10, gamma=0.1, to classify the occluded object as glass, plastic or metal.For glass, an alpha blending algorithm was used with an initial alpha value of 0.7. For translucent plastics, a contrast-limited adaptive histogram equalization (CLAHE) algorithm was employed with a block size of 8x8 pixels and a contrast limit threshold of 3. For opaque metals, a bilateral filter was applied with a spatial standard deviation of 3 and a range standard deviation of 50. Algorithm parameters were optimized through a grid search, searching for alpha values ​​within the range of 0.5-0.9 in steps of 0.1. CLAHE block sizes ranged from 4x4 to 16x16, and the two standard deviations of the bilateral filter were searched within the ranges of 1-5 and 30-70, respectively. The structural similarity index (SSIM) was calculated after each iteration, stopping when the SSIM exceeded 0.9 or the number of iterations reached 50. Finally, the edge strength and texture complexity of the target product area before and after the treatment were compared, and the percentage improvement in visibility was calculated. A value greater than 30% was considered significant and used as a basis for assessing the impact of occlusion.

[0080] For opaque occluders, the true value of the occluded pixel is estimated by searching for similar image blocks in the neighborhood of the occluded area, and the original image content is replaced with the repaired result. For semi-transparent and transparent occluders, multiple frames of images of the occluded area are obtained, and the image changes caused by the movement of the occluder between different frames are used to estimate the transmittance of the occluder. The transmittance is combined with the background image information to restore the target product image without the occlusion effect.

[0081] A real-time video sequence captured by a camera in a workshop is obtained, wherein the video sequence includes multiple continuous frame images; the continuous frame images are processed using a U-Net semantic segmentation algorithm to obtain the occlusion area in each frame image; a pre-trained ResNet model is used to classify the occlusion area and determine the transparency attribute of the occlusion object; if the occlusion object is an opaque occlusion object, the PatchMatch algorithm is used to perform image restoration on the occlusion area; if the occlusion object is a semi-transparent or transparent occlusion object, the multi-scale Lucas-Kanade method is used to estimate the optical flow field of the continuous frame images; background modeling is performed using a Gaussian mixture model, and a weighted linear mixture model is established based on the optical flow field estimation result and the background model; the inverse problem of the weighted linear mixture model is solved to obtain a target product image with occlusion removed; adaptive histogram equalization is applied to the target product image to obtain a restored image with enhanced contrast; a structural similarity index between the restored image and the original image is calculated to determine whether the structural similarity index is lower than a preset threshold; if the structural similarity index is lower than the preset threshold, the image is reprocessed.

[0082] For example, a real-time video sequence captured by a high-definition camera in a workshop is acquired. Occlusions in each frame are extracted using the U-Net semantic segmentation algorithm. A pre-trained ResNet model is used to classify the occlusions and determine their transparency. Based on the classification results, the occlusions are classified into three categories: opaque, semi-transparent, and transparent, providing a basis for subsequent processing. For opaque occlusions, a search radius is set around the occlusion area, and the PatchMatch algorithm is used for image inpainting. A nearest neighbor field is constructed, and matching results are iteratively optimized through random sampling and local propagation. The image patch with the highest similarity score is selected as the inpainting source, and bilinear interpolation is used to estimate the true value of the occluded pixel. Poisson image editing techniques are used to seamlessly blend the inpainted result into the original image, replacing the content of the occluded area. For semi-transparent and transparent occlusions, multiple consecutive frames containing the target product are extracted from the video sequence. An image pyramid is constructed, and the multi-scale Lucas-Kanade method is used to estimate optical flow and calculate the motion vector of the occlusion between consecutive frames. This iterative solution, layer by layer, from coarse to fine, improves the accuracy and robustness of motion estimation. Based on motion vectors, pixel changes in occluded areas between frames are analyzed to establish a set of transmittance estimation equations. These equations are solved using the least squares method to obtain transmittance estimates for each pixel of the occluded object. A Gaussian mixture model is used to model the background, initializing multiple Gaussian components to represent the background distribution at each pixel. The model parameters are iteratively updated using the expectation-maximization algorithm to adapt to scene changes. A weighted linear mixture model is constructed by combining the estimated transmittance with the background model. The inverse problem of the mixture model is solved to isolate the effects of the occluded object and restore the target product image without the occlusion. Adaptive histogram equalization is applied to the restored image to enhance image contrast. Finally, a structural similarity index is calculated between the restored image and the original image to assess the restoration quality. If the index falls below a preset threshold, a reprocessing mechanism is triggered. A 4K HD camera with a resolution of 1920x1080 pixels is installed in the workshop, capturing a real-time video stream at 30 fps. Each frame is processed using the U-Net semantic segmentation algorithm, with a network input size of 512x512 pixels. Occluded regions are extracted using a five-layer encoder and decoder architecture. Subsequently, a pre-trained ResNet-50 model was used to classify the occluded areas. The input size was 224x224 pixels, and the last layer was replaced with 3 output nodes, corresponding to the three categories of opaque, semi-transparent, and transparent. The classification threshold was set to 0.7, and the category with a confidence level higher than the threshold was used as the final judgment result. For opaque occluders, the PatchMatch algorithm set the search radius to 1.5 times the side length of the occluded area, the patch size to 8x8 pixels, and the number of iterations to 5. In each iteration, 100 candidate matching points were randomly sampled, and the best match was selected as the repair source. Bilinear interpolation was used to estimate the occluded pixel values, and then Poisson image editing was applied, and the number of iterations of solving the Poisson equation was set to 50.For semi-transparent and transparent occluders, a four-layer image pyramid was constructed, with a scale factor of 0.5 per layer. The Lucas-Kanade method used a 3x3 pixel local window, with a maximum number of iterations set to 20 and a convergence threshold of 0.01 pixel. Transmittance was estimated using the least squares method with a regularization parameter λ set to 0.1. Background modeling used five Gaussian components, a learning rate α of 0.01, and a background threshold T of 0.7. The expectation-maximization algorithm was run for 50 iterations or stopped when the log-likelihood function changed by less than 1e-6. During image restoration, the weights of the mixture model were solved using the ADMM algorithm with a maximum number of iterations of 100 and a convergence threshold of 1e-4. Adaptive histogram equalization was performed with a block size of 8x8 pixels and a contrast limit parameter of 3. Finally, the SSIM index was calculated. If it fell below 0.85, the process was retried up to three times. The restoration process ran on a workstation equipped with an NVIDIA RTX 3090 GPU, with an average processing time of 50 ms per frame, meeting real-time requirements.

[0083] In some embodiments, in S300, the steps of obtaining an estimated distance and azimuth between a target product and a Bluetooth beacon, detecting the position of an obscured target product based on a preprocessed video image, spatially registering the target product's position detection result with the Bluetooth beacon data, obtaining an estimated distance and azimuth of the Bluetooth beacon relative to the target product based on Bluetooth positioning, detecting and locating the target product by constructing a spatial topological relationship between the product and the beacon, and obtaining the target product's position coordinates and bounding box information may include:

[0084] S310, obtaining signal strength data transmitted by Bluetooth beacons, selecting a signal strength attenuation model based on the signal strength data; and calculating an estimated distance between the target product and each Bluetooth beacon based on the signal strength attenuation model;

[0085] For example, the signal strength data transmitted by the Bluetooth beacons arranged in the workshop is obtained, and the signal strength attenuation model is adaptively selected based on environmental factors. The free space path loss model is used in open space, and the logarithmic distance path loss model is used in multi-obstacle environments to calculate the estimated distance between the target product and each Bluetooth beacon. The formula for the free space path loss model is:

[0086]

[0087] Where PL fs represents the free space path loss, d represents the distance between the transmitting point and the receiving point, f represents the frequency of the signal, and c represents the speed of light.

[0088] The formula for the logarithmic distance path loss model is:

[0089]

[0090] Where PL log represents the logarithmic distance path loss, PL0 represents the path loss at the reference distance d0, n represents the path loss exponent, and d represents the distance between the transmitting point and the receiving point.

[0091] S320, analyzing the preprocessed image to obtain position and size information of the target product in the image, and determining an initial position of the target product by combining the estimated distance and the position and size information of the target product in the image;

[0092] S330, calculating the distance and azimuth of the target product relative to each beacon, and using an adaptive weighting method based on signal strength to perform a weighted average of the distance and azimuth of the target product relative to each beacon to obtain a positioning result for the target product; wherein the weight in the adaptive weighting method is positively correlated with the signal strength and is mapped to a range of 0 to 1 using a sigmoid function;

[0093] S340: Establishing a spatial topological relationship between the target product and the Bluetooth beacon based on the initial position and positioning result of the target product. Based on the spatial topological relationship and in combination with the initial position and positioning result of the target product, employing an extended Kalman filter algorithm to estimate and track the position of the target product in real time; wherein the state vector includes position, velocity, and acceleration, and the observation vector includes the image detection position and the Bluetooth positioning result;

[0094] S350 uses an extended Kalman filter algorithm to estimate the initial position of the target product in real time to optimize the target product's position coordinates. Furthermore, a particle filter algorithm is introduced to address occlusion situations. This algorithm maintains multiple hypothetical particles to handle multimodal distributions and improve tracking stability under occlusion. The state vector of the extended Kalman filter algorithm includes position, velocity, and acceleration, and the observation vector includes image detection position and Bluetooth positioning results.

[0095] S360 maps the optimized position coordinates back to the image plane and combines them with the bounding box information of the target detection to obtain the precise position and size of the target product in the image.

[0096] Exemplarily, the pre-processed image is used as input, and the YOLOv5 target detection algorithm is used to detect the obscured target product to obtain the position and size information of the target product in the image. The signal strength estimated distance and the image detection result are combined to roughly estimate the initial position of the target product through triangulation. According to the internal and external parameter matrix of the camera, the target product detection result in the image coordinate system is converted to the world coordinate system. The known coordinate information of the Bluetooth beacon is used, combined with the estimated distance calculated by signal strength, and the initial position estimate of the target product is optimized by the least squares method. For the deviation between the optimization result and the detection result, the iterative nearest point algorithm is used for spatial registration. Feature points in the image and three-dimensional space are extracted, and the corresponding relationship is established. The rigid body transformation matrix is ​​calculated by SVD decomposition, and iterative optimization is performed until convergence or the maximum number of iterations is reached. The signal strength data of at least three Bluetooth beacons are used, combined with the triangulation method to calculate the precise distance of the target product relative to each beacon, and the azimuth of the target product relative to each beacon is calculated by combining the data of the Bluetooth beacon and the triangulation method. The formula for solving the azimuth of the target product relative to each beacon is:

[0097]

[0098] Where θ i represents the azimuth of the target product relative to the i-th beacon, x i and y i They represent the x and y coordinates of the i-th beacon, respectively, and x0 and y0 represent the x and y coordinates of the target product.

[0099] An adaptive weighting method based on signal strength is used to perform a weighted average of multiple positioning results. The weight is positively correlated with signal strength and is mapped to a range of 0 to 1 using a sigmoid function. A spatial topological relationship diagram is established between the target product and the Bluetooth beacon based on the fused positioning results and the registered detection results. Based on this established spatial topological relationship, the extended Kalman filter algorithm is used to estimate and track the target product's position in real time, combining image detection results and Bluetooth positioning results. The state vector contains position, velocity, and acceleration, and the observation vector contains image detection position and Bluetooth positioning results. The target product's position coordinates are continuously optimized through an iterative process of prediction and update. To address occlusion, a particle filter algorithm is introduced to handle multimodal distributions by maintaining multiple hypothetical particles, improving tracking stability under occlusion. The optimized position coordinates are mapped back to the image plane and combined with the bounding box information from target detection to obtain the target product's precise position and size in the image.

[0100] Ten Bluetooth 5.1 beacons were deployed throughout the workshop, with a spacing of 5 meters and a transmit power of 0 dBm. An adaptive signal strength attenuation model was employed: a free-space path loss model with a path loss exponent of n = 2 in open areas and a logarithmic distance path loss model with a path loss exponent of n = 3.5 in areas with multiple obstacles. The estimated distance between the target product and each beacon was calculated using the received signal strength indicator (RSSI) value, with a measurement error within ±0.5 meters. Images were captured using a high-definition camera with a resolution of 1920x1080 pixels and a frame rate of 30 fps. The YOLOv5s model was used to detect target products, with an input image size of 640x640 pixels. The model output the bounding box coordinates of each detected target product, with a confidence threshold of 0.5 and an NMS threshold of 0.4. Combining the RSSI distance estimate with the image detection results, triangulation was used to preliminarily estimate the target product's position, with an accuracy of approximately ±1 meter. The camera was calibrated using the Zhang Zhengyou calibration method, with a checkerboard grid size of 9x6, and each square measuring 25mmx25mm. The camera's intrinsic and extrinsic parameter matrices were calculated using the DLT algorithm, with reprojection error kept within 0.1 pixel. Spatial registration was performed using the ICP algorithm with SVD decomposition, with a maximum number of iterations set to 50 and a convergence threshold of 0.001 meter. Trilateration and the triangle centroid method were combined for precise positioning, using a sigmoid function based on RSSI for weight calculation, with parameters k = 0.1 and x0 = -70dBm. In the extended Kalman filter, the state vector contains 3D position, velocity, and acceleration, for a total of nine variables. The observation vector contains 2D coordinates from image detection and 3D coordinates from Bluetooth positioning, for a total of five variables. The process noise covariance matrix Q and the measurement noise covariance matrix R were obtained through offline data analysis. To address occlusion, a 500-particle SIR particle filter was used, with the resampling threshold set to 50% of the number of valid particles. The final target product positioning accuracy reaches ±0.2 meters, and the bounding box detection accuracy reaches above IoU0.85.

[0101] In some embodiments, in S400, the tracking and predicting of the motion state of the target product based on the location coordinates and bounding box information of the target product in combination with Bluetooth positioning data to obtain the motion trajectory and speed information of the target product includes:

[0102] S410, obtaining the location coordinates, bounding box information, and Bluetooth positioning data of the target product;

[0103] S420, using a Kalman filter algorithm to fuse the position coordinates, the bounding box information, and the Bluetooth positioning data to obtain fused position data; wherein the state vector of the Kalman filter algorithm includes a three-dimensional position and velocity, and the observation vector includes a two-dimensional coordinate and a three-dimensional coordinate;

[0104] S430: Fitting the target product's motion trajectory using the least squares method based on the fused position data. Constructing a motion state model of the target product based on the motion trajectory and the fused position data; wherein the motion state model includes a linear regression model and a long short-term memory network model;

[0105] S440 : Predicting the future trajectory of the target product based on the motion state model. The linear regression model is used to predict the short-term future position, and the long short-term memory network model is used to capture the long-term motion pattern. The future trajectory is optimized using a particle filter algorithm to obtain an optimized future trajectory.

[0106] For example, the target product's location coordinates, bounding box information, and Bluetooth positioning data are obtained. The image detection results and Bluetooth positioning data are fused using a Kalman filter algorithm. The state vector contains the 3D position and velocity, and the observation vector contains the 2D coordinates from the image detection and the 3D coordinates from the Bluetooth positioning. A sliding window method is used to process multiple frames of data, with a window size of 10 frames and a step size of 1 frame. A time series model is constructed to capture the target product's motion trends. Based on the fused position data, the least squares method is used to fit the target product's motion trajectory. By comparing the position differences between adjacent time points, the target product's velocity components in the x, y, and z directions are calculated. The calculated velocity is smoothed using an exponentially weighted moving average algorithm with a weighting factor of 0.2 to eliminate noise and obtain a more stable velocity estimate. Based on the historical trajectory and velocity information, a motion state model of the target product is constructed. A linear regression algorithm is used to predict the short-term future position, while a long short-term memory network is used to capture long-term motion patterns, enabling prediction of the target product's future trajectory. The LSTM network consists of two hidden layers, each with 128 neurons, and its input features are the position and velocity data from the past 20 time steps. By weightedly merging the two prediction results, a comprehensive predicted trajectory is generated. Prediction accuracy is assessed using root mean square error (RMS) and mean absolute percentage error (MAPE). If the error exceeds a preset threshold, an anomaly detection mechanism is triggered. A particle filter algorithm is used to optimize the predicted trajectory, taking into account environmental constraints such as factory layout and equipment location. 1000 particles are set to represent possible future states, and particle weights are continuously updated through importance sampling and resampling. By combining the predicted trajectory with actual observation data, the prediction model parameters are dynamically adjusted to improve prediction accuracy. The optimized target product trajectory and velocity information are output for subsequent production scheduling and quality control. A sliding variance detection method is used to identify outliers for sudden changes in motion patterns. Model retraining is triggered when the variance exceeds three times the normal range. A high-precision camera system and Bluetooth positioning network are deployed throughout the workshop. The cameras have a 4K resolution (3840x2160 pixels) and a capture frame rate of 60 fps. The Bluetooth beacons use BLE 5.1 ​​technology with a 50ms signal transmission interval. The target product's location coordinates are obtained through image processing, achieving centimeter-level accuracy. Bounding box information is provided by the YOLOv5 object detection algorithm, with a model input size of 416x416 pixels and a detection confidence threshold of 0.6. The Kalman filter state vector contains six variables: [x, y, z, vx, vy, vz], and the observation vector contains five variables: [x_img, y_img, x_ble, y_ble, z_ble]. The sliding window size is set to 10 frames, totaling approximately 167ms. A quadratic polynomial curve is fitted using the least squares method as the local trajectory. Velocity is calculated using the central difference method with a time step of 16.7ms. The exponentially weighted moving average smoothing factor α is 0.2 to suppress velocity noise.The LSTM network input is a 20-time-step sequence of [x, y, z, vx, vy, vz]. The two hidden layers each contain 128 LSTM units. The output layer is a fully connected layer that predicts the position for the next five time steps. Linear regression and LSTM predictions are weighted and fused in a 6:4 ratio. Prediction performance is evaluated using the RMSE and MAPE metrics, with thresholds set to 0.1 meters and 5%, respectively. The particle filter uses 1000 particles, each containing position and velocity information. The resampling threshold is 50% of the number of valid particles. Environmental constraints consider a 3D factory model, with equipment occupied space as a no-go zone. Anomaly detection uses a 20-frame sliding window to calculate variance, with anomalies identified when the variance exceeds three times the mean.

[0107] In some embodiments, in S500, during the target product tracking process, if an obstruction blocks the target product, resulting in tracking failure, a local search is performed in an area where the target product may appear based on the target product's historical motion trajectory and the current Bluetooth positioning estimated position until the target product's motion trajectory and speed information are recaptured, including:

[0108] S510, receiving consecutive frame images and target detection confidence information, and determining whether occlusion has caused tracking failure based on the consecutive frame images and target detection confidence information. If the target detection confidence is lower than a preset threshold or the number of consecutive undetected frames exceeds a preset number of frames, a target recapture mechanism is triggered;

[0109] S520, obtaining the location and time information of the last successful tracking of the target product, and using the Kalman filter algorithm to predict the current location of the target product;

[0110] S530, receiving target product position estimation information, fusing the Kalman filter prediction result and the Bluetooth positioning result through a covariance-based adaptive weight allocation method to obtain a fused estimate of the target product position;

[0111] S540, determining a local search area based on the fused estimate of the target product position, and employing an adaptive grid search algorithm within the local search area to recapture the target; if the target is detected within a certain grid, performing a refined search on the grid; receiving continuous multi-frame tracking results, and confirming successful target capture if the number of continuous tracking frames exceeds a preset frame number threshold;

[0112] S550 , obtaining historical trajectory information, Bluetooth positioning data, and new visual tracking results of the target product, back-estimating the motion state of the target product during the occlusion period, and obtaining the motion trajectory and speed information of the target product.

[0113] Exemplarily, during the target tracking process, continuous frame image analysis and target detection confidence evaluation are used to determine whether tracking failure occurs due to occlusion. If the confidence output by the target detection algorithm is lower than 0.5 or the target is not detected for 5 consecutive frames, the target recapture mechanism based on Bluetooth positioning is triggered. The location and time information of the last successful tracking of the target product are obtained as the starting point for recapture. Based on the historical motion trajectory data of the target product, the Kalman filter algorithm is used to predict the current possible location of the target product. At the same time, the current location of the target product estimated by the Bluetooth positioning system is obtained. The Kalman filter prediction results and the Bluetooth positioning results are fused through a covariance-based adaptive weight allocation method to obtain the most likely location estimate of the target product. Based on the fused position estimate and its uncertainty, a local search area centered on the position is determined. Within the determined local search area, an adaptive grid search algorithm is used to recapture the target.

[0114] The initial grid size is set to 1 meter by 1 meter and dynamically adjusted based on the uncertainty of the predicted position. Object detection is performed at each grid center using a modified YOLOv5 algorithm, incorporating a spatial attention mechanism to improve small object detection. The detection threshold is lowered to 0.3 to improve recall. If an object is detected within a grid, that area becomes the focus search zone, and the grid is refined to 0.2 meter by 0.2 meter for precise localization. A preliminary evaluation of the search results is performed, calculating the deviation between the detected object and the expected position. Once the target product is redetected during the local search, a multi-frame confirmation mechanism is immediately initiated, continuously tracking the target for at least five frames to verify the reliability of the capture. If capture is confirmed, normal tracking is resumed, updating the target product's trajectory and velocity information. Simultaneously, a particle filter algorithm using 1000 particles is used to back-estimate the target product's motion state during occlusion, combining historical trajectory data, Bluetooth positioning data, and new visual tracking results. Particle weights are updated based on consistency with observed data, using a resampling strategy with a threshold set to 50% of the number of valid particles. If the target is not reacquired after 20 consecutive frames, the search range is doubled and other available sensor data, such as infrared or ultrasonic sensors, is used for auxiliary positioning. A high-definition camera array with a 4K resolution (3840x2160 pixels) and a frame rate of 60fps is deployed in the factory environment, covering the entire production area. A Bluetooth 5.1 beacon network, spaced 5 meters apart, is also installed to achieve centimeter-level positioning accuracy. Object tracking uses the YOLOv5 algorithm with a channel attention mechanism, and the model input size is 640x640 pixels. During normal tracking, the detection confidence threshold is set to 0.5. Reacquisition is triggered when the detection confidence falls below the threshold or the target is not detected for 5 consecutive frames (approximately 83ms). The Kalman filter state vector contains six variables, including position and velocity, and the observation vector contains three variables, including position. Bluetooth positioning data is updated every 50ms. Adaptive weight fusion uses a weight calculation method based on the Mahalanobis distance, with weights inversely proportional to the covariance matrix. The initial search area is set to a 5x5m area around the last known position, with a grid size of 1mx1m. During the recapture phase, the YOLOv5 model lowers the detection threshold to 0.3 and adds additional attention weight to objects smaller than 50% of their original size. The grid refinement strategy is to divide the grid into four equal parts after detecting the target, refining it to a minimum of 0.2m x 0.2m. A multi-frame confirmation mechanism requires stable detection for approximately 83ms over five consecutive frames to be considered a successful capture. The particle filter uses 1000 particles, each containing position and velocity information, and the weight update frequency matches the image frame rate of 60Hz. The resampling threshold is 500 valid particles. If recapture is not achieved within approximately 333ms after 20 frames, the search range is expanded to 10m x 10m, and an infrared thermal imaging camera with a resolution of 640x480 and a frame rate of 30fps is activated to assist in the search.The recapture process runs on an edge server equipped with NVIDIA RTX3090 GPU, with an average processing delay of less than 20ms, meeting real-time requirements.

[0115] If the target product is partially obscured, the particle filter algorithm is used to predict its position and motion, combining it with the visible features of the target product to perform a match and correct the prediction. If the target product is completely obscured, the particle filter algorithm alone is used to predict its position and motion. While the target product is obscured, the prediction results and the position estimated by Bluetooth positioning are used to predict the motion area of ​​the obstructing object, and a local search is performed within this area.

[0116] Real-time image data of the target product is acquired and segmented using a U-Net semantic segmentation network to determine the visible area ratio of the target product. If the visible area ratio is less than a preset threshold and greater than zero, the target product is considered partially occluded; if the visible area is zero, the target product is considered fully occluded. Based on the results, a particle filter algorithm is used to predict the target product's position and motion for partial occlusion. The number of particles is set to a preset value, and each particle contains position, velocity, and acceleration information. The particle swarm is initialized using historical motion data and its state is updated. For full occlusion, a weighted fusion method is used to predict the target product's position and motion, combined with Bluetooth positioning data. The Bluetooth positioning results are introduced as additional observations into the particle filter update process. Particles are updated using sequential importance resampling to maintain particle diversity. A probability distribution map of the target product's motion area is constructed based on the particle filter prediction results, the Bluetooth positioning estimated position, and the predicted motion of the obstructing object. This probability distribution map is discretized into a grid with a preset grid size. Grids that are within a preset percentage of the probability value are selected as key search areas. A spiral search pattern with adaptive step size is used, expanding outward from the point of highest probability. The probability distribution map is updated during the search process until the target product is recaptured or the preset maximum number of searches is reached.

[0117] For example, real-time image data of a target product is acquired. The target product is segmented using a U-Net semantic segmentation network, and the area ratio of the visible portion of the target product is calculated. If the visible area ratio is less than 30% but greater than zero, it is considered partially occluded; if the visible area is zero, it is considered fully occluded. Based on the determination results, a corresponding tracking strategy is selected. For partial occlusion, a particle filter algorithm is used to predict the target product's position and motion information. The number of particles is set to 1000, with each particle containing position, velocity, and acceleration information. The particle swarm is initialized using historical motion data, and the state of each particle is updated using a motion model. Features of the visible portion of the target product are extracted, including color histograms and HOG features. The HOG features have a cell size of 8x8 pixels and a block size of 2x2 cells. These features are matched to the predicted particle positions, and the weight of each particle is calculated. The root mean square error (RMS) is used to evaluate prediction accuracy. If the error exceeds a preset threshold, the number of particles is increased or the noise parameters are adjusted. For fully occluded cases, predictions are made solely using the particle filter algorithm. The number of particles is increased to 2000, expanding the particle distribution range to account for greater uncertainty. Combined with Bluetooth positioning data, the adaptive weight fusion method of covariance intersection criterion is adopted to introduce Bluetooth positioning results as additional observations into the particle filter update process. Particles are updated through sequential importance resampling to maintain particle diversity. At the same time, An optical flow algorithm estimates the motion of occluders and predicts their possible motion areas. Based on the particle filter predictions, the Bluetooth location estimate, and the occluder motion predictions, a probability distribution map of the target product's possible motion areas is constructed. The probability distribution map is discretized into a grid, each sized 0.5 m x 0.5 m. Grids with the top 10% probability values ​​are selected as key search areas, and a localized search is conducted within these areas. A spiral search pattern with an adaptive step size is employed, with an initial step size of 0.5 m, increasing by 0.1 m with each rotation, expanding outward from the point of highest probability. The probability distribution map is continuously updated during the search process until the target product is recaptured or a maximum of 100 searches are completed. A 4K resolution, 3840 x 2160 pixel high-speed camera with a frame rate of 60 fps is deployed in the factory environment. A U-Net semantic segmentation network is used, with an input size of 512 x 512 pixels, a four-layer encoder and decoder structure, and a frame rate of 30 ms. Partial occlusion is triggered when the visible area of ​​the target product falls below 30%, while full occlusion is triggered when the visible area falls below 0%. The particle filter algorithm initializes 1000 particles, and the state vector contains 3D position, velocity, and acceleration, for a total of 9 variables. When extracting HOG features, the cell size is set to 8x8 pixels, the block size is 2x2 cells, and the step size is 4 pixels, resulting in a 3780-dimensional feature vector. The color histogram uses three RGB channels, with 16 bins per channel, for a total of 48-dimensional features. Particle weights are calculated using the Euclidean distance of feature matching, normalized, and importance resampling performed. The prediction RMSE threshold is set to 0.5 meters; if exceeded, the number of particles is increased to 1500. In the case of complete occlusion, the number of particles increases to 2000, and the particle distribution range is expanded to a 3-meter area around the last known position. The Bluetooth 5.1 positioning system provides centimeter-level accuracy data and uses the covariance intersection criterion to calculate adaptive weights, ranging from 0.1 to 0.9. The optical flow algorithm parameters were set to 3 pyramid levels, 15 window sizes, and 3 iterations. The probability distribution map resolution was centimeter-level, with a 100x100 grid covering a 5mx5m area. The spiral search had an initial step size of 0.5 meters, increasing by 0.1 meters per turn. The maximum number of searches was 100, or a coverage area of ​​25 square meters. The algorithm ran on an NVIDIA Jetson AGX Xavier with an average processing latency of less than 16.7ms (1 / 60 second), meeting real-time requirements.

[0118] In some embodiments, in S600, based on the motion trajectory and speed information of the target product, the depth information of the target product is obtained through a depth camera, and combined with the distance and azimuth estimated by Bluetooth positioning, a three-dimensional representation of the target product is constructed, and the position and tracking motion information of the indoor target product is generated and transmitted to the indoor workshop production management system to achieve target positioning of indoor products obscured during the production process.

[0119] S610, obtaining motion trajectory and speed information of the target product, and smoothing the information using a Kalman filter algorithm to obtain filtered motion data;

[0120] S620, collecting a depth image of the target product through a depth camera and extracting a depth profile of the target product;

[0121] S630, performing illumination unevenness correction on the depth image using a three-scale Retinex algorithm, and improving detail representation of the depth image using a CLAHE algorithm;

[0122] S640, constructing a three-dimensional point cloud representation of the target product based on the processed depth information and Bluetooth positioning data;

[0123] S650: align the 3D point cloud representation with a pre-established CAD model of the target product to determine the precise pose of the target product. If the mean absolute error of the precise pose exceeds a preset threshold, an anomaly detection is triggered.

[0124] S660: Pack the precise position and filtered motion data and transmit them to an indoor workshop production management system via industrial Ethernet.

[0125] For example, the target product's trajectory and velocity are acquired, and the Kalman filter algorithm is used to smooth the data and eliminate noise. A depth camera captures a depth image of the target product, and an adaptive threshold segmentation algorithm is used to extract its depth contour. The window size is set to 51x51 pixels, and the C value is 2. Combined with distance information estimated by the Bluetooth positioning system, triangulation is used to calculate the target product's initial 3D coordinates. To address the complex lighting conditions in the workshop, a three-scale Retinex algorithm is used to correct for uneven illumination in the depth image. The scales are 15, 80, and 250 pixels, with a weighting ratio of 1:2:1. The CLAHE algorithm is used to enhance detail in the depth image, with a block size of 8x8 pixels and a contrast threshold of 3. Morphological opening is used to remove noise and small areas from the depth image, with a structuring element size of 3x3 pixels. The signal-to-noise ratio of the processed depth image is evaluated, and if it falls below a preset threshold of 15dB, a re-acquisition mechanism is triggered. A 3D point cloud representation of the target product is constructed based on the processed depth information and Bluetooth positioning data. Multi-frame point cloud data was registered using an iterative closest point algorithm, with a maximum number of iterations set to 50 and a convergence threshold of 0.001 meters. The point cloud was downsampled using a voxel grid filtering algorithm, with a voxel size set to 0.01 meters, to reduce data volume while preserving geometric features. The resulting 3D model was aligned with a pre-established CAD model of the target product using a robust model matching method based on RANSAC, with an inlier threshold set to 0.005 meters and 1000 iterations. Principal component analysis was used to determine the principal axis orientations of the model, and the rigid body transformation matrix was calculated to obtain the precise pose of the target product. High-precision position and tracking motion information was generated by combining the motion trajectory and velocity information processed by Kalman filtering. Positioning accuracy was assessed using mean absolute error and root mean square error, with thresholds set to 0.05 meters and 0.08 meters, respectively. If the error exceeded the threshold, an anomaly detection mechanism was triggered, using the sliding variance method to identify sudden positioning errors. The data was packaged into a specified format and transmitted to the indoor workshop production management system via Industrial Ethernet, enabling the localization of products obscured by the production process. Intel RealSense D455 depth cameras were deployed throughout the workshop, with a resolution of 1920x1080 pixels, a frame rate of 30fps, and a depth range of 0.6-6 meters. A Bluetooth 5.1 positioning system was also installed, with a beacon spacing of 3 meters and a positioning accuracy of ±10 cm. The target product's motion trajectory data was collected at a frequency of 50 Hz, and the Kalman filter state vector contained 3D position and velocity, a total of 6 variables. Adaptive threshold segmentation used a 51x51 pixel window with a C value of 2, resulting in a processing time of less than 5 milliseconds. The three-scale Retinex algorithm was set to scales of 15, 80, and 250 pixels, with a weighting of 1:2:1, and a runtime of approximately 20 milliseconds. The CLAHE algorithm used 8x8 pixel blocks, a contrast limit threshold of 3, and a processing time of 7 milliseconds. The structuring element of the morphological opening operation was a 3x3 pixel square.The depth image signal-to-noise ratio threshold is 15dB. A value below this triggers resampling, with a maximum of three retries. The ICP algorithm has a maximum iteration count of 50, a convergence threshold of 0.001 meters, and an average registration time of 100 milliseconds. The voxel grid filter has a voxel size of 0.01 meters, and the point cloud downsampling rate is approximately 80%. The RANSAC model uses an inlier threshold of 0.005 meters, 1000 iterations, and a matching time of approximately 200 milliseconds. The PCA principal axis orientation is calculated using singular value decomposition, with the eigenvector corresponding to the maximum singular value. Positioning accuracy is evaluated using a 20-frame sliding window, with a MAE threshold of 0.05 meters and a RMSE threshold of 0.08 meters. Anomaly detection uses a 5-second sliding window, and anomalies are identified when the variance exceeds three standard deviations. Data is packaged in JSON format and transmitted over 100Mbps industrial Ethernet with a latency of less than 10 milliseconds. The indoor workshop production management system runs on edge computing devices equipped with NVIDIA Jetson Xavier NX, with an average processing latency of 33 milliseconds, meeting real-time requirements.

[0126] and Figure 1 Corresponding to the method, refer to Figure 2 , an embodiment of the present invention provides an indoor target positioning system based on machine vision and deep learning, comprising:

[0127] at least one processor;

[0128] at least one memory for storing at least one program;

[0129] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0130] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0131] In addition, an embodiment of the present invention further discloses a computer program product or computer program, which is stored in a computer-readable storage medium. A processor of a computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the above-mentioned method. Similarly, the contents of the above-mentioned method embodiment are applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.

[0132] Those skilled in the art will appreciate that all or some of the methods disclosed above, the system can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data) and is volatile and non-volatile, removable and non-removable media. Computer storage media includes but is not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0133] The above is a specific description of the preferred implementation of the present disclosure, but the present disclosure is not limited to the above-mentioned implementation mode. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A method for indoor target positioning based on machine vision and deep learning, characterized in that: The method comprises the following steps: S100: Acquire real-time video images of the workshop, analyze the movement trajectory and size of obstructions in the images, determine the frequency and duration of the obstructions, and determine the degree of impact of the obstructions on the target product; S200 , preprocessing the video image based on the degree of impact of the occlusion on the target product, analyzing the transparency and refractive index properties of the pixels in the occluded area, inferring the material type of the occluded object, and selecting a occluded area processing method for that material type to obtain a preprocessed video image; S300: Obtain an estimated distance and azimuth between the target product and the Bluetooth beacon, perform position detection on the obscured target product based on the preprocessed video image, spatially align the target product's position detection result with the Bluetooth beacon data, obtain an estimated distance and azimuth of the Bluetooth beacon relative to the target product based on Bluetooth positioning, and detect and locate the target product by constructing a spatial topological relationship between the product and the beacon, obtaining the target product's position coordinates and bounding box information. S400: Tracking and predicting the motion state of the target product based on the location coordinates and bounding box information of the target product in combination with Bluetooth positioning data to obtain the motion trajectory and speed information of the target product; S500: During the target product tracking process, if an obstruction obstructs the target product, causing tracking failure, a local search is performed within an area where the target product may appear, based on the target product's historical motion trajectory and the current Bluetooth positioning estimated position, until the target product's motion trajectory and velocity information are recaptured; the area where the target product may appear is a local search area centered on a fused estimate of the target product's position, obtained by fusing a Kalman filter prediction result and a Bluetooth positioning result using a covariance-based adaptive weight allocation method; S600, based on the target product's motion trajectory and speed information, obtains the target product's depth information through a depth camera, combines it with the distance and azimuth estimated by Bluetooth positioning, constructs a three-dimensional representation of the target product, generates the position and tracking motion information of the indoor target product, and transmits it to the indoor workshop production management system to achieve target positioning of indoor products that are obscured during the production process.

2. The method according to claim 1, characterized in that In S100, the real-time video images of the workshop are acquired, and the frequency and duration of the occlusions are determined by analyzing the movement trajectory and size of the occlusions in the images, thereby determining the degree of impact of the occlusions on the target products. This includes: S110, acquiring real-time video streams captured by multiple cameras, and performing image preprocessing on the video streams, wherein the image preprocessing includes median filtering, histogram equalization, and Sobel operator edge sharpening; S120, constructing a scene background using a Gaussian mixture model, performing a difference operation between the current frame and the scene background to obtain an initial outline of the occluder; S130, matching and associating the initial outline of the occluder between consecutive frames to obtain a motion trajectory of the occluder on the image plane; S140, performing a time series analysis on the movement trajectory of the obstruction, calculating the number of times the obstruction appears in a specific area and the length of time it stays there, and obtaining statistical data on the frequency and duration of the obstruction; S150, based on the spatial position and time distribution of the obstruction, find the affected target products and analyze the impact of the obstruction on each target product; set differentiated impact weights for different production stages to determine the degree of impact of the obstruction on product quality and production efficiency.

3. The method according to claim 1, characterized in that In S200, based on the degree of impact of the occlusion on the target product, the video image is preprocessed, the transparency and refractive index properties of the pixels in the occluded area are analyzed, the material type of the occluded object is inferred, and the occluded area processing method of the material type is selected to obtain the preprocessed video image, including: S210, acquiring a real-time video image captured by a camera, locating the outline of the obstruction, and obtaining a pixel set of the obstruction area; S220, calculating brightness distribution characteristics and gradient information based on the pixel set of the occluded area, analyzing the spatial relationship between the occluded area and the target product in combination with the location information of the target product, and determining a preliminary impact range; S230, extracting a material feature vector based on the texture features and color histogram of the occluded area, and mapping the feature vector to a predefined material category using a support vector machine classifier; if the material category is glass, processing the occluded area using an alpha blending algorithm; if the material category is translucent plastic, processing the occluded area using a transparency-based image enhancement algorithm; if the material category is opaque metal, processing the occluded area using an edge-preserving filtering algorithm; S240 , optimizing parameters of the occluded area processing method, traversing the parameter space using a grid search method, evaluating the quality of the processing result using a structural similarity index, and applying the optimized method to the occluded area of ​​the original image to obtain a preprocessed image.

4. The method according to claim 1, wherein In S300, the estimated distance and azimuth between the target product and the Bluetooth beacon are obtained, the position of the obscured target product is detected based on the preprocessed video image, the position detection result of the target product is spatially aligned with the Bluetooth beacon data, the estimated distance and azimuth of the Bluetooth beacon relative to the target product are obtained based on Bluetooth positioning, and the target product is detected and located by constructing a spatial topological relationship between the product and the beacon to obtain the position coordinates and bounding box information of the target product, including: S310, obtaining signal strength data transmitted by Bluetooth beacons, selecting a signal strength attenuation model based on the signal strength data; and calculating an estimated distance between the target product and each Bluetooth beacon based on the signal strength attenuation model; S320, analyzing the preprocessed image to obtain position and size information of the target product in the image, and determining an initial position of the target product by combining the estimated distance and the position and size information of the target product in the image; S330, calculating the distance and azimuth of the target product relative to each beacon, and using an adaptive weighting method based on signal strength to perform a weighted average of the distance and azimuth of the target product relative to each beacon to obtain a positioning result for the target product; wherein the weight in the adaptive weighting method is positively correlated with the signal strength and is mapped to a range of 0 to 1 using a sigmoid function; S340: Establishing a spatial topological relationship between the target product and the Bluetooth beacon based on the initial position and positioning result of the target product. Based on the spatial topological relationship and in combination with the initial position and positioning result of the target product, employing an extended Kalman filter algorithm to estimate and track the position of the target product in real time; wherein the state vector includes position, velocity, and acceleration, and the observation vector includes the image detection position and the Bluetooth positioning result; S350 uses an extended Kalman filter algorithm to estimate the initial position of the target product in real time to optimize the target product's position coordinates. Furthermore, a particle filter algorithm is introduced to address occlusion situations. This algorithm maintains multiple hypothetical particles to handle multimodal distributions and improve tracking stability under occlusion. The state vector of the extended Kalman filter algorithm includes position, velocity, and acceleration, and the observation vector includes image detection position and Bluetooth positioning results. S360 maps the optimized position coordinates back to the image plane and combines them with the bounding box information of the target detection to obtain the precise position and size of the target product in the image.

5. The method according to claim 1, characterized in that In S400, the motion state of the target product is tracked and predicted based on the position coordinates and bounding box information of the target product in combination with the Bluetooth positioning data to obtain the motion trajectory and speed information of the target product, including: S410, obtaining the location coordinates, bounding box information, and Bluetooth positioning data of the target product; S420, using a Kalman filter algorithm to fuse the position coordinates, the bounding box information, and the Bluetooth positioning data to obtain fused position data; wherein the state vector of the Kalman filter algorithm includes a three-dimensional position and velocity, and the observation vector includes a two-dimensional coordinate and a three-dimensional coordinate; S430, fitting the motion trajectory of the target product using the least squares method based on the fused position data; constructing a motion state model of the target product based on the motion trajectory and the fused position data; wherein the motion state model includes a linear regression model and a long short-term memory network model; S440, predicting the future trajectory of the target product based on the motion state model; wherein the linear regression model is used to predict the short-term future position, and the long short-term memory network model is used to capture the long-term motion pattern; and optimizing the future trajectory using a particle filter algorithm to obtain an optimized future trajectory.

6. The method according to claim 1, wherein In S500, during the target product tracking process, if an obstruction blocks the target product, resulting in tracking failure, a local search is performed in the area where the target product may appear based on the historical motion trajectory of the target product and the current Bluetooth positioning estimated position until the motion trajectory and speed information of the target product are recaptured, including: S510, receiving consecutive frame images and target detection confidence information, and determining whether occlusion has caused tracking failure based on the consecutive frame images and target detection confidence information. If the target detection confidence is lower than a preset threshold or the number of consecutive undetected frames exceeds a preset number of frames, a target recapture mechanism is triggered; S520, obtaining the location and time information of the last successful tracking of the target product, and using the Kalman filter algorithm to predict the current location of the target product; S530, receiving target product position estimation information, fusing the Kalman filter prediction result and the Bluetooth positioning result through a covariance-based adaptive weight allocation method to obtain a fused estimate of the target product position; S540, determining a local search area based on the fused estimate of the target product position, and employing an adaptive grid search algorithm within the local search area to recapture the target; if the target is detected within a certain grid, performing a refined search on the grid; receiving continuous multi-frame tracking results, and confirming successful target capture if the number of continuous tracking frames exceeds a preset frame number threshold; S550 , obtaining historical trajectory information, Bluetooth positioning data, and new visual tracking results of the target product, back-estimating the motion state of the target product during the occlusion period, and obtaining the motion trajectory and speed information of the target product.

7. The method according to claim 1, characterized in that In S600, based on the target product's motion trajectory and speed information, the depth information of the target product is obtained through the depth camera, combined with the distance and azimuth estimated by Bluetooth positioning, to construct a three-dimensional representation of the target product, generate the position and tracking motion information of the indoor target product, and transmit it to the indoor workshop production management system to achieve target positioning of indoor products obscured during the production process, including: S610, obtaining motion trajectory and speed information of the target product, and smoothing the information using a Kalman filter algorithm to obtain filtered motion data; S620, collecting a depth image of the target product through a depth camera and extracting a depth profile of the target product; S630, performing illumination unevenness correction on the depth image using a three-scale Retinex algorithm, and improving detail representation of the depth image using a CLAHE algorithm; S640, constructing a three-dimensional point cloud representation of the target product based on the processed depth information and Bluetooth positioning data; S650, registering the three-dimensional point cloud representation with a pre-established CAD model of the target product to determine the precise pose of the target product; if the mean absolute error of the precise pose exceeds a preset threshold, triggering anomaly detection; S660: Pack the precise position and filtered motion data and transmit them to an indoor workshop production management system via industrial Ethernet.

8. An indoor target positioning system based on machine vision and deep learning, characterized in that: The system comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

Images