Defect projection tracking control method based on space-time coordinate mapping and displacement vector compensation

Abstract

The present application relates to the technical field of industrial control system, disclose a kind of defect projection tracking control method based on space-time coordinate mapping and displacement vector compensation.The present application aims to solve the technical problems of existing projection cursor lagging behind in dynamic tracking process and stretching deformation of projection pattern.The present application converts the defect bound with time stamp into original space coordinates by establishing the space conversion ratio of camera and projector;Combined with running speed, operation time consumption and mechanical response time, the displacement vector is calculated, and the one-way displacement compensation is completed by superimposing the original coordinates;According to the deflection angle, the reverse compensation coefficient is calculated, the coordinates are scaled and adjusted, and the tracking is updated cyclically.Overcome the position drift caused by system and mechanical time delay, offset the stretching distortion caused by projection tilt, realize the alignment of cursor and continuous moving target.

Description

Technical Field

[0001] This invention relates to the field of industrial control system technology, and more specifically, to a defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation. Background Technology

[0002] In industrial automated production and quality inspection processes, surface defect detection and projection indication technology based on machine vision is widely used. Industrial control systems typically use industrial cameras to acquire real-time images of materials continuously moving on conveyor belts. Through image analysis, the location of defects on the material surface is identified, and then a projection device is controlled to project a cursor or outline onto the material surface to assist in manual re-inspection, precise cutting, or automatic rejection operations in the later stages of the production line. This non-contact visual positioning and light and shadow indication collaborative mechanism constitutes the typical target tracking control basis in modern continuous automated production lines.

[0003] However, in existing industrial control applications, when tracking materials on a continuously moving conveyor belt, the tracking cursor projected by the projection device often fails to accurately and stably align with the actual defect. This easily leads to noticeable cursor lag, actual landing point deviation, and severe stretching and deformation of the projected light and shadow pattern. This uncontrollable spatial misalignment and geometric deviation during dynamic tracking not only causes defect indicator marks to malfunction but also seriously affects the accuracy of subsequent processing steps and overall production efficiency of the entire automated production line. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation, which solves the technical problems of lag and offset of the projection cursor and stretching and deformation of the projection pattern in the existing dynamic tracking process.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation, which includes the following steps: S1: Spatial coordinate alignment: Acquire spatial calibration images of the calibration board placed on the conveyor belt and the light and shadow pattern of the projector captured by the industrial camera; establish the conversion ratio between the pixels of the industrial camera, the light emission point of the projector and the spatial distance based on the spatial calibration images; and unify the field of view of the industrial camera and the projection range of the projector into the spatial coordinate system. S2: Spatiotemporal coordinate generation: Control the industrial camera to capture an image of the material surface on the conveyor belt, bind the exposure time as a timestamp to the material surface image, identify defects in the material surface image, convert the pixel coordinates of the defects into original spatial coordinates based on the conversion ratio, and fuse the original spatial coordinates with the timestamp to generate spatiotemporal coordinate data. S3: Displacement Vector Calculation: Obtain the running speed of the conveyor belt, subtract the timestamp from the current time to obtain the computation time; extract the actual mechanical time obtained from receiving the command to the complete stop of the deflection control mechanism inside the projector under the maximum deflection stroke condition, add the actual mechanical time to the preset error fluctuation range to determine the fixed mechanical response time; add the computation time to the fixed mechanical response time to obtain the total delay time, and multiply the running speed by the total delay time to obtain the displacement vector; S4: Projection Dynamic Tracking: The displacement vector is superimposed on the original spatial coordinates to obtain the preliminary target coordinates. The preliminary target coordinates are deformed and corrected according to the deflection angle to obtain the final target coordinates. The projector is controlled to project the tracking cursor to the final target coordinates, and the final target coordinates are updated cyclically before the defect leaves the projection range.

[0007] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of acquiring spatial calibration images of the calibration board captured by the industrial camera and the light and shadow pattern of the projector includes: placing the calibration board with a black and white checkerboard pattern printed on its surface flat within a designated area of ​​the conveyor belt while the conveyor belt is stationary, ensuring that the pattern on the calibration board is completely within the effective field of view of the industrial camera; controlling the projector to turn on the projection light source, accurately projecting a red grid light and shadow pattern with a fixed spacing onto the surface of the calibration board. The red grid light and shadow pattern and the original black and white checkerboard pattern on the calibration board are made to form a composite visual feature that overlaps and interweaves in space; an image acquisition command is sent to the industrial camera, the exposure parameters of the industrial camera are adjusted to adapt to the surface reflected light of the calibration board and the high-brightness light and shadow projected by the projector, and the industrial camera is controlled to capture a high-definition digital image containing both the black and white checkerboard pattern and the red grid light and shadow pattern at the same time; the captured high-definition digital image is transmitted to the control terminal through a communication line, and the received high-definition digital image is established as a spatial calibration image.

[0008] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of establishing the conversion ratio between the pixels of the industrial camera, the light source of the projector, and the spatial distance based on the spatial calibration image, and unifying the field of view of the industrial camera and the projection range of the projector into a spatial coordinate system, includes: performing denoising and grayscale preprocessing on the received spatial calibration image; using edge contour extraction to locate the edge information of the pattern on the calibration board in the preprocessed spatial calibration image; performing sub-pixel corner feature recognition based on the extracted edge information, and locking the first pixel coordinates of each intersection corner point in the calibration board pattern in the field of view; extracting specific color channel data from the spatial calibration image, filtering out background interference information, identifying the nodes where the light and shadow patterns intersect in the separated color channel image, and locating each intersection node in the light and shadow patterns in the field of view. The second pixel coordinates are obtained; the known spatial distance values ​​between adjacent intersection points on the calibration board, which are pre-determined in the real world, are retrieved; the pixel span between the located first pixel coordinates is registered and compared with the corresponding known spatial distance values ​​to calculate the actual horizontal and vertical spatial lengths represented by the pixels in the field of view; the inherent arrangement coordinates of each light-emitting unit in the light-emitting array inside the projector are obtained; the located second pixel coordinates are spatially mapped and bound to the inherent arrangement coordinates; combining the calculated actual horizontal and vertical spatial lengths, and the mapping relationship between the second pixel coordinates and the inherent arrangement coordinates, a conversion ratio integrating the camera view, projection view, and real physical scale is generated; and based on the conversion ratio, the field of view boundary of the industrial camera is aligned with the maximum projection range boundary of the projector and unified into the spatial coordinate system.

[0009] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of identifying defects in the material surface image includes: acquiring the material surface image bound with the timestamp; performing smoothing and denoising processing on the material surface image to eliminate noise interference; uniformly converting the denoised material surface image into a single-channel grayscale digital image, and reading the grayscale value corresponding to each pixel in the grayscale digital image; retrieving a reference grayscale feature interval pre-determined and established on a normal material surface, wherein the reference grayscale feature interval includes the allowable fluctuation range of color depth of the normal material under standard lighting conditions; and scanning according to a preset method. The grayscale values ​​of each pixel read from the grayscale digital image are compared with the baseline grayscale feature range. When an abnormal pixel is found whose grayscale value exceeds or falls below the baseline grayscale feature range, a connectivity diffusion search is performed from the abnormal pixel to its surrounding adjacent pixels. When the total area of ​​a continuous region formed by adjacent abnormal pixels exceeds a preset defect misjudgment filtering threshold, the continuous region is confirmed as a real abnormal region, and the outline of the outermost pixel surrounding the continuous region is extracted as a closed boundary. The complete image region formed by all pixels contained within the closed boundary is determined to be a defect.

[0010] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, wherein: the step of converting the pixel coordinates of the defect into original spatial coordinates based on the conversion ratio includes: after determining the closed boundary of the defect, traversing all pixels located inside the closed boundary as internal pixels, and extracting the horizontal pixel axis value and vertical pixel axis value of each internal pixel in the field of view; summing the extracted horizontal pixel axis values ​​of all internal pixels to obtain a total horizontal value, and simultaneously summing the extracted vertical pixel axis values ​​of all internal pixels to obtain a total vertical value; counting the total number of pixels actually contained inside the closed boundary; dividing the calculated total horizontal value by... The total number of pixels is used to determine the horizontal column coordinates of the defect's geometric center on the material surface image. The sum of the calculated vertical values ​​is divided by the total number of pixels to determine the vertical row coordinates of the defect's geometric center on the material surface image, thus locking the defect's center position on the material surface image as its pixel coordinates. The actual horizontal and vertical spatial lengths of the pixels included in the conversion ratio are retrieved. Using the pre-set reference zero point of the spatial coordinate system as the origin, the horizontal column coordinates are multiplied by the actual horizontal spatial length to obtain the defect's absolute horizontal physical position in real space, and the vertical row coordinates are multiplied by the actual vertical spatial length to obtain the defect's absolute vertical physical position in real space. The calculated absolute lateral physical position and the absolute longitudinal physical position are combined to form the original spatial coordinates.

[0011] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of obtaining the running speed of the conveyor belt includes: receiving in real time the high-frequency pulse electrical signal continuously output by the rotary encoder installed on the active drive wheel axle of the conveyor belt during continuous operation following the conveyor belt; performing hardware filtering and digital de-jitter processing on the received high-frequency pulse electrical signal to eliminate false pulse spikes caused by electromagnetic interference; setting a lower time limit constraint based on the minimum number of pulses actually received within the fixed time sampling period satisfying the minimum speed resolution requirement of the control system, and an upper time limit constraint based on the hysteresis time generated during the speed measurement feedback process not exceeding a preset maximum tracking delay constant; establishing the duration of the fixed time sampling period between the lower time limit constraint and the upper time limit constraint; and accurately counting the cumulative effective pulse signals actually received within each fixed time sampling period. The system reads the mechanical transmission parameters pre-configured in the internal registers of the control chip. These parameters include the total number of standard single-turn pulses generated by the rotary encoder rotating a full circle, and the outer circumference of the drive wheel of the conveyor belt, which has been precisely measured. The cumulative number of effective pulse signals is divided by the total number of standard single-turn pulses to obtain the number of rotations of the drive wheel. The number of rotations is then multiplied by the outer circumference to obtain the distance the conveyor belt travels within the fixed-time sampling period. The calculated distance is divided by the duration of the fixed-time sampling period to convert it into the current instantaneous linear speed of the conveyor belt. A moving average filter is applied to the instantaneous linear speeds calculated over multiple consecutive fixed-time sampling periods to smooth out and eliminate brief speed fluctuations generated by the conveyor belt during mechanical transmission. The stable speed value obtained after the moving average filter is used as the operating speed.

[0012] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of subtracting the timestamp from the current time to obtain the computation time, and adding the computation time to the fixed mechanical response time of the projector to obtain the total delay time, includes: before the drive command is prepared to be sent to the projector, calling the real-time hardware clock inside the processing chip and reading the absolute time value at this moment as the current time; extracting the timestamp bound to the material surface image corresponding to the defect, and parsing the absolute time value of the material surface image at the moment of exposure; subtracting the parsed absolute time value from the read current time to obtain the time difference, and using the time difference as the computation time; extracting the actual mechanical time obtained by testing the deflection control mechanism inside the projector from receiving the command to completely stopping under the maximum deflection stroke condition; adding the actual mechanical time to a preset error fluctuation range, and determining the value obtained after addition as the fixed mechanical response time; adding the computation time to the fixed mechanical response time of the projector to obtain the total delay time.

[0013] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, wherein: the step of superimposing the displacement vector onto the original spatial coordinates to obtain preliminary target coordinates includes: determining the running direction of the conveyor belt and mapping the running direction to the longitudinal displacement principal axis direction in the spatial coordinate system; extracting the scalar value of the displacement vector in the direction parallel to the longitudinal displacement principal axis based on the displacement vector obtained by multiplying the running speed by the total delay time, and establishing the scalar value as the longitudinal displacement unidirectional compensation amount; parsing the generated original spatial coordinates... The coordinates are decoupled and separated into horizontal spatial coordinates perpendicular to the running direction and vertical spatial coordinates parallel to the running direction; the decoupled horizontal spatial coordinates are kept unchanged; the decoupled vertical spatial coordinates are added to the extracted vertical displacement unidirectional compensation amount to calculate the new vertical physical position of the defect after the total delay time, where it is moved forward by the conveyor belt; the unchanged horizontal spatial coordinates are combined with the updated vertical physical position to generate the preliminary target coordinates.

[0014] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of obtaining the final target coordinates by deforming the preliminary target coordinates according to the deflection angle includes: extracting the reference vertical projection point pre-calibrated in the spatial coordinate system by the physical center of the projector's projection lens; calculating the lateral spatial deviation distance of the preliminary target coordinates from the reference vertical projection point in the horizontal axis direction and the longitudinal spatial deviation distance in the vertical axis direction; calling the fixed vertical installation height parameter of the projector from the conveyor belt surface; and using the arctangent value of the quotient obtained by dividing the lateral spatial deviation distance by the fixed vertical installation height parameter as the horizontal deflection angle. The vertical deflection angle is determined by dividing the longitudinal spatial deviation distance by the fixed vertical installation height parameter, using the arctangent of the quotient as the vertical deflection angle value. The horizontal deflection angle value and the vertical deflection angle value are used as the deflection angle. The cosine of the horizontal deflection angle value is calculated to obtain the lateral distortion reverse compensation coefficient, and the cosine of the vertical deflection angle value is calculated to obtain the longitudinal distortion reverse compensation coefficient. The lateral coordinate value in the initial target coordinates is multiplied by the lateral distortion reverse compensation coefficient to obtain the adjusted lateral coordinate value, and the longitudinal coordinate value in the initial target coordinates is multiplied by the longitudinal distortion reverse compensation coefficient to obtain the adjusted longitudinal coordinate value. The adjusted lateral coordinate value and the adjusted longitudinal coordinate value are combined to obtain the final target coordinates.

[0015] As a preferred embodiment of the defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation described in this invention, the step of cyclically updating the final target coordinates before the defect leaves the projection range includes: pre-defining an effective tracking coverage area along the conveyor belt's travel direction in the spatial coordinate system based on the maximum mechanical travel of the deflection control mechanism and the optical divergence limit of the lens, and clearly defining the farthest boundary of the effective tracking coverage area as the maximum longitudinal spatial boundary warning value; after each generation of the final target coordinates and the issuance of a drive command to the deflection control mechanism, extracting the longitudinal coordinate value from the latest generated final target coordinates; comparing and monitoring the extracted longitudinal coordinate value with the pre-defined maximum longitudinal spatial boundary warning value in real time; when it is determined that the longitudinal coordinate value is within the maximum longitudinal spatial boundary warning value, triggering the next round of current time reading, total delay time calculation, displacement compensation superposition, and deformation correction process; when it is determined that the longitudinal coordinate value exceeds the maximum longitudinal spatial boundary warning value, stopping the sending of drive commands for the defect and terminating the current dynamic tracking cycle.

[0016] The beneficial effects of this invention are as follows: It establishes a spatial conversion ratio between the camera and the projector, transforming the time-stamped defects into original spatial coordinates; it calculates the displacement vector by combining the running speed, computation time, and mechanical response time, and superimposes it with the original coordinates to complete unidirectional displacement compensation; it calculates the reverse compensation coefficient based on the deflection angle, scales and adjusts the coordinates, and cyclically updates the tracking. This overcomes the position drift caused by system and mechanical delays, cancels the stretching distortion caused by projection tilt, and achieves alignment between the cursor and continuously moving targets. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is the overall flowchart of a defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation.

[0019] Figure 2 This diagram illustrates the application scenarios and equipment interaction of the defect projection tracking control method.

[0020] Figure 3 This is a logic block diagram for spatiotemporal coordinate mapping and displacement vector compensation.

[0021] Figure 4 This is a block diagram showing the state switching between projection deformation correction and dynamic tracking. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "example" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the invention. The appearance of an embodiment in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0025] Example 1: Refer to Figures 1-4This is one embodiment of the present invention, which provides a defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation, including the following steps: S1. Spatial coordinate alignment: Acquire spatial calibration images of the calibration board placed on the conveyor belt and the light and shadow pattern of the projector captured by the industrial camera. Based on the spatial calibration images, establish the conversion ratio between the pixels of the industrial camera, the light source of the projector and the spatial distance, and unify the field of view of the industrial camera and the projection range of the projector into the spatial coordinate system.

[0026] Acquiring spatial calibration images of a calibration board and a projector's light and shadow pattern captured by an industrial camera includes: placing a calibration board with a black and white checkerboard pattern printed on its surface flat within a designated area of ​​the conveyor belt while the conveyor belt is stationary, ensuring that the pattern on the calibration board is completely within the effective field of view of the industrial camera; controlling the projector to turn on its projection light source, precisely projecting a red grid light and shadow pattern with a fixed spacing onto the surface of the calibration board, so that the red grid light and shadow pattern and the original black and white checkerboard pattern on the calibration board form a composite visual feature that overlaps and interweaves in space; sending an image acquisition command to the industrial camera, adjusting the exposure parameters of the industrial camera to adapt to the surface reflected light of the calibration board and the high-brightness light and shadow projected by the projector, and controlling the industrial camera to capture a high-definition digital image containing both the black and white checkerboard pattern and the red grid light and shadow pattern at the same time; transmitting the captured high-definition digital image to the control terminal through a communication line, and establishing the received high-definition digital image as the spatial calibration image.

[0027] Based on spatial calibration images, a conversion ratio is established between the pixels of an industrial camera, the light source of a projector, and the spatial distance. This unifies the field of view of the industrial camera and the projection range of the projector into a spatial coordinate system. This includes: denoising and grayscale preprocessing of the received spatial calibration images; using edge contour extraction to locate the edge information of the pattern on the calibration board in the preprocessed spatial calibration images; performing sub-pixel corner feature recognition based on the extracted edge information to lock the first pixel coordinates of each intersection corner point in the calibration board pattern within the field of view; extracting specific color channel data from the spatial calibration images, filtering out background interference, identifying the nodes where light and shadow patterns intersect in the separated color channel images, and locating the second pixel coordinates of each intersection node in the light and shadow patterns within the field of view; retrieving the pre-determined known spatial distance values ​​between adjacent intersection corner points on the calibration board in the real physical world, and matching the pixel span between the located first pixel coordinates with the corresponding known spatial distance. Numerical registration and comparison are performed to calculate the actual horizontal and vertical spatial lengths represented by pixels in the field of view. Two-dimensional projection control coordinates are extracted when the projector generates the red grid light and shadow pattern. The located second pixel coordinates are mapped one-to-one with the two-dimensional projection control coordinates to establish a spatial mapping binding relationship. Lens radial distortion correction is performed on the first pixel coordinates using known spatial distance values ​​to obtain the corrected first pixel coordinates. Based on the corrected first pixel coordinates, a homography transformation matrix containing rotation and translation parameters is calculated. Combining the calculated actual horizontal and vertical spatial lengths, spatial mapping binding relationship, and homography transformation matrix, a joint mapping model covering camera coordinates, projection control coordinates, and three-dimensional spatial coordinates is constructed. This joint mapping model is determined as the conversion ratio, and based on this ratio, the field of view boundary of the industrial camera is aligned with the maximum projection range boundary of the projector, unifying them into a spatial coordinate system.

[0028] Due to the influence of the actual lighting environment in the production site and the reflective properties of the physical material of the conveyor belt surface, image sensors are prone to local overexposure or dark colors during the acquisition phase. By adjusting the exposure time and sensitivity parameters of the industrial camera, the black and white checkerboard pattern in the captured image can maintain a clear contrast between light and dark, and the red grid light and shadow projected on the calibration plate surface will not suffer from edge blurring or light spot halos.

[0029] After acquiring the spatially calibrated image, since image sensors and transmission lines typically introduce random electrical signal noise, a new pixel value is calculated by reading the values ​​of multiple neighboring pixels in the vicinity of the target pixel, using either the median or a neighborhood weighted average, to replace the original target pixel, thus achieving smooth image denoising. Next, the image is converted from a multi-channel color format to a single-channel grayscale format to reduce data dimensionality and computational burden.

[0030] When extracting edge information, the horizontal and vertical grayscale differences between adjacent pixels are read, and the global maximum gradient value of the grayscale difference in the entire image is calculated. Combining the optical low-pass filtering characteristics of the camera lens with the true transition band width of the calibration board pattern, a value between 0.6 and 0.8 is selected as the edge sensitivity coefficient. If the coefficient is too low, minute textures on the material surface will be misjudged as edges; if the coefficient is too high, the extracted continuous edges will be broken. The global maximum gradient value is multiplied by the selected edge sensitivity coefficient to obtain the dynamic edge boundary. A set of continuous pixels with a grayscale difference greater than the dynamic edge boundary is determined as the pattern edge. For the location of intersection corners, based on the spatial distribution gradient change trend of pixel grayscale values ​​within the edge intersection area, a quadratic parabolic numerical fitting method is used to calculate the specific coordinates where the grayscale change reaches an extreme value, overcoming the physical limitations of the integer pixel grid and locking the first pixel coordinates with sub-pixel accuracy.

[0031] For the light and shadow pattern projected by the projector, the red, green, and blue color channel data of the spatially calibrated image are separated, retaining the red channel data while significantly reducing the data weights of the green and blue channels. The brightness values ​​of all pixels in the red channel are extracted, and the global average brightness value and the peak brightness value within the red channel are calculated. The brightness difference between the peak brightness value and the global average brightness value is calculated. Considering the Gaussian attenuation characteristic of the energy distribution of the projection beam when projected onto a plane, which is strong at the center and weak at the edges, a value between 0.3 and 0.5 is selected as a weighting coefficient to ensure that the extracted light and shadow nodes have sufficient pixel area for calculating a stable centroid. The brightness difference is multiplied by the selected weighting coefficient to obtain the dynamic tolerance value. The global average brightness value and the dynamic tolerance value are added to determine the red light brightness threshold. Then, the brightness values ​​of each pixel in the red channel are compared with the red light brightness threshold value, and background interference pixels below the red light brightness threshold value are removed to obtain a red light and shadow image with background color removed. Next, the brightness centroid is calculated in the area where mutually perpendicular light bands intersect, and the location of the centroid is determined as the second pixel coordinate of the intersection node.

[0032] In real space, the side lengths of each square in the black and white checkerboard are predetermined and known. The actual spatial dimensions of each pixel in the horizontal and vertical directions are calculated by dividing the actual straight-line distance between adjacent corner points by the difference in the number of horizontal pixels and the difference in the number of vertical pixels between the first pixel coordinates of the two corner points.

[0033] When generating the red grid light and shadow pattern, the control software ensures that each intersection node of the grid has precise two-dimensional projection control coordinates on the digital image canvas. By matching the second pixel coordinates located in the camera image with the two-dimensional projection control coordinates of the corresponding grid node sent by the software, the software overcomes the physical limitations of the hardware device and directly establishes the correspondence between the visual acquisition point and the cursor projection point at the software control end.

[0034] Furthermore, due to the curvature of the camera lens glass, the calibration plate pattern at the edge of the image will exhibit barrel-shaped or pincushion-shaped bending errors. A coordinate offset polynomial is established using pre-determined known spatial distance values ​​to perform numerical compensation on the extracted first pixel coordinates, restoring the bent coordinates to their true linear positions and completing the lens radial distortion correction. Based on the corrected first pixel coordinates, a homography transformation matrix specifically describing the tilt of the spatial plane and perspective angle is solved. Finally, the corrected first pixel coordinates are converted into homogeneous coordinate vector form and multiplied with the solved homography transformation matrix. During matrix multiplication, rotation parameters within the homography transformation matrix are used to correct the trapezoidal perspective distortion caused by the camera tilt angle, and translation parameters are used to compensate for the actual distance deviation between the camera lens optical axis and the origin of the spatial coordinate system, calculating the precise position equation of the pixel mapped to the real spatial plane. The above matrix multiplication transformation rules based on homogeneous vectors, along with the corresponding relationship between the visual acquisition point and the cursor projection point, are integrated to form a complete set of coordinate transformation calculation rules. This set of calculation rules, which clearly defines the specific matrix operation logic, replaces a single proportional constant as the conversion ratio, completely opening up the cross-dimensional conversion channel between the two-dimensional image pixel plane, the digital projection control plane, and the real three-dimensional space, and achieving precise alignment of the perspectives of various devices.

[0035] S2. Spatiotemporal coordinate generation: Control the industrial camera to capture images of the material surface on the conveyor belt, bind the exposure time as a timestamp to the material surface image, identify defects in the material surface image, convert the pixel coordinates of the defects into original spatial coordinates based on the conversion ratio, and fuse the original spatial coordinates with the timestamp to generate spatiotemporal coordinate data.

[0036] Identifying defects in material surface images includes: acquiring a time-stamped material surface image; smoothing and denoising the material surface image to eliminate noise interference; converting the denoised material surface image into a single-channel grayscale digital image and reading the grayscale value corresponding to each pixel in the grayscale digital image; retrieving a reference grayscale feature range pre-determined and established on a normal material surface, which includes the allowable fluctuation range of color depth of normal materials under standard lighting conditions; and sequentially scanning each pixel read from the grayscale digital image according to a preset scanning direction. The grayscale value of a pixel is compared with the baseline grayscale feature range. When an abnormal pixel is found whose grayscale value exceeds or falls below the baseline grayscale feature range, a connectivity diffusion search is performed from the abnormal pixel to its surrounding adjacent pixels. When the total area of ​​a continuous region formed by adjacent abnormal pixels exceeds a pre-set defect misjudgment filtering threshold, the continuous region is confirmed as a real abnormal region, and the outline of the outermost pixels surrounding the continuous region is extracted as a closed boundary. The complete image region formed by all pixels contained within the closed boundary is determined to be a defect.

[0037] Based on a conversion ratio, the pixel coordinates of the defect are converted into their original spatial coordinates. This process includes: after determining the closed boundary of the defect, traversing all pixels located inside the closed boundary as internal pixels, and extracting the horizontal and vertical pixel axis values ​​of each internal pixel in the field of view; summing the extracted horizontal pixel axis values ​​of all internal pixels to obtain a total horizontal value, and summing the extracted vertical pixel axis values ​​of all internal pixels to obtain a total vertical value; counting the total number of pixels actually contained inside the closed boundary; dividing the calculated total horizontal value by the total number of pixels to obtain the horizontal column coordinates of the defect's geometric center position on the material surface image; and then... The sum of the calculated vertical values ​​is divided by the total number of pixels to obtain the vertical row coordinates of the defect's geometric center on the material surface image. This determines the center position of the defect on the material surface image as the pixel coordinates. The actual horizontal and vertical spatial lengths of the pixels included in the conversion ratio are retrieved. Using the pre-set reference zero point of the spatial coordinate system as the origin, the horizontal column coordinates are multiplied by the actual horizontal spatial length to obtain the absolute horizontal physical position of the defect in real space, and the vertical row coordinates are multiplied by the actual vertical spatial length to obtain the absolute vertical physical position of the defect in real space. The calculated absolute horizontal and absolute vertical physical positions are combined to form the original spatial coordinates.

[0038] In this process, the industrial camera captures an image of the conveyor belt surface, and the underlying hardware generates a level trigger pulse. The time value of the hardware clock at the moment the level trigger pulse is generated is read and written into the additional data segment of the image file, thus completing the binding of the timestamp with the image of the material surface.

[0039] Due to slight fluctuations in ambient light at the production site and thermal noise inherent in the image sensor itself, direct analysis of the original image can easily lead to misjudgments. A local matrix is ​​formed by extracting multiple neighboring pixels around the target pixel, calculating the average or median of the grayscale values ​​of all pixels within the local matrix, and replacing the original value of the target pixel with the calculated value to complete the image smoothing and denoising process.

[0040] Multiple flawless images of normal materials were pre-captured under identical lighting conditions. The maximum, minimum, and standard deviation of the global grayscale values ​​in each image were extracted. Confidence coefficients were selected based on the surface reflectivity of the normal materials. For matte materials exhibiting diffuse reflectivity, confidence coefficients ranged from 2.0 to 3.0, while for smooth, highly reflective materials, confidence coefficients ranged from 1.2 to 1.5. The standard deviation was multiplied by the confidence coefficients to calculate the dynamic grayscale tolerance value. Grayscale tolerance values ​​were then added to both ends of the maximum and minimum values ​​as upper and lower limits to establish a baseline grayscale feature range.

[0041] The denoised and grayscale image of the material surface is read pixel by pixel in a left-to-right, top-to-bottom order. When an abnormal pixel with a grayscale value greater than the upper limit or less than the lower limit is encountered, the adjacent pixels in the eight directions directly adjacent to the abnormal pixel are checked. If the adjacent pixels also show grayscale abnormalities, they are grouped into the same abnormal set, and the process continues to check outwards from the newly added abnormal pixel as the center until no new abnormal pixels are found, forming a continuous region.

[0042] Tiny dust particles or minute material texture color differences are not considered defects that need to be tracked. To avoid wasting computational resources and mistracking, the minimum acceptable physical area of ​​a defect, as defined in the production process specifications, is extracted. This minimum acceptable physical area is divided by the pre-calibrated actual physical area of ​​a single pixel, and the quotient is rounded down. This yields the defect misjudgment filtering threshold, representing the lower limit of the number of pixels per area. For example, when the minimum acceptable physical area of ​​a defect is 1 square millimeter and the actual area represented by a single pixel is 0.05 square millimeters, the defect misjudgment filtering threshold is strictly determined to be 20 pixels. The number of pixels contained in a continuous region is counted, and continuous regions with a number of pixels below the defect misjudgment filtering threshold are removed. For the remaining true abnormal regions, the peripheral pixels that are directly adjacent to at least one normal background pixel are selected. These peripheral pixels are then connected end-to-end in spatial connectivity order to construct a closed boundary.

[0043] When calculating the center position of a defect, the horizontal column numbers of all pixels inside the closed boundary are summed, and then divided by the total number of pixels inside to obtain the horizontal column coordinates of the geometric center. The vertical row coordinates are calculated similarly. The calculation formula is as follows: in, and These represent the horizontal column coordinates and vertical row coordinates of the geometric center of the defect on the material surface image, respectively. This represents the total number of pixels actually contained within the closed boundary. and They represent the first and second parts inside the closed boundary, respectively. The horizontal and vertical pixel axis values ​​of each internal pixel.

[0044] Subsequently, using the effective field-of-view boundary vertices established during the spatial calibration phase as the baseline zero point, and utilizing the actual spatial dimensions of each pixel calculated in the previous stage, the horizontal column number and vertical row number are converted into absolute horizontal and absolute vertical positions expressed in physical length units through multiplication operations. The conversion formula is as follows: in, and These represent the absolute horizontal and absolute vertical physical positions of the defect in real space, respectively. and These represent the actual horizontal and vertical lengths of the pixel, respectively.

[0045] Finally, the absolute horizontal position, absolute vertical position, and timestamp values ​​bound to the image are concatenated and encapsulated to generate spatiotemporal coordinate data with both time and spatial orientation attributes.

[0046] S3, Displacement Vector Calculation: Obtain the running speed of the conveyor belt, subtract the timestamp from the current time to obtain the calculation time; extract the actual mechanical time obtained from receiving the command to the complete stop of the internal deflection control mechanism of the projector under the maximum deflection stroke condition, add the actual mechanical time to the preset error fluctuation range to determine the fixed mechanical response time; add the calculation time to the fixed mechanical response time to obtain the total delay time, and multiply the running speed by the total delay time to obtain the displacement vector.

[0047] The process of obtaining the conveyor belt's operating speed includes: receiving in real-time high-frequency pulse signals continuously output by a rotary encoder mounted on the conveyor belt's drive axle during continuous operation; performing hardware filtering and digital de-jitter processing on the received high-frequency pulse signals to eliminate false pulse spikes caused by electromagnetic interference; establishing a lower time limit constraint based on the minimum number of pulses actually received within a fixed time sampling period meeting the control system's minimum speed resolution requirement, and an upper time limit constraint based on the hysteresis time generated during speed feedback not exceeding a preset maximum tracking delay constant; establishing the duration of the fixed time sampling period between the lower and upper time limits; and accurately counting the cumulative number of valid pulse signals actually received within each fixed time sampling period; and reading the pre-configured mechanical transmission parameters from the control chip's internal registers, including the standard pulses generated by the rotary encoder rotating a full revolution. The total number of pulses per revolution and the precisely measured outer circumference of the drive wheel of the conveyor belt are used as the basis for the following calculations: The cumulative number of effective pulse signals is divided by the total number of standard pulses per revolution to obtain the number of revolutions of the drive wheel; the number of revolutions is then multiplied by the outer circumference to obtain the distance the conveyor belt travels within a fixed time sampling period; the calculated travel distance is divided by the duration of the fixed time sampling period to convert it into the current instantaneous linear speed of the conveyor belt; the inherent jitter period of the conveyor belt during transmission is obtained; the quotient of the inherent jitter period divided by the fixed time sampling period is rounded up to obtain the moving average window length; the instantaneous linear speed calculated from a number of consecutive fixed time sampling periods equal to the moving average window length is used to perform moving average filtering calculations to smooth out and eliminate the brief speed jitter generated by the conveyor belt during transmission; the stable speed value obtained after moving average filtering calculation is used as the running speed.

[0048] The computation time is obtained by subtracting the timestamp from the current time. The computation time is then added to the projector's fixed mechanical response time to obtain the total delay time. This includes: calling the real-time hardware clock inside the processing chip before the drive command is sent to the projector, reading the absolute time value at this moment as the current time; extracting the timestamp bound to the material surface image corresponding to the defect, and parsing the absolute time value of the material surface image at the moment of exposure; subtracting the parsed absolute time value from the read current time to obtain the time difference, and using the time difference as the computation time; extracting the actual mechanical time obtained from receiving the command to the complete stop of the deflection control mechanism inside the projector under the maximum deflection stroke condition, adding the actual mechanical time to the preset error fluctuation range, and determining the value obtained after addition as the fixed mechanical response time; and adding the computation time to the projector's fixed mechanical response time to obtain the total delay time.

[0049] In industrial settings, the start-up and shutdown of electromechanical equipment can cause electromagnetic interference, resulting in high-frequency false spikes in the pulse signal output by the rotary encoder. A hardware filter circuit consisting of resistors and capacitors is used to filter out high-frequency noise. A process involving repeatedly checking the level to ensure consistency before confirming a level reversal eliminates transient interference signals and ensures accurate pulse counting. For each rotation of the drive wheel, the rotary encoder emits a fixed number of standard pulses. The number of valid pulses counted within a fixed sampling period is divided by the total number of standard pulses per rotation to calculate the specific number of rotations of the drive wheel. Multiplying the pre-measured outer circumference by the number of rotations yields the linear distance the conveyor belt travels within a single sampling period. Dividing this by the duration of the sampling period gives the current instantaneous linear speed.

[0050] When setting sampling parameters, a boundary constraint principle balancing resolution and real-time performance is followed. Specifically, the lower time limit constraint is set at the minimum number of pulses actually received within a fixed sampling period that meets the minimum speed resolution requirement of the control system. If the set period is lower than this lower time limit, the number of effective pulses counted in a single measurement will be too small, easily leading to serious quantization errors and jumps in speed calculation values. Simultaneously, the upper time limit constraint is set at the hysteresis time generated during the speed measurement feedback process not exceeding the preset maximum tracking delay constant. If the set period is higher than this upper time limit, the collected speed information will be excessively delayed, failing to reflect the actual dynamic speed changes of the conveyor belt in a timely manner. The duration of the fixed-time sampling period is dynamically established between the above lower and upper time limits to ensure a balance between speed measurement accuracy and system response capability. The calculation formulas for the instantaneous linear speed of the conveyor belt and the stable operating speed after moving average filtering are as follows: in, Indicates the conveyor belt at the 1st Instantaneous linear velocity within a fixed time sampling period; Indicates the first The cumulative number of valid pulse signals actually received within a fixed time sampling period; This represents the outer circumference parameter of the conveyor belt's drive wheel; This indicates the total number of standard single-turn pulses generated by a rotary encoder during a full rotation. This indicates the specific duration of the fixed-time sampling period. The mechanical linear velocity of the active drive wheel is calculated by statistically analyzing the proportion of effective pulses per unit time.

[0051] To eliminate brief speed fluctuations, the calculation process for moving average filtering is as follows: in, This indicates the running speed obtained after filtering and smoothing. This represents the moving average window length obtained by rounding up the quotient of the inherent jitter period divided by the fixed sampling period. (This is achieved by applying continuous...) The instantaneous speed of each cycle is calculated by arithmetic average to filter out speed measurement errors caused by high-frequency mechanical vibration.

[0052] Furthermore, due to the influence of gear meshing clearance and belt elastic deformation, the instantaneous linear motion speed will generate short-frequency alternating jitter. The inherent jitter period of the conveyor belt during transmission is obtained, and the quotient of dividing the inherent jitter period by the fixed time sampling period is rounded up to obtain the moving average window length. This ensures that the set window length can just cover a complete mechanical jitter sine waveform, thus effectively neutralizing the calculation errors caused by speed peaks and troughs. A numerical queue with the same length as the moving average window is established within the processing chip. Specifically, each time a newly calculated instantaneous linear motion speed is obtained, it is stored in the queue, while the oldest stored value is discarded. The arithmetic mean of all speed values ​​in the queue is calculated, and the arithmetic mean is used to replace the instantaneous value to obtain a smooth and stable operating speed.

[0053] From the moment the industrial camera's image sensor completes exposure to the moment the central processing chip prepares to issue the projection drive command, a series of data processing steps occur, including image data transmission, grayscale threshold comparison, connectivity retrieval, and center coordinate conversion. This process consumes a period of time that dynamically varies depending on the chip's computational load. The absolute time value of the current moment is obtained by calling the hardware clock at the processing chip's underlying level, and then subtracted from the timestamp value accompanying the image exposure, to measure the specific computational time. When the projector receives the digital command, its internal micro-servo motor starts, causing the reflector to deflect against mechanical inertia until the reflector reaches the designated angle position and the minor oscillations are completely eliminated and the projector comes to a complete stop. The entire mechanical transmission process has a relatively stable time consumption. The actual mechanical time consumption from receiving the command to the complete stabilization of the mechanical structure is extracted under the maximum deflection stroke condition. To ensure that delay compensation covers the mechanical oscillation time under extreme conditions, the actual mechanical time consumption is added to a pre-set preset error fluctuation range, and the sum is determined as the fixed mechanical response time. The dynamically changing computation time is added to the fixed mechanical response time to obtain the total delay time from when the defect is captured by the camera to when the projector cursor is ready to land. During this total delay time, the conveyor belt continuously carries the defect forward. The smoothed running speed is multiplied by the total delay time to obtain the specific straight-line distance the defect moves forward with the conveyor belt during this period. This straight-line distance is used as the displacement vector to provide numerical basis for subsequent spatial coordinate superposition. The specific calculation process for estimating the total delay time and superimposing the longitudinal displacement unidirectional compensation is as follows: in, Indicates the total delay time; This indicates that the absolute time value at this moment (i.e., the current time) is being read. This represents the timestamp bound to the material surface image corresponding to the defect; This represents the fixed mechanical response time determined by superimposing a preset error fluctuation range under the projector's maximum deflection stroke condition. The computation time is obtained by subtracting timestamps, and then the fixed mechanical action time is compensated for, achieving precise quantification of the end-to-end time delay.

[0054] Further calculations were performed to determine the new longitudinal position of the defect after the total delay time. The calculation process is as follows: in, This represents the unidirectional longitudinal displacement compensation amount of the extracted displacement vector in the direction of the longitudinal displacement principal axis; This indicates the new vertical physical position after the addition update; This represents the absolute vertical physical location of the defect in real space. The physical distance moved is obtained by multiplying the running speed by the total delay time, and then linearly added to the original absolute vertical physical coordinates to achieve a closed loop of spatiotemporal compensation.

[0055] S4. Projection Dynamic Tracking: The displacement vector is superimposed on the original spatial coordinates to obtain the initial target coordinates. The initial target coordinates are deformed and corrected according to the deflection angle to obtain the final target coordinates. The projector is controlled to project the tracking cursor to the final target coordinates, and the final target coordinates are updated cyclically before the defect leaves the projection range.

[0056] The initial target coordinates are obtained by superimposing the displacement vector onto the original spatial coordinates, including: determining the running direction of the conveyor belt and mapping the running direction to the longitudinal displacement principal axis direction in the spatial coordinate system; extracting the scalar value of the displacement vector in the direction parallel to the longitudinal displacement principal axis based on the displacement vector obtained by multiplying the running speed by the total delay time, and establishing the scalar value as the longitudinal displacement unidirectional compensation amount; parsing the generated original spatial coordinates and decoupling them into lateral spatial coordinate values ​​perpendicular to the running direction and longitudinal spatial coordinate values ​​parallel to the running direction; keeping the decoupled lateral spatial coordinate values ​​unchanged; performing a numerical addition operation between the decoupled longitudinal spatial coordinate values ​​and the extracted longitudinal displacement unidirectional compensation amount to calculate the new longitudinal physical position of the defect after the total delay time, where it has been moved forward by the conveyor belt; and recombining the unchanged lateral spatial coordinate values ​​with the newly updated longitudinal physical position to generate the initial target coordinates.

[0057] The final target coordinates are obtained by deforming the initial target coordinates based on the deflection angle. This includes: extracting the reference vertical projection point pre-calibrated in the spatial coordinate system from the physical center of the projector lens; calculating the lateral spatial deviation distance of the initial target coordinates from the reference vertical projection point along the horizontal axis and the longitudinal spatial deviation distance along the vertical axis; calling the fixed vertical installation height parameter of the projector from the conveyor belt surface; using the arctangent value of the quotient obtained by dividing the lateral spatial deviation distance by the fixed vertical installation height parameter as the horizontal deflection angle value, and the arctangent value of the quotient obtained by dividing the longitudinal spatial deviation distance by the fixed vertical installation height parameter as the final target coordinates. The arctangent value of the quotient is used as the vertical deflection angle value. The horizontal deflection angle value and the vertical deflection angle value are used as the deflection angle. The cosine value of the horizontal deflection angle value is calculated to obtain the lateral distortion reverse compensation coefficient, and the cosine value of the vertical deflection angle value is calculated to obtain the longitudinal distortion reverse compensation coefficient. The lateral coordinate value in the initial target coordinates is multiplied by the lateral distortion reverse compensation coefficient to obtain the adjusted lateral coordinate value. The longitudinal coordinate value in the initial target coordinates is multiplied by the longitudinal distortion reverse compensation coefficient to obtain the adjusted longitudinal coordinate value. The adjusted lateral coordinate value and the adjusted longitudinal coordinate value are combined as the final target coordinates.

[0058] The final target coordinates are updated cyclically before the defect leaves the projection range. This includes: pre-defining the effective tracking coverage area along the conveyor belt's direction of travel in the spatial coordinate system based on the maximum mechanical travel of the deflection control mechanism and the optical divergence limit of the lens, and clearly defining the farthest boundary of the effective tracking coverage area as the maximum longitudinal spatial boundary warning value; after each generation of the final target coordinates and the issuance of drive commands to the deflection control mechanism, extracting the longitudinal coordinate value from the latest generated final target coordinates; comparing and monitoring the extracted longitudinal coordinate value with the pre-defined maximum longitudinal spatial boundary warning value in real time; when the longitudinal coordinate value is determined to be within the maximum longitudinal spatial boundary warning value, triggering the next round of current time reading, total delay time calculation, displacement compensation superposition, and deformation correction process; when the longitudinal coordinate value is determined to exceed the maximum longitudinal spatial boundary warning value, stopping the issuance of drive commands for the defect and terminating the current dynamic tracking cycle.

[0059] The conveyor belt runs linearly along a fixed guide rail, and the material placed on the conveyor belt does not experience lateral relative slippage during its movement. The longitudinal displacement unidirectional compensation value, representing the main axis of the conveyor belt, is extracted and directly added to the longitudinal spatial coordinate value in the original spatial coordinate system. This calculates the new longitudinal position of the defect after the total time elapsed due to software calculations and mechanical delays. Simultaneously, the lateral spatial coordinate value is retained without modification. The two are then combined to generate the preliminary target coordinates, completing the first round of compensation for the position drift error.

[0060] The projected beam does not always illuminate the material surface perpendicularly. When the deflection control mechanism tilts the projected beam, the cursor pattern falling on the planar material will experience trapezoidal stretching and geometric diffusion distortion. The coordinates of the reference point corresponding vertically downwards from the center point of the projector lens are extracted, and the actual distance differences between the initial target coordinates and the reference point coordinates in the horizontal and vertical directions are calculated. Combined with the pre-measured vertical installation height of the projector lens from the conveyor belt surface, the specific horizontal and vertical deflection angles required for the internal reflector of the projector are calculated using the tangent function in a Cartesian coordinate system. Next, the cosine values ​​of the horizontal and vertical deflection angles are calculated to obtain horizontal and vertical distortion inverse compensation coefficients with values ​​less than one. When the beam tilts and falls, it produces image magnification and stretching effects. By multiplying the calculated horizontal and vertical distortion inverse compensation coefficients by the horizontal and vertical coordinates of the initial target coordinates, respectively, the coordinate values ​​are pre-adjusted non-linearly during data processing to counteract the aforementioned stretching and magnification effects, ensuring that the actual projected tracking cursor accurately falls on the defect location. The specific calculation process for estimating the deflection angle of the projected beam and generating the deformation correction for the final target coordinates is as follows: in, and These represent the horizontal and vertical deflection angles when the projector's beam reaches the initial target coordinates, respectively. Indicates the absolute lateral physical location of the defect in real space; This indicates the new vertical physical position after the addition update; and These represent the horizontal and vertical coordinates of the reference vertical projection point, which is pre-calibrated in the spatial coordinate system, representing the physical center of the projector's projection lens. This indicates the fixed vertical installation height parameter between the projector and the conveyor belt surface. Using the tangent relationship of a right triangle, the tilt angle of the light beam in real three-dimensional physical space is accurately reproduced.

[0061] The calculation process for the distortion reverse compensation coefficient and the final target coordinates based on the calculated deflection angle is as follows: in, and These represent the reverse compensation coefficients for lateral distortion and longitudinal distortion, respectively. and These represent the horizontal and vertical coordinate values ​​in the final target coordinate system, respectively. Indicates the absolute lateral physical location of the defect in real space; This represents the new vertical physical position after the addition update. Since the tilted beam illuminating the plane will cause the projected pattern to be stretched (the magnification is related to the reciprocal of the cosine of the deflection angle), the target coordinates are multiplied by the corresponding cosine value in advance for reverse pre-compression, so that the actual projected tracking cursor can perfectly fit the defect contour and complete the deformation correction.

[0062] The deflection control mechanism's internal drive motor has a rotation angle limit, and the optical lens's beam has an effective focusing range. The first longitudinal spatial coordinate corresponding to the deflection control mechanism reaching its maximum safe travel distance is extracted. Simultaneously, the second longitudinal spatial coordinate corresponding to the diameter of the optical divergence circle of the projected light spot reaching the upper limit allowed by the process is tested. The first and second longitudinal spatial coordinates are compared numerically, and the minimum value is selected as the maximum longitudinal spatial boundary warning value. The conveyor belt continuously transports material forward. After each generation of the final target coordinates, the longitudinal coordinate value in the final target coordinates is immediately read and compared with the maximum longitudinal spatial boundary warning value. As long as the longitudinal coordinate value is less than or equal to the maximum longitudinal spatial boundary warning value, it indicates that the defect is still within the effective tracking coverage area. A new round of hardware clock reading, delay time calculation, and coordinate update operations is then initiated to achieve continuous dynamic tracking of the defect. Once the comparison reveals that the longitudinal coordinate value is greater than the maximum longitudinal spatial boundary warning value, it means that the defect has moved out of the projection control range with the conveyor belt. The drive command channel for the current defect is immediately cut off, the dynamic tracking loop task ends, and the computing resources of the processing chip are released to process the next defect that enters the field of view.

[0063] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation, characterized in that, Performed by a computer device, including the following steps: S1: Spatial coordinate alignment: Acquire spatial calibration images of the calibration board placed on the conveyor belt and the light and shadow pattern of the projector captured by the industrial camera; establish the conversion ratio between the pixels of the industrial camera, the light emission point of the projector and the spatial distance based on the spatial calibration images; and unify the field of view of the industrial camera and the projection range of the projector into the spatial coordinate system. S2: Spatiotemporal coordinate generation: Control the industrial camera to capture an image of the material surface on the conveyor belt, bind the exposure time as a timestamp to the material surface image, identify defects in the material surface image, convert the pixel coordinates of the defects into original spatial coordinates based on the conversion ratio, and fuse the original spatial coordinates with the timestamp to generate spatiotemporal coordinate data. S3: Displacement Vector Calculation: Obtain the running speed of the conveyor belt, subtract the timestamp from the current time to obtain the computation time; extract the actual mechanical time obtained from receiving the command to the complete stop of the deflection control mechanism inside the projector under the maximum deflection stroke condition, add the actual mechanical time to the preset error fluctuation range to determine the fixed mechanical response time; add the computation time to the fixed mechanical response time to obtain the total delay time, and multiply the running speed by the total delay time to obtain the displacement vector; S4: Projection Dynamic Tracking: The displacement vector is superimposed on the original spatial coordinates to obtain the preliminary target coordinates. The preliminary target coordinates are deformed and corrected according to the deflection angle to obtain the final target coordinates. The projector is controlled to project the tracking cursor to the final target coordinates, and the final target coordinates are updated cyclically before the defect leaves the projection range.

2. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The acquisition of spatial calibration images of the calibration board and the light and shadow pattern of the projector captured by the industrial camera includes: With the conveyor belt in a stationary state where it is not running, the calibration plate with a black and white checkerboard pattern printed on its surface is placed flat in the designated area of ​​the conveyor belt, ensuring that the pattern on the calibration plate is completely within the effective field of view of the industrial camera. The projector is controlled to turn on the projection light source, and a red grid light and shadow pattern with a fixed spacing is accurately projected onto the surface of the calibration board, so that the red grid light and shadow pattern and the original black and white checkerboard pattern of the calibration board form a composite visual feature that overlaps and interweaves in space. Send an image acquisition command to the industrial camera, adjust the exposure parameters of the industrial camera to adapt to the surface reflected light of the calibration plate and the high-brightness light and shadow projected by the projector, and control the industrial camera to capture a high-definition digital image containing both the black and white checkerboard pattern and the red grid light and shadow pattern at the same time. The captured high-definition digital images are transmitted to the control terminal via communication lines, and the received high-definition digital images are established as spatial calibration images.

3. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 2, characterized in that, The step of establishing a conversion ratio between the pixels of the industrial camera, the light source of the projector, and the spatial distance based on the spatial calibration image, and unifying the field of view of the industrial camera and the projection range of the projector into a spatial coordinate system, includes: The received spatial calibration image is preprocessed by denoising and grayscale conversion. In the preprocessed spatial calibration image, edge contour extraction is used to locate the edge information of the pattern on the calibration board. Based on the extracted edge information, sub-pixel corner feature recognition is performed to lock the first pixel coordinates of each intersection corner point in the calibration board pattern in the field of view; Specific color channel data is extracted from the spatial calibration image, background interference information is filtered out, and the nodes where the light and shadow patterns intersect are identified in the separated color channel image. The second pixel coordinates of each intersection node in the light and shadow pattern in the field of view are located. The known spatial distance values ​​between adjacent corner points on the calibration board, which are pre-determined in the real world, are retrieved. The pixel span between the first pixel coordinates is registered and compared with the corresponding known spatial distance values. The actual horizontal and vertical spatial lengths represented by the pixels in the field of view are calculated. Extract the two-dimensional projection control coordinates when the projector generates the red grid light and shadow pattern, and establish a spatial mapping binding relationship by matching the located second pixel coordinates with the two-dimensional projection control coordinates one by one. Using the known spatial distance value, a lens radial distortion correction operation is performed on the first pixel coordinates to obtain the corrected first pixel coordinates. Based on the corrected first pixel coordinates, a homography transformation matrix containing rotation and translation parameters is calculated. Combining the calculated actual length of the horizontal space, the actual length of the vertical space, the spatial mapping binding relationship, and the homography transformation matrix, a joint mapping model covering camera coordinates, projection control coordinates, and three-dimensional spatial coordinates is constructed. The joint mapping model is determined as the conversion ratio. Based on the conversion ratio, the field of view boundary of the industrial camera and the maximum projection range boundary of the projector are aligned and unified into the spatial coordinate system.

4. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The identification of defects in the material surface image includes: Obtain the material surface image bound with the timestamp, and perform smoothing and denoising processing on the material surface image to eliminate noise interference; The denoised material surface image is uniformly converted into a single-channel grayscale digital image, and the grayscale value corresponding to each pixel in the grayscale digital image is read. A reference grayscale feature range, pre-measured and established on the surface of a normal material, is retrieved. This reference grayscale feature range includes the allowable fluctuation range of color depth of the normal material under standard lighting conditions. According to the preset scanning direction, the gray value of each pixel read from the grayscale digital image is compared with the reference grayscale feature range one by one; When the comparison finds abnormal pixels whose gray values ​​exceed or fall below the reference gray value feature range, a connectivity diffusion search is performed from the abnormal pixel as the center to the surrounding adjacent pixels. When the total area of ​​a continuous region formed by adjacent abnormal pixels exceeds a preset defect misjudgment filtering threshold, the continuous region is confirmed as a real abnormal region, and the outline of the pixels surrounding the outermost edge of the continuous region is extracted as a closed boundary. The complete image region consisting of all pixels contained within the closed boundary is determined to be a defect.

5. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 4, characterized in that, The process of converting the pixel coordinates of the defect into original spatial coordinates based on the conversion ratio includes: After determining the closed boundary of the defect, all pixels located inside the closed boundary are traversed as internal pixels, and the horizontal pixel axis value and vertical pixel axis value of each internal pixel in the field of view are extracted respectively. The horizontal pixel axis values ​​of all extracted internal pixels are summed to obtain the total horizontal value, and the vertical pixel axis values ​​of all extracted internal pixels are summed to obtain the total vertical value. The total number of pixels actually contained inside the closed boundary is counted; the sum of the calculated horizontal values ​​is divided by the total number of pixels to obtain the horizontal column coordinates of the geometric center of the defect on the material surface image. Divide the sum of the calculated vertical values ​​by the total number of pixels to obtain the vertical row coordinates of the geometric center of the defect on the material surface image, thereby locking the center position of the defect on the material surface image as the pixel coordinates. Retrieve the actual horizontal and vertical spatial lengths of the pixels included in the conversion ratio; Using the preset reference zero point of the spatial coordinate system as the starting origin, the absolute horizontal physical position of the defect in the real space is obtained by multiplying the horizontal column coordinate by the actual length of the horizontal space, and the absolute vertical physical position of the defect in the real space is obtained by multiplying the vertical row coordinate by the actual length of the vertical space. The calculated absolute lateral physical position and the absolute longitudinal physical position are combined to form the original spatial coordinates.

6. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The step of obtaining the operating speed of the conveyor belt includes: The high-frequency pulse electrical signals continuously output by the rotary encoder installed on the drive wheel shaft of the conveyor belt during its continuous operation following the conveyor belt are received in real time. The received high-frequency pulse electrical signal is subjected to hardware filtering and digital dejitter processing to eliminate false pulse spikes caused by electromagnetic interference. The minimum number of pulses actually received within the fixed time sampling period is used to meet the minimum speed resolution requirement of the control system as the lower time constraint, and the hysteresis time generated by the speed measurement feedback process is not greater than the preset maximum tracking delay constant as the upper time constraint. The duration of the fixed time sampling period is established between the lower time constraint and the upper time constraint, and the cumulative number of effective pulse signals actually received within each fixed time sampling period is accurately counted. The mechanical transmission parameters pre-configured in the internal registers of the control chip are read. These parameters include the total number of standard single-turn pulses generated by the rotary encoder rotating a full circle, and the outer circumference of the conveyor belt drive wheel, which has been precisely measured. The number of rotations of the drive wheel is obtained by dividing the cumulative number of effective pulse signals obtained by the total number of standard single-circle pulses. Then, the number of rotations is multiplied by the outer circumference parameter to obtain the distance the conveyor belt moves within the fixed time sampling period. The calculated moving distance is divided by the duration of the fixed time sampling period to convert it into the current instantaneous linear speed of the conveyor belt; The inherent jitter period of the conveyor belt during transmission is obtained. The quotient of the inherent jitter period divided by the fixed time sampling period is rounded up to obtain the moving average window length. The instantaneous linear motion speed calculated by the fixed time sampling period that is consecutively equal to the moving average window length is retrieved and a moving average filter is performed to smooth out and eliminate the brief speed jitter generated by the conveyor belt during transmission. The stable speed value obtained after the moving average filter calculation is taken as the running speed.

7. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The step of subtracting the timestamp from the current time to obtain the computation time, and adding the computation time to the fixed mechanical response time of the projector to obtain the total delay time, includes: Before the drive command is sent to the projector, the real-time hardware clock inside the processing chip is called to read the absolute time value at this moment as the current time. Extract the timestamp bound to the material surface image corresponding to the defect, and parse the absolute time value of the material surface image at the moment of exposure; Subtract the parsed absolute time value from the current time to obtain the time difference, and use the time difference as the calculation time. Extract the actual mechanical time taken by the internal deflection control mechanism of the projector from receiving the command to coming to a complete stop under the maximum deflection stroke condition; add the actual mechanical time to a preset error fluctuation range, and determine the value obtained by the addition as the fixed mechanical response time; The total delay time is obtained by adding the computation time to the fixed mechanical response time of the projector.

8. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The step of superimposing the displacement vector onto the original spatial coordinates to obtain the preliminary target coordinates includes: Determine the running direction of the conveyor belt and map the running direction to the longitudinal displacement principal axis direction in the spatial coordinate system; Based on the displacement vector obtained by multiplying the running speed by the total delay time, the scalar value of the displacement vector in the direction parallel to the longitudinal displacement principal axis is extracted, and the scalar value is established as the longitudinal displacement unidirectional compensation amount. The generated original spatial coordinates are analyzed and decoupled into horizontal spatial coordinate values ​​perpendicular to the running direction and vertical spatial coordinate values ​​parallel to the running direction. The decoupled horizontal spatial coordinate values ​​are kept in their original state without any changes; the decoupled vertical spatial coordinate values ​​are added to the extracted vertical displacement unidirectional compensation value to calculate the new vertical physical position of the defect after the total delay time. The unchanged horizontal spatial coordinates are combined with the newly updated vertical physical position to generate the initial target coordinates.

9. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The process of obtaining the final target coordinates by deforming the preliminary target coordinates based on the deflection angle includes: Extract the reference vertical projection point, which is pre-calibrated in the spatial coordinate system, from the physical center of the projector's projection lens; Calculate the lateral spatial deviation of the preliminary target coordinates from the reference vertical projection point in the horizontal axis direction, and the longitudinal spatial deviation of the preliminary target coordinates from the reference vertical projection point in the vertical axis direction. The fixed vertical installation height parameter of the projector from the surface of the conveyor belt is called; the arctangent value of the quotient obtained by dividing the lateral spatial deviation distance by the fixed vertical installation height parameter is used as the horizontal deflection angle value, and the arctangent value of the quotient obtained by dividing the longitudinal spatial deviation distance by the fixed vertical installation height parameter is used as the vertical deflection angle value. The horizontal deflection angle value and the vertical deflection angle value are used as the deflection angle. The cosine of the horizontal deflection angle is calculated to obtain the lateral distortion reverse compensation coefficient, and the cosine of the vertical deflection angle is calculated to obtain the longitudinal distortion reverse compensation coefficient. The adjusted horizontal coordinate value is obtained by multiplying the horizontal coordinate value in the initial target coordinate by the horizontal distortion inverse compensation coefficient. The adjusted vertical coordinate value is obtained by multiplying the vertical coordinate value in the initial target coordinate by the vertical distortion inverse compensation coefficient. The adjusted horizontal coordinate value and the adjusted vertical coordinate value are combined as the final target coordinate.

10. The defect projection tracking control method based on spatiotemporal coordinate mapping and displacement vector compensation according to claim 1, characterized in that, The step of cyclically updating the final target coordinates before the defect leaves the projection range includes: Based on the maximum mechanical travel of the deflection control mechanism and the optical divergence limit of the lens, an effective tracking coverage area along the direction of travel of the conveyor belt is pre-defined in the spatial coordinate system, and the farthest boundary of the effective tracking coverage area is clearly defined as the maximum longitudinal spatial boundary warning value. After each generation of the final target coordinates and the issuance of a drive command to the deflection control mechanism, the longitudinal coordinate value in the latest generated final target coordinates is extracted; The extracted longitudinal coordinate values ​​are compared and monitored in real time with the predefined maximum longitudinal spatial boundary warning values. When it is determined that the longitudinal coordinate value is within the maximum longitudinal spatial boundary warning value, the next round of current time reading, total delay time calculation, displacement compensation superposition, and deformation correction process is triggered. When the longitudinal coordinate value is determined to exceed the maximum longitudinal spatial boundary warning value, the sending of drive commands for the defect is stopped, and the current dynamic tracking loop is terminated.

Images