A monocular vision-based color 3D dynamic projection positioning method and device

Abstract

The application discloses a monocular vision-based color 3D dynamic projection positioning method and device, which comprises a monocular camera, a projector, an industrial computer and a computer host, the computer host is signal-connected with the industrial computer, the industrial computer is signal-connected with the monocular camera and the projector respectively, and the photographing range of the monocular camera and the projection range of the industrial computer at least partially coincide. The application has the effects of reducing the positioning time of feature points and improving the tracking positioning and projection efficiency.

Description

Technical Field

[0001] This application relates to the field of 3D dynamic projection technology, and in particular to a color 3D dynamic projection positioning method and apparatus based on monocular vision. Background Technology

[0002] Color 3D dynamic projection positioning is a technology that uses computer vision to track the pose of a target in real time and combines optical projection to accurately fit virtual content onto the surface of a dynamic object. Its core objective is to achieve high-precision three-dimensional spatial positioning and real-time correction of projected content during the movement of a physical object, thus achieving "dynamic fusion of virtual and real".

[0003] In existing technologies, color 3D dynamic projection positioning takes a relatively long time to locate the projected target, and the target tracking effect of dynamic projection is not good, so there is room for further improvement. Summary of the Invention

[0004] This invention provides a color 3D dynamic projection positioning method and apparatus based on monocular vision for achieving color 3D dynamic projection of target objects using an industrial white light projector. The method determines the pose relationship between the target object and the monocular camera using monocular vision, and then establishes a pose relationship between the target object and the industrial white light projector by utilizing a pre-calibrated pose relationship between the monocular camera and the projector. Furthermore, this invention proposes predicting the location of feature points based on their motion trends, reducing the positioning time of feature points, improving the efficiency of tracking, positioning, and projection, and solving the problem of coordinate reference failure caused by mold displacement during production line operation. This enables high frame rate color 3D dynamic projection, allowing for real-time positioning and projection tasks.

[0005] The present invention proposes a system based on a combined single monocular camera and an industrial white light projector, which consists of an industrial monocular camera, a projector, an industrial control computer, and a computer host; wherein, the computer host and the industrial control computer are connected via a network cable, and the industrial control computer is electrically connected to both the binocular industrial monocular camera and the projector.

[0006] The color 3D dynamic projection positioning method and device based on monocular vision provided in this application adopts the following technical solution: A color 3D dynamic projection positioning method based on monocular vision includes the following steps: Step 1: The projector sequentially projects structured light coded images onto the calibration board; the monocular camera sequentially acquires images of the calibration board; wherein, the calibration board has multiple poses; the images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; the calibration plate has a feature circle; the calibration plate image is analyzed; the monocular camera coordinates of the center of the feature circle are obtained. m is the number of feature circles in the calibration plate; calculate the initial intrinsic values ​​of the model parameters for the monocular camera and the projector respectively. and ; Calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and The initial pose of the monocular camera relative to the projector was also calculated. ; Step 2: The target object requiring dynamic tracking and projection positioning has feature points that match the content to be projected; the projector projects onto the target object; the monocular camera samples the image of the target object at its initial position; the image is analyzed and the actual spatial coordinates of the feature points at the initial position are calculated; the pose of the target object is estimated; the first frame is projected. Step 3: Determine the approximate region of the feature points that are adapted to the content to be projected in the second frame and project the second frame; accurately extract the center of the feature points adapted to the second frame and calculate the actual spatial coordinates of the center of the feature points; Step 4: Use the difference order to predict and sequentially obtain the actual spatial coordinates of the feature point centers that are adapted to subsequent projection frames; Step 5: Register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the CAD model of the feature points, adjust the coordinates of the target object in the monocular camera coordinate system in the current frame; calculate the pose of the target object relative to the projector; obtain the virtual image of the CAD model of the target object on the projector.

[0007] By employing the above technical solution, the pose relationship between a pre-calibrated industrial monocular camera and an industrial white light projector is utilized. The monocular camera acquires the coordinates of feature points of a dynamically moving object and the relationship between these coordinates and known theoretical feature points, enabling the device to obtain the pose of the dynamically moving target object. Based on this, the industrial monocular camera continuously samples pre-arranged feature points on the projected target object. Leveraging the continuous change in feature points between frames, and under the condition of initial frame recognition and positioning, the dynamic feature points are rapidly reconstructed in 3D, continuously tracking the target object's pose. Ultimately, the industrial projector can also perceive the dynamic pose of the target object in real time and project and position the dynamic target contour on the object. This invention determines the location of feature points by predicting their movement trends, reducing feature point positioning time, improving tracking, positioning, and projection efficiency, and solving the problem of coordinate reference failure caused by mold displacement during production line operation. It achieves high frame rate color 3D dynamic projection, enabling real-time positioning and projection tasks. Through efficient and accurate projection positioning of static or dynamic workpieces, it provides strong support for digital and intelligent assisted manufacturing. Furthermore, feature point extraction is optimized through feature point tracking and prediction technology, reducing the overall algorithm overhead during the positioning process while ensuring positioning accuracy. The reduced computational power consumption of a monocular camera compared to traditional binocular systems significantly lowers the reliance on auxiliary hardware performance, requiring only a single camera module, thus lowering hardware costs compared to binocular systems. It also eliminates the need for complex baseline calibration procedures, supporting plug-and-play deployment for greater convenience. Moreover, in this invention, the scope of the target object (including the calibration board and the target object) is limited only by the combined imaging range of the monocular camera and the projection range of the industrial control computer, resulting in a larger overall detectable area compared to binocular cameras. This makes this invention highly suitable for real-time 3D dynamic projection in scenarios where the target object's position changes rapidly.

[0008] Preferably, step one specifically includes the following steps: Step 1.1: The computer host sends a signal command to the industrial control computer. After receiving the signal command, the industrial control computer controls the projector to project the pre-prepared structured light coded image onto the calibration board and controls the monocular camera to acquire the image of the calibration board according to the preset sequence. Step 1.2: Under the shared field of view of the monocular camera and the projector, place the calibration board at several positions and repeat Step 1.1 to acquire several sets of calibration board images projected with structured light coded patterns. The images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; Step 1.3: By comparing the calibration plate image Image processing is performed to obtain the coordinates of the feature center of the calibration board in the monocular camera coordinate system. m is the number of characteristic circles in the calibration plate; Step 1.4: Utilize The initial intrinsic parameters of the monocular camera and the projector models can be calculated separately. and It can also calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and ; Step 1.5: Using the calculated and This allows for further calculation of the monocular camera's position relative to the projector. .

[0009] Preferably, step two specifically includes the following steps: Step 2.1: Control the monocular camera to sample the initial position image of the target object that needs dynamic tracking and projection positioning. Through image processing, find the local region of feature points arranged on the target object. Within this local region, accurately extract the center of the feature points to obtain the imaging coordinates of the feature point center in the monocular camera coordinate system, denoted as […]. , Given the number of feature points, extract the center image position of the initial feature points; Step 2.2: Based on the obtained feature points on the target object, use the monocular camera coordinates. with model intrinsic parameters initial values and theoretical feature point coordinates Calculate the spatial coordinates of the feature points This allows for the estimation of the target object's pose and the completion of spatial localization of the initial tracking position. Step 2.3: The initial frame is defined as the first frame image when the spatial coordinates of the feature points are reacquired.

[0010] Preferably, step three specifically includes the following steps: based on the spatial coordinates of the feature points of the target object in the initial frame state. The process involves identifying the feature point regions on the target object that correspond to the second frame image; precisely extracting the centers of the feature points within these regions to obtain the monocular camera coordinates; and then utilizing the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the spatial coordinates of the feature points adapted to the second frame image. .

[0011] Preferably, step three specifically includes the following steps: based on the spatial coordinates of the feature points of the target object in the initial frame state. The process involves identifying the feature point regions on the target object that correspond to the second frame image; precisely extracting the centers of the feature points within these regions to obtain the monocular camera coordinates; and then utilizing the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the spatial coordinates of the feature points adapted to the second frame image. .

[0012] Preferably, step four specifically includes the following steps: Step 4.1: After obtaining the feature point spatial coordinates adapted to the first and second frames, use the first-order difference order to obtain the predicted spatial coordinates of the feature points adapted to the third frame. The system identifies feature point regions on the target object that match the third frame image, accurately extracts the center of each feature point within these regions, and obtains the monocular camera coordinates of these center points. Then, it utilizes the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the actual spatial coordinates of the feature points adapted to the third frame. ; Step 4.2: After obtaining the three sets of feature point spatial coordinates adapted to the first, second, and third frame images, the predicted spatial coordinates of the feature points adapted to the fourth frame image are obtained using the second-order difference order. The system identifies the feature point regions on the target object that match the fourth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fourth frame. ; Step 4.3: After obtaining the spatial coordinates of the four feature points adapted to the first, second, third, and fourth frames of the image, the third-order difference order is used to obtain the predicted spatial coordinates of the feature points adapted to the fourth frame image. The system identifies the feature point regions on the target object that match the fifth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fifth frame. ; Step 4.4: After sampling the four actual spatial coordinates of the initial frame, the predicted spatial coordinates of the feature points adapted to the subsequent frames can all be obtained by the third-order difference order.

[0013] Preferably, step five specifically includes the following steps: Step 5.1: Based on the actual spatial coordinates obtained in Steps 3 and 4, register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the feature points on the CAD model to complete the spatial positioning of the target object in the monocular camera coordinate system in the current frame state. Step 5.2: Based on the obtained pose of the monocular camera relative to the projector, calculate the pose of the target object relative to the projector in the current frame state. Using this pose and the intrinsic parameters of the projector, obtain the CAD digital model of the target object and virtually image it on the projector according to the monocular camera imaging model.

[0014] Preferably, if step two through four fails to extract the feature point center, step two will be entered to reposition the initial frame.

[0015] A color 3D dynamic projection positioning device based on monocular vision, operating using the aforementioned color 3D dynamic projection positioning method, includes a monocular camera, a projector, an industrial control computer, and a computer host. The computer host is signal-connected to the industrial control computer, and the industrial control computer is signal-connected to both the monocular camera and the projector. The imaging range of the monocular camera at least partially overlaps with the projection range of the industrial control computer.

[0016] By adopting the above technical solution, the monocular camera reduces the computing power consumption compared to traditional binocular systems, significantly reducing the dependence on auxiliary hardware performance. It only requires a single camera module, reducing hardware costs compared to binocular systems. Furthermore, it eliminates the need for complex baseline calibration procedures, supports plug-and-play deployment, and is more convenient. In addition, in this invention, the range of the target object (including the calibration board and the target object) is limited only by the combined imaging range of the monocular camera and the projection range of the industrial control computer, making the overall detectable range of the target object larger than that of a binocular camera. This makes this invention very suitable for real-time 3D dynamic projection in scenarios where the position of the target object changes rapidly.

[0017] In summary, this application includes at least one of the following beneficial technical effects: By utilizing the pre-calibrated pose relationship between an industrial monocular camera and an industrial white light projector, the device acquires the coordinates of feature points of a dynamically moving object and the relationship between these coordinates and known theoretical feature points. This allows the device to obtain the pose of the dynamically moving target object. Based on this, the industrial monocular camera continuously samples pre-arranged feature points on the projected target object. Leveraging the continuous change in feature points between frames, and under the condition of initial frame recognition and positioning, the device rapidly performs 3D reconstruction of the dynamic feature points, continuously tracking the target object's pose. Ultimately, this enables the industrial projector to perceive the target object's dynamic pose in real time and project and locate the dynamic target contour. This invention determines the location of feature points by predicting their movement trends, reducing feature point positioning time, improving tracking, positioning, and projection efficiency, and solving the problem of coordinate reference failure caused by mold displacement during production line operation. It achieves high frame rate color 3D dynamic projection, enabling real-time positioning and projection tasks. Through efficient and accurate projection positioning of static or dynamic workpieces, it provides strong support for digital and intelligent assisted manufacturing. The reduced computing power consumption of a monocular camera compared to traditional binocular systems significantly reduces the reliance on auxiliary hardware performance for this invention. It requires only a single camera module, resulting in lower hardware costs compared to binocular systems. Furthermore, it eliminates the need for complex baseline calibration processes, supports plug-and-play deployment, and is more convenient. The range of objects (including calibration plates and target objects) is limited only by the combined imaging range of the monocular camera and the projection range of the industrial control computer, making the overall detectable range of objects larger compared to that of binocular cameras. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the calibration plate in Example 2; Figure 2 This is a schematic diagram of the calibration based on the calibration plate in Example 2; Figure 3 This is a flowchart of the color 3D dynamic projection positioning process in Example 2; Figure 4 This is a flowchart of the feature point tracking process in Example 2; Figure 5 The object to be projected in Example 2; Figure 6 This is a schematic diagram illustrating the spatial pose alignment principle between the color 3D dynamic projection device and the target object in Example 2. Figure 7 In Example 2 Figure 6 3D dynamic projection of the actual object; Figure 8 This is a schematic diagram of the connection of the color 3D dynamic projection device in Example 2. Detailed Implementation

[0019] The following is in conjunction with the appendix Figures 1-8 This application will be described in further detail.

[0020] Example 1 This application discloses a color 3D dynamic projection positioning method based on monocular vision.

[0021] Reference Figures 1-8 This includes the following steps: Step 1: The projector sequentially projects structured light coded images onto the calibration board; the monocular camera sequentially acquires images of the calibration board; the calibration board has multiple poses; the images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; the calibration plate has a feature circle; analyze the calibration plate image; obtain the monocular camera coordinates of the center of the feature circle. m is the number of feature circles in the calibration plate; calculate the initial intrinsic values ​​of the model parameters for the monocular camera and the projector respectively. and ; Calculate the pose of the calibration plate relative to the monocular camera and projector at different positions. and It also calculated the initial pose of the monocular camera relative to the projector. ; Step 2: The target object requiring dynamic tracking and projection positioning has feature points that match the content to be projected; the projector projects onto the target object; the monocular camera samples the image of the target object at the initial position; the image is analyzed and the actual spatial coordinates of the feature points at the initial position are calculated; the pose of the target object is estimated; the first frame is projected. Step 3: Determine the approximate region of feature points that fit the content to be projected in the second frame and project the second frame; accurately extract the center of the feature points that fit the second frame and calculate the actual spatial coordinates of the feature point center; Step 4: Use the difference order to predict and sequentially obtain the actual spatial coordinates of the feature point centers that are adapted to subsequent projection frames; Step 5: Register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the CAD model of the feature points, adjust the coordinates of the target object in the monocular camera coordinate system in the current frame; calculate the pose of the target object relative to the projector; obtain the virtual image of the CAD model of the target object on the projector.

[0022] By utilizing the pre-calibrated pose relationship between an industrial monocular camera and an industrial white light projector, the device acquires the coordinates of feature points of a dynamically moving object and the relationship between these coordinates and known theoretical feature points. This allows the device to obtain the pose of the dynamically moving target object. Based on this, the industrial monocular camera continuously samples pre-arranged feature points on the projected target object. Leveraging the continuous change in feature points between frames, and under the condition of initial frame recognition and positioning, it rapidly reconstructs the dynamic feature points in 3D, continuously tracking the target object's pose. Ultimately, this allows the industrial projector to perceive the dynamic pose of the target object in real time and project and locate the dynamic target contour. This invention determines the location of feature points by predicting their movement trends, reducing feature point positioning time, improving tracking, positioning, and projection efficiency, and solving the problem of coordinate reference failure caused by mold displacement during production line operation. It achieves high frame rate color 3D dynamic projection, enabling real-time positioning and projection tasks. Through efficient and accurate projection positioning of static or dynamic workpieces, it provides strong support for digital and intelligent assisted manufacturing.

[0023] Furthermore, feature point extraction is optimized through feature point tracking and prediction technology, reducing the overall algorithm overhead during the positioning process while ensuring positioning accuracy. The reduced computational power consumption of a monocular camera compared to traditional binocular systems significantly lowers the reliance on auxiliary hardware performance, requiring only a single camera module, thus lowering hardware costs compared to binocular systems. It also eliminates the need for complex baseline calibration procedures, supporting plug-and-play deployment for greater convenience. Moreover, in this invention, the scope of the target object (including the calibration board and the target object) is limited only by the combined imaging range of the monocular camera and the projection range of the industrial control computer, resulting in a larger overall detectable area compared to binocular cameras. This makes this invention highly suitable for real-time 3D dynamic projection in scenarios where the target object's position changes rapidly.

[0024] Furthermore, in this embodiment, step one specifically includes the following steps: Step 1.1: The computer host sends a signal command to the industrial control computer. After receiving the signal command, the industrial control computer controls the projector to project the pre-prepared structured light coded image onto the calibration board and controls the monocular camera to acquire the image of the calibration board according to the preset sequence. Step 1.2: Under the shared field of view of the monocular camera and the projector, place the calibration board at several positions and repeat Step 1.1 to acquire several sets of calibration board images projected with structured light coded patterns. The images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; Step 1.3: By comparing the calibration plate image Image processing is performed to obtain the coordinates of the center of the feature circle of the calibration board in the monocular camera coordinate system. m is the number of characteristic circles in the calibration plate; Step 1.4: Utilize The initial intrinsic parameters of the monocular camera and the projector models can be calculated separately. and It can also calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and ; Step 1.5: Using the calculated and This allows for further calculation of the monocular camera's position relative to the projector. .

[0025] Furthermore, in this embodiment, step two specifically includes the following steps: Step 2.1: Control the monocular camera to sample the initial position image of the target object that needs dynamic tracking and projection positioning. Through image processing, find the local region of feature points arranged on the target object. Within this local region, accurately extract the center of the feature points to obtain the imaging coordinates of the feature point center in the monocular camera coordinate system, denoted as […]. , Given the number of feature points, extract the center image position of the initial feature points; Step 2.2: Based on the obtained feature points on the target object, use the monocular camera coordinates. with model intrinsic parameters initial values and theoretical feature point coordinates Calculate the spatial coordinates of the feature points This allows for the estimation of the target object's pose and the completion of spatial localization of the initial tracking position. Step 2.3: The initial frame is defined as the first frame image when the spatial coordinates of the feature points are reacquired.

[0026] Furthermore, in this embodiment, step three specifically includes the following steps: based on the spatial coordinates of the feature points of the target object in the initial frame state. The process involves identifying the feature point regions on the target object that correspond to the second frame image; precisely extracting the centers of the feature points within these regions to obtain the monocular camera coordinates; and then utilizing the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the spatial coordinates of the feature points adapted to the second frame image. .

[0027] Furthermore, in this embodiment, step four specifically includes the following steps: Step 4.1: After obtaining the feature point spatial coordinates adapted to the first and second frames, use the first-order difference order to obtain the predicted spatial coordinates of the feature points adapted to the third frame. The system identifies feature point regions on the target object that match the third frame image, accurately extracts the center of each feature point within these regions, and obtains the monocular camera coordinates of these center points. Then, it utilizes the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the actual spatial coordinates of the feature points adapted to the third frame. ; Step 4.2: After obtaining the three sets of feature point spatial coordinates adapted to the first, second, and third frame images, the predicted spatial coordinates of the feature points adapted to the fourth frame image are obtained using the second-order difference order. The system identifies the feature point regions on the target object that match the fourth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fourth frame. ; Step 4.3: After obtaining the spatial coordinates of the four feature points adapted to the first, second, third, and fourth frames of the image, the third-order difference order is used to obtain the predicted spatial coordinates of the feature points adapted to the fourth frame image. The system identifies the feature point regions on the target object that match the fifth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fifth frame. ; Step 4.4: After sampling the four actual spatial coordinates of the initial frame, the predicted spatial coordinates of the feature points adapted to the subsequent frames can all be obtained by the third-order difference order.

[0028] Furthermore, in this embodiment, step five specifically includes the following steps: Step 5.1: Based on the actual spatial coordinates obtained in Steps 3 and 4, register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the feature points on the CAD model to complete the spatial positioning of the target object in the monocular camera coordinate system in the current frame state. Step 5.2: Based on the obtained pose of the monocular camera relative to the projector, calculate the pose of the target object relative to the projector in the current frame state. Using this pose and the intrinsic parameters of the projector, obtain the CAD digital model of the target object and virtually image it on the projector according to the monocular camera imaging model.

[0029] Furthermore, in this embodiment, if step two through four fails to extract the feature point center, step two will be entered to reposition the initial frame.

[0030] Furthermore, in this embodiment, the monocular camera is a monocular industrial camera; the projector is a white light industrial projector; and the industrial computer is a micro industrial computer.

[0031] Example 2 This application discloses a color 3D dynamic projection positioning device based on monocular vision.

[0032] Reference Figures 1-8 It includes a monocular camera, a projector, an industrial control computer, and a computer host. The computer host is connected to the industrial control computer via signal, and the industrial control computer is connected to the monocular camera and the projector via signal. The shooting range of the monocular camera and the projection range of the industrial control computer at least partially overlap.

[0033] The color 3D dynamic projection positioning method of the color 3D dynamic projection positioning device includes the following steps: Step 1: The projector sequentially projects structured light coded images onto the calibration board; the monocular camera sequentially acquires images of the calibration board; the calibration board has multiple poses; the images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; the calibration plate has a feature circle; analyze the calibration plate image; obtain the monocular camera coordinates of the center of the feature circle. m is the number of feature circles in the calibration plate; calculate the initial intrinsic values ​​of the model parameters for the monocular camera and the projector respectively. and ; Calculate the pose of the calibration plate relative to the monocular camera and projector at different positions. and It also calculated the initial pose of the monocular camera relative to the projector. ; Step 2: The target object requiring dynamic tracking and projection positioning has feature points that match the content to be projected; the projector projects onto the target object; the monocular camera samples the image of the target object at the initial position; the image is analyzed and the actual spatial coordinates of the feature points at the initial position are calculated; the pose of the target object is estimated; the first frame is projected. Step 3: Determine the approximate region of feature points that fit the content to be projected in the second frame and project the second frame; accurately extract the center of the feature points that fit the second frame and calculate the actual spatial coordinates of the feature point center; Step 4: Use the difference order to predict and sequentially obtain the actual spatial coordinates of the feature point centers that are adapted to subsequent projection frames; Step 5: Register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the CAD model of the feature points, adjust the coordinates of the target object in the monocular camera coordinate system in the current frame; calculate the pose of the target object relative to the projector; obtain the virtual image of the CAD model of the target object on the projector.

[0034] By utilizing the pre-calibrated pose relationship between an industrial monocular camera and an industrial white light projector, the device acquires the coordinates of feature points of a dynamically moving object and the relationship between these coordinates and known theoretical feature points. This allows the device to obtain the pose of the dynamically moving target object. Based on this, the industrial monocular camera continuously samples pre-arranged feature points on the projected target object. Leveraging the continuous change in feature points between frames, and under the condition of initial frame recognition and positioning, it rapidly reconstructs the dynamic feature points in 3D, continuously tracking the target object's pose. Ultimately, this allows the industrial projector to perceive the dynamic pose of the target object in real time and project and locate the dynamic target contour. This invention determines the location of feature points by predicting their movement trends, reducing feature point positioning time, improving tracking, positioning, and projection efficiency, and solving the problem of coordinate reference failure caused by mold displacement during production line operation. It achieves high frame rate color 3D dynamic projection, enabling real-time positioning and projection tasks. Through efficient and accurate projection positioning of static or dynamic workpieces, it provides strong support for digital and intelligent assisted manufacturing.

[0035] Furthermore, feature point extraction is optimized through feature point tracking and prediction technology, reducing the overall algorithm overhead during the positioning process while ensuring positioning accuracy. The reduced computational power consumption of a monocular camera compared to traditional binocular systems significantly lowers the reliance on auxiliary hardware performance, requiring only a single camera module, thus lowering hardware costs compared to binocular systems. It also eliminates the need for complex baseline calibration procedures, supporting plug-and-play deployment for greater convenience. Moreover, in this invention, the scope of the target object (including the calibration board and the target object) is limited only by the combined imaging range of the monocular camera and the projection range of the industrial control computer, resulting in a larger overall detectable area compared to binocular cameras. This makes this invention highly suitable for real-time 3D dynamic projection in scenarios where the target object's position changes rapidly.

[0036] Furthermore, in this embodiment, step one specifically includes the following steps: Step 1.1: The computer host sends a signal command to the industrial control computer. After receiving the signal command, the industrial control computer controls the projector to project the pre-prepared structured light coded image onto the calibration board and controls the monocular camera to acquire the image of the calibration board according to the preset sequence. Step 1.2: Under the shared field of view of the monocular camera and the projector, place the calibration board at several positions and repeat Step 1.1 to acquire several sets of calibration board images projected with structured light coded patterns. The images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; Step 1.3: By comparing the calibration plate image Image processing is performed to obtain the coordinates of the center of the feature circle of the calibration board in the monocular camera coordinate system. m is the number of characteristic circles in the calibration plate; Step 1.4: Utilize The initial intrinsic parameters of the monocular camera and the projector models can be calculated separately. and It can also calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and ; Step 1.5: Using the calculated and This allows for further calculation of the monocular camera's position relative to the projector. .

[0037] Furthermore, in this embodiment, step two specifically includes the following steps: Step 2.1: Control the monocular camera to sample the initial position image of the target object that needs dynamic tracking and projection positioning. Through image processing, find the local region of feature points arranged on the target object. Within this local region, accurately extract the center of the feature points to obtain the imaging coordinates of the feature point center in the monocular camera coordinate system, denoted as […]. , Given the number of feature points, extract the center image position of the initial feature points; Step 2.2: Based on the obtained feature points on the target object, use the monocular camera coordinates. with model intrinsic parameters initial values and theoretical feature point coordinates Calculate the spatial coordinates of the feature points This allows for the estimation of the target object's pose and the completion of spatial localization of the initial tracking position. Step 2.3: The initial frame is defined as the first frame image when the spatial coordinates of the feature points are reacquired.

[0038] Furthermore, in this embodiment, step three specifically includes the following steps: based on the spatial coordinates of the feature points of the target object in the initial frame state. The process involves identifying the feature point regions on the target object that correspond to the second frame image; precisely extracting the centers of the feature points within these regions to obtain the monocular camera coordinates; and then utilizing the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the spatial coordinates of the feature points adapted to the second frame image. .

[0039] Furthermore, in this embodiment, step four specifically includes the following steps: Step 4.1: After obtaining the feature point spatial coordinates adapted to the first and second frames, use the first-order difference order to obtain the predicted spatial coordinates of the feature points adapted to the third frame. The system identifies feature point regions on the target object that match the third frame image, accurately extracts the center of each feature point within these regions, and obtains the monocular camera coordinates of these center points. Then, it utilizes the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the actual spatial coordinates of the feature points adapted to the third frame. ; Step 4.2: After obtaining the three sets of feature point spatial coordinates adapted to the first, second, and third frame images, the predicted spatial coordinates of the feature points adapted to the fourth frame image are obtained using the second-order difference order. The system identifies the feature point regions on the target object that match the fourth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fourth frame. ; Step 4.3: After obtaining the spatial coordinates of the four feature points adapted to the first, second, third, and fourth frames of the image, the third-order difference order is used to obtain the predicted spatial coordinates of the feature points adapted to the fourth frame image. The system identifies the feature point regions on the target object that match the fifth frame image. Within these regions, the centers of the feature points are precisely extracted. After obtaining the coordinates of the feature point centers, the initial intrinsic parameters of the monocular camera model are used. Calculate the actual spatial coordinates of the feature points adapted to the fifth frame. ; Step 4.4: After sampling the four actual spatial coordinates of the initial frame, the predicted spatial coordinates of the feature points adapted to the subsequent frames can all be obtained by the third-order difference order.

[0040] Furthermore, in this embodiment, step five specifically includes the following steps: Step 5.1: Based on the actual spatial coordinates obtained in Steps 3 and 4, register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the feature points on the CAD model to complete the spatial positioning of the target object in the monocular camera coordinate system in the current frame state. Step 5.2: Based on the obtained pose of the monocular camera relative to the projector, calculate the pose of the target object relative to the projector in the current frame state. Using this pose and the intrinsic parameters of the projector, obtain the CAD digital model of the target object and virtually image it on the projector according to the monocular camera imaging model.

[0041] Furthermore, in this embodiment, if step two through four fails to extract the feature point center, step two will be entered to reposition the initial frame.

[0042] Furthermore, in this embodiment, the monocular camera is a monocular industrial camera; the projector is a white light industrial projector; and the industrial computer is a micro industrial computer.

[0043] Furthermore, in this embodiment, the target object is a hexagonal prism based on a rotatable, conventionally common mechanism. Each side of the hexagonal prism has a set of reflective positioning target points and a preset projection image. When the hexagonal prism rotates, a color 3D dynamic projection positioning operation is performed on one side of the prism. Since the position of the object to be projected is constantly changing, it is difficult to guarantee the real-time positioning and projection effect of the projection image. Therefore, in this example, the position of the next frame is predicted based on the movement trend of feature points, reducing the positioning time of feature points and improving the efficiency of tracking, positioning, and projection. This enables real-time positioning and projection, as shown in the specific positioning and projection effect. Figure 8 As shown.

[0044] It should be noted that the various embodiments of this application can be arbitrarily combined into new embodiments, provided that the solutions do not conflict and the technical solutions can coexist.

[0045] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A color 3D dynamic projection positioning method based on monocular vision, characterized in that: Includes the following steps: Step 1: The projector sequentially projects structured light coded images onto the calibration board; the monocular camera sequentially acquires images of the calibration board; wherein, the calibration board has multiple poses; the images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; the calibration plate has a feature circle; the calibration plate image is analyzed; the monocular camera coordinates of the center of the feature circle are obtained. m is the number of feature circles in the calibration plate; calculate the initial intrinsic values ​​of the model parameters for the monocular camera and the projector respectively. and ; Calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and The initial pose of the monocular camera relative to the projector was also calculated. ; Step 2: The target object requiring dynamic tracking and projection positioning has feature points that match the content to be projected; the projector projects onto the target object; the monocular camera samples the image of the target object at its initial position; the image is analyzed and the actual spatial coordinates of the feature points at the initial position are calculated; the pose of the target object is estimated; the first frame is projected. Step 3: Determine the approximate region of the feature points that are adapted to the content to be projected in the second frame and project the second frame; accurately extract the center of the feature points adapted to the second frame and calculate the actual spatial coordinates of the center of the feature points; Step 4: Use the difference order to predict and sequentially obtain the actual spatial coordinates of the feature point centers that are adapted to subsequent projection frames; Step 5: Register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the CAD model of the feature points, adjust the coordinates of the target object in the monocular camera coordinate system in the current frame; calculate the pose of the target object relative to the projector; obtain the virtual image of the CAD model of the target object on the projector.

2. The color 3D dynamic projection positioning method based on monocular vision according to claim 1, characterized in that: Step one specifically includes the following steps: Step 1.1: The computer host sends a signal command to the industrial control computer. After receiving the signal command, the industrial control computer controls the projector to project the pre-prepared structured light coded image onto the calibration board and controls the monocular camera to acquire the image of the calibration board according to the preset sequence. Step 1.2: Under the shared field of view of the monocular camera and the projector, place the calibration board at several positions and repeat Step 1.1 to acquire several sets of calibration board images projected with structured light coded patterns. The images of the calibration board acquired by the monocular camera are denoted as follows: n is the number of calibration plate placement positions; Step 1.3: By comparing the calibration plate image Image processing is performed to obtain the coordinates of the center of the feature circle of the calibration board in the monocular camera coordinate system. m is the number of characteristic circles in the calibration plate; Step 1.4: Utilize The initial intrinsic parameters of the monocular camera and the projector models can be calculated separately. and It can also calculate the pose of the calibration plate relative to the monocular camera and the projector at different positions. and ; Step 1.5: Using the calculated and This allows for further calculation of the monocular camera's position relative to the projector. .

3. The color 3D dynamic projection positioning method based on monocular vision according to claim 1, characterized in that: Step two specifically includes the following steps: Step 2.1: Control the monocular camera to sample the initial position image of the target object that needs dynamic tracking and projection positioning. Through image processing, find the local region of feature points arranged on the target object. Within this local region, accurately extract the center of the feature points to obtain the imaging coordinates of the feature point center in the monocular camera coordinate system, denoted as […]. , Given the number of feature points, extract the center image position of the initial feature points; Step 2.2: Based on the obtained feature points on the target object, use the monocular camera coordinates. with model intrinsic parameters initial values and theoretical feature point coordinates Calculate the spatial coordinates of the feature points This allows for the estimation of the target object's pose and the completion of spatial localization of the initial tracking position. Step 2.3: The initial frame is defined as the first frame image when the spatial coordinates of the feature points are reacquired.

4. The color 3D dynamic projection positioning method based on monocular vision according to claim 1, characterized in that: Step three specifically includes the following steps: based on the spatial coordinates of the feature points of the target object in the initial frame state... The process involves identifying the feature point regions on the target object that correspond to the second frame image; precisely extracting the centers of the feature points within these regions to obtain the monocular camera coordinates; and then utilizing the initial values ​​of the monocular camera's model intrinsic parameters. Calculate the spatial coordinates of the feature points adapted to the second frame image. .

5. The color 3D dynamic projection positioning method based on monocular vision according to claim 1, characterized in that: Step five specifically includes the following steps: Step 5.1: Based on the actual spatial coordinates obtained in Steps 3 and 4, register the measured spatial coordinates of the feature points with the theoretical spatial coordinates of the feature points on the CAD model to complete the spatial positioning of the target object in the monocular camera coordinate system in the current frame state. Step 5.2: Based on the obtained pose of the monocular camera relative to the projector, calculate the pose of the target object relative to the projector in the current frame state. Using this pose and the intrinsic parameters of the projector, obtain the CAD digital model of the target object and virtually image it on the projector according to the monocular camera imaging model.

6. The color 3D dynamic projection positioning method based on monocular vision according to claim 1, characterized in that: If step two through four fails to extract the feature point center, step two will be resumed to reposition the initial frame.

7. A color 3D dynamic projection positioning device based on monocular vision, characterized in that: The method operates using the monocular vision-based color 3D dynamic projection positioning method as described in any one of claims 1-6, comprising a monocular camera, a projector, an industrial control computer, and a computer host. The computer host is signal-connected to the industrial control computer, and the industrial control computer is signal-connected to both the monocular camera and the projector. The imaging range of the monocular camera at least partially overlaps with the projection range of the industrial control computer.

Images