Acupoint projection method and system
By calibrating the grid at multiple depth locations and generating the homography matrix and transformation function, the projection deviation problem caused by the assumption of a fixed depth in traditional acupoint location methods is solved, and accurate projection and efficient location of human acupoints are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MEDICAL LATITUDE INTELLIGENT TECHNOLOGY (CHENGDU) CO LTD
- Filing Date
- 2026-05-07
- Publication Date
- 2026-07-28
AI Technical Summary
In existing technologies, traditional acupoint location methods assume that the human body is at a fixed height or uniform depth, resulting in a significant deviation between the projected position of the acupoint and its actual position, which cannot adapt to the height differences of the complex three-dimensional structure of the human body.
The calibration is performed at multiple depth locations using a calibration mesh, the coordinates in the depth camera and projector are obtained, a homography matrix is generated, a transformation function is calculated, and the actual 3D coordinates are mapped to the projector coordinate system for acupoint projection.
It achieves precise projection of different acupoints on the human body, adapts to the three-dimensional structure of the human body, improves the accuracy and efficiency of acupoint positioning, and supports the versatility and practicality of multiple parts.
Smart Images
Figure CN122473263A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of acupoint projection technology, and in particular to an acupoint projection method and system. Background Technology
[0002] Acupoints are important stimulation points in Traditional Chinese Medicine (TCM) diagnosis and treatment. Accurate acupoint location is a key factor in ensuring the efficacy of TCM external therapies such as acupuncture, massage, and tuina. With the development of computer vision technology, acupoint location using image recognition technology has become possible.
[0003] Existing technologies, such as human pose estimation, can detect key points of posture in various parts of the body in real time, as well as corresponding points of acupoints throughout the body. These technologies provide a foundation for automated acupoint localization. Meanwhile, the collection and annotation of acupoint localization data is fundamental to building acupoint localization models. Traditional acupoint localization relies on the experience and judgment of professional physicians, and different physicians may have different localizations of the same acupoint. To train an accurate and reliable acupoint localization model, a large amount of acupoint localization data confirmed by professional physicians is needed. This data needs to cover acupoints in all parts of the body, including the hands, feet, face, and torso.
[0004] With the development of computer vision technology, existing technologies include using cameras to capture images of various parts of the human body, using an interactive positioning pen to specify acupoint locations, and combining coordinate transformation technology to achieve acupoint marking and projection positioning. This technology provides a visual aid for acupoint positioning. However, traditional projector calibration methods typically assume the target object is at a fixed height or use a uniform average depth for coordinate transformation of the entire human body. Because the human body is a complex three-dimensional structure, different acupoints (such as fingertips, nose tips, and toes) have significant differences in depth relative to the camera. Using a uniform depth will lead to a significant deviation between the projected position and the actual position. Summary of the Invention
[0005] In view of this, the present invention provides an acupoint projection method and system, wherein the method can adapt to the height differences of different acupoints in the human body and can achieve precise projection of various key acupoints.
[0006] A method for acupoint projection, the method comprising: The calibration grid is used to perform calibration at multiple depth locations, and the first coordinates of each calibration point in the depth camera and the second coordinates in the projector are obtained at different calibration depths. Based on the first and second coordinates of each calibration point at the same calibration depth, generate the first homography matrix corresponding to that calibration depth; Based on the first homography matrix corresponding to each calibration depth, calculate the transformation function between the calibration depth and each element of the corresponding first homography matrix; Obtain the actual three-dimensional coordinates of the acupoint to be projected under the depth camera, and calculate the second homography matrix corresponding to the acupoint depth in the actual three-dimensional coordinates using the transformation function; Based on the second homography matrix, the actual three-dimensional coordinates are mapped to the projector coordinate system to obtain the projected coordinates, so as to control the projector to project the acupoints according to the projected coordinates.
[0007] Optionally, in the above method, obtaining the first and second coordinates of each calibration point at different depths includes: When calibrating at multiple depth locations using a calibration grid, the calibration components within the calibration grid are tracked; When the dwell time of the calibration component at any position exceeds the preset time, the current dwell position of the calibration component is determined as the calibration point, and the first coordinate and second coordinate of the calibration point at different calibration depths are obtained.
[0008] Optionally, in the above method, tracking the calibration elements within the calibration grid includes: Check whether the calibration component is equipped with a marker; When the calibration component is equipped with markers, a preset image recognition algorithm is applied to detect the position of the markers in real time; In the absence of markers on the calibration component, the tip of the calibration component is identified, and the position of the tip is detected in real time using the image recognition algorithm.
[0009] Optionally, in the above method, generating the first homography matrix corresponding to the calibration depth based on the first and second coordinates of each calibration point at the same calibration depth includes: Obtain the mapping combination corresponding to each calibration depth. The mapping combination contains multiple mapping pairs. The mapping pairs are first coordinates and second coordinates that have a mapping relationship under the same calibration depth. A preset direct linear transformation algorithm is applied to calculate each of the mapping combinations to obtain the first homography matrix corresponding to each calibration depth.
[0010] Optionally, in the above method, calculating the transformation function between the calibration function and each element of the first homography matrix includes: Extract the elements at the same position from each of the first homography matrices to form an element sequence; The transformation function is obtained by fitting each element in the element sequence with its corresponding calibration depth using a preset least squares method.
[0011] Optionally, in the above method, calculating the transformation function between each calibration depth and each element of the first homography matrix includes: Obtain a pre-set base matrix, which is the matrix corresponding to the center depth of the calibration range; Based on the first homography matrix corresponding to each calibration depth, calculate the depth offset corresponding to the calibration depth; The transformation function is obtained by adding the basic matrix to the depth offset corresponding to the calibration depth.
[0012] Optionally, in the above method, calculating the depth offset corresponding to each calibration depth includes: Extract the elements at the same position in the first homography matrix corresponding to each calibration depth to form an element sequence; The least squares method, which removes the initial constant term, is used to fit each element in the element sequence to its corresponding calibration depth to obtain the depth offset.
[0013] Optionally, in the above method, calculating the depth offset corresponding to the calibration depth includes: The depth range is divided into multiple depth intervals, wherein the depth range includes all calibration depths and each depth interval includes at least two calibration depths; For any calibration depth, determine the depth interval to which the calibration depth belongs, and apply a preset linear interpolation algorithm to calculate the calibration depth and the endpoints of the depth interval to obtain the depth offset corresponding to the calibration depth.
[0014] Optionally, in the above method, calculating the depth offset corresponding to the calibration depth includes: The calibration depth and its corresponding first homography matrix are input into a preset neural network model to obtain the depth offset corresponding to the calibration depth output by the neural network model. The neural network model is a model that automatically learns the mapping relationship between the implicit features of the calibration depth and the depth offset through a large amount of training data.
[0015] An acupoint projection system, the system comprising: Projector, depth camera, and processor; The depth camera is used to acquire the first coordinates of each calibration point of the calibration grid at different calibration depths and the actual three-dimensional coordinates of the acupoint to be projected, and send them to the processor. The projector is used to project the calibration grid onto reference planes at different depths, collect the second coordinates of each calibration point of the calibration grid at different calibration depths and send them to the processor; upon receiving the projection coordinates sent by the processor, the acupoints are projected according to the projection coordinates. The processor is used to execute the acupoint projection method described above.
[0016] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform the aforementioned acupoint projection method.
[0017] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the above-described acupoint projection method.
[0018] Compared with the prior art, the present invention has the following advantages: This invention provides a method for acupoint projection, comprising: calibrating at multiple depth locations using a calibration grid; obtaining the first coordinates of each calibration point in a depth camera and the second coordinates in a projector at different calibration depths; generating a first homography matrix corresponding to the calibration depth based on the first and second coordinates at the same calibration depth; calculating a transformation function between the calibration depth and each element of the first homography matrix based on the first homography matrix corresponding to each calibration depth; obtaining the actual three-dimensional coordinates of the acupoint to be projected in the depth camera; calculating the corresponding second homography matrix based on the acupoint depth in the actual three-dimensional coordinates using the transformation function; and mapping the actual three-dimensional coordinates to the projector coordinate system based on the second homography matrix to obtain the projected coordinates. Applying the method provided by this invention can adapt to the height differences of different acupoints on the human body, enabling precise projection of various key acupoints. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0020] Figure 1 A flowchart of an acupoint projection method provided in an embodiment of the present invention; Figure 2 This is a system structure diagram of an acupoint projection system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0023] This invention can be used in a wide variety of general-purpose or special-purpose computing environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.
[0024] This invention provides an acupoint projection method, which is applied to a processor. The flowchart of the method is shown below. Figure 1 As shown, it specifically includes: S1: Use the calibration grid to perform calibration at multiple depth locations, and obtain the first coordinates of each calibration point in the depth camera and the second coordinates in the projector at different calibration depths; S2: Based on the first and second coordinates of each calibration point at the same calibration depth, generate the first homography matrix corresponding to that calibration depth; S3: Based on the first homography matrix corresponding to each calibration depth, calculate the transformation function between each calibration depth and each element of the first homography matrix; S4: Obtain the actual 3D coordinates of the acupoint to be projected under the depth camera, and calculate the second homography matrix corresponding to the acupoint depth in the actual 3D coordinates through the transformation function; S5: Based on the second homography matrix, the actual three-dimensional coordinates are mapped to the projector coordinate system to obtain the projected coordinates, so as to control the projector to project the acupoints according to the projected coordinates.
[0025] In this invention, before projecting acupoints on a specific body part of a user's body, depth information of various body parts (e.g., hands, feet, face, torso, whole body) of a first human body is acquired using a depth camera. Specifically, a human pose estimation method is used to identify various body parts of the first human body and key acupoints within each body part, and the locations of these key acupoints are marked. Color and depth images of each body part are acquired using the depth camera, and depth alignment is performed between the color and depth images to ensure spatial consistency. The two-dimensional coordinates (x, y) of each key acupoint are obtained from the color image, and the three-dimensional coordinates (x, y, d) of each key acupoint are obtained from the depth image. The acupoint depth is then determined using the three-dimensional coordinates.
[0026] Specifically, after obtaining the two-dimensional coordinates of each key acupoint from the color image, the depth value of each key acupoint is obtained from the depth image, thereby obtaining the three-dimensional coordinates of each key acupoint.
[0027] The aforementioned depth information is used to construct the mapping relationship between the depth camera and the projector. Since the depth values of key acupoints differ across different body parts, to ensure the accuracy of acupoint projection by the projector, a mapping relationship needs to be constructed for each different depth value. This mapping relationship is reflected through a coordinate transformation function between the depth camera and the projector. Using a depth-adaptive approach, the corresponding homography matrix is independently calculated for each depth value. During the actual transformation process, the pixel coordinates of the key acupoints in the depth camera are converted into the projected coordinates of the projector, thus controlling the projector to project the acupoints according to these coordinates.
[0028] After obtaining depth information for different body parts, a transformation function between depth values and homography matrix is constructed based on the calibration data corresponding to each depth value. This transformation function is used to reflect the mapping relationship between depth values and the depth camera and projector.
[0029] Based on the above, the present invention provides the following specific description of steps S1-S5: S1: Use the calibration grid to perform calibration at multiple depth locations, and obtain the first coordinates of each calibration point in the depth camera and the second coordinates in the projector at different calibration depths.
[0030] It should be noted that a calibration grid is projected onto a reference plane at multiple depth positions using a projector. Each depth position corresponds to a calibration depth, and multiple points are selected on the calibration grid as calibration points to obtain the first and second coordinates of the calibration points at different calibration depths. The depths of all selected calibration points under the same reference plane are consistent.
[0031] Optionally, the depth values corresponding to various acupoints in any part of the body can be selected as the depth reference basis for the calibration grid projection.
[0032] It should be noted that there is a mapping relationship between the first and second coordinates of the same calibration point at the same calibration depth. That is, at the same calibration depth d, the first coordinates of the same calibration point... With the second coordinate A mapping relationship exists: .
[0033] Specifically, when the projector projects the calibration grid onto the reference plane, the calibration element within the calibration grid is tracked. This calibration element can be a movable device on the reference plane, such as a positioning pen. During the tracking of the calibration element, if the dwell time of the calibration element at any position exceeds a preset time, the current dwell position is determined as the calibration point. The three-dimensional coordinates of the calibration point under the depth camera (i.e., the first coordinates) are collected by the depth camera, and the coordinates of the current dwell position in the projector (i.e., the second coordinates) are also determined.
[0034] In this invention, a depth camera can be used to track calibration components. During the tracking process, image processing is used to identify calibration components in the calibration grid and to detect whether the calibration components have markers (e.g., a prominent mark on the tip of a positioning pen). If markers are present, an image recognition algorithm is applied to track the markers and detect their positions in real time. If no markers are present, a preset target detection model is used to identify the tip of the calibration component (i.e., the tip of the positioning pen), and the image recognition algorithm is then used to track the tip and detect its position in real time.
[0035] When markers are set on the calibration component, the markers in the image are directly identified using image recognition algorithms. During marker tracking, Kalman filtering or feature-point-based tracking algorithms can be used to achieve stable tracking of the markers across consecutive frames. When no markers are set on the calibration component, the tip of the calibration component is set as the tracking target. The tip is detected using a target detection model. After the tip is identified, its position in the image can be detected in real time using computer vision algorithms, or its movement across consecutive frames can be determined using optical flow tracing algorithms to achieve tip localization.
[0036] During the tracking of the calibration component using a depth camera, the images acquired by the depth camera are filtered and denoised to extract the contour boundaries of connected components in the images, identify whether the calibration component contains markers, and identify the sharp points in the calibration component.
[0037] Optionally, besides using a depth camera to acquire images for tracking the calibration device, electromagnetic materials can be used as the calibration device. For example, an electromagnetic induction positioning pen can be used as the calibration device, and the three-dimensional position coordinates of the pen tip can be obtained through an electromagnetic sensor array. Specifically, if an electromagnetic induction positioning pen is used as the calibration device, the reference plane needs to be set as a plane with an electromagnetic transmitting coil array. The transmitting coil generates an alternating electromagnetic field in the plane, and an induction coil is set inside the positioning pen. When the pen tip approaches the reference plane, the transmitting coil in the reference plane and the induction coil inside the pen are induced. By observing the changes in the phase and amplitude of the induced signal, the position change of the positioning pen can be determined, thereby achieving tracking of the positioning pen.
[0038] In this invention, by tracking the calibration component, the position of the calibration component can be precisely aligned with the position of the projector mark, ensuring that the coordinate points collected by the depth camera are consistent with the coordinate points collected by the projector, thereby guaranteeing the accuracy of subsequent calculation results.
[0039] S2: Based on the first and second coordinates of each calibration point at the same calibration depth, generate the first homography matrix corresponding to that calibration depth.
[0040] The first homography matrix contains multiple elements, each element being associated with a first coordinate and a second coordinate at the calibration depth corresponding to the first homography matrix, which have a mapping relationship. Specifically, for each calibration point in the same reference plane, there is a mapping relationship between the first coordinate of the calibration point under the depth camera and the second coordinate under the projector. The first coordinate and the second coordinate with the mapping relationship form a mapping pair. The mapping pairs of multiple calibration points at the same calibration depth are combined to form a mapping combination at the same calibration depth: Where m is the number of calibration points in the same reference plane. The direct linear transformation algorithm is applied to calculate the mapping combination at each calibration depth to obtain the first homography matrix corresponding to each calibration depth. , where k is the number of calibration depths.
[0041] It should be noted that by generating the first homography matrix corresponding to different calibration depths, a series of discrete depth-homography matrix mapping pairs are obtained: That is, each calibration depth corresponds to a homography matrix.
[0042] For example, the first homography matrix to The expressions are as follows: S3: Based on the first homography matrix corresponding to each calibration depth, calculate the transformation function between each calibration depth and each element of the first homography matrix.
[0043] It should be noted that the transformation function can be a polynomial function, or a predicted value with a continuous function form based on the fundamental matrix and the offset matrix.
[0044] In this invention, one method for calculating the transformation function is as follows: After obtaining the first homography matrix corresponding to each calibration depth, elements at the same position in the first homography matrix corresponding to each calibration depth are extracted to form an element sequence. For example, the first homography matrix is extracted separately. to elements in Constituent element sequence Since each element in the element sequence is a known quantity obtained from the first homography matrix corresponding to different calibration depths, the least squares method is applied to fit each element in the element sequence to the corresponding calibration depth to obtain a transformation function, which is a polynomial function.
[0045] It is understandable that the elements at the same position in each of the first homography matrices are used to form an observation sequence (i.e., an element sequence). Each element in this sequence is treated as a data point. The least squares method is then used to fit each data point in the sequence so that the result of the transformation function is as close as possible to each data point in the sequence.
[0046] Therefore, the transformation function calculated using the least squares method is: in, The first homography matrix selected from each first homography matrix Line 1 A sequence of elements formed by the elements of a column. , The depth value for calibration depth. The order is typically 3 to 4, chosen to balance accuracy and generalization ability. The fitting coefficients are denoted as .
[0047] By combining each element in the sequence of elements, the polynomial coefficients in the transformation function can be determined. For the first homography matrix of 3×3, the number of polynomial coefficients is: indivual.
[0048] It should be noted that after solving for the polynomial coefficients in the transformation function, Given the depth camera's 3D coordinates, when converting them to projected coordinates in the projector's coordinate system, we only need to substitute the depth value and polynomial coefficients from the 3D coordinates into the transformation function to obtain the second homography matrix that matches the depth value. Then, we can obtain the projected coordinates using the second homography matrix and the 3D coordinates.
[0049] In this invention, another way to calculate the transformation function is as follows: Get the pre-set basic matrix and the depth offset corresponding to the calibration depth The transformation function is calculated. The fundamental matrix is the matrix corresponding to the center depth of the calibration range; the depth offset is the depth offset corresponding to the calibration depth calculated based on the calibration depth and the homography matrix; the transformation function corresponding to the calibration depth is obtained by adding the fundamental matrix and the depth offset corresponding to the calibration depth. That is, the depth value at a certain calibration depth. The corresponding transformation function is: .
[0050] in, This is the second homography matrix required in the transformation function.
[0051] Specifically, the depth offset can be obtained through polynomial calculation or prediction using a neural network model.
[0052] In a first optional embodiment, the depth offset can be obtained through polynomial calculation as follows: in, These are the polynomial coefficients. Each matrix element independently establishes a set of polynomial functions, totaling... coefficients (and) The total of 9 elements (Number of parameters).
[0053] It should be noted that the formula for calculating the depth offset using the polynomial above is similar to the formula for calculating the transformation function using the least squares method. In this embodiment, the polynomial calculation method for the depth offset is to use the least squares method after removing the initial constant term, i.e., the above formula: In the middle, setting ,Right now, This is the initial constant term in the least squares method.
[0054] In this invention, the depth offset exhibits a smooth, non-linear relationship with depth, and the coefficient... The "curvature" and "trend" of this change are described.
[0055] It should be noted that before using the least squares method to remove the initial constant term for calculation, it is also necessary to extract the elements at the same position in the first homography matrix corresponding to the calibration depth to form an element sequence, so that each data point in the element sequence can be used as a known quantity to calculate the coefficients of each polynomial. The specific processing procedure will not be described here.
[0056] In a second alternative embodiment, the depth offset can be calculated using linear interpolation as follows: The depth range is divided into multiple depth intervals, which include all calibration depths. Each depth interval contains at least two calibration depths. When it is necessary to calculate the depth offset corresponding to a certain calibration depth, a linear interpolation algorithm is applied to calculate the calibration depth and the endpoints of the depth interval to obtain the depth offset corresponding to the calibration depth.
[0057] Determine the depth range to which each calibration depth belongs, and divide the depth range into multiple intervals in ascending order of calibration depth, for example: ,in After dividing the depth range into different intervals, calibration data is used to determine each endpoint (e.g., depth interval) within each interval. The endpoints include the upper endpoint. and lower endpoint The endpoint offsets are then used to calculate the depth offset corresponding to a calibration depth. After obtaining the endpoint offsets, when it is necessary to calculate the depth offset corresponding to a certain calibration depth, a linear interpolation algorithm is applied to calculate the calibration depth, the endpoints in the depth interval to which the calibration depth belongs, and the endpoint offsets to obtain the depth offset corresponding to the calibration depth.
[0058] For example, calibration depth Belongs to the depth range The depth offset corresponding to this calibration depth is: in, This represents the endpoint offset at the endpoint of the depth interval, which is determined using calibration data. When hour, .
[0059] In this invention, when the depth-offset relationship exhibits obvious nonlinear or abrupt characteristics, piecewise linearity can better capture local characteristics and avoid oscillations of higher-order polynomials in the boundary region.
[0060] In a third optional embodiment, the method for calculating the depth offset corresponding to the calibration depth using a neural network model can be: The calibration depth and its corresponding first homography matrix are input into a preset neural network model to obtain the depth offset corresponding to the calibration depth output by the neural network model. The neural network model is a model that automatically learns the mapping relationship between the implicit features of the calibration depth and the depth offset through a large amount of training data.
[0061] Understandably, by collecting a large amount of data on the relationship between depth values and depth offsets as training data and inputting it into a neural network model, the neural network model automatically learns the implicit features of depth values and the mapping relationship between depth offsets using the input training data. This allows for adjustments to the network parameters within the neural network model, ultimately leading to its application. Build depth offset: in, For depth value, These are network parameters.
[0062] It should be noted that the neural network model can be a fully connected network or a one-dimensional convolutional network. In this invention, when the depth-offset relationship is complex and difficult to describe with simple mathematical functions, the neural network, due to its stronger expressive power, can automatically learn the mapping between the implicit features of depth and the matrix offset.
[0063] S4: Obtain the actual 3D coordinates of the acupoint to be projected under the depth camera, and calculate the second homography matrix corresponding to the acupoint depth in the actual 3D coordinates through the transformation function.
[0064] In one embodiment, when it is necessary to project key acupoints on any body part, the three-dimensional coordinates of the acupoints to be projected are acquired using a depth camera. To obtain the depth value of the acupoint to be projected under the depth camera. Substitute this depth value into the corresponding transformation function. Since the polynomial coefficients in the transformation function have been solved through step S3 above, By simply substituting the depth value, the second homography matrix can be obtained. Using the second homography matrix and the three-dimensional coordinates, the projection coordinates of the acupoint to be projected on the projector can be derived. The projector is controlled to project acupoints onto key acupoints on the target body part based on the projection coordinates.
[0065] In another embodiment, when it is necessary to project key acupoints on any body part, the three-dimensional coordinates of the acupoints to be projected are acquired using a depth camera. To obtain the depth value of the acupoint to be projected under the depth camera. Substitute this depth value into the corresponding transformation function. Since the depth offsets corresponding to different depths in the transformation function have already been solved through step S3 above, we only need to substitute the acupoint depth to obtain the second homography matrix. The projection coordinates of the acupoints to be projected in the projector are derived using the second homography matrix and three-dimensional coordinates. The system then controls the projector to project acupoints onto the target body part based on these projection coordinates. For example, when the depth offset is calculated using a polynomial formula, the calculation is performed according to the formula... Calculate the depth offset corresponding to the acupoint to be projected; when the depth offset is calculated using an interval calculation method, determine the depth interval to which the acupoint to be projected belongs, and then apply the formula... The depth offset corresponding to the acupoint to be projected is calculated. When the depth offset is calculated using a neural network model, the acupoint depth of the acupoint to be projected is input into the neural network model to obtain the depth offset corresponding to the acupoint to be projected output by the neural network model.
[0066] S5: Based on the second homography matrix, the actual three-dimensional coordinates are mapped to the projector coordinate system to obtain the projected coordinates, so as to control the projector to project the acupoints according to the projected coordinates.
[0067] It should be noted that the transformation from the 3D coordinates of the depth camera to the projected coordinates of the projector satisfies the following transformation formula: in, The pixel coordinates of the acupoint to be projected in the depth camera; The second homography matrix is a 3×3 matrix, which can be used to represent the depth value in the three-dimensional coordinates of the acupoint to be projected. The result is obtained by substituting into the transformation function; These are the coordinates in the projector coordinate system after the transformation. 1 represents the homogeneous coordinate factor, and 1 represents the third component added to the homogeneous coordinate.
[0068] In this invention, the first homography matrix is a matrix obtained by direct linear transformation of the mapping combination corresponding to the calibration depth; the second homography matrix is a matrix obtained by transformation function.
[0069] In the method provided in this embodiment of the invention, after generating the transformation formula, a professional physician can locate and label the acupoints for correction, and determine the error of the transformation function based on the location or labeling results, and then correct the transformation function.
[0070] Based on the acupoint projection method provided in the embodiments of the present invention, the technical solution of the present invention has the following advantages: (1) High-precision positioning. Since each key point is transformed using its own actual depth, systematic errors caused by uniform depth are avoided.
[0071] (2) Adapting to the three-dimensional structure of the human body. The human body is a complex three-dimensional structure, and there are significant height differences between different acupoints. The point-by-point depth transformation method of the present invention can perfectly adapt to this three-dimensional curvature, so that acupoints in various parts of the body (hands, feet, face, etc.) can obtain accurate projection mapping.
[0072] (3) Improved calibration efficiency. The polynomial fitting method makes full use of the depth information of all calibration points, with each point participating in the modeling, and a smooth depth-transformation mapping can be obtained without dense calibration. Compared with the traditional method of averaging depth by group, it eliminates the problem of depth information loss.
[0073] (4) Smooth and continuous projection. A continuous function relationship is established by polynomial fitting. The transformation matrix of the key point can be obtained accurately regardless of the depth value. The projection is continuous and smooth, without discrete jump phenomenon.
[0074] (5) Versatility across multiple body parts. The same technical framework is applicable to acupoint positioning projection on various parts of the body, such as the hands, feet, face, and torso, and has good scalability.
[0075] (6) The annotation process is consistent with the diagnosis and treatment scenario. This invention collects data in the actual diagnosis and treatment environment. Professional physicians use the same positioning method as in actual diagnosis and treatment (using a pen to specify acupoints). The collected data truly reflects the diagnosis and treatment process and has higher practical value and scenario consistency.
[0076] (7) Significantly improved annotation efficiency. Compared with the traditional manual annotation method, the present invention automatically completes data collection and annotation while the physician designates the acupoints, greatly shortening the annotation time per session and significantly improving efficiency.
[0077] (8) Enhanced integrity of annotation information. This invention simultaneously records multi-dimensional data such as acupoint location, depth information, multi-angle images, timestamps, and site type, and the integrity of annotation information far exceeds that of traditional methods.
[0078] (9) Supports in-situ verification. Through the projection verification unit, the annotation results can be projected onto the human body surface in real time for verification, ensuring the accuracy of the annotation and supporting physicians to confirm or correct it on the spot.
[0079] and Figure 1 Corresponding to the method described above, embodiments of the present invention also provide an acupoint projection system for... Figure 1 The specific implementation of the method is shown in the following structural diagram. Figure 2 As shown, it specifically includes: Projector, depth camera, and processor; The depth camera is used to acquire the first coordinates of each calibration point of the calibration grid at different calibration depths and the actual three-dimensional coordinates of the acupoint to be projected, and send them to the processor. The projector is used to project the calibration grid onto reference planes at different depths, collect the second coordinates of each calibration point of the calibration grid at different calibration depths and send them to the processor; upon receiving the projection coordinates sent by the processor, the acupoints are projected according to the projection coordinates. The processor is used to execute the acupoint projection method described above.
[0080] Optionally, the system further includes: A calibration element is used to move within the calibration grid and indicate the position of calibration points within the calibration grid.
[0081] The specific working process of each device and component in the acupoint projection system disclosed in the above embodiments of the present invention can be found in the corresponding content of the acupoint projection method disclosed in the above embodiments of the present invention, and will not be repeated here.
[0082] This invention also provides a storage medium, which includes stored instructions, wherein the execution of the instructions controls the device containing the storage medium to perform the above-described acupoint projection method.
[0083] This invention also provides an electronic device, the structural schematic of which is shown below. Figure 3 As shown, it specifically includes a memory 301 and one or more instructions 302, wherein one or more instructions 302 are stored in the memory 301 and configured to be executed by one or more processors 303 to perform the following operations: The calibration grid is used to perform calibration at multiple depth locations, and the first coordinates of each calibration point in the depth camera and the second coordinates in the projector are obtained at different calibration depths. Based on the first and second coordinates of each calibration point at the same calibration depth, generate the first homography matrix corresponding to that calibration depth; Based on the first homography matrix corresponding to each calibration depth, calculate the transformation function between each calibration depth and each element of the first homography matrix; Obtain the actual three-dimensional coordinates of the acupoint to be projected under the depth camera, and calculate the second homography matrix corresponding to the acupoint depth in the actual three-dimensional coordinates using the transformation function; Based on the second homography matrix, the actual three-dimensional coordinates are mapped to the projector coordinate system to obtain the projected coordinates, so as to control the projector to project the acupoints according to the projected coordinates.
[0084] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0085] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both.
[0086] To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality above. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0087] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for acupoint projection, characterized in that, include: The calibration grid is used to perform calibration at multiple depth locations, and the first coordinates of each calibration point in the depth camera and the second coordinates in the projector are obtained at different calibration depths. Based on the first and second coordinates of each calibration point at the same calibration depth, generate the first homography matrix corresponding to that calibration depth; Based on the first homography matrix corresponding to each calibration depth, calculate the transformation function between the calibration depth and each element of the first homography matrix; Obtain the actual three-dimensional coordinates of the acupoint to be projected under the depth camera, and calculate the second homography matrix corresponding to the acupoint depth in the actual three-dimensional coordinates using the transformation function; Based on the second homography matrix, the actual three-dimensional coordinates are mapped to the projector coordinate system to obtain the projected coordinates, so as to control the projector to project the acupoints according to the projected coordinates.
2. The method according to claim 1, characterized in that, The process of obtaining the first and second coordinates of each calibration point at different depths includes: When calibrating at multiple depth locations using a calibration grid, the calibration components within the calibration grid are tracked; When the dwell time of the calibration component at any position exceeds the preset time, the current dwell position of the calibration component is determined as the calibration point, and the first coordinate and second coordinate of the calibration point at different calibration depths are obtained.
3. The method according to claim 2, characterized in that, The tracking of calibration elements within the calibration grid includes: Check whether the calibration component is equipped with a marker; When the calibration component is equipped with markers, a preset image recognition algorithm is applied to detect the position of the markers in real time; In the absence of markers on the calibration component, the tip of the calibration component is identified, and the position of the tip is detected in real time using the image recognition algorithm.
4. The method according to claim 1, characterized in that, The process of generating the first homography matrix corresponding to the calibration depth based on the first and second coordinates of each calibration point at the same calibration depth includes: Obtain the mapping combination corresponding to each calibration depth. The mapping combination contains multiple mapping pairs. The mapping pairs are first coordinates and second coordinates that have a mapping relationship under the same calibration depth. A preset direct linear transformation algorithm is applied to calculate each of the mapping combinations to obtain the first homography matrix corresponding to each calibration depth.
5. The method according to claim 1, characterized in that, The transformation function between the calibration depth and the elements of the first homography matrix includes: Extract the elements at the same position from each of the first homography matrices to form an element sequence; The transformation function is obtained by fitting each element in the element sequence with its corresponding calibration depth using a preset least squares method.
6. The method according to claim 1, characterized in that, The transformation function between the calibration depth and the elements of the first homography matrix includes: Obtain a pre-set base matrix, which is the matrix corresponding to the center depth of the calibration range; Based on the first homography matrix corresponding to each calibration depth, calculate the depth offset corresponding to the calibration depth; The transformation function is obtained by adding the basic matrix to the depth offset corresponding to the calibration depth.
7. The method according to claim 6, characterized in that, The calculation of the depth offset corresponding to the calibration depth includes: Extract the elements at the same position in the first homography matrix corresponding to each calibration depth to form an element sequence; The least squares method, which removes the initial constant term, is used to fit each element in the element sequence to its corresponding calibration depth to obtain the depth offset.
8. The method according to claim 6, characterized in that, The calculation of the depth offset corresponding to the calibration depth includes: The depth range is divided into multiple depth intervals, wherein the depth range includes all calibration depths and each depth interval includes at least two calibration depths; For any calibration depth, determine the depth interval to which the calibration depth belongs, and apply a preset linear interpolation algorithm to calculate the calibration depth and the endpoints of the depth interval to obtain the depth offset corresponding to the calibration depth.
9. The method according to claim 6, characterized in that, The calculation of the depth offset corresponding to the calibration depth includes: The calibration depth and its corresponding first homography matrix are input into a preset neural network model to obtain the depth offset corresponding to the calibration depth output by the neural network model. The neural network model is a model that automatically learns the mapping relationship between the implicit features of the calibration depth and the depth offset through a large amount of training data.
10. An acupoint projection system, characterized in that, The system includes: Projector, depth camera, and processor; The depth camera is used to acquire the first coordinates of each calibration point of the calibration grid at different calibration depths and the actual three-dimensional coordinates of the acupoint to be projected, and send them to the processor. The projector is used to project the calibration grid onto reference planes at different depths, collect the second coordinates of each calibration point of the calibration grid at different calibration depths and send them to the processor; upon receiving the projection coordinates sent by the processor, the acupoints are projected according to the projection coordinates. The processor is used to execute the acupoint projection method as described in any one of claims 1-9.