A method for overlaying and annotating acupoint locations in traditional Chinese medicine teaching scenarios

Abstract

This invention discloses a method for image overlay annotation of acupoint location in traditional Chinese medicine teaching scenarios, applied to teaching terminals. The method acquires teaching image streams, student image streams, current teaching mode information, and target acupoint information. Based on key points of the human body, it constructs a semantic coordinate system of the human lateral aspect for both the instructor and the student. Depending on whether the teaching is in the same direction or a face-to-face mirror teaching mode, it establishes a mapping relationship between the target lateral aspect, the teaching display lateral aspect, and the student's practice lateral aspect, generating acupoint overlay annotation information in the corresponding image streams. During student practice, it performs position and lateral aspect verification based on the positional relationship between the operation point and the acupoint anchor point or projection area, as well as the human lateral aspect to which the operation point belongs. It outputs practice confirmation, lateral aspect correction, or position repositioning annotations, thereby reducing acupoint practice errors caused by inconsistencies between the left and right sides of the image and the human lateral aspect in mirror teaching.

Description

Technical Field

[0001] This application relates to the field of TCM teaching technology, and in particular to a method for superimposing and annotating acupoint location images for TCM teaching scenarios. Background Technology

[0002] Acupuncture and moxibustion, as well as the study of meridians and acupoints, are important components of traditional Chinese medicine education. During acupoint instruction, teachers typically demonstrate the target acupoint's location on the body surface, its corresponding side, and its relationship to surrounding landmarks. Students then practice imitating the teacher's demonstration to complete acupoint identification and location training.

[0003] Existing technologies already include methods for teaching acupoints using image overlay, augmented reality, virtual reality, or mixed reality. These methods can display acupoint locations, meridian pathways, and local body surface structures on human images or virtual human models, thereby enhancing the intuitiveness and interactivity of acupoint teaching.

[0004] In addition, existing technologies include mirroring human body images, flipping instructional videos horizontally, and guiding users to imitate movements based on mirrored images in follow-up exercises. These solutions typically improve the user's viewing experience of the instructional images and enhance the ease of movement imitation to some extent.

[0005] However, in TCM teaching, especially in the context of acupoint instruction, teachers often use face-to-face demonstrations. In this case, the left side of the body the teacher is demonstrating might appear as the right side in the student's view; similarly, the right side might appear as the left side. For acupoints like Lieque, Hegu, Neiguan, and Waiguan, which are symmetrical and share the same name, if the system only superimposes the acupoint label based on the left and right positions displayed in the image, or simply mirrors the demonstration without semantically distinguishing between the left and right sides in an anatomical sense, students may easily develop the mistaken impression that the image positions are correct but the body's sides are incorrect.

[0006] Furthermore, existing acupoint display solutions focus more on how to display acupoints at corresponding positions in human body images. However, for face-to-face mirror teaching conditions, there is still a lack of targeted processing mechanisms for how to correspond the display semantics in the teacher's teaching screen, the student's observation semantics, and the student's actual practice semantics, and how to identify situations where the position is basically correct but the side identification is incorrect during the student's practice.

[0007] Therefore, it is necessary to provide a method for superimposing and annotating acupoint location images for TCM teaching scenarios to solve the problem that the displayed semantics of symmetrical acupoints with the same name are inconsistent with the students' observation semantics under face-to-face mirror teaching conditions, which leads to students making mistakes in side identification during practice. Summary of the Invention

[0008] This application provides a method for superimposing and annotating acupoint location images in traditional Chinese medicine teaching scenarios to solve the problem that the displayed semantics of symmetrical acupoints with the same name are inconsistent with the students' observation semantics under face-to-face mirror teaching conditions, which leads to students selecting the corresponding position in the picture but actually misidentifying the acupoints during practice.

[0009] Firstly, this application provides a method for image overlay annotation of acupoint location in traditional Chinese medicine teaching scenarios, which can be executed by a teaching terminal. In this method, the teaching terminal acquires a teaching image stream, a student image stream, current teaching mode information, and current target acupoint information. Based on the teaching image stream and the student image stream, the teaching terminal determines the lateral semantic information of the instructor's body and the student's body, respectively, and establishes a mapping relationship between the target lateral of the current target acupoint and the lateral of the instructor's body and the student's body, respectively, according to the current teaching mode information. Based on the current target acupoint information and the mapping relationship, the teaching terminal generates acupoint overlay annotation information in the teaching image stream and the student image stream, respectively. Responding to the student's practice operation, the teaching terminal performs lateral and position verification on the student's practice operation based on the lateral and positional relationships corresponding to the student's practice operation, and outputs the corresponding annotation results.

[0010] In the above method, the teaching terminal does not directly label acupoints based on the left-right position on the image display plane. Instead, it first determines the lateral semantic information of the instructor's and student's bodies, and establishes a mapping relationship between the target acupoints on the instructor's side and the student's side based on the teaching mode. This distinguishes the displayed position in the image from the anatomical side of the human body. In this way, even under face-to-face mirror teaching conditions, the teaching terminal can still accurately prompt and verify the student's side during practice, preventing the student from practicing the target acupoint on the opposite side due to judging solely by the left-right position on the screen. This improves the consistency of instruction and the accuracy of practice during acupoint teaching.

[0011] In one possible design, establishing the mapping relationship between the target side of the current target acupoint and the side of the instructor's body and the student's body based on the current teaching mode information includes: determining the target mapping rule from preset side mapping rules based on the current teaching mode information. The preset side mapping rules include side mapping rules corresponding to the same-direction teaching mode and side mapping rules corresponding to the face-to-face mirror teaching mode. Through this design, the teaching terminal can invoke different side interpretation rules for different teaching methods, thereby avoiding side misguidance caused by mechanically applying the same set of left-right judgment logic to different teaching scenarios.

[0012] In one possible design, the lateral semantic information is a human lateral semantic coordinate system constructed based on human key points; the human key points include at least the left shoulder key point, right shoulder key point, left hip key point, and right hip key point. With this design, the teaching terminal can determine the left and right regions based on the human body's own structure, rather than solely relying on left and right positions in image coordinates, thereby improving the stability of lateral recognition.

[0013] In one possible design, the current target acupoint information includes acupoint identifiers and target lateral orientations. The step of generating acupoint overlay annotation information in the teaching image stream and the student image stream, respectively, based on the current target acupoint information and mapping relationships, includes: determining the lateral orientation of the teaching body and the student's practice side according to the target lateral orientation and target mapping rules, and overlaying the acupoint identifiers in the teaching image stream and the student image stream, respectively. Through this design, the teaching terminal can output lateralized acupoint annotation results matching the current teaching mode to both the teacher's teaching screen and the student's practice screen.

[0014] In one possible design, the acupoint overlay information includes the acupoint identifier, the side of the person demonstrating the lesson, and the side of the student practicing. With this design, the teaching terminal can output the acupoint location and the corresponding side information simultaneously, providing students with clearer side identification prompts when observing the teacher's demonstration.

[0015] In one possible design, when the current target acupoint is a bilaterally symmetrical acupoint with the same name, the acupoint overlay labeling information also includes a lateral differentiation prompt; the lateral differentiation prompt includes at least one of arrow symbols, text symbols, and split-screen symbols. Through this design, for scenarios where bilaterally symmetrical acupoints with the same name are easily confused, the teaching terminal can add explicit differentiation information, reducing the probability that students will mistakenly identify the same acupoint as the opposite side.

[0016] In one possible design, the response to the student's practice operation, based on the lateral and positional relationships corresponding to the student's practice operation, includes performing lateral and positional verification on the student's practice operation. This includes: acquiring the operation area corresponding to the student's practice operation; and determining the human lateral and operation coordinates corresponding to the student's practice operation based on the position of the operation area in the student's image stream and lateral semantic information. Through this design, the teaching terminal can perform a correspondence analysis between the student's actual operation and the target acupoint, so that lateral and positional verification can be performed subsequently.

[0017] In one possible design, the position verification includes: comparing the distance between the operation coordinates and the coordinates of the acupoint anchor point corresponding to the current target acupoint, or determining whether the operation coordinates fall within the acupoint projection area corresponding to the current target acupoint; when the comparison result meets the preset position conditions, it is determined that the position verification meets the preset conditions. Through this design, the teaching terminal can determine from a spatial position perspective whether the student has operated on the corresponding area of ​​the target acupoint.

[0018] In one possible design, the output of the corresponding annotation result includes: when the position verification meets the preset conditions but the lateral verification does not meet the preset conditions, outputting a lateral correction annotation, and maintaining the superimposed annotation of the current target acupoint on the body surface area corresponding to the student's lateral practice. Through this design, the teaching terminal can identify situations where the student's position is basically correct but the lateral alignment is incorrect, and perform specific corrections for such situations, preventing the system from misjudging the practice as correct based solely on proximity.

[0019] In one possible design, the method further includes: during the continuous updating of the teaching image stream and the student image stream, if changes in human key points are detected to exceed a preset update threshold, or if changes in the current teaching mode information are detected, then the lateral semantic information is redefined, the mapping relationship is re-established, and the acupoint overlay annotation information is updated. Through this design, the teaching terminal can promptly correct the lateral semantics and annotation results in cases of changes in the posture of the instructor or student, or switching of teaching modes, to ensure the continuity and stability of the teaching process.

[0020] Secondly, this application also provides a teaching terminal, which includes multiple functional modules; the multiple functional modules interact with each other to implement the methods described in the first aspect and its various embodiments. The multiple functional modules can be implemented based on software, hardware, or a combination of both.

[0021] Thirdly, this application also provides an apparatus including at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the apparatus is in operation, the at least one processor executes the methods described in the first aspect and its various embodiments.

[0022] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the methods described in the first aspect and its various embodiments.

[0023] Compared with existing technologies, this application has at least the following beneficial effects: The teaching terminal establishes a lateral mapping relationship of the target acupoint based on the semantic information of the human body's lateral aspect and the teaching mode information, so that the lateral aspect of the teaching body and the lateral aspect of the student's practice can be distinguished from each other, which can avoid the error of the lateral aspect of practice caused by the inconsistency between the left and right positions of the screen and the anatomical lateral aspect under face-to-face mirror teaching conditions; at the same time, the teaching terminal combines position verification and lateral aspect verification, which can identify the situation where the student's position is correct but the lateral aspect is incorrect, and output targeted correction marks, thereby improving the accuracy and stability of acupoint teaching and practice. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a method for overlaying and annotating acupoint locations in a traditional Chinese medicine teaching setting, provided in this application embodiment; Figure 2 A schematic diagram of a teaching scenario provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the construction of a human lateral semantic coordinate system provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the relationship between a teaching mode and a lateral mapping, provided in an embodiment of this application. Figure 5 A schematic diagram illustrating the generation of acupoint anchor point coordinates and acupoint projection area provided in this application embodiment; Figure 6 This application provides a schematic diagram for verifying a student's practice operation. Figure 7 This is a schematic diagram of the structure of a teaching terminal provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0026] This application provides a method for overlaying and annotating acupoint location images in traditional Chinese medicine teaching scenarios. The method can be applied to a teaching terminal. The teaching terminal can be any of the following: an electronic whiteboard device, a tablet terminal, an all-in-one machine, a head-mounted display device, or a computing device with image acquisition and display capabilities. The teaching terminal can acquire images of teacher demonstrations and student practice, and perform acupoint location image overlay annotation and practice verification within the corresponding images.

[0027] In some implementations, the teaching scenarios applicable to the teaching terminal may include unidirectional teaching scenarios and face-to-face mirror teaching scenarios. In unidirectional teaching scenarios, the teacher and student face roughly the same direction, and the target side the teacher is currently teaching on is consistent with the target side the student is currently practicing. In face-to-face mirror teaching scenarios, the teacher and student stand opposite each other, and the display position of the target side the teacher is currently teaching on in the student's view may differ from the target side the student should be practicing; therefore, it is necessary to distinguish between display semantics and practice semantics.

[0028] Figure 1 This is a flowchart illustrating a method for overlaying and annotating acupoint locations in a traditional Chinese medicine teaching setting, as provided in an embodiment of this application. (Reference) Figures 1 to 6 The method includes the following steps: S101: The teaching terminal acquires the teaching image stream, the student image stream, the current teaching mode information, and the current target acupoint information.

[0029] Among them, the teaching image stream is used to represent the sequence of human images in the current teaching process of the teacher, the student image stream is used to represent the sequence of human images in the current practice process of the student, the current teaching mode information is used to indicate whether the current teaching belongs to the same-direction teaching mode or the face-to-face mirror teaching mode, and the current target acupoint information is used to indicate the acupoint content corresponding to this round of teaching or practice.

[0030] In some embodiments, the current target acupoint information may include an acupoint identifier and a target side. For example, the acupoint identifier may be the name of any acupoint among Lieque, Hegu, Neiguan, and Waiguan, and the target side may be the left or right side. Optionally, the current target acupoint information may also include acupoint anchor point coordinates, acupoint projection area, acupoint description information, and information about the body surface area to which it belongs.

[0031] In some embodiments, the current teaching mode information can be obtained by the teaching terminal according to a preset teaching process, or it can be selected and input by the teacher through the interface. For example, before starting the lesson, the teacher can select a face-to-face mirror teaching mode or a same-direction teaching mode, and the teaching terminal will determine the subsequent side mapping method accordingly.

[0032] S102: The teaching terminal determines the lateral semantic information of the instructor's body and the student's body based on the teaching image stream and the student's image stream, respectively, and establishes a mapping relationship between the target lateral of the current target acupoint and the teaching lateral of the instructor's body and the student's practice lateral of the student's body according to the current teaching mode information.

[0033] In this embodiment, the teaching terminal does not directly determine the lateral aspect of the human body based on the left and right pixel positions in the teaching image or student image. Instead, it first determines the key structures of the human body and then establishes the semantic information of the lateral aspect. Optionally, the semantic information of the lateral aspect can be obtained through human key point recognition.

[0034] In one optional implementation, the teaching terminal performs human keypoint recognition in both the teaching image stream and the student image stream to obtain the human keypoints of the current human body. The human keypoints include at least the left shoulder keypoint, right shoulder keypoint, left hip keypoint, and right hip keypoint. In some embodiments, they may also include at least one of the following: neck keypoint, suprasternal notch keypoint, navel keypoint, and pelvic center point.

[0035] The teaching terminal constructs a lateral semantic coordinate system of the human body based on the aforementioned key points. Specifically, the teaching terminal calculates the midpoint between the left and right shoulder key points, denoted as the shoulder midpoint; calculates the midpoint between the left and right hip key points, denoted as the hip midpoint; uses the line connecting the shoulder midpoint and the hip midpoint as the longitudinal reference line of the human body, and uses the midline passing through the shoulder midpoint and the hip midpoint as the midline of the human torso.

[0036] In one optional implementation, the teaching terminal establishes a local two-dimensional semantic coordinate system for the human body by using the direction of the line connecting the midpoint of the hip and the midpoint of the shoulder as the longitudinal direction vector and the direction from the right shoulder key point to the left shoulder key point as the lateral direction vector u. Using the midpoint M of the line connecting the midpoint of the shoulder and the midpoint of the hip as the reference point for the midline of the torso, for any point P in the image, the teaching terminal calculates the lateral sign value s(P) of point P relative to the midline of the torso, where s(P) = (PM)·u, and the symbol "·" represents the vector dot product. When the lateral sign value s(P) is greater than or equal to a preset zero threshold, point P is determined to be located in the semantic region on the left side of the human body; when the lateral sign value s(P) is less than the preset zero threshold, point P is determined to be located in the semantic region on the right side of the human body. In some embodiments, the preset zero threshold is 0; in other embodiments, the preset zero threshold is a tolerance value close to 0.

[0037] To reduce lateral jitter caused by slight body swaying, the teaching terminal performs smooth updates on the shoulder midpoint, hip midpoint, and torso midline in consecutive frames. Optionally, the torso midline position in the current frame is determined by a weighted average of the detection results from the current frame and the previous frame. For example, the weight for the current frame can be between 0.6 and 0.8, and the weight for the previous frame can be between 0.4 and 0.2.

[0038] In one optional implementation, when the confidence level of any human key point is lower than a first preset confidence threshold, the teaching terminal does not directly use the human key point to participate in the reconstruction of the human lateral semantic coordinate system. Instead, it calls the position of the corresponding human key point in the previous valid frame, or performs compensation estimation based on the relative positional relationship between adjacent human key points on the same side and the midline of the torso. For example, the first preset confidence threshold can be 0.55, 0.60, or 0.65.

[0039] Through this method, the semantic regions of the left and right sides of the human body determined by the teaching terminal no longer depend on the static left and right positions on the image display plane, but rather on the current structural distribution of the human body itself. Therefore, even when the teacher turns or shifts their body position, or when the student's standing position is tilted, the left and right sides can still be stably distinguished in an anatomical sense.

[0040] In one optional implementation, the teaching terminal pre-stores the correspondence between teaching modes and lateral mapping rules. The teaching modes include at least a same-direction teaching mode and a face-to-face mirror teaching mode. The lateral mapping rules include at least the correspondence rules between the target lateral, the teaching body lateral, and the student's practice lateral.

[0041] When the current teaching mode is a same-direction teaching mode, the teaching terminal determines the side of the teaching body and the side of the student's practice according to the same-side mapping rule. Specifically, when the target side of the current target acupoint is the left side, the side of the teaching body is determined to be the left side of the teacher's body, and the side of the student's practice is determined to be the left side of the student's body; when the target side of the current target acupoint is the right side, the side of the teaching body is determined to be the right side of the teacher's body, and the side of the student's practice is determined to be the right side of the student's body.

[0042] When the current teaching mode is a face-to-face mirror teaching mode, the teaching terminal separately determines the side of the person being taught and the side of the student practicing. Specifically, when the target side of the current acupoint is the left side of the teacher's body, the teaching terminal determines the corresponding area on the left side of the teacher's body in the teaching image stream as the teaching side, and determines the corresponding area on the left side of the student's body in the student image stream as the student's practice side; when the target side of the current acupoint is the right side of the teacher's body, the teaching terminal determines the corresponding area on the right side of the teacher's body in the teaching image stream as the teaching side, and determines the corresponding area on the right side of the student's body in the student image stream as the student's practice side. Simultaneously, the teaching terminal also outputs side indication information corresponding to the observation display position, indicating the left or right display position of the currently taught side in the observation screen. Thus, the teaching terminal distinguishes between the left or right side displayed on the screen and the actual left or right side where the person should be practicing.

[0043] In an optional implementation, the teaching terminal may also configure the lateral mapping rules to be stored in the form of a rule table. For example, the rule table includes at least the following fields: teaching mode field, target lateral field, demonstration body lateral field, student practice lateral field, and observation display position field. After receiving the current teaching mode information and the current target acupoint information, the teaching terminal looks up the table to obtain a set of mapping results matching the current teaching mode and target lateral, and generates subsequent overlay annotation information accordingly.

[0044] S103: The teaching terminal generates acupoint overlay annotation information in the teaching image stream and the student image stream respectively, based on the current target acupoint information and mapping relationship.

[0045] In this embodiment of the application, the teaching terminal can determine the teaching body side in the teaching image stream and the student practice side in the student image stream based on the acupoint identifier and target side in the current target acupoint information and the mapping relationship obtained in step S102, and output acupoint superimposed annotation information in the corresponding body surface area respectively.

[0046] In one optional implementation, the teaching terminal pre-stores an acupoint template library. For each target acupoint, the acupoint template library records the body surface region to which the target acupoint belongs, a set of reference landmarks used to locate the target acupoint, and a normalized offset parameter relative to the set of reference landmarks. The set of reference landmarks may include at least one of the following: key points on the body surface, local boundary points, center points of transverse creases, bony edge points, and center points of intertendinous spaces.

[0047] After the teaching terminal acquires the information of the current target acupoint, it first determines a local reference region based on the body surface area to which the target acupoint belongs, and then extracts a set of reference marker points corresponding to the current target acupoint from the local reference region. Based on the set of reference marker points and a normalized offset parameter, the teaching terminal calculates the acupoint anchor point coordinates of the current target acupoint in the current image.

[0048] In one optional calculation method, the teaching terminal selects a first reference point R1 and a second reference point R2 from the set of reference markers. Using the line connecting R1 and R2 as a local reference axis, the coordinates A of the target acupoint anchor point are expressed as: A = R1 + α × (R2 - R1) + β × n. Here, α represents the normalized distance coefficient along the local reference axis, β represents the normalized offset coefficient perpendicular to the local reference axis, and n represents the unit normal vector of the local reference axis. α and β are pre-given by the acupoint template library. In an optional implementation, the unit normal vector n is the normal vector obtained by rotating the local reference axis 90 degrees according to a preset rotation direction; exemplarily, the preset rotation direction is a counterclockwise direction from the first reference point R1 to the second reference point R2. When the target acupoint is located on the other side of the local reference axis, the corresponding β takes a negative value.

[0049] In one optional implementation, the teaching terminal further generates an acupoint projection area based on the coordinates of the acupoint anchor points. For acupoints displayed as dots, the teaching terminal can generate a circular projection area with the acupoint anchor point coordinates A as the center and the distance obtained by multiplying the local reference length L by the scaling factor γ as the projection radius. For elongated body surface areas, the teaching terminal can generate an elliptical projection area with the acupoint anchor point coordinates A as the center, based on the major axis length, minor axis length, and local reference axis direction. For example, the scaling factor γ can be taken as 0.03 to 0.08.

[0050] For example, for target acupoints in the forearm region, the teaching terminal can use the midpoint of the wrist crease as the first reference point R1, the midpoint of the elbow crease as the second reference point R2, and the direction of the line connecting R1 and R2 as the local reference axis, and determine the acupoint anchor point coordinates according to the template parameters; for target acupoints in the back of the hand region, the teaching terminal can use the base of the first metacarpal bone and the base of the second metacarpal bone as a set of reference markers, and calculate the acupoint anchor point coordinates according to the corresponding acupoint template.

[0051] In one optional implementation, when a reference marker in a local reference area cannot be reliably extracted in the current frame, the teaching terminal retrieves the position of the corresponding reference marker from the previous valid frame, or estimates it using the relative proportions between adjacent reference markers. The teaching terminal only updates the acupoint anchor coordinates of the current target acupoint when the set of reference markers meets a second preset completeness condition; otherwise, it maintains the acupoint anchor coordinates from the previous valid frame. For example, the second preset completeness condition may be that at least 70% or 80% of the points in the set of reference markers are reliably detected.

[0052] In the above way, the coordinates of the acupoint anchor point and the acupoint projection area corresponding to the current target acupoint are no longer abstract inputs, but are calculated from the set of reference markers and acupoint template parameters in the current image, thus providing a clear spatial reference for position verification.

[0053] Optionally, the acupoint overlay annotation information may include acupoint identifiers, the teaching body side, and the student practice side. The teaching body side can be used to indicate the acupoint side highlighted in the teacher's current screen, and the student practice side can be used to indicate the body side that the student should currently be operating on.

[0054] In some implementations, when the current target acupoint is a bilaterally symmetrical acupoint with the same name, the teaching terminal can also output lateral identification prompts. These lateral identification prompts include at least one of a lateral identification distinction indicator and observation prompts. The lateral identification distinction indicator can include at least one of an arrow indicator, a text indicator, and a split-screen indicator. For example, the teaching terminal can display a prompt indicating the teacher's left side in the teaching image stream, and a prompt indicating whether the student should operate on the right side or the student should operate on the left side in the student image stream, or it can use both arrows and text to indicate the target lateral identification.

[0055] S104: The teaching terminal responds to the student's practice operation, performs side and position verification on the student's practice operation based on the human side and position relationship corresponding to the student's practice operation, and outputs the corresponding annotation results.

[0056] In this embodiment, the student's practice operation can be any of the following: clicking, touching, pointing, hovering, or needle-prick simulation operations performed by the student on their corresponding body surface area. The teaching terminal can obtain the operation area corresponding to the practice operation from the student's image stream and determine the human side and operation coordinates corresponding to the operation area.

[0057] In one optional implementation, the acquisition method for student practice operations includes at least one of the following: screen touch acquisition, fingertip pointing acquisition, and instrument tip acquisition. For screen touch acquisition, the teaching terminal directly acquires the touch point coordinates as the operation coordinates; for fingertip pointing acquisition, the teaching terminal extracts the fingertip key points from the student's hand image and uses the projection points of these key points onto the student's image stream as the operation coordinates; for instrument tip acquisition, the teaching terminal extracts the position of the instrument tip in the student's image stream and uses it as the operation coordinates.

[0058] The teaching terminal uniformly records the operation coordinates obtained from the different acquisition methods as operation point P. Based on the position of operation point P relative to the student's lateral semantic coordinate system, the teaching terminal determines the lateral aspect of the body to which operation point P belongs; simultaneously, based on the spatial relationship between operation point P and the coordinates A of the acupoint anchor point corresponding to the current target acupoint or the acupoint projection area, it determines the position verification result.

[0059] In one optional implementation, the teaching terminal can determine the actual body side to which the student's practice operation belongs based on the position of the operation area in the student's image stream and the student's body side semantic information determined in step S102. For example, when the operation area is located to the left of the student's torso midline, the corresponding body side can be determined to be left; when the operation area is located to the right of the torso midline, the corresponding body side can be determined to be right.

[0060] In one optional implementation, when using acupoint anchor point coordinates A for position verification, the teaching terminal calculates the Euclidean distance D between the operation point P and the acupoint anchor point coordinates A; when D is less than or equal to a preset position threshold T1, the position verification is determined to meet the preset condition. The preset position threshold T1 can be adaptively determined based on the local reference length L. For example, T1 = λ × L, where λ can be from 0.03 to 0.10.

[0061] In another optional implementation, when using the acupoint projection area for position verification, if the operation point P is located inside the acupoint projection area, the position verification is determined to meet the preset conditions; if the operation point P is located outside the acupoint projection area, the position verification is determined to not meet the preset conditions.

[0062] The teaching terminal compares the human side corresponding to operation point P with the side the student is practicing to obtain a side alignment verification result. When the two match, the side alignment verification is deemed to meet the preset conditions; when they do not match, the side alignment verification is deemed to not meet the preset conditions. Thus, the teaching terminal obtains both the position verification result and the side alignment verification result.

[0063] In one optional implementation, when the position verification meets preset conditions but the lateral verification does not, the teaching terminal outputs a lateral correction annotation; when the position verification does not meet preset conditions but the lateral verification meets preset conditions, the teaching terminal outputs a position repositioning annotation; when both the position verification and lateral verification meet preset conditions, the teaching terminal outputs a practice confirmation annotation; and when neither the position verification nor the lateral verification meets preset conditions, the teaching terminal outputs a re-observation and teaching annotation. In this way, the teaching terminal can distinguish between multiple different error types, avoiding mixing different error scenarios into the same feedback result.

[0064] In some embodiments, if the position verification meets preset conditions but the lateral deviation verification does not, the teaching terminal outputs a lateral deviation correction label and maintains the superimposed label of the current target acupoint on the body surface area corresponding to the student's practice lateral deviation. For example, the teaching terminal can indicate that the position is close to correct, but the student should switch to the other side for practice. In this way, the teaching terminal can identify cases where the position is correct but the lateral deviation is incorrect, avoiding judging the practice operation as correct solely based on spatial proximity.

[0065] In some embodiments, if both position verification and lateral verification meet preset conditions, the teaching terminal outputs a practice confirmation mark. For example, the teaching terminal may display a prompt message indicating that the target acupoint is correctly located.

[0066] S105: During the continuous image update process, the teaching terminal updates the acupoint overlay annotation information according to changes in human body key points or teaching mode.

[0067] In this embodiment, the instructor or student may change posture during the lesson, or the teacher may switch teaching modes. To maintain the accuracy of lateral prompts and acupoint markings, the teaching terminal can detect whether there are significant changes in key human body points and whether the current teaching mode information changes during the continuous updating of the teaching image stream and student image stream.

[0068] In one optional implementation, the teaching terminal continuously detects changes in key human body points and changes in the teaching mode. When the change in at least one of the shoulder midpoint, hip midpoint, trunk midline direction, or local reference axis direction relative to the previous valid frame exceeds a third preset update threshold, the teaching terminal re-determines the lateral semantic information of the human body, recalculates the coordinates of acupoint anchor points and acupoint projection areas, and updates the acupoint overlay annotation information. For example, the third preset update threshold may be: the displacement of the key human body point exceeds 8% to 12% of the local reference length, or the change in the trunk midline direction exceeds 8 degrees to 12 degrees.

[0069] In one optional implementation, if a change in the current teaching mode information is detected, the teaching terminal first calls back the lateral mapping rules, and then refreshes the acupoint overlay annotation information in the teaching image stream and the student image stream according to the updated lateral of the teaching human body and the student practice lateral.

[0070] In some embodiments, if the detected change in human body key points does not exceed the third preset update threshold and the current teaching mode information does not change, the teaching terminal maintains the lateral semantic information, acupoint anchor coordinates, acupoint projection area and acupoint superimposed annotation information corresponding to the previous valid frame, so as to reduce annotation flickering caused by slight shaking in consecutive frames.

[0071] In one exemplary implementation, when a teacher explains the left Neiguan acupoint using a face-to-face mirror teaching mode, the teaching terminal first identifies the key points of the teacher's left shoulder, right shoulder, left hip, and right hip from the teaching image stream, constructing a semantic coordinate system for the teacher's body's lateral aspect. It then identifies the corresponding key points of the student's body from the student's image stream, constructing a semantic coordinate system for the student's body's lateral aspect. Subsequently, based on the lateral mapping rules corresponding to the face-to-face mirror teaching mode, the teaching terminal determines the corresponding area on the left side of the teacher's body as the lateral aspect of the teaching body and the corresponding area on the left side of the student's body as the lateral aspect of the student's body. It then outputs the acupoint overlay annotation information corresponding to the left Neiguan acupoint in both the teaching image stream and the student's image stream.

[0072] In this example, the teaching terminal uses the midpoint of the student's wrist crease as the first reference point R1 and the midpoint of the elbow crease as the second reference point R2. It calculates the coordinates A of the acupoint anchor point of the left Neiguan acupoint based on the template parameters α and β corresponding to the left Neiguan acupoint, and determines the acupoint projection area using the local reference length L and the scaling factor γ.

[0073] When a student points to an operation point P during practice, the teaching terminal calculates the Euclidean distance D between operation point P and the coordinates A of the acupoint anchor point, and determines the side of the body to which operation point P belongs. If D is less than or equal to a preset position threshold T1, but the side of the body to which operation point P belongs is inconsistent with the side of the student's practice, the teaching terminal determines that the position verification meets the preset conditions, but the side verification does not meet the preset conditions, and outputs a side correction label.

[0074] Through the above implementation methods, the teaching terminal can distinguish and correspond the semantics of acupoint display in the teacher's demonstration screen, the semantics of student observation, and the semantics of student actual practice in the context of TCM teaching. It can also specifically verify and correct errors in acupoint identification during student practice, thereby improving the accuracy and stability of acupoint teaching in face-to-face mirror teaching scenarios.

[0075] Figure 7 This is a schematic diagram of the structure of a teaching terminal provided in an embodiment of this application. The teaching terminal may include an image acquisition module, a lateral semantic determination module, a mapping relationship generation module, an acupoint annotation module, a practice verification module, and a display output module. The image acquisition module is used to acquire teaching image streams and student image streams; the lateral semantic determination module is used to determine human lateral semantic information based on human key points; the mapping relationship generation module is used to establish a mapping relationship between the target lateral, the teaching human lateral, and the student's practice lateral based on teaching mode information; the acupoint annotation module is used to generate acupoint superimposed annotation information; the practice verification module is used to perform position verification and lateral verification; and the display output module is used to output the corresponding annotation results.

[0076] In some embodiments, the teaching terminal may further include a processor, a memory, and a communication interface. The processor is used to execute program instructions stored in the memory to implement the acupoint location image overlay annotation method in the above embodiments; the communication interface is used to interact with external image acquisition devices, display devices, or teaching management devices.

[0077] Those skilled in the art will understand that the types of human body key points, reference marker sets, template parameters, preset thresholds, acquisition methods, and annotation formats mentioned in the above embodiments are merely examples, and this application is not limited to the specific values ​​and implementation forms described above. Without departing from the concept of this application, those skilled in the art can adjust and replace them according to specific teaching parts, hardware conditions, and interaction methods.

[0078] Those skilled in the art will also understand that the order of steps, module division, and parameter expression in the embodiments of this application are not intended to limit this application. Any splitting, combination, substitution, or order adjustment of any step without departing from the technical concept of this application shall be deemed to fall within the protection scope of this application.

[0079] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for superimposing and annotating acupoint location images for TCM teaching scenarios, characterized in that, The method, applied to a teaching terminal, includes: acquiring a teaching image stream, a student image stream, current teaching mode information, and current target acupoint information; determining the lateral semantic information of the instructor's body and the student's body based on the teaching image stream and the student image stream, respectively, and establishing a mapping relationship between the target lateral of the current target acupoint and the teaching lateral of the instructor's body and the student's practice lateral of the student's body according to the current teaching mode information; generating acupoint overlay annotation information in the teaching image stream and the student image stream according to the current target acupoint information and the mapping relationship; and, in response to the student's practice operation, performing lateral and position verification on the student's practice operation based on the lateral and positional relationship corresponding to the student's practice operation, and outputting the corresponding annotation results.

2. The method according to claim 1, characterized in that, The step of establishing a mapping relationship between the target side of the current target acupoint and the teaching side of the instructor's body and the student's practice side based on the current teaching mode information includes: determining the target mapping rule from the preset side mapping rules based on the current teaching mode information. The preset side mapping rules include the side mapping rules corresponding to the same-direction teaching mode and the side mapping rules corresponding to the face-to-face mirror teaching mode.

3. The method according to claim 1, characterized in that, The lateral semantic information is a human lateral semantic coordinate system constructed based on human key points; the human key points include at least the left shoulder key point, the right shoulder key point, the left hip key point, and the right hip key point.

4. The method according to claim 3, characterized in that, The current target acupoint information includes acupoint identifier and target side; the step of generating acupoint overlay annotation information in the teaching image stream and the student image stream according to the current target acupoint information and the mapping relationship includes: determining the side of the teaching human body and the side of the student practice according to the target side and the target mapping rule, and overlaying the acupoint identifier in the teaching image stream and the student image stream respectively.

5. The method according to claim 4, characterized in that, The acupoint overlay labeling information includes the acupoint identifier, the side of the teaching body, and the side of the student's practice.

6. The method according to claim 4, characterized in that, When the current target acupoint is a symmetrical acupoint with the same name on both sides, the acupoint superimposed label information also includes a side differentiation prompt; the side differentiation prompt includes at least one of arrow symbols, text symbols, and split-screen symbols.

7. The method according to any one of claims 6, characterized in that, The response to the student's practice operation includes performing lateral and position verification on the student's practice operation based on the human lateral and positional relationships corresponding to the student's practice operation. This includes: obtaining the operation area corresponding to the student's practice operation; and determining the human lateral and operation coordinates corresponding to the student's practice operation based on the position of the operation area in the student's image stream and the lateral semantic information.

8. The method according to claim 7, characterized in that, The position verification includes: comparing the distance between the operation coordinates and the coordinates of the acupoint anchor point corresponding to the current target acupoint, or determining whether the operation coordinates fall into the acupoint projection area corresponding to the current target acupoint; when the distance is less than or equal to a preset position threshold, or the operation coordinates fall into the acupoint projection area, it is determined that the position verification meets the preset conditions.

9. The method according to claim 1, characterized in that, The corresponding annotation results are as follows: when the position verification meets the preset conditions and the lateral verification does not meet the preset conditions, output the lateral correction annotation and maintain the superimposed annotation of the current target acupoint on the body surface area corresponding to the student's practice lateral side.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: during the continuous updating of the teaching image stream and the student image stream, if changes in human body key points are detected to exceed a preset update threshold, or if changes in the current teaching mode information are detected, then the lateral semantic information is redefined, the mapping relationship is re-established, and the acupoint overlay annotation information is updated.

Images