Mouth opening training method and system

Abstract

The embodiment of the invention relates to a mouth opening training method and system. The mouth opening training method comprises the steps that three-dimensional structured light data of a user in the mouth opening training process are obtained in real time based on a structured light camera; acquiring a real-time position of a lower jaw point of the user in the three-dimensional structured light data in a personal reference coordinate system, wherein the personal reference coordinate system takes the center of a connecting line of two inner canthus of the user in the three-dimensional structured light data as an original point; based on the real-time position of the lower jaw point of the user in the personal reference coordinate system, real-time chin motion parameters of the lower jaw point are determined, and the real-time chin motion parameters are used for indicating real-time position deviation of the user in the personal reference coordinate system; based on the real-time chin motion parameters, mouth opening training prompt information is generated, and the mouth opening training prompt information is used for guiding the user to adjust the mouth opening state. According to the embodiment of the invention, the user can autonomously perform mouth opening training without manual cooperation of medical care, and the trainer does not need to be put into the oral cavity of the user, so that discomfort of the user is avoided.

Description

Technical Field

[0001] This disclosure relates to the field of medical technology, and in particular to a method and system for mouth opening training. Background Technology

[0002] Radiation therapy (RT) is a common treatment for head and neck cancers such as nasopharyngeal carcinoma and laryngeal carcinoma. This treatment uses high-energy rays to kill cancer cells, thus achieving a therapeutic effect. However, this treatment can cause some damage to the surrounding normal tissues of the head and neck. For example, the lateral pterygoid muscle or suprahyoid muscle group may become fibrotic and lose elasticity, leading to limited mouth opening.

[0003] In existing technologies, for the aforementioned damage caused by head and neck radiotherapy, a mouth opening trainer is typically placed in the patient's mouth under the guidance of medical staff. The trainer then manually guides the patient to practice opening their mouth to help restore their original biting function. However, this method requires manual operation by medical staff; patients cannot perform mouth opening exercises independently and require their assistance. Furthermore, the mouth opening trainer may touch other parts of the patient's mouth during training, such as the base of the tongue or the throat, causing discomfort.

[0004] Therefore, it is necessary to provide a new technical solution to improve one or more of the problems existing in the above solutions.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a mouth-opening training method and system that allows users to perform mouth-opening training independently without the need for manual assistance from medical staff, and avoids the need to place the trainer into the user's mouth, thus preventing user discomfort.

[0007] According to a first aspect of the present disclosure, an mouth-opening training method is provided, comprising: The system uses a structured light camera to acquire real-time 3D structured light data of the user during the mouth-opening training process. The real-time position of the user's chin point in the personal reference coordinate system is obtained from the three-dimensional structured light data, with the center of the line connecting the two inner corners of the user's eyes in the three-dimensional structured light data as the origin. Based on the real-time position of the user's mandibular point in the personal reference coordinate system, the real-time chin movement parameters of the mandibular point are determined, and the real-time chin movement parameters are used to indicate the real-time positional offset of the user in the personal reference coordinate system. Based on the real-time chin movement parameters, mouth opening training prompts are generated to guide the user in adjusting their mouth opening state.

[0008] In an exemplary embodiment of this disclosure, obtaining the real-time position of the user's chin point in a personal reference coordinate system from the three-dimensional structured light data includes: Construct a personal baseline coordinate system; The user's chin point was identified from the three-dimensional structured light data; Obtain the real-time position of the mandibular point in the personal reference coordinate system.

[0009] In an exemplary embodiment of this disclosure, determining the real-time chin movement parameters of the mandible point based on its real-time position in a personal reference coordinate system includes: Obtain the reference position of the user's chin point in the personal reference coordinate system when the user's mouth is closed; Based on the reference position and real-time position of the user's mandibular point in the personal reference coordinate system, the real-time chin movement parameters of the mandibular point are determined.

[0010] In an exemplary embodiment of this disclosure, the real-time chin motion parameters include at least one of the following: Real-time chin opening, real-time chin expansion and contraction, and real-time chin lateral offset.

[0011] In an exemplary embodiment of this disclosure, obtaining the reference position of the user's chin point in the personal reference coordinate system when the user is not opening their mouth includes: The reference positions of the user's chin point in the first, second, and third directions of the personal reference coordinate system are obtained when the user's mouth is closed. The first direction is the user's height direction, the second direction is the user's front-back direction, and the third direction is the left-right hand direction. Accordingly, determining the real-time chin movement parameters of the mandible point based on the reference position and the real-time position of the user's mandible point in the personal reference coordinate system includes: The real-time chin opening degree is determined based on the displacement of the user's chin point in the first direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. The real-time chin extension is determined based on the displacement of the user's chin point in the second direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. The real-time chin offset is determined based on the displacement of the user's chin point in the third direction of the personal reference coordinate system relative to the reference position in the closed-mouth state.

[0012] In an exemplary embodiment of this disclosure, generating mouth-opening training prompt information based on the real-time chin movement parameters includes: Obtain reference chin movement parameters for the mandibular point, wherein the reference chin movement parameters are the chin movement parameters of the user in the maximum mouth opening state; Based on the ratio of the real-time chin movement parameters to the reference chin movement parameters, the user's mouth opening training prompt information is generated.

[0013] In an exemplary embodiment of this disclosure, obtaining the reference chin movement parameters of the mandibular point includes: Obtain the reference position of the user's chin point in the personal reference coordinate system when the user is in the maximum mouth opening state; Based on the reference position and the reference position of the user's mandibular point in the personal reference coordinate system, the reference chin movement parameters of the mandibular point are determined.

[0014] In an exemplary embodiment of this disclosure, the reference chin motion parameter includes at least one of the following: Reference chin opening, reference chin extension and contraction, and reference chin lateral offset; Accordingly, generating the user's mouth-opening training prompt information based on the ratio of the real-time chin movement parameters to the reference chin movement parameters includes: When the ratio of the real-time chin opening to the reference chin opening is less than the preset chin opening, a mouth-opening training prompt message is generated to instruct the user to increase the size of the mouth opening in the first direction. When the ratio of the real-time chin extension to the reference chin extension is not within the preset chin extension range, a mouth-opening training prompt message is generated to instruct the user to extend or retract the chin in the second direction. When the ratio of the real-time chin offset to the reference chin offset is not within the preset chin offset range, a prompt message is generated instructing the user to practice opening their mouth by moving their chin in the third direction.

[0015] In an exemplary embodiment of this disclosure, both the reference position and the reference location are obtained before the user undergoes radiotherapy.

[0016] According to a second aspect of the present disclosure, an opening-mouth training system is provided, the system comprising: A structured light camera, positioned facing the user's facial area, is used to acquire three-dimensional structured light data of the user in real time during mouth-opening training. A processing device, electrically connected to the structured light camera, is used to perform the method as described in any of the preceding methods; An information prompting device, electrically connected to the processing device, is used to output the mouth-opening training prompt information.

[0017] The technical solution provided in this disclosure may include the following beneficial effects: In the embodiments of this disclosure, three-dimensional structured light data of the user during mouth-opening training is acquired in real time, and then the real-time position of the user's mandibular point in the personal reference coordinate system is obtained from the three-dimensional structured light data. Based on the real-time position of the user's mandibular point in the personal reference coordinate system, the real-time chin movement parameters of the mandibular point can be determined. These real-time chin movement parameters indicate the user's real-time positional offset in the personal reference coordinate system. That is, by performing the above processing on the three-dimensional structured light data, the positional offset of the user's mandibular point during mouth-opening training can be determined. Furthermore, based on the real-time chin movement parameters, mouth-opening training prompts are generated to guide the user in adjusting their mouth-opening state. The entire process does not require manual assistance from medical staff, allowing the user to perform mouth-opening training independently, and it does not require placing the trainer in the user's mouth, thus avoiding discomfort for the user.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 This is a schematic diagram of the structure of an mouth-opening training system provided in an embodiment of this application; Figure 2 This is a schematic diagram of another mouth-opening training system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a head limiting component provided in an embodiment of this application; Figure 4 A structural block diagram of another mouth-opening training system provided in this application embodiment; Figure 5 A flowchart illustrating an oral opening training method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a processing device provided in an embodiment of this application. Detailed Implementation

[0021] The disclosed embodiments will now be described more fully with reference to the accompanying drawings. However, the disclosed embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the disclosed embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more implementations.

[0022] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processing device devices and / or microprocessor device devices.

[0023] This disclosure provides an mouth-opening training system, which serves as the implementation environment for a mouth-opening training method illustrated in this disclosure. (Refer to...) Figure 1 As shown, the mouth-opening training system includes a structured light camera 100, a processing device 200, and an information prompting device 300.

[0024] Among them, the structured light camera 100 is a visual sensor that acquires the three-dimensional shape and depth information (three-dimensional coordinates) of an object in real time by actively projecting a specially coded optical pattern and analyzing the deformation of the pattern on the surface of the object. It has the advantages of non-contact and high-precision three-dimensional measurement.

[0025] In this embodiment of the disclosure, the structured light camera 100 is positioned facing the facial area of ​​the user (e.g., a patient who needs to perform mouth-opening training), ensuring that the user's facial area is completely within the field of view of the structured light camera 100, in order to acquire three-dimensional structured light data of the user during the mouth-opening training process in real time. This three-dimensional structured light data indicates the patient's mouth-opening state.

[0026] For example, the structured light camera 100 can be positioned 0.3 to 1.5 meters away from the user's head, at a height level with the user's mouth and nose, and opposite to the user's facial area.

[0027] The location of the structured light camera 100 is not limited in this embodiment of the disclosure. For example, it can be installed on the ceiling of the room where the mouth opening training system is located, on the information prompting device 300, or placed on a table using a support frame.

[0028] The processing device 200 is electrically connected to the structured light camera 100 and is used to process the three-dimensional structured light data of the user during the mouth opening training process acquired in real time by the structured light camera 100, and generate mouth opening training prompt information. For details, please refer to the mouth opening training method provided in the following embodiments.

[0029] In this embodiment, the processing device 200 may be at least one of the following: a smartphone, a smartwatch, a desktop computer, a laptop, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop computer. In other embodiments, the processing device 200 may be a standalone physical server, a server cluster or distributed file system composed of multiple physical servers, or at least one of the following cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data or artificial intelligence platforms. This embodiment does not limit the scope of the application.

[0030] The information prompting device 300 is electrically connected to the processing device 200 and is used to output mouth-opening training prompt information to guide the user to adjust the mouth-opening state.

[0031] In this embodiment of the disclosure, the information prompting device 300 can be at least one of a display and a voice player. The display can be used to display mouth-opening training prompts and can be positioned relative to the user. The voice player can be used to play the mouth-opening training prompts aloud. Specifically, it can be a loudspeaker, which can be placed in the room where the mouth-opening training system is located, or it can be headphones that the user can wear. The information prompting device 300 can also be a virtual reality device, such as wearable virtual reality glasses. Of course, the information prompting device 300 can also be other devices, and this embodiment of the disclosure does not specifically limit them.

[0032] In this embodiment of the disclosure, the electrical connection between the structured light camera 100, the processing device 200, and the information prompting device 300 can be either a wired connection or a wireless connection, and this embodiment of the disclosure does not limit this.

[0033] The aforementioned mouth-opening training system uses a structured light camera to acquire real-time 3D structured light data of the user during mouth-opening training. A processing device executes the mouth-opening training method, generating mouth-opening training prompts using the 3D structured light data. These prompts are then displayed to the user during training to guide them in adjusting their mouth-opening posture, enabling them to complete active mouth-opening training under the guidance of these prompts. This mouth-opening training system requires no manual assistance from medical staff, allowing users to perform mouth-opening training independently, and eliminates the need to insert the trainer into the user's mouth, thus avoiding discomfort.

[0034] In this embodiment of the disclosure, such as Figure 2 As shown, the system may also include a head-limiting component 400 for restricting the user's head to a preset position. The head-limiting component 400 restricts the user's head to the preset position, which can be a position within a range of 0.3 meters to 1.5 meters away from the structured light camera 100, and can be set and adjusted according to parameters such as the type and field of view of the structured light camera 100.

[0035] In one disclosed embodiment, reference is made to Figure 3 As shown, the head limiting component 400 includes a bracket 410 and a fixing strip 420 disposed on the bracket 410; the fixing strip 420 is arc-shaped, so that the fixing strip 420 is adapted to the shape of the user's head. When the user performs mouth opening training, the fixing strip 420 fits against the user's forehead to limit the user's head position.

[0036] In one disclosed embodiment, reference is made to Figure 3 As shown, the bracket 410 includes a support base plate 411 and two support columns 412 spaced apart on the support base plate 411. The fixing strip 420 is connected to the two support columns 412 at both ends along its length. The support base plate facilitates flexible placement of the head limiting component and ensures the stability of the bracket for the head limiting component.

[0037] In one disclosed embodiment, reference is made to Figure 4 As shown in the diagram, the mouth opening training system of this embodiment further includes a user feedback device 500; the user feedback device 500 is electrically connected to the processing device 200 and is used to acquire user feedback information and send the feedback information to the processing device 200. By setting up the user feedback device 500, the user can provide real-time feedback information to the processing device 200 during the mouth opening training process. For example, the feedback information may be: whether the user can meet the training requirements according to the mouth opening training prompts, and whether the user has started mouth opening training.

[0038] In one disclosed embodiment, the user feedback device 500 includes at least one of: a user touch button; a user touch touch screen; and a camera disposed at a position opposite to the head restraint component.

[0039] When the user feedback device 500 includes a user touch button, the user can provide feedback to the processing device 200 by pressing the button. When the user feedback device 500 includes a user touchscreen, the user can provide feedback to the processing device 200 by touching the touchscreen. When the user feedback device 500 includes a camera, the user can provide feedback to the processing device 200 by making different types of hand or facial gestures.

[0040] For example, taking the user touch button as an example, if the user clicks the user touch button, the processing device 200 will receive feedback that the user has started mouth opening training; if the user presses the user touch button twice in succession, the processing device 200 will receive feedback that the user has met the training requirements according to the mouth opening training prompts; if the user presses the user touch button more than twice in succession, the processing device 200 will receive feedback that the user has not met the training requirements according to the mouth opening training prompts.

[0041] It should be noted that the embodiments disclosed herein do not impose any restrictions on the specific triggering feedback mechanisms for user touch buttons, user touch screens, and cameras.

[0042] The following description uses a processing device as an example to illustrate the mouth-opening training method provided in this embodiment. Figure 5 This is a flowchart of a mouth-opening training method provided in an embodiment of the present disclosure, wherein the executing entity is... Figure 1 The processing device in the mouth-opening training system shown, such as Figure 5 As shown, the method includes the following steps S201-S204.

[0043] Step S201: Real-time acquisition of three-dimensional structured light data of the user during the mouth-opening training process using a structured light camera.

[0044] It should be explained that the mouth opening training process includes at least one mouth opening training session. One complete mouth opening and closing action by the user constitutes one mouth opening training session. In order to accurately judge the training status of each mouth opening training session, the structured light camera acquires the user's three-dimensional structured light data in real time and continuously during the mouth opening training process. This three-dimensional structured light data can be real-time three-dimensional point cloud data of the user's facial area surface, which can indicate the patient's mouth opening status.

[0045] Step S202: Obtain the real-time position of the user's chin point in the personal reference coordinate system from the three-dimensional structured light data.

[0046] Among them, the personal reference coordinate system is a three-dimensional coordinate system constructed with the center of the line connecting the two inner corners of the user's eyes in the three-dimensional structured light data as the origin, which fundamentally filters out the deviation caused by slight shaking of the user's head.

[0047] The three-dimensional coordinate system includes a first coordinate axis, a second coordinate axis, and a third coordinate axis that are perpendicular to each other. The first coordinate axis is parallel to the third direction, which is the direction of the user's left and right hands; the second coordinate axis is parallel to the first direction, which is the direction of the user's height; and the third coordinate axis is parallel to the second direction, which is the direction of the user's front and back.

[0048] Among them, the chin point is the lowest point of the user's chin. The user's chin point or the center of the line connecting the two inner corners of the user's eyes in the three-dimensional structured light data (or three-dimensional point cloud data) can be identified from the three-dimensional structured light data based on geometric features, contour extraction, curvature surface analysis, statistical shape model (SSM) or deep learning, etc., without specific limitations.

[0049] Step S203: Determine the real-time chin movement parameters of the mandibular point based on the real-time position of the user's mandibular point in the personal reference coordinate system.

[0050] Among them, the real-time chin motion parameters represent the current chin motion parameters, which are used to indicate the user's real-time positional offset in the personal reference coordinate system.

[0051] The chin movement parameters may include at least one of chin opening, chin extension and retraction, and chin lateral offset. Correspondingly, the real-time chin movement parameters may include at least one of real-time chin opening, real-time chin extension and retraction, and real-time chin lateral offset.

[0052] Here, chin opening degree is used to indicate the positional deviation of the user's mouth opening in the first direction, reflecting the amplitude of the mandibular point's movement in the user's height direction, and can be used to quantitatively assess the user's mouth opening size; chin extension degree is used to indicate the positional deviation of the user's chin extension in the second direction, reflecting the amplitude of the mandibular point's movement in the user's front-back direction, and can be used to quantitatively assess the front-back extension of the mandibular point when opening the mouth; chin lateral offset is used to indicate the positional deviation of the user's chin in the third direction, reflecting the amplitude of the mandibular point's movement in the user's left-right direction, and can be used to quantitatively assess the left-right offset of the mandibular point when opening the mouth.

[0053] Step S204: Generate mouth opening training prompts based on real-time chin movement parameters.

[0054] Among them, the mouth-opening training prompts are used to guide users to adjust their mouth-opening state.

[0055] It should be noted that the prompts for mouth opening training can be in any of the following forms: graphic prompts, text prompts, or sound prompts.

[0056] In the embodiments of this disclosure, by acquiring the user's three-dimensional structured light data during the mouth-opening training process in real time, the real-time position of the user's chin point in the personal reference coordinate system is obtained from the three-dimensional structured light data. Establishing the personal reference coordinate system with the center of the line connecting the user's two inner corners of the eyes as the origin can eliminate the influence of slight head shaking. Based on the real-time position of the user's chin point in the personal reference coordinate system, the real-time chin movement parameters of the chin point are determined. These real-time chin movement parameters indicate the user's real-time positional offset in the personal reference coordinate system. That is, by processing the three-dimensional structured light data as described above, the positional offset of the user's chin point during the mouth-opening training process can be determined. Furthermore, based on the real-time chin movement parameters, mouth-opening training prompts are generated to guide the user in adjusting their mouth-opening state. The entire process does not require manual assistance from medical staff, allowing the user to perform mouth-opening training independently, and it does not require placing the trainer in the user's mouth, thus avoiding user discomfort.

[0057] The steps of the methods described in the embodiments of this disclosure will now be explained in more detail.

[0058] In one embodiment, step S202, acquiring the real-time position of the user's chin point in the personal reference coordinate system from the three-dimensional structured light data, includes the following steps: Step S2021: Construct a personal baseline coordinate system.

[0059] The personal reference coordinate system has been explained in step S202 and will not be repeated here.

[0060] Step S2022: Identify the user's chin point from the 3D structured light data.

[0061] Identifying a user's chin point from 3D structured light data (or 3D point cloud data) can be based on geometric features, contour extraction, curvature surface analysis, statistical shape model (SSM), or deep learning, etc., without specific limitations here.

[0062] Taking the identification of a user's chin point from three-dimensional structured light data based on deep learning as an example, it can include: inputting the acquired three-dimensional structured light data into a deep neural network model, which can be a convolutional neural network based on multi-channel images, and using the trained deep neural network model to extract the chin point from numerous point cloud data of the three-dimensional structured light data.

[0063] Step S2023: Obtain the real-time position of the mandibular point in the personal reference coordinate system.

[0064] After identifying the user's mandibular point, the real-time position of the mandibular point in the personal reference coordinate system can be obtained. The real-time position can be in three-dimensional coordinates.

[0065] In one embodiment, step S203, based on the real-time position of the user's mandibular point in a personal reference coordinate system, determines the real-time chin movement parameters of the mandibular point, which may include the following steps: Step S2031: Obtain the reference position of the user's chin point in the personal reference coordinate system when the user's mouth is closed.

[0066] It should be explained that the reference position of the user's chin point in the personal reference coordinate system when the user's mouth is closed was obtained before the user underwent radiation therapy. This reference position, as the benchmark for measuring chin point movement, can be expressed as three-dimensional coordinates in the personal reference coordinate system.

[0067] For example, when the user is in a closed-mouth state, three-dimensional structured light data of the user in a closed-mouth state is acquired using a structured light camera. The user's mandibular point is identified from this three-dimensional structured light data, and the coordinate position of the mandibular point is obtained. The coordinate position of the mandibular point in the closed-mouth state is transformed into the personal reference coordinate system, thereby obtaining the reference position of the user's mandibular point in the personal reference coordinate system.

[0068] Step S2032: Based on the reference position and real-time position of the user's mandibular point in the personal reference coordinate system, determine the real-time chin movement parameters of the mandibular point.

[0069] Among them, the real-time chin motion parameters are used to indicate the amount or degree of offset of the user's real-time position relative to the reference position in the personal reference coordinate system.

[0070] In one embodiment, the real-time chin movement parameters include at least one of real-time chin opening degree, real-time chin extension degree, and real-time chin lateral offset. Specifically, the real-time chin opening degree corresponds to the real-time positional offset of the mandibular point displacement in a first direction of the personal reference coordinate system during mouth opening training, directly reflecting the amplitude of the mandibular point movement in the user's height direction, and can be used to quantitatively assess the user's mouth opening size; the real-time chin extension degree corresponds to the real-time positional offset of the mandibular point displacement in a second direction of the personal reference coordinate system during mouth opening training, directly reflecting the amplitude of the mandibular point movement in the user's anterior-posterior direction, and can be used to quantitatively assess the anterior-posterior extension of the mandibular point during mouth opening; and the real-time chin lateral offset corresponds to the real-time positional offset of the mandibular point displacement in a third direction of the personal reference coordinate system during mouth opening training, directly reflecting the amplitude of the mandibular point movement in the user's left-right direction, and can be used to quantitatively assess the left-right offset of the mandibular point during mouth opening.

[0071] In one embodiment, when the real-time chin movement parameters include at least one of real-time chin opening degree, real-time chin extension degree, and real-time chin lateral offset, step S2031, obtaining the reference position of the user's mandible point in the personal reference coordinate system when the user is not opening their mouth, may include: Step S20311: Obtain the reference positions of the user's mandible in the first, second, and third directions of the personal reference coordinate system when the user's mouth is closed; accordingly, Step S2032, based on the user's chin point's reference position and real-time position in the personal reference coordinate system, determines the real-time chin movement parameters of the chin point, which may include the following steps: Step S20321: Determine the real-time chin opening based on the displacement of the user's mandibular point in the first direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the real-time position and the reference position are in the first direction.

[0072] Step S20322: Determine the real-time chin extension / retraction based on the displacement of the user's chin point in the second direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the real-time position and the reference position are in the second direction.

[0073] Step S20323: Determine the real-time chin offset based on the displacement of the user's mandibular point in the third direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the real-time position and the reference position are in the third direction.

[0074] It should be noted that the real-time positions of the user's mandible in the first direction of the personal reference coordinate system, the real-time positions of the user's mandible in the second direction of the personal reference coordinate system, and the real-time positions of the user's mandible in the third direction of the personal reference coordinate system can be obtained through steps S2021-S2023, and will not be elaborated here.

[0075] The above uses the reference position of the user's mandibular point in the personal reference coordinate system as the reference. The real-time position of the user's mandibular point in the first direction, second direction and third direction of the personal reference coordinate system is compared with the reference position of the user's mandibular point in the first direction, second direction and third direction of the personal reference coordinate system to obtain the real-time chin opening degree, real-time chin expansion degree and real-time chin lateral offset.

[0076] In one embodiment, step S204, which generates mouth-opening training prompts based on real-time chin movement parameters, may include the following steps: Step S2041: Obtain reference chin movement parameters for the mandibular point.

[0077] Among them, the reference chin movement parameters are the chin movement parameters of the user in the maximum mouth opening state.

[0078] It should be explained that the reference chin movement parameters for the mandibular point can be chin movement parameters obtained before the user undergoes radiotherapy. These parameters reflect the maximum mouth opening that the user can achieve before radiotherapy. Using the chin movement parameters at the maximum mouth opening that the user can achieve before radiotherapy as a reference can more accurately measure the user's condition after radiotherapy and generate more targeted mouth opening training prompts for the user, thereby improving the training effect.

[0079] Step S2042: Generate mouth-opening training prompts for the user based on the ratio of real-time chin motion parameters to reference chin motion parameters.

[0080] It should be explained that the real-time chin movement parameters are obtained by the user after undergoing radiotherapy and during mouth opening training, and they reflect the mouth opening state that the user can achieve during the radiotherapy recovery period.

[0081] The above method uses the maximum mouth opening that the user can achieve before radiotherapy as a reference. By comparing the ratio of real-time chin movement parameters to reference chin movement parameters, the method assesses the mouth opening that the user can achieve during the radiotherapy recovery period. This achieves individualized mouth opening training and more scientifically and accurately assesses the recovery of each user.

[0082] Here, the ratio of real-time chin motion parameters to reference chin motion parameters can be in percentage or decimal form.

[0083] Furthermore, step S2041, obtaining reference chin movement parameters for the mandibular point, may include the following steps: Step S20411: Obtain the reference position of the user's mandible point in the personal reference coordinate system when the user is in the maximum mouth opening state.

[0084] It should be noted that the reference position of the user's mandibular point in the personal reference coordinate system is a three-dimensional coordinate, that is, the coordinate values ​​of the user's mandibular point in the first, second, and third directions of the personal reference coordinate system when the user is in the maximum mouth opening state. The specific implementation method of step S20411 can be referred to the method of steps S2021-S2023, which will not be repeated here.

[0085] Step S20412: Based on the reference position and reference position of the user's mandibular point in the personal reference coordinate system, determine the reference chin motion parameters of the mandibular point.

[0086] It should be explained that the reference position of the user's mandibular point in the personal reference coordinate system was obtained before the user underwent radiation therapy and when the user's mouth was closed. The reference chin movement parameters of the mandibular point quantify the maximum mouth opening achieved by the user before radiation therapy.

[0087] The aforementioned method of constructing reference chin movement parameters for the mandibular point before radiotherapy provides an objective reference benchmark for the entire radiotherapy and rehabilitation cycle, thus providing a quantitative scientific basis for assessing the user's rehabilitation progress. Furthermore, the benchmark and reference positions are derived using the exact same coordinate system and calculation logic, ensuring that all data are within the same measurement system and guaranteeing the accuracy and reliability of the reference chin movement parameters for the mandibular point.

[0088] For example, in step S20411, the user needs to be guided to put their mouth in the maximum open state. The three-dimensional structured light data of the user in the maximum open state is acquired based on the structured light camera. The user's mandibular point is identified from the three-dimensional structured light data. The coordinate position of the mandibular point is the coordinate position of the user in the maximum open state. The coordinate position of the user in the maximum open state is converted to the personal reference coordinate system to obtain the reference position of the user's mandibular point in the personal reference coordinate system.

[0089] In one embodiment, the reference chin movement parameters include at least one of reference chin opening degree, reference chin extension degree, and reference chin lateral offset. Specifically, the reference chin opening degree corresponds to the positional displacement of the mandibular point in a first direction of the personal reference coordinate system when the user is in maximum mouth opening before radiotherapy, serving as a reference standard for quantitatively assessing the user's chin opening degree; the reference chin extension degree corresponds to the positional displacement of the mandibular point in a second direction of the personal reference coordinate system when the user is in maximum mouth opening before radiotherapy, serving as a reference standard for quantitatively assessing the user's chin extension degree; and the reference chin lateral offset corresponds to the positional displacement of the mandibular point in a third direction of the personal reference coordinate system when the user is in maximum mouth opening before radiotherapy, serving as a reference standard for quantitatively assessing the user's chin lateral offset.

[0090] In one embodiment, when the reference chin movement parameters include at least one of reference chin opening, reference chin extension, and reference chin lateral offset, step S20411, obtaining the reference position of the user's mandibular point in the personal reference coordinate system when the user is in the maximum mouth opening state, may include: Step S204111: Obtain the reference position of the user's chin point in the first, second, and third directions of the personal reference coordinate system when the user is in the maximum mouth opening state. The first direction is the user's height direction, the second direction is the user's front-back direction, and the third direction is the left-right hand direction.

[0091] Accordingly, step S20412, based on the reference position and reference position of the user's mandibular point in the personal reference coordinate system, determines the reference chin movement parameters of the mandibular point, which may include: Step S204121: Determine the reference chin opening based on the displacement of the user's mandibular point in the first direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the reference position and the reference position are in the first direction.

[0092] Step S204122: Determine the reference chin extension / retraction based on the displacement of the user's chin point in the second direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the reference position and the reference position are in the second direction.

[0093] Step S204123: Determine the reference chin offset based on the displacement of the user's mandibular point in the third-direction reference position of the personal reference coordinate system relative to the reference position in the closed-mouth state. It is understood that both the reference position and the reference position are in the third-direction direction.

[0094] The above uses the reference position of the user's mandibular point in the personal reference coordinate system as the reference. The reference positions of the user's mandibular point in the first direction, second direction and third direction of the personal reference coordinate system are compared with the reference positions of the user's mandibular point in the first direction, second direction and third direction of the personal reference coordinate system to obtain the reference chin opening, reference chin expansion and contraction and reference chin lateral offset.

[0095] In one embodiment, when the reference chin movement parameters include at least one of reference chin opening degree, reference chin extension degree, and reference chin lateral offset, step S2042, based on the ratio of real-time chin movement parameters to reference chin movement parameters, generates mouth opening training prompts for the user, which may include the following steps: Step S20421: When the ratio of the real-time chin opening degree to the reference chin opening degree is less than the preset chin opening degree, generate a mouth opening training prompt message instructing the user to increase the mouth opening size in the first direction.

[0096] In step S20421 above, when the ratio of the real-time chin opening to the reference chin opening is less than the preset chin opening, it indicates that the user's current mouth opening size in the first direction is not up to standard. Under the guidance of the mouth opening training prompt, the user needs to be guided to increase the mouth opening size in the first direction.

[0097] It should be explained that the preset chin opening can be a fixed preset value, or the preset chin opening for the current training stage can be dynamically determined based on the user's real-time chin opening during the previous training stage, and can be expressed in the form of percentage, fraction, etc.

[0098] Step S20422: When the ratio of the real-time chin extension / retraction to the reference chin extension / retraction is not within the preset chin extension / retraction range, generate a prompt message to instruct the user to extend and retract the chin in the second direction for mouth opening training.

[0099] In step S20422 above, when the ratio of the real-time chin extension and contraction to the reference chin extension and contraction is not within the preset chin extension and contraction range, it indicates that the user's current chin point movement amplitude in the second direction exceeds the preset standard range. Under the prompt of the mouth opening training prompt information, the user needs to be guided to extend and contract the chin in the second direction.

[0100] It should be explained that the preset chin extension range can be a fixed preset range, or the preset chin extension range for the current training stage can be dynamically determined based on the real-time chin extension during the user's previous training phase, and can be expressed in the form of a percentage, score, or other similar range.

[0101] Step S20423: When the ratio of the real-time chin offset to the reference chin offset is not within the preset chin offset range, generate a prompt message to instruct the user to move the chin in a third direction for mouth opening training.

[0102] In step S20423 above, when the ratio of the real-time chin offset to the reference chin offset is not within the preset chin offset range, it indicates that the user's current chin point movement in the third direction exceeds the preset standard range. Under the prompt of the mouth opening training prompt, the user needs to be guided to move and adjust the chin in the third direction.

[0103] It should be explained that the preset chin offset range can be a fixed preset range, or the preset chin offset range for the current training stage can be dynamically determined based on the real-time chin offset during the user's previous training phase, and can be expressed in the form of a percentage, fraction, or other similar range.

[0104] In one embodiment, the reference and baseline positions of the user's mandibular point in the personal reference coordinate system are obtained before the user undergoes radiotherapy.

[0105] The reference position of the user's mandibular point in the personal reference coordinate system is obtained when the user's mouth is closed; the reference position of the user's mandibular point in the personal reference coordinate system is obtained when the user's mouth is fully open.

[0106] The aforementioned establishment of the reference and baseline positions of the user's mandibular point in the personal reference coordinate system before radiotherapy provides an accurate and reliable standard for assessing the user's recovery after radiotherapy.

[0107] In one embodiment, real-time chin movement parameters may further include real-time chin movement trajectory and / or real-time chin movement speed. The real-time chin movement trajectory is used to evaluate the smoothness and symmetry of the mouth-opening training, while the real-time chin movement speed is used to evaluate the user's facial muscle control ability.

[0108] For example, the real-time chin movement trajectory is determined based on the real-time position of the user's mandibular point in a personal reference coordinate system. For instance, the real-time position of the user's mandibular point in the personal reference coordinate system is obtained over a continuous time period from closing to opening and back to closing, and the real-time chin movement trajectory is generated based on this position.

[0109] It should be noted that the real-time chin movement trajectory is visualized and projected onto a two-dimensional analysis plane to evaluate the smoothness of the movement during mouth-opening training. For example, it can be projected onto multiple two-dimensional analysis planes with time as the horizontal axis and the real-time position in the first direction, the real-time position in the second direction, and the real-time position in the third direction as the vertical axis to evaluate the smoothness of the real-time chin movement trajectory in the first, second, and third directions.

[0110] Additionally, the smoothness of the mouth-opening training motion can be assessed by calculating the curvature at each point on the real-time chin movement trajectory and evaluating the gentleness of the curvature changes. If there are abrupt changes in the curvature of the trajectory, the mouth-opening training motion is not smooth; if the curvature changes gently, the mouth-opening training motion is smooth.

[0111] It should be noted that the real-time chin movement trajectory corresponding to the continuous time period from closing to opening is compared with the real-time chin movement trajectory corresponding to the time period from opening to closing to evaluate the symmetry of mouth opening training.

[0112] For example, based on the real-time position of the user's chin point in the personal reference coordinate system during the continuous time period from closing to opening and then back to closing, the real-time chin movement displacement is determined. The real-time chin movement velocity is obtained by differentiating the real-time chin movement displacement, which includes the mouth opening velocity and the mouth closing velocity.

[0113] It should be noted that if the real-time chin movement speed is uniform, the user's facial muscle control ability is relatively good; if the real-time chin movement speed is non-uniform, the user's facial muscle control ability is relatively poor.

[0114] In one embodiment, when the real-time chin movement parameters include the real-time chin movement trajectory, step S204 generates mouth-opening training prompt information based on the real-time chin movement parameters, including: When the rate of curvature change of the real-time chin movement trajectory is not within the preset curvature change range, a mouth-opening training prompt message is generated to instruct the user to move their chin.

[0115] It should be explained that the mouth-opening training prompt can be a visually dynamic moving icon on the display screen, which can simultaneously display the real-time position of the user's chin and the dynamic moving icon.

[0116] By setting a preset range of curvature changes, the system can prompt users to move their chin along a dynamic movement icon when their mouth opening movements are uncoordinated, thus ensuring the smoothness of mouth opening training.

[0117] Optionally, the preset curvature variation range includes a preset opening curvature variation range and a preset closing curvature variation range. Correspondingly, when the rate of change of curvature of the real-time chin movement trajectory is outside the preset curvature variation range, a mouth-opening training prompt message is generated to instruct the user to move their chin, including: If the rate of change of curvature of the real-time chin movement trajectory during the opening phase is not within the preset range of opening curvature change, a prompt message is generated to instruct the user to move the chin for mouth opening training; if the rate of change of curvature of the real-time chin movement trajectory during the closing phase is not within the preset range of closing curvature change, a prompt message is generated to instruct the user to move the chin for mouth opening training.

[0118] By setting preset ranges for opening curvature and closing curvature, the system can prompt users on how to move their chin when their mouth movements are uncoordinated during the opening and closing phases. This ensures smoothness of mouth-opening training and facilitates symmetry control.

[0119] In one embodiment, when the real-time chin movement parameters include the real-time chin movement velocity, step S204 generates mouth-opening training prompt information based on the real-time chin movement parameters, including: When the real-time chin movement speed is outside the preset speed range, a mouth-opening training prompt message is generated to instruct the user to move their chin.

[0120] It should be explained that the mouth-opening training prompt can be an interval prompt sound, or a prompt word instructing the user to increase or decrease the speed of chin movement, etc.

[0121] By setting a preset speed range, the system can prompt users on how to adjust the speed of their chin movement when their mouth opening speed is uneven, thereby improving their facial muscle control.

[0122] Figure 6 This is a schematic diagram of a processing device provided in an embodiment of this application. Figure 6 As shown, the processing device 600 includes, but is not limited to, a processor 601 and a memory 602.

[0123] The aforementioned memory 602 is used to store the executable instructions of the aforementioned processor 601. It is understood that the processor 601 is configured to execute instructions to implement the mouth-opening training method in the above embodiments.

[0124] It should be noted that those skilled in the art will understand that Figure 6 The processing device structure shown does not constitute a limitation on the processing device; the processing device may include more than Figure 6 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.

[0125] Processor 601 is the control center of the processing device. It connects various parts of the processing device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 602, and by calling data stored in memory 602, it performs various functions and processes data, thereby providing overall monitoring of the processing device. Processor 601 may include one or more processing units. Optionally, processor 601 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into processor 601.

[0126] The memory 602 can be used to store software programs and various data. The memory 602 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required by at least one functional module (such as a determination unit, processing unit, etc.), etc. Furthermore, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0127] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 602 including instructions, which can be executed by a processor 601 of a processing device 600 to implement the mouth-opening training method in the above embodiments.

[0128] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0129] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by the processor 601 of the processing device to complete the mouth-opening training method in the above embodiments.

[0130] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of the processing device, they implement the various processes of the above method embodiments and achieve the same technical effect as the above method. To avoid repetition, they will not be described again here.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the classified units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, essentially, or the part that contributes to the prior art, or a complete or partial classification of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0136] In the description of the embodiments of this application, specific features, structures, materials or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0137] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training mouth opening, characterized in that, include: The system uses a structured light camera to acquire real-time 3D structured light data of the user during the mouth-opening training process. The real-time position of the user's chin point in the personal reference coordinate system is obtained from the three-dimensional structured light data, with the center of the line connecting the two inner corners of the user's eyes in the three-dimensional structured light data as the origin. Based on the real-time position of the user's mandibular point in the personal reference coordinate system, the real-time chin movement parameters of the mandibular point are determined, and the real-time chin movement parameters are used to indicate the real-time positional offset of the user in the personal reference coordinate system. Based on the real-time chin movement parameters, mouth opening training prompts are generated to guide the user in adjusting their mouth opening state.

2. The mouth-opening training method according to claim 1, characterized in that, The process of obtaining the real-time position of the user's chin point in the personal reference coordinate system from the three-dimensional structured light data includes: Construct a personal baseline coordinate system; The user's chin point was identified from the three-dimensional structured light data; Obtain the real-time position of the mandibular point in the personal reference coordinate system.

3. The mouth-opening training method according to claim 1, characterized in that, The determination of real-time chin movement parameters based on the real-time position of the user's chin point in the personal reference coordinate system includes: Obtain the reference position of the user's chin point in the personal reference coordinate system when the user's mouth is closed; Based on the reference position and real-time position of the user's mandibular point in the personal reference coordinate system, the real-time chin movement parameters of the mandibular point are determined.

4. The mouth-opening training method according to claim 3, characterized in that, The real-time chin motion parameters include at least one of the following: Real-time chin opening, real-time chin expansion and contraction, and real-time chin lateral offset.

5. The mouth-opening training method according to claim 4, characterized in that, The step of obtaining the reference position of the user's chin point in the personal reference coordinate system when the user's mouth is closed includes: The reference positions of the user's chin point in the first, second, and third directions of the personal reference coordinate system are obtained when the user's mouth is closed. The first direction is the user's height direction, the second direction is the user's front-back direction, and the third direction is the left-right hand direction. Accordingly, determining the real-time chin movement parameters of the mandible point based on the reference position and the real-time position of the user's mandible point in the personal reference coordinate system includes: The real-time chin opening degree is determined based on the displacement of the user's chin point in the first direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. The real-time chin extension is determined based on the displacement of the user's chin point in the second direction of the personal reference coordinate system relative to the reference position in the closed-mouth state. The real-time chin offset is determined based on the displacement of the user's chin point in the third direction of the personal reference coordinate system relative to the reference position in the closed-mouth state.

6. The mouth-opening training method according to claim 5, characterized in that, The step of generating mouth-opening training prompts based on the real-time chin movement parameters includes: Obtain reference chin movement parameters for the mandibular point, wherein the reference chin movement parameters are the chin movement parameters of the user in the maximum mouth opening state; Based on the ratio of the real-time chin movement parameters to the reference chin movement parameters, the user's mouth opening training prompt information is generated.

7. The mouth-opening training method according to claim 6, characterized in that, The process of obtaining the reference chin movement parameters for the mandibular point includes: Obtain the reference position of the user's chin point in the personal reference coordinate system when the user is in the maximum mouth opening state; Based on the reference position and the reference position of the user's mandibular point in the personal reference coordinate system, the reference chin movement parameters of the mandibular point are determined.

8. The mouth-opening training method according to claim 6, characterized in that, The reference chin motion parameters include at least one of the following: Reference chin opening, reference chin extension and contraction, and reference chin lateral offset; Accordingly, generating the user's mouth-opening training prompt information based on the ratio of the real-time chin movement parameters to the reference chin movement parameters includes: When the ratio of the real-time chin opening to the reference chin opening is less than the preset chin opening, a mouth-opening training prompt message is generated to instruct the user to increase the size of the mouth opening in the first direction. When the ratio of the real-time chin extension to the reference chin extension is not within the preset chin extension range, a mouth-opening training prompt message is generated to instruct the user to extend or retract the chin in the second direction. When the ratio of the real-time chin offset to the reference chin offset is not within the preset chin offset range, a prompt message is generated instructing the user to practice opening their mouth by moving their chin in the third direction.

9. The mouth-opening training method according to claim 7, characterized in that, Both the baseline position and the reference position were obtained before the user underwent radiotherapy.

10. An opening-mouth training system, characterized in that, The system includes: A structured light camera, positioned facing the user's facial area, is used to acquire three-dimensional structured light data of the user in real time during mouth-opening training. A processing device, electrically connected to the structured light camera, is used to perform the method according to any one of claims 1-9; An information prompting device, electrically connected to the processing device, is used to output the mouth-opening training prompt information.

Images