Doctor-patient communication agent interaction method and system based on natural language processing

Abstract

The invention relates to the technical field of agent interaction, provides a doctor-patient communication agent interaction method and system based on natural language processing, and is used for solving the problems of inaccurate affected part positioning, low participation degree in a patient diagnosis and treatment process and insufficient visualization of a treatment principle. The method comprises the following steps: receiving voice information of a user for describing symptoms, analyzing to generate semantic feature data, matching the semantic feature data with a disease database, and sending affected part positioning guidance; activating an image acquisition device to obtain spectral reflection data, and comparing to generate an abnormal distribution diagram; based on the position of the target abnormal area, controlling a display device to mark a real affected part and trigger voice explanation of the reflection intensity difference; and generating drug particle dynamic demonstration in combination with the spatial structure characteristics, and automatically switching to three-dimensional operation demonstration of the treatment equipment when watching exceeds a preset time length. According to the technical scheme provided by the invention, full-process interaction from speech symptom analysis, spectrum precise positioning to augmented reality visual treatment is realized, and the affected part recognition precision and patient treatment cognition are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent agent interaction technology, and in particular to a method and system for intelligent agent interaction in doctor-patient communication based on natural language processing. Background Art

[0002] Chronic disease patients require long-term medication management, but they face problems such as low medication compliance, insufficient dynamic monitoring of drug side effects, and conflicts in medication regimens due to the coexistence of multiple diseases. Such scenarios require personalized medication guidance through natural language interaction, requiring intelligent bodies to dynamically analyze patient complaints (such as symptom changes, drug reactions), integrate electronic medical records and real-time health monitoring data (such as blood pressure, blood sugar), and generate adaptive medication recommendations based on the evidence-based medical knowledge base, while providing patients with easy-to-understand medication explanations and risk warnings to improve the safety and effectiveness of long-term treatment.

[0003] The current mainstream solution uses a dialogue system based on a pre-trained language model (such as BERT, GPT series) and a fusion of medical knowledge graphs: using NLP technology to analyze the medication problems and health status in the patient's natural language input, and to associate drug indications, contraindications and interaction rules through the knowledge graph to generate preliminary suggestions; at the same time, a reinforcement learning framework is introduced to optimize the response logic based on historical dialogue feedback. The system can provide standardized instructions for drug usage and automatically answer questions about common side effects.

[0004] However, the pre-trained language model has a biased semantic understanding of long-tail medical issues (such as side effects of rare drugs and conflicts in medication for multiple diseases), which can easily lead to erroneous recommendations that contradict clinical guidelines; the knowledge graph relies on a static rule base, which makes it difficult to dynamically integrate patients' real-time biological indicators (such as mutated liver function data) and personalized variables (such as differences in diet and daily routine), resulting in rigid medication recommendations. In addition, the system lacks the ability to adaptively adjust patients' emotions and cognitive levels, and cannot clarify ambiguous descriptions through multiple rounds of dialogue (such as whether "dizziness" is related to postural hypotension or drug overdose), which may lead to the risk of misjudgment. Summary of the invention

[0005] The present application provides a method and system for interaction between medical and patient communication agents based on natural language processing, which are used to solve the problems in the prior art of inaccurate positioning of affected areas, low patient participation in the diagnosis and treatment process, and insufficient visualization of treatment principles.

[0006] In a first aspect, the present application provides a method for interaction between a doctor-patient communication agent based on natural language processing, comprising: Receiving voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parsing the voice information to generate semantic feature data, matching the semantic feature data with a corresponding disease feature database, and sending affected area location guidance information including medical guidance to the user terminal according to the matching result; In response to the user completing the affected part alignment operation according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area; According to the position information of the target abnormal area in the abnormal distribution map, the display device is controlled to display the marking information corresponding to the actual affected part, and after the marking information reaches a preset display effect, the corresponding relationship between the reflection intensity difference and the abnormal degree is converted into voice explanation information; A drug action demonstration is generated based on the spatial structural characteristics of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

[0007] Optionally, in response to the user completing the affected part alignment operation according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflectance data while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the difference in reflection intensity between the abnormal area and the normal area, including: After the user completes the affected area alignment operation according to the affected area positioning mark map of the medical guidance, a spectral scanning instruction of the image acquisition device is triggered to obtain spectral reflectance data of the affected area in real time; Extracting a spectral reflection intensity sequence of an abnormal area from the spectral reflection data, and simultaneously extracting a spectral reflection intensity sequence of a pre-calibrated normal area; The reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area are calculated one by one at the same wavelength point to generate a reflection intensity difference value; According to the difference value of each wavelength point in the reflection intensity difference value, a reflection intensity difference heat map of the abnormal area at different wavelengths is drawn; The wavelength points whose reflection intensity difference values ​​in the reflection intensity difference thermodynamic map exceed a preset threshold are spatially superimposed to generate an abnormal distribution map including the abnormal area position and the abnormal degree.

[0008] Optionally, according to the position information of the target abnormal area in the abnormal distribution map, controlling the display device to display the marking information corresponding to the real affected part, and after the marking information reaches a preset display effect, converting the corresponding relationship between the reflection intensity difference and the abnormality degree into voice explanation information, including: Extracting the reflection intensity difference value of each area from the abnormal distribution map, selecting the area with the largest reflection intensity difference value as the target abnormal area, and recording the position information of the target abnormal area; According to the distribution range of the position information in the abnormal distribution map, the transparency of the marking information corresponding to the actual affected part displayed in the display device is calculated, and the maximum reflection intensity difference value of the distribution range corresponds to the minimum transparency; Controlling the marking information to use the rate of change of the reflection intensity difference value over time as the gradual rate of the transparency, and continuously reducing the transparency until the reflection intensity difference value of the target abnormal area stops increasing; Extracting the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining area in the abnormal distribution map, converting the absolute difference into a percentage level according to the distribution range, and generating the degree of abnormality; The position information of the target abnormal area is converted into a percentage position of the display device screen, and the percentage position is spliced ​​with the abnormality degree to generate voice commentary information including a screen position description of the target abnormal area and an abnormality level label corresponding to the abnormality degree.

[0009] Optionally, the calculating, according to the distribution range of the position information in the abnormal distribution map, the transparency of the marking information corresponding to the actual affected part displayed in the display device includes: Extracting a numerical set of reflection intensity difference values ​​of all regions from the abnormal distribution map, screening a maximum reflection intensity difference value and a minimum reflection intensity difference value in the numerical set, and generating a reflection intensity difference distribution range; Calculate the relative difference between the reflection intensity difference value and the minimum reflection intensity difference value in the reflection intensity difference distribution range according to the reflection intensity difference value corresponding to the position information of the target abnormal area; The relative difference is mapped to the initial transparency of the target abnormal area, wherein the initial transparency corresponding to the maximum reflection intensity difference value in the reflection intensity difference distribution range is the lowest initial transparency, and the initial transparency corresponding to the minimum reflection intensity difference value is the highest initial transparency.

[0010] Optionally, a drug action demonstration is generated based on the spatial structural features of the target abnormal area, in which the motion state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating the affected area for treatment is displayed, including: Extracting surface depression depth parameters and edge diffusion range parameters from the spatial structural features of the target abnormal area to generate three-dimensional morphological parameters of the abnormal area; Calculating the initial movement path of the drug particles in the abnormal area according to the surface depression depth parameter in the three-dimensional morphological parameter, and adjusting the coverage density of the movement path according to the edge diffusion range parameter; Acquire direction sensor data of changes in user observation angles in real time, convert the direction sensor data into a deflection angle of a drug particle motion path, and control the drug particles to move along the deflected path; A timing device is started during the movement of the drug particle motion path, and when the accumulated time of the timing device exceeds a preset time, a three-dimensional demonstration operation of the treatment device is triggered; In the three-dimensional demonstration operation, the operation trajectory of the treatment device in the abnormal area is generated according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional morphological parameters, and the operation trajectory is controlled to overlap and display with the coverage density area of ​​the drug particle motion path.

[0011] Optionally, the direction sensor data includes a horizontal rotation angle and a vertical rotation angle; The step of converting the direction sensor data into a deflection angle of the drug particle motion path and controlling the drug particles to move along the deflected path includes: Converting the horizontal rotation angle into a horizontal offset angle of the drug particle motion path, and converting the vertical rotation angle into a vertical offset angle of the drug particle motion path, to generate a deflection angle of the drug particle motion path; According to the horizontal offset angle and the vertical offset angle in the deflection angle, the moving direction of the drug particle moving path is adjusted to generate a deflected drug particle moving path, and the drug particles are controlled to move along the deflected drug particle moving path; The method further comprises: In the process of controlling the movement of the drug particles along the movement path of the deflected drug particles, the change rate of the direction sensor data is monitored in real time, and the update frequency of the deflection angle is synchronously adjusted according to the change rate; When the change rate of the direction sensor data is lower than a preset threshold, the drug particle movement path corresponding to the current deflection angle is locked, and the particle position is updated in the display device based on the deflected drug particle movement path.

[0012] Optionally, receiving voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parsing the voice information to generate semantic feature data, matching the semantic feature data with a corresponding disease feature database, and sending affected area location guidance information including medical guidance to the user terminal according to the matching result, including: Receive voice information input by a user, and extract a keyword set describing abnormal features in the voice information, wherein the keyword set includes abnormal symptom description words such as redness, swelling, desquamation, and itching, and corresponding symptom location description words; Performing weight matching on the abnormal symptom description words in the keyword set and the symptom associations in a preset disease feature database to generate semantic feature data including symptom association weights; According to the semantic feature data, the target disease feature data with the highest association weight is screened out from the disease feature database, and the affected part positioning rule corresponding to the target disease feature data is extracted; Performing spatial mapping between the symptom location descriptors in the keyword set and the affected part positioning rules to generate positioning guidance information including the affected part coordinate offset and the surface diffusion range; According to the diffusion range in the positioning guidance information and the display device resolution of the user terminal, an affected part positioning mark map adapted to the screen size is generated, and the affected part positioning guidance information containing medical guidance is sent to the user terminal.

[0013] In a second aspect, the present application provides a doctor-patient communication agent interaction system based on natural language processing, comprising: A receiving module, configured to receive voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parse the voice information to generate semantic feature data, match the semantic feature data with a corresponding disease feature database, and send affected area location guidance information including medical guidance to the user terminal according to the matching result; a comparison module, configured to respond to the user completing the affected part alignment operation according to the medical guidance, activate the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generate an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area; a conversion module, for controlling a display device to display marking information corresponding to a real affected part according to the position information of a target abnormal area in the abnormal distribution map, and converting the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information after the marking information reaches a preset display effect; The display module is used to generate a drug action demonstration based on the spatial structural characteristics of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

[0014] In a third aspect, an embodiment of the present application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a medical-patient communication intelligent agent interaction method based on natural language processing as described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a doctor-patient communication agent interaction method based on natural language processing as described in the first aspect.

[0016] Beneficial effects of this application: The present application receives voice information input by a user, wherein the voice information includes symptom description information of the user's abnormal characteristics, parses the voice information to generate semantic feature data, matches the semantic feature data with a corresponding disease feature database, and sends affected part positioning guidance information including medical guidance to the user terminal according to the matching result, thereby realizing a preliminary diagnosis of the user's self-described symptoms based on the precise matching of symptom semantics and disease characteristics, and dynamically generating affected part positioning guidance instructions; in response to the user completing the affected part alignment operation according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the difference in reflection intensity between the abnormal area and the normal area, thereby combining image acquisition and spectral reflection analysis technology to improve the accuracy of abnormal area detection and positioning efficiency; By controlling the display device to display marking information corresponding to the actual affected area according to the position information of the target abnormal area in the abnormal distribution map, and converting the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information after the marking information reaches the preset display effect, it is possible to achieve visual annotation of the abnormal area and synchronous feedback of multimodal information, thereby assisting the user to intuitively understand the degree of the lesion; by generating a drug action demonstration based on the spatial structural characteristics of the target abnormal area, the movement state of the drug particles in the drug action demonstration is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment equipment operating and treating the affected area is displayed, which can enhance the user's understanding depth of the treatment plan and treatment compliance through dynamic interactive drug action simulation and treatment process demonstration.

[0017] Furthermore, through the dynamic collection of spectral reflectance data and multi-wavelength point difference analysis, combined with the point-by-point comparison of the reflection intensity sequence of abnormal and normal areas, accurate quantitative detection of lesion characteristics and spatial distribution analysis can be achieved; based on the superposition generation mechanism of the reflection intensity difference heat map, the limitation of the traditional single-point spectral detection on the blurred boundary of abnormal areas can be broken through, and the multi-wavelength abnormal response characteristics can be simultaneously integrated to construct a three-dimensional distribution model with position resolution and abnormal degree classification. Its technical effect is: through the dynamic calculation and spatial visualization mapping of spectral reflectance differences, the subjective errors of manual visual interpretation can be eliminated, the sensitivity and positioning accuracy of lesion detection can be improved, and the differences in deep structural characteristics of lesions can be revealed based on multi-wavelength collaborative analysis, providing high-resolution objective quantitative basis for disease grading diagnosis, and optimizing the targeting of subsequent treatment plans and the scientific nature of intervention parameter setting through visual marking of abnormal area distribution and intensity threshold screening.

[0018] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flowchart of a method for interaction between a doctor and a patient communication agent based on natural language processing provided by the present application is shown; Figure 2 A schematic diagram of the structure of a doctor-patient communication agent interaction system based on natural language processing provided by the present application is shown; Figure 3 A schematic diagram of the structure of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0022] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0023] Researchers have found that existing remote diagnosis systems for skin abnormalities generally have problems such as the separation of voice interaction and optical detection data, insufficient accuracy in positioning and guiding the affected area, and the static nature of treatment demonstration content, which leads to user misunderstanding. Based on this, a dynamic diagnosis and treatment interaction method for skin abnormalities is provided, which can integrate voice semantic analysis, spectral reflectance difference analysis, and three-dimensional dynamic demonstration technology to achieve closed-loop interaction between diagnosis and treatment guidance and treatment feedback. The technical solution of this application can be applied to scenarios such as remote assisted diagnosis and treatment of skin diseases, home health management equipment, etc., which require high-precision affected area positioning and visual treatment guidance.

[0024] The entire R&D process reflects the technical route of multimodal data fusion and dynamic guidance collaboration, aiming to overcome the defects of isolated analysis of voice-optical data and disconnection between treatment demonstration and user operation in existing solutions. Through the closed-loop linkage mechanism of voice guidance, optical feedback and treatment demonstration, user operation compliance and treatment awareness are significantly improved, providing a dynamic interactive solution with both accuracy and immersion for the diagnosis and treatment of skin abnormalities.

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] Figure 1 A flowchart of a method for interacting with a doctor-patient communication agent based on natural language processing is provided for an embodiment of the present application. Figure 1 As shown, the following contents of this application are mainly illustrated for patients with skin diseases, and do not mean that this application scheme is only applicable to the doctor-patient communication intelligent agent interaction method for patients with skin diseases, but the doctor-patient communication intelligent agent interaction method for other diseases may also be applicable.

[0027] The method includes: 101. Receive voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parse the voice information to generate semantic feature data, match the semantic feature data with the corresponding disease feature database, and send the affected area location guidance information including medical guidance to the user terminal according to the matching result; in this step, the voice information refers to the oral description of abnormal skin characteristics (such as redness, swelling, desquamation, and itching) input by the user through the terminal device. Symptom description information refers to keywords or phrases (such as "persistent peeling" and "local burning sensation") extracted from the voice information, which are used to characterize the clinical manifestations of the user's skin problems. Semantic feature data refers to a vector or label set generated after structured analysis of symptom description information through natural language processing technology (such as entity recognition and sentiment analysis). Disease feature database refers to a standardized data set containing skin disease names, typical symptoms, affected area locations, and medical guidelines. Matching results refer to the correlation ranking after comparing the user's semantic feature data with the entries in the disease feature database through semantic similarity calculation (such as cosine similarity and knowledge graph retrieval). Medical guidance refers to text or voice suggestions generated based on matching results (such as "suspected eczema, it is recommended to check the elbow"). Affected area positioning guidance information refers to operating instructions that guide users to align the affected area with the detection device through visual arrows, highlighted areas or voice prompts.

[0028] In the embodiment of the present application, first, the system receives the voice information input by the user through the terminal device, converts the voice into text data using voice recognition technology, and extracts keywords (such as "erythema", "itching") through the natural language processing algorithm to generate structured semantic feature data. Subsequently, the semantic feature data is matched with the preset disease feature database for similarity, and the potential disease types (such as eczema, dermatitis) are screened out using the fuzzy query algorithm, and medical guidance content is generated based on the matching results. Finally, a graphic information containing guidance for positioning the affected area (such as "Please aim the camera at the erythema area on the inside of the arm") is sent to the user terminal to guide the user to adjust the device to the affected area.

[0029] A user found irregular erythema on his face and described the symptoms through voice on a smart device: "red patches on the cheeks, slight desquamation, and occasional itching." After receiving the voice information, the system parses the semantic feature data, identifies keywords such as "erythema," "desquamation," and "itching," and matches them with the skin disease features in the disease feature database. The comparison found that this symptom combination has the highest similarity with seborrheic dermatitis and contact dermatitis, and then sends medical guidance to the user terminal: "Please point the mobile phone camera at the affected area, keep a distance of ten centimeters, and ensure sufficient light." At the same time, the voice prompts to adjust the shooting angle: "Slowly move the device so that the red area is fully displayed in the dotted box on the screen," guiding the user to complete the initial positioning of the affected area.

[0030] 102. In response to the user completing the operation of aligning the affected part according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflection data while the user keeps aligning the affected part, and generating an abnormal distribution map by comparing the difference in reflection intensity between the abnormal area and the normal area; In this step, the affected area alignment operation refers to the action of the user adjusting the body position or device angle according to the guidance information to bring the abnormal skin area into the field of view of the image acquisition device. The image acquisition device refers to a hardware device that integrates a multispectral camera or infrared sensor, which is used to capture the optical properties of the skin surface. Spectral reflectance data refers to the matrix of reflection intensity values ​​recorded after the skin is irradiated with light of different wavelengths (such as visible light, near infrared). The reflection intensity difference refers to the difference in reflectivity between the abnormal skin area and the surrounding normal skin at the same wavelength, which is used to quantify the change in the absorption characteristics of the lesion area. The abnormal distribution map refers to the visualization result that displays the spatial distribution of the reflection intensity difference in the form of a heat map or contour lines, identifying the potential range of lesions.

[0031] In the embodiment of the present application, first, when the user aligns the affected area with the camera according to the guidance, the system triggers the image acquisition device to start multi-spectral scanning, and simultaneously obtains visible light images and near-infrared spectral reflectance data. Secondly, the abnormal skin area is separated from the visible light image through the image segmentation algorithm, and the spectral reflectance value of the corresponding position is extracted. Then, the difference in reflection intensity between the abnormal area and the adjacent normal area is calculated, and the difference value is mapped to a color gradient (such as red for high difference and blue for low difference). Finally, an abnormal distribution map superimposed on the visible light image is generated, and the range and severity of the lesion are dynamically annotated to provide a spatial positioning basis for subsequent marking display.

[0032] According to the guidance, the user aligns the facial erythema area with the image acquisition device, and the system activates the multi-spectral scanning function. In a stable focus state, by detecting the difference in reflection intensity of light of different wavelengths on the skin surface, it is found that the abnormal area shows a sudden drop in reflectivity in a specific band. The system automatically delineates the boundaries of the erythema and generates an abnormal distribution map with a gradient from dark red to light yellow, in which the difference in reflection intensity in the core area reaches the warning threshold. At this time, the device vibrates to indicate that the scan is complete, and the screen displays "Three suspected inflammation areas have been locked", laying a data foundation for subsequent precise marking.

[0033] 103. According to the position information of the target abnormal area in the abnormal distribution map, control the display device to display the marking information corresponding to the real affected part, and after the marking information reaches a preset display effect, convert the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information; In this step, the location information of the target abnormal area refers to the pixel coordinates or area contours whose reflection intensity difference exceeds the threshold value, which are screened out from the abnormal distribution map. Marking information refers to the virtual mark (such as red border, flashing light spot) superimposed on the user's skin surface through augmented reality (AR) technology, which is used to accurately indicate the actual location of the affected area. The preset display effect refers to the visual stability standard that the marking information must meet (such as continuous display for 3 seconds without offset). The correspondence between the reflection intensity difference and the degree of skin abnormality refers to the mapping rule between the reflectivity difference range calibrated by clinical data and the severity of the disease (such as mild, moderate, and severe). Voice commentary information refers to the explanatory voice content generated based on the above correspondence (such as "The current area has a strong inflammatory reaction, and it is recommended to take medication in time").

[0034] In an embodiment of the present application, first, based on the coordinate information of the target abnormal area in the abnormal distribution map, the augmented reality engine is called to render a dynamic marking box (such as a flashing red border) in real time on the user's screen, and the image tracking algorithm is used to fit it to the edge of the affected area. Secondly, the display stability (such as the position offset rate is lower than the threshold) and transparency of the marking box are continuously detected, and the voice conversion module is triggered when the preset effect is achieved (such as the transparency gradually changes to 50%). Then, the reflection intensity difference value is matched with the preset abnormality grading rule (such as a difference value of 0.3 corresponds to moderate inflammation) to generate a natural language description text. Finally, the text is converted into a voice commentary (such as "the current area The degree of inflammation is moderate") through speech synthesis technology and played through the terminal speaker.

[0035] Based on the positioning data of the abnormal distribution map, the system superimposes augmented reality markers on the user's face in real time: three flashing ring cursors accurately frame the core area of ​​erythema, and the semi-transparent highlight layer at the edge shows the trend of inflammation spreading. When the user keeps his head still for more than three seconds, the marker automatically turns into a continuously displayed amber outline. The voice commentary starts simultaneously: "There is abnormal accumulation of keratin in the epidermis of the current marked area, and the capillaries in the dermis are significantly dilated. It is recommended to give priority to the central area." The commentary content is dynamically adjusted according to the gradient changes of the reflection intensity difference, emphasizing the pathological differences between the central area and the surrounding skin.

[0036] 104. Generate a drug action demonstration based on the spatial structural features of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

[0037] In this step, the spatial structural features refer to the volume, surface curvature and depth information of the target abnormal area extracted by three-dimensional reconstruction technology. The drug action demonstration refers to a virtual animation that simulates the penetration and diffusion of drug ingredients (such as anti-inflammatory molecules) in the skin layer through dynamic particle effects. The motion state of drug particles refers to the movement path, speed and visual effect of the interaction with skin cells of drug particles in the animation. The synchronization of observation angle changes refers to the augmented reality technology that adjusts the movement perspective of drug particles in real time with the device posture when the user moves the terminal device. The preset duration refers to the time threshold (such as 30 seconds) that the user must meet to continuously watch the drug demonstration to trigger the three-dimensional demonstration content. The three-dimensional demonstration content of the treatment device operation treatment refers to an interactive animation that uses modeling to show the virtual treatment of the affected area by medical devices (such as lasers and microneedles), which is used to illustrate the treatment process and expected results.

[0038] In the embodiment of the present application, first, a drug diffusion model is constructed based on the three-dimensional contour data (such as depth and area) of the target abnormal area, and the particle system is used to simulate the movement trajectory of drug particles on the surface of the skin. Secondly, the gyroscope is used to obtain the observation angle change data of the user device in real time, and the movement direction of the drug particles is dynamically adjusted (such as the particles diffuse to the left when the device is tilted to the left). Then, if the user watches the demonstration continuously for more than a preset time (such as 30 seconds), the three-dimensional treatment scene loading module is triggered, and the preset laser or microneedle device model is called to generate a three-dimensional animation of the treatment device operating on the affected area (such as focusing the laser beam to ablate the diseased tissue). Finally, the dynamic demonstration screen is rendered to the user terminal in real time and synchronously superimposed on the image of the affected area, completing the visual guidance of the entire process from drug action to treatment.

[0039] For the marked core area, the system generates a three-dimensional drug action demonstration: virtual liquid particles penetrate from the edge of the marked area in the form of green light spots and diffuse into the deep layer along the skin texture. When the user turns his face to change the viewing angle, the particle movement trajectory presents a perspective effect in real time, clearly showing the accumulation process of the drug in the hair follicle channel. The demonstration is interspersed with flashing prompts: "The white highlighted area is the targeted treatment area", guiding users to understand the key areas of drug action. After watching for two minutes, the interface automatically pops up the treatment progress bar to prepare for the transition to the advanced treatment demonstration. When the drug action demonstration reaches the preset duration, the system switches to the phototherapy device operation demonstration: the three-dimensional model of the virtual photon rejuvenation device slides in from the edge of the screen, and the blue treatment spot completely overlaps with the previously marked area. In the demonstration, the pulsed light beam emitted by the device forms a dynamic grid on the skin surface, the red light area shows the focus range of thermal energy, and the purple ripples indicate the cell repair process. When the user follows the instructions to rotate the device to observe the angle of the treatment head, the system marks the "optimal irradiation angle" prompt line in real time and adds in voice: "Maintaining this angle allows the light energy to evenly cover the abnormal area." After completing the full process demonstration, the interface generates a customized plan including treatment frequency and key care points, forming a closed loop of diagnosis and treatment.

[0040] In summary, steps 101 to 104 realize the intelligence and precision of the entire process of doctor-patient interaction. By receiving patient voice information and parsing semantic features, the system can quickly understand the description of skin symptoms, automatically match the disease database to generate a personalized guidance plan for the location of the affected area, and significantly improve the accuracy of the patient's self-examination. Combined with the image acquisition device, the spectral data is synchronously acquired and the abnormal distribution map is generated, which realizes the quantitative analysis of the skin lesion area and provides a visual basis for subsequent treatment. Through the coordinated output of dynamic marking and voice explanation, the patient's intuitive understanding of the disease is enhanced. At the same time, the three-dimensional drug action demonstration based on spatial structure characteristics helps patients to deeply understand the treatment plan, forming a complete closed-loop service from symptom identification to treatment display.

[0041] In order to accurately identify abnormal skin areas through dynamic comparison of spectral reflectance data, the spectral scan is triggered to obtain real-time reflectance data after the user completes the positioning and marking of the affected area. The reflection intensity sequences of the abnormal and normal areas are extracted to calculate the wavelength level difference to quantify the spectral difference. A multi-wavelength heat map is generated based on the difference value and the frequency band exceeding the standard is screened. Finally, a visual map containing the abnormal position distribution and the degree of lesions is formed through spatial superposition and fusion, which realizes the spectral feature extraction and spatial mapping of the skin lesion boundary and the degree of abnormality, providing objective optical basis and quantitative indicators for skin pathological diagnosis.

[0042] In some embodiments, in step 102, in response to the user completing the affected part alignment operation according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflectance data while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the difference in reflection intensity between the abnormal skin area and the normal skin area, including: 201. After the user completes the affected part alignment operation according to the affected part positioning mark map of the medical guidance, a spectral scanning instruction of the image acquisition device is triggered to obtain spectral reflectance data of the affected part area in real time; In step 201, the spectral scanning instruction refers to a control signal that triggers the image acquisition device to start scanning the affected skin area point by point with multi-wavelength light (such as visible light and near infrared). Spectral reflectance data refers to a set of reflection intensity values ​​of the skin surface at different wavelengths obtained by scanning, and the reflectance of each pixel is recorded in a matrix form.

[0043] In the embodiment of the present application, first, when the user completes the alignment operation of the affected part according to the affected part positioning mark map of the medical guidance, the system detects that the device has stabilized through the camera posture sensor, triggering the image acquisition device to start multi-spectral scanning. Secondly, the device continuously emits light sources of different bands in a preset wavelength range (such as visible light to near infrared), and synchronously receives the reflection signal from the skin surface. Then, the light signal is converted into digital spectral reflection data through an analog-to-digital converter, and stored as a time series data stream according to the wavelength point classification. Finally, the raw spectral data is transmitted to the data processing module in real time to provide input for subsequent abnormal area extraction.

[0044] 202. Extracting a spectral reflection intensity sequence of an abnormal area from the spectral reflection data, and simultaneously extracting a spectral reflection intensity sequence of a pre-calibrated normal area; In step 202, the spectral reflection intensity sequence of the abnormal skin area refers to a one-dimensional array of reflection intensities of the diseased area at continuous wavelengths (such as 400nm to 1000nm) extracted from the spectral reflection data, arranged in wavelength order. The spectral reflection intensity sequence of the normal skin area refers to a reflection intensity reference sequence established by pre-collecting spectral data of a healthy user or a non-lesioned skin area of ​​the same user, for comparative analysis.

[0045] In the embodiment of the present application, first, from the spectral reflectance data obtained in step 201, the image segmentation algorithm is called to identify the boundary range of the abnormal skin area, and the spectral reflectance intensity sequence of all pixel points in the area is extracted (such as the reflectance value of each wavelength point). Secondly, the area not marked as abnormal is selected as the normal reference area in the same field of view of the user's skin, and its spectral reflectance intensity sequence is extracted synchronously. Then, the spectral data of the abnormal and normal areas are aligned according to the spatial coordinates to ensure that the wavelength points of the subsequent difference calculation correspond one to one. Finally, the reflection intensity sequence data set of the abnormal area and the normal area is generated, and its position and wavelength attributes are marked.

[0046] 203. Calculate the difference between the reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area at the same wavelength point one by one to generate a reflection intensity difference value; In step 203, the difference calculation refers to performing a numerical subtraction operation on the reflection intensity values ​​of the abnormal skin area and the normal skin area at the same wavelength point to obtain the reflection intensity difference value at the wavelength. The reflection intensity difference value refers to the reflectivity difference corresponding to a single wavelength point, which is used to quantify the degree of deviation of the absorption characteristics of the lesion area and the normal area.

[0047] In the embodiment of the present application, first, the reflection intensity sequence of the abnormal area is matched with the reflection intensity sequence of the normal area at the same wavelength point, and the absolute difference of the reflectivity value is calculated for each wavelength point. For example, for the reflectivity of a wavelength of 500nm, the abnormal area is 65%, and the normal area is 80%, then the difference is 15%. Secondly, after traversing all wavelength points to complete the difference calculation, a list containing the difference values ​​of each wavelength point is generated. Then, the difference value is normalized to eliminate the influence of the difference in the base value of the reflectivity of different wavelengths. Finally, a set of reflection intensity difference values ​​with wavelength labels is output to provide a data basis for drawing a thermal map.

[0048] 204. Draw a reflection intensity difference heat map of the abnormal area at different wavelengths according to the difference value of each wavelength point in the reflection intensity difference value; In step 204, the reflection intensity difference heat map refers to a two-dimensional image that uses wavelength as the horizontal axis and spatial position as the vertical axis to visualize the difference value of each wavelength point through a color gradient (such as red for high difference and blue for low difference). A wavelength point refers to a data node corresponding to a specific wavelength (such as 550nm, 780nm) selected in a spectral scan.

[0049] In the embodiment of the present application, first, according to the set of reflection intensity difference values ​​generated in step 203, the difference values ​​are mapped to color codes in the order of wavelength (e.g., the larger the difference value, the closer the color is to red, and the smaller the difference value, the closer the color is to blue). Secondly, the heat map rendering engine is called to superimpose color blocks of different wavelength points on the corresponding affected area image on the user terminal screen to form a dynamic heat map with multiple wavelengths superimposed. Then, the currently displayed wavelength range is controlled by a sliding window (e.g., gradually switching from 400nm to 900nm), so that the user can observe abnormal responses in specific bands. Finally, a heat map sequence containing the full wavelength difference distribution is generated to identify the wavelength sensitivity characteristics of the abnormal area.

[0050] 205. Spatially superimpose the wavelength points whose reflection intensity difference values ​​in the reflection intensity difference heat map exceed a preset threshold value to generate an abnormal distribution map including the abnormal area position and the abnormal degree.

[0051] In step 205, the preset threshold refers to the critical value of the difference in reflection intensity set according to clinical experimental data. If the value exceeds this value, it is considered that the corresponding wavelength has significant pathological characteristics. Spatial superposition refers to the logical OR operation of the spatial positions corresponding to the difference values ​​exceeding the threshold at different wavelengths, and the merging generates a comprehensive abnormal area. The abnormal distribution map refers to a thermal map containing the abnormal area boundary and the degree of abnormality (such as the accumulation of difference values) generated by the superposition result, which is used to guide treatment positioning.

[0052] In an embodiment of the present application, first, wavelength points whose reflection intensity difference values ​​exceed a preset threshold (such as wavelength points whose difference values ​​are greater than 20%) are screened from the heat map sequence, and their corresponding spatial position coordinates are extracted. Secondly, the coordinates of different wavelength points that exceed the standard are superimposed and fused, and the number of wavelengths that exceed the standard at each spatial position point is counted. Then, according to the number of exceeding standards, it is mapped to the degree of abnormality (such as exceeding the standard by more than 5 wavelength points is a severe abnormality), and the degree of abnormality is marked with color depth on the image of the affected area. Finally, a comprehensive abnormality distribution map is generated, which simultaneously displays the location boundary and severity grade of the abnormal area, providing a visual basis for subsequent diagnosis and treatment decisions.

[0053] Here is a specific example: In the follow-up visit scenario of COPD patients in community hospitals, a certain intelligent doctor-patient interaction system constructs a semantic dynamic parsing framework through natural language processing technology. When the patient complains that "there is a whistle in the throat when gasping for breath recently, and I wake up three times at night with coughing", the voice interaction module first triggers the semantic collection instruction (corresponding to step 201), converts the characteristic words such as "whistle" and "wake up with cough at night" described by the patient into symptom entities, and extracts standard symptom descriptions such as "mild cough" and "shortness of breath during daytime activities" of typical stable COPD patients from the knowledge base (step 202). The system performs semantic difference analysis on the abnormal features of the current symptoms such as "whistle (wheeze)" and "frequent at night" with the standard symptom library (step 203), and generates difference marks such as "wheeze intensity +3 levels" and "night symptom frequency exceeds threshold". A multidimensional semantic weight distribution map is constructed based on the difference value (step 204), in which "night symptoms" are highlighted because they exceed the acute exacerbation warning threshold. Finally, key difference dimensions such as abnormal respiratory sounds and diurnal distribution of symptoms are integrated (step 205) to generate a decision map containing labels such as "high-risk acute exacerbation" and "need for enhanced anti-inflammatory treatment", driving the system to automatically push sputum examination recommendations and emergency referral instructions, and at the same time generate an interactive health education animation of "avoid cold air stimulation", helping doctors complete the entire process from symptom collection to intervention plan formulation within 8 minutes.

[0054] In summary, steps 201 to 205 achieve multi-spectral precision detection and analysis of skin lesion areas. The reflection intensity data of different wavelengths are obtained through spectral scanning, and the system uses a difference calculation method to effectively distinguish the difference in reflection characteristics between abnormal and normal skin tissues. The abnormal distribution map generated by the thermal map overlay technology not only accurately calibrates the spatial position of the lesion, but also intuitively presents the distribution of abnormalities at different wavelengths, providing multi-dimensional quantitative indicators for skin disease diagnosis. This technology breaks through the limitations of traditional image recognition in the detection of microscopic tissue changes, significantly improves the sensitivity of early lesion identification through spectral feature comparison, and provides a reliable basis for defining the operating range of subsequent treatment equipment.

[0055] In some embodiments, as described in step 103, according to the position information of the target abnormal area in the abnormal distribution map, controlling the display device to display the marking information corresponding to the actual affected part, and after the marking information reaches a preset display effect, converting the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information, includes: 301. Extracting the reflection intensity difference value of each area from the abnormal distribution map, selecting the area with the largest reflection intensity difference value as the target abnormal area, and recording the position information of the target abnormal area; In step 301, the reflection intensity difference value refers to the specific value of the reflectivity corresponding to each pixel or area extracted from the abnormal distribution map deviating from the normal area, usually expressed as a percentage or an absolute value. The target abnormal area refers to a set of continuous pixels whose reflection intensity difference value reaches the maximum value after screening, which may correspond to severe lesions such as inflammation and ulceration. The position information refers to the geometric boundary (such as the upper left corner coordinates + width / height) or the center point coordinates of the target abnormal area in the image coordinate system.

[0056] In the embodiment of the present application, first, the system reads the reflection intensity difference value of each area from the abnormal distribution map, and extracts the values ​​one by one by traversing the pixel points or grid units. Secondly, the maximum value screening algorithm is used to compare the difference values ​​of all areas, and the area with the largest difference value is identified as the target abnormal area. Then, the image coordinate parsing module is called to obtain the vertex coordinates and boundary range of the area, and the center point position and coverage area are recorded. Finally, the position information of the target abnormal area is stored as structured data, including the horizontal and vertical coordinate offsets, to provide input for subsequent transparency calculations.

[0057] 302. Calculate the transparency of the marking information corresponding to the actual affected part displayed in the display device according to the distribution range of the position information in the abnormal distribution map, wherein the maximum reflection intensity difference value in the distribution range corresponds to the minimum transparency; In step 302, the distribution range refers to the coverage area or the total number of pixels occupied by the target abnormal area in the abnormal distribution map. The maximum reflection intensity difference value refers to the peak value of all pixel difference values ​​in the area. The minimum transparency refers to the complete opacity of the marking information in the display device (such as 0% transparency), which is used to most prominently mark the core area of ​​the lesion during visualization.

[0058] In the embodiment of the present application, first, the distribution range of the target abnormal area in the abnormal distribution map is determined according to the location information of the target abnormal area (such as covering a 100×100 pixel area). Secondly, the reflection intensity difference value of all pixels within the range is extracted, and the maximum value is found as the basis for transparency adjustment. Then, the difference value is mapped to a transparency parameter through normalization processing (such as the maximum value corresponds to a transparency of 10%, and the minimum value corresponds to 90%). Finally, the transparency control instruction of the display device is generated according to the mapping result to ensure that the greater the difference, the more conspicuous the regional marking information is.

[0059] 303. Control the mark information to use the rate of change of the reflection intensity difference value over time as the gradual rate of the transparency, and continuously reduce the transparency until the reflection intensity difference value of the target abnormal area stops increasing; In step 303, the gradual change rate of transparency refers to the speed at which the transparency of the marking information changes linearly or nonlinearly over time, and its value is equal to the change in the reflection intensity difference value of the target abnormal area per unit time (such as per second) (such as the difference value increases from 30% to 35% corresponding to a decrease in transparency of 5% per second). Stopping the growth means that the real-time monitoring value of the difference value does not change for more than a preset time (such as 5 seconds) or reaches the upper limit of the instrument range.

[0060] In the embodiment of the present application, first, the rate of change of the reflection intensity difference value of the target abnormal area over time is monitored in real time, and the difference value increment per second is calculated by time series difference. Secondly, the change rate is converted into a transparency gradient rate (e.g., the higher the rate, the faster the transparency decreases). Then, the transparency of the marking information is dynamically adjusted in the display device so that it continues to decrease at a gradient rate. Finally, when the difference value increment approaches zero (i.e., stops growing), the transparency parameter is frozen and the current display effect is maintained.

[0061] 304. Extract the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining area in the abnormal distribution map, convert the absolute difference into a percentage level according to the distribution range, and generate an abnormality degree; In step 304, the absolute difference refers to the absolute value of the subtraction between the current difference value of the target abnormal area and the average difference value of all pixels in the remaining area, which is used to quantify the prominence of the local lesion. The percentage level refers to the discretized score after mapping the difference to a preset interval (such as 0-10% for level 1, 10-20% for level 2), and the level threshold is calibrated by clinical data. The degree of skin abnormality refers to the final generated level label (such as "mild abnormality" and "severe abnormality").

[0062] In the embodiment of the present application, first, the absolute difference between the reflection intensity difference value of the target abnormal area and the average difference value of the remaining area in the abnormal distribution map is calculated. Secondly, the difference is proportionally converted according to the maximum difference value of the distribution range (for example, if the difference is 50 and the maximum difference value is 100, the percentage level is 50%). Then, a skin abnormality degree label is generated according to the preset abnormality degree grading rules (such as 0-30% is mild, 31-70% is moderate, and 71% or more is severe). Finally, the percentage level is associated with the label, and a structured abnormality degree report is output.

[0063] 305. Convert the position information of the target abnormal area into a percentage position on the display device screen, and concatenate the percentage position with the abnormality degree to generate voice commentary information including a screen position description of the target abnormal area and an abnormality level label corresponding to the abnormality degree.

[0064] In step 305, the percentage position of the display device screen refers to converting the physical coordinates of the target abnormal area into the relative proportions of the horizontal and vertical axes of the screen (such as 50% on the X axis and 70% on the Y axis) to adapt the mark position to different device resolutions. The abnormal level label refers to the semantic description generated in combination with the percentage level (such as "high abnormality requires priority processing"). Voice commentary information refers to the composite content broadcasted through speech synthesis technology (such as "the abnormal area is located at 60% of the lower right corner of the screen, and the severity level is level three").

[0065] In the embodiment of the present application, first, the position coordinates of the target abnormal area are converted into the percentage position of the display device screen (such as horizontal coordinate 300 pixels / screen width 600 pixels = 50%). Secondly, the percentage position and the skin abnormality degree label are spliced ​​in a fixed format (such as "50% position in the middle of the right arm, abnormality degree: moderate"). Then, the splicing result is converted into descriptive text through natural language generation technology. Finally, the speech synthesis engine is called to convert the text into voice commentary information, which is played through the terminal device, and the corresponding position label is highlighted on the screen synchronously.

[0066] Here is a specific example: In the diabetic foot screening scenario in a community hospital, an intelligent doctor-patient interaction system uses natural language processing technology to achieve dynamic symptom analysis. When the patient complains that "there is always a numb feeling on the sole of the right foot, and the color is a little dark when soaking the foot", the semantic analysis module first captures the core symptoms such as "numbness" and "local discoloration" from the dialogue flow as semantic abnormal areas (corresponding to step 301). The system compares the symptom with the typical early manifestations of diabetic foot, "symmetrical numbness" and "intact skin", and determines that "local discoloration" breaks through the baseline threshold to become a key semantic abnormal point, and automatically increases its display priority in the interactive interface (step 302). As the patient adds the description that "the dark red area has become larger this week", the system enhances the display intensity of the semantic point in real time, so that the corresponding warning box gradually changes from opaque yellow to translucent red (step 303). By calculating the deviation between the semantic weight of the "local discoloration" symptom and the mean of conventional symptoms, the severity level of "moderate tissue hypoxia risk" is generated (step 304). Finally, the semantic focus is mapped to the projection area of ​​the third metatarsal bone of the virtual foot model, and a voice prompt of "moderate ischemia signs appear in the middle of the right forefoot" is synthesized, and a red light flashes synchronously at the corresponding coordinate point of the three-dimensional model (step 305), driving the doctor to quickly perform transcutaneous oxygen partial pressure testing, thereby discovering hidden pre-ulcer lesions 3 days earlier than the traditional consultation mode.

[0067] In summary, steps 301 to 305 achieve the coordinated guidance of visual marking of abnormal areas and intelligent voice interpretation. The system converts spectral difference data into visual marking information through dynamic transparency adjustment technology, so that patients can intuitively perceive the location and severity changes of lesions. Based on the percentage grade evaluation system generated by the reflection intensity difference value, professional medical indicators are converted into easy-to-understand voice interpretation, realizing the natural language conversion of medical test results. This technology creatively establishes a mapping relationship between optical feature data and user cognitive experience, and significantly improves the efficiency of patients' understanding of disease judgment and treatment recommendations through the spatiotemporal synchronization of visual marking and voice interpretation.

[0068] In some embodiments, as described in step 302, calculating the transparency of the marking information corresponding to the actual affected part displayed in the display device according to the distribution range of the position information in the abnormal distribution map includes: 401. Extract a numerical set of reflection intensity difference values ​​of all regions from the abnormal distribution map, select a maximum reflection intensity difference value and a minimum reflection intensity difference value in the numerical set, and generate a reflection intensity difference distribution range; In step 401, the numerical set of reflection intensity difference values ​​refers to an array or list consisting of all data of reflectance difference values ​​extracted from all pixels or regions in the abnormal distribution map. The maximum reflection intensity difference value refers to the highest value in the set, reflecting the degree of deviation of the light absorption characteristics of the most serious abnormal area. The minimum reflection intensity difference value refers to the lowest value in the set, corresponding to the reflection characteristics close to normal skin. The reflection intensity difference distribution range refers to the interval composed of the maximum and minimum values, which is used to quantify the span of the overall abnormality.

[0069] In an embodiment of the present application, first, the system traverses the pixel points or grid cells of all the marked areas in the abnormal distribution map, extracts the reflection intensity difference values ​​one by one, and forms a set containing all the values. Secondly, the values ​​in the set are arranged in ascending order through a sorting algorithm, and the first and last two values ​​after the arrangement are screened out as the minimum reflection intensity difference value and the maximum reflection intensity difference value, respectively. Then, these two extreme values ​​are encapsulated into a data structure of the reflection intensity difference distribution range in a fixed format (such as "minimum value X, maximum value Y"). Finally, the distribution range is stored in the memory to provide a benchmark parameter for the subsequent relative difference calculation.

[0070] 402. Calculate, according to the reflection intensity difference value corresponding to the position information of the target abnormal area, a relative difference between the reflection intensity difference value and the minimum reflection intensity difference value in the reflection intensity difference distribution range; In step 402, the relative difference refers to the ratio of the difference between the reflection intensity difference value of the target abnormal area and the minimum value of the distribution range to the entire distribution range (maximum value - minimum value), and the calculation formula is (target value - minimum value) / (maximum value - minimum value), which is used to normalize the abnormality degree of the target area.

[0071] In the embodiment of the present application, first, according to the location information of the target abnormal area, the corresponding reflection intensity difference value is located from the abnormal distribution map. Secondly, the reflection intensity difference distribution range generated in step 401 is called, and the minimum reflection intensity difference value therein is read. Then, the difference between the difference value of the target area and the minimum value is calculated to obtain the relative difference amount (for example, if the target value is 80 and the minimum value is 20, the relative difference amount is 60). Finally, the relative difference amount is proportionally associated with the maximum span of the distribution range (i.e., the maximum value minus the minimum value) to provide input parameters for transparency mapping.

[0072] 403. Map the relative difference to the initial transparency of the target abnormal area, wherein the initial transparency corresponding to the maximum reflection intensity difference value in the reflection intensity difference distribution range is the lowest initial transparency, and the initial transparency corresponding to the minimum reflection intensity difference value is the highest initial transparency.

[0073] In step 403, the initial transparency refers to the initial visible transparency value assigned to the target abnormal area according to the relative difference. The lower the value (such as 0%), the more opaque it is, which is used to highlight the high abnormal area. The lowest initial transparency corresponds to the maximum reflection intensity difference value (when the relative difference is 1), and the marking information is completely visible. The highest initial transparency corresponds to the minimum reflection intensity difference value (when the relative difference is 0), and the marking information is close to completely transparent. The mapping method usually uses linear interpolation or nonlinear functions (such as exponential functions) to convert the relative difference into a transparency range (such as 0%-100%).

[0074] In the embodiment of the present application, first, according to the relative difference amount calculated in step 402 and its proportional relationship with the maximum span of the distribution range, the proportional value is converted into a transparency parameter through a linear mapping algorithm. For example, if the relative difference amount accounts for 50% of the maximum span, the initial transparency is set to the middle value. Secondly, the mapping rule is set: when the relative difference amount reaches the maximum span of the distribution range (that is, the target value is equal to the maximum value), the initial transparency is the preset minimum value (such as 10%); when the relative difference amount is zero (that is, the target value is equal to the minimum value), the initial transparency is the highest value (such as 90%). Then, the transparency value is dynamically adjusted according to the real-time relative difference amount of the target area. Finally, the transparency parameter is bound to the display mark of the target abnormal area to complete the real-time update of the visual effect.

[0075] Here is a specific example: In the follow-up scenario of hypertension patients in community hospitals, an intelligent doctor-patient interaction system constructs a semantic dynamic evaluation model through natural language processing. When the patient complains that "I feel dizzy recently, like taking a boat, and my left hand is occasionally numb, which gets better after taking medicine", the semantic engine first extracts the abnormal feature set in the symptom description (corresponding to step 401), and compares abnormal semantic units such as "dizziness like taking a boat" and "intermittent limb numbness" with "mild dizziness" and "no abnormality in limbs" in the baseline health state, and determines the maximum value (dizziness) and minimum value (mild dizziness) of the semantic abnormality intensity. The system calculates the semantic deviation of the core abnormal point "dizziness like taking a boat" (step 402), generates priority parameters according to the degree to which the symptom deviates from the normal description, and drives the interactive interface to set the initial transparency of the warning box corresponding to the symptom to the lowest (i.e., the most eye-catching). As the patient adds the description of "dizziness is most severe in the morning", the system dynamically increases the semantic weight of the symptom (step 403), causing the warning box to gradually change from translucent yellow to opaque red, and simultaneously generates a pop-up reminder containing "need to exclude posterior circulation ischemia" on the doctor's terminal. This dynamic prompt mechanism based on semantic intensity distribution enables doctors to identify the risk of vertebral basilar artery insufficiency caused by blood pressure fluctuations within 3 minutes, which is 50% more efficient than traditional consultations and ensures timely adjustment of antihypertensive plans to prevent stroke.

[0076] In summary, steps 401 to 403 realize the dynamic optimization presentation of medical image visualization effects. By establishing a mapping model between the reflection intensity difference value and the transparency parameter, the system can automatically adjust the display effect according to the optical characteristic difference of the lesion area. The gradient rate control technology is used to convert the data change process into a smooth transition of the visualization parameters, so that the patient can clearly perceive the dynamic development process of the lesion area. This technology effectively solves the contradiction between the loss of detail information and visual interference in the presentation of medical images. Through the intelligent transparency adjustment algorithm, the complete layering of the image is maintained while ensuring the prominence of key information.

[0077] In some embodiments, as described in step 104, a drug action demonstration is generated based on the spatial structural characteristics of the target abnormal area, the movement state of the drug particles in the drug action demonstration is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating the affected area for treatment is displayed, including: 501. Extracting a surface depression depth parameter and an edge diffusion range parameter from the spatial structural features of the target abnormal region to generate a three-dimensional morphological parameter of the abnormal region; In step 501, the surface depression depth parameter refers to the vertical depression distance of the target abnormal area relative to the normal skin surface extracted by three-dimensional reconstruction technology, which is usually measured in millimeters to quantify the depth of depressed lesions such as ulcers and scars. The edge diffusion range parameter refers to the radial distance of the abnormal area boundary extending outward, which is used to characterize the lateral range of the spread of inflammation or infection. The three-dimensional morphological parameter refers to the vector composed of the depression depth and the diffusion range, which describes the spatial geometric characteristics of the abnormal area (such as bowl shape, crater shape).

[0078] In the embodiment of the present application, first, the system obtains the spatial structure data of the target abnormal area through the three-dimensional scanning module, uses the depth sensor to measure the vertical height difference of the surface depression area, and extracts the surface depression depth parameter (such as the maximum depth of 2 mm). Secondly, the edge detection algorithm is called to analyze the boundary transition data between the abnormal area and the normal skin, and the diffusion range parameter is calculated (such as the outward diffusion radius of 5 mm). Then, the depth parameter and the diffusion range parameter are integrated according to the three-dimensional coordinate system to generate a three-dimensional morphological parameter data set that describes the three-dimensional morphology of the abnormal area. Finally, the data set is stored as the input benchmark for subsequent drug path planning and treatment demonstration.

[0079] 502. Calculate the initial movement path of the drug particles in the abnormal area according to the surface depression depth parameter in the three-dimensional morphological parameter, and adjust the coverage density of the movement path according to the edge diffusion range parameter; In step 502, the initial motion path refers to the virtual motion trajectory of the drug particles generated according to the depth of the surface depression (such as a spiral descending path). The greater the depth, the longer the path length and curvature, simulating the need for deep penetration of the drug. Coverage density refers to the number of drug particles distributed per unit area. The larger the edge diffusion range, the lower the density, so as to achieve wide coverage of the diffusion area.

[0080] In the embodiment of the present application, first, the depth of the initial motion path of the drug particles is determined according to the surface depression depth parameter. The greater the depth, the greater the downward inclination angle of the path, and the gravity sedimentation trajectory of the particles in the depression structure is simulated by the physical engine. Secondly, the regional grid is divided based on the edge diffusion range parameter, and the particle coverage density of each grid unit is calculated (such as the density increases by 10% for every 1 mm increase in the diffusion range). Then, the density parameter is injected into the particle system to dynamically adjust the distribution number of particles in the diffusion edge area. Finally, a motion path network with a density gradient is generated to ensure that the distribution of particles in the core depression area and the diffusion edge area meets the treatment needs.

[0081] 503. Acquire direction sensor data of changes in user observation angles in real time, convert the direction sensor data into a deflection angle of a drug particle motion path, and control the drug particles to move along the deflected path; In step 503, the direction sensor data refers to the real-time attitude angle (such as pitch angle and yaw angle) of the user's handheld device obtained by the gyroscope or accelerometer. The deflection angle refers to the deviation of the drug particle path dynamically calculated based on the sensor data (such as 15° deflection to the left), ensuring that the particle movement direction is consistent with the user's perspective.

[0082] In the embodiment of the present application, first, the directional sensor data of the user's observation angle changes are collected in real time through the built-in gyroscope of the device (such as horizontal rotation of 15 degrees and vertical tilt of 10 degrees). Secondly, the angle data is converted into a direction vector in three-dimensional space, and the deflection angle of the drug particle motion path is calculated through a coordinate transformation algorithm (such as horizontal rotation corresponds to X-axis offset, vertical tilt corresponds to Y-axis offset). Then, the particle motion direction vector is dynamically modified in the particle system to move along the new angle trajectory, and the particle position update animation is rendered in real time. Finally, the adjusted path coordinates are continuously output to maintain the synchronization between the user's perspective and the particle motion direction.

[0083] 504. Start a timing device during the movement of the drug particles along the motion path, and trigger a three-dimensional demonstration operation of the treatment device when the accumulated time of the timing device exceeds a preset time; In step 504, the timing device refers to a module that records the cumulative time that the user continues to watch the animation of the movement of drug particles. The preset duration refers to the threshold time (such as 30 seconds) that triggers the treatment demonstration, which is used to ensure that the user fully understands the effect of the drug before showing the treatment process. The three-dimensional demonstration operation refers to the virtual treatment process of simulating a treatment device (such as a laser probe) through particle effects or robotic arm animation.

[0084] In an embodiment of the present application, first, a timing device is started when the particles begin to move, and the continuous time for the user to watch the demonstration is accumulated. When the duration exceeds a preset threshold (such as 30 seconds), a three-dimensional treatment scene loading instruction is triggered. Secondly, the operating depth of the treatment device (such as a microneedle) is generated according to the surface depression depth parameter, and the treatment coverage area is set in combination with the edge diffusion range parameter (such as the needle movement radius matching the diffusion range). Then, the device operation trajectory is generated through the animation engine (such as the microneedle spiraling down in the depression area), and the trajectory is controlled to overlap with the high-density area of ​​the particle motion path. Finally, the dual visualization effects of the particle path and the treatment trajectory are superimposed on the user terminal to complete the collaborative demonstration of the treatment process.

[0085] 505. In the three-dimensional demonstration operation, an operation trajectory of the treatment device in the abnormal area is generated according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional morphological parameters, and the operation trajectory is controlled to overlap and display with the coverage density area of ​​the drug particle motion path.

[0086] In step 505, the operation trajectory refers to the movement route (such as a circular scanning trajectory) planned by the treatment device in the three-dimensional model, and its depth and radius are controlled by the surface depression depth and edge diffusion range parameters respectively. Coverage density area overlap display refers to superimposing the treatment trajectory with the high-density coverage area of ​​the drug particles to intuitively demonstrate the synergistic effect of "drug targeting + device treatment".

[0087] In the embodiments of the present application, first, the operating depth level of the therapeutic device is determined according to the surface depression depth parameter (such as the depth value determines the distance the microneedle probes downward), and the treatment coverage radius is set in combination with the edge diffusion range parameter. Secondly, the movement path of the therapeutic device is planned in the three-dimensional model (such as spiral expansion from the center of the depression outward) to ensure that the path completely covers the high-density area of ​​particle movement. Then, a dynamic collision detection algorithm is used to avoid conflicts between the device operation trajectory and the particle path, and the trajectory direction is adjusted to achieve seamless superposition. Finally, the treatment trajectory is rendered with a translucent highlight effect in the display device, and played synchronously with the drug particle path to intuitively demonstrate the synergistic effect of treatment and drug action.

[0088] Here is a specific example: In the follow-up visit scenario of asthma patients in primary hospitals, a certain intelligent doctor-patient interaction system constructs a dynamic semantic topological model through natural language processing technology. When the patient complains that "I can't breathe at night recently, and it takes half an hour to calm down after taking medicine", the semantic parsing module first extracts core symptom entities such as "nocturnal attacks" and "delayed drug onset" from the dialogue flow (corresponding to step 501), and analyzes their semantic depth (symptom severity) and correlation diffusion range (complication risk). The system generates an initial consultation path based on the symptom depth, focusing on the possibility of drug compliance and inhalation technique defects (step 502). When the patient mentions "I always feel that I haven't inhaled the medicine to the bottom", the direction sensor-style semantic tracking immediately captures the potential signal of operation error and shifts the focus of the dialogue to the technical guidance module (step 503). If the patient's description is continuously monitored and no key symptom changes are mentioned for more than 90 seconds (step 504), the intelligent agent automatically triggers the three-dimensional demonstration command and generates a comparison map including the decomposition animation of the standard inhalation action and the deviation points reported by the patient (step 505), in which the incorrect operation nodes and the correct operation trajectories are overlapped and highlighted, helping the doctor to identify the patient's insufficient drug deposition problem caused by the deviation of the inhalation angle within 5 minutes, and simultaneously generate a corrective training plan with voice commentary, so that the patient's nocturnal attack frequency is reduced by 70% in the next follow-up period.

[0089] In summary, steps 501 to 505 realize the three-dimensional dynamic demonstration and interactive operation of the treatment process. Based on the motion path of the drug particles generated by the three-dimensional morphological parameters of the lesion, the system creates a treatment visualization effect with a sense of spatial immersion by collecting the user's observation angle data in real time. The personalized demonstration plan generated by combining the surface depression depth and edge diffusion range parameters enables patients to intuitively understand the mechanism of action of the drug. When the demonstration time reaches the preset value, it automatically switches to the treatment equipment operation demonstration, forming a full-process visual explanation system from drug treatment to physical therapy, which significantly improves patients' acceptance of complex treatment plans.

[0090] In some embodiments, as described in step 503, the direction sensor data includes a horizontal rotation angle and a vertical rotation angle; The step of converting the direction sensor data into a deflection angle of the drug particle motion path and controlling the drug particles to move along the deflected path includes: 601. Convert the horizontal rotation angle into a horizontal offset angle of the drug particle motion path, and convert the vertical rotation angle into a vertical offset angle of the drug particle motion path, to generate a deflection angle of the drug particle motion path; In step 601, the horizontal rotation angle refers to the angle value of the device rotating around the vertical axis (Z axis) obtained by a direction sensor (such as a gyroscope) (for example, 30° left or 45° right). The vertical rotation angle refers to the angle value of the device rotating around the horizontal axis (X axis or Y axis) (for example, 20° looking up or 15° looking down). The horizontal offset angle refers to the horizontal rotation angle converted into the offset of the drug particle motion path in the horizontal direction (left and right) according to a preset ratio (for example, a 30° rotation corresponds to a 6-pixel rightward offset of the path). The vertical offset angle refers to the vertical rotation angle converted into the offset of the path in the vertical direction (up and down) (for example, a 15° downward angle corresponds to a 3-pixel downward offset of the path). The deflection angle refers to a two-dimensional vector synthesized by the horizontal offset angle and the vertical offset angle (for example, [6,-3]).

[0091] In the embodiment of the present application, first, the system obtains the horizontal rotation angle and vertical rotation angle data of the device in real time through a direction sensor (such as a gyroscope). Secondly, a linear mapping algorithm is used to convert the horizontal rotation angle into a horizontal offset angle of the drug particle motion path in proportion (such as a 30-degree left rotation of the device corresponds to a 15-degree left deflection of the particle path). Next, the vertical rotation angle is converted into a vertical offset angle in the same way to ensure that the tilting action of the device corresponds to the direction of particle movement. Finally, the horizontal and vertical offset angles are combined into deflection angle parameters in three-dimensional space to generate a complete directional description of the particle motion path, providing input for path adjustment.

[0092] 602. According to the horizontal offset angle and the vertical offset angle in the deflection angle, adjust the moving direction of the drug particle moving path, generate a deflected drug particle moving path, and control the drug particles to move along the deflected drug particle moving path; In step 602, the movement direction adjustment refers to modifying the initial movement direction vector of the drug particle according to the deflection angle (for example, the original direction [0,1] becomes [6,-3]). The movement path of the drug particle after deflection refers to the updated particle movement trajectory (for example, moving 9 pixels to the lower right per second). Particle movement control refers to the process of driving the particle to move along a new path through an animation engine or a physics engine.

[0093] In the embodiment of the present application, first, based on the deflection angle parameters generated in step 601, the path generation algorithm is called to recalculate the three-dimensional motion trajectory of the drug particles. For example, the original path is a straight line forward, and the horizontal offset angle makes it tilt to the lower left. Secondly, the moving direction vector of the particle is dynamically modified through the particle system, and the new path parameters are injected into the physical engine to simulate the movement of the particles. Then, the animation effect of the particles moving along the new path is rendered in real time on the display device to ensure that it is synchronized with the rotation of the device. Finally, the updated particle position coordinates are continuously output to provide a data stream for subsequent dynamic monitoring.

[0094] The method further comprises: 603. In the process of controlling the movement of the drug particles along the movement path of the deflected drug particles, monitoring the change rate of the direction sensor data in real time, and synchronously adjusting the update frequency of the deflection angle according to the change rate; In step 603, the change rate of the orientation sensor data refers to the change in the device attitude angle (such as horizontal / vertical rotation angle) per unit time (for example, 5° deflection per second). The update frequency refers to the deflection angle calculation frequency dynamically adjusted according to the change rate (for example, 30 updates per second when changing at a high speed).

[0095] In the embodiment of the present application, first, during the movement of the particles, the rate of change of the direction sensor data (such as the angle change per second) is collected in real time, and the rotational acceleration of the current device is calculated by the time series difference algorithm. Secondly, the update frequency of the deflection angle is dynamically adjusted according to the rate of change. When the device rotates quickly, the update frequency is increased (such as 60 frames per second), and when it rotates slowly, the frequency is reduced (such as 30 frames per second). Then, the adjusted frequency parameters are synchronized to the path generation module to ensure that the response delay of the particle movement direction and the device action is controllable. Finally, the monitoring and frequency adaptation process is continuously looped to maintain the fluency and accuracy of the motion demonstration.

[0096] 604. When the change rate of the direction sensor data is lower than a preset threshold, the drug particle motion path corresponding to the current deflection angle is locked, and the particle position is updated in the display device based on the deflected drug particle motion path.

[0097] In step 604, the preset threshold refers to a critical value of the rate of change for determining whether the user's viewing angle tends to be stationary (e.g., a change of less than 1° per second). Locking the deflection angle refers to an operation of stopping responding to changes in sensor data and fixing the particle movement path to the current direction. Particle position update refers to the process of continuously refreshing the particle position in the display device based on the locked path.

[0098] In the embodiment of the present application, first, when the rate of change of the direction sensor data is continuously lower than the preset threshold (such as the angle change is less than 5 degrees per second), the device is determined to enter a stable state. Secondly, the current deflection angle parameters are immediately frozen, the path update calculation is stopped, and the particle motion trajectory is locked. Then, according to the final locked path parameters, the particle position is fixed in the display device through the rendering engine, and the end point of the path is highlighted. Finally, the "path is locked" prompt message is superimposed on the interface to complete the full process switch from dynamic adjustment to static display.

[0099] Here is a specific example: In the inhalation treatment guidance scenario for asthma patients in community hospitals, an intelligent doctor-patient interaction system constructs a dynamic semantic navigation model through natural language processing. When the patient complains that "I always feel the powder stuck in my throat when I inhale medicine", the semantic parsing engine converts the "stuck throat feeling" described by the patient into the horizontal deviation parameter of drug delivery (corresponding to step 601), maps "inspiratory weakness" into the vertical path deviation parameter, and generates an interactive path with inhalation angle calibration as the core. The system then adjusts the three-dimensional trajectory of the virtual demonstration (step 602), and dynamically displays the orange guide line of the standard inhalation action at an elevation angle of 30 degrees in the interface. When the patient repeatedly asks "how high should I tilt my head when I inhale medicine", which causes the semantic flow fluctuation to intensify, the system increases the trajectory fine-tuning frequency to 3 times per second in real time, and helps the patient understand by flashing green contrast paths with different elevation angles at high frequency (step 603). When the patient stares at the screen for more than 5 seconds without asking any questions, the intelligent agent automatically locks the optimal 45-degree elevation path, superimposes the golden steady-state trajectory on the patient's actual medicine holder in the augmented reality glasses (step 604), and generates a voice prompt of "lower jaw slightly raised when deep inhalation". This semantic-action linkage guidance mechanism enabled patients to successfully achieve an 80% drug deposition rate in the lungs for the first time, improving operational accuracy by three times compared to traditional education methods.

[0100] In summary, steps 601 to 604 achieve intelligent adaptation of 3D demonstration content and user interaction behavior. By converting device orientation sensor data into particle motion path parameters in real time, the system creates an immersive interactive observation experience. Dynamic frequency adjustment technology is used to ensure that the demonstration content and user operations are synchronized in real time. When the user stops operating, the current viewing angle is automatically locked to ensure the stable presentation of key treatment scenes. This technology breaks through the one-way playback mode of traditional 3D demonstrations. By establishing a two-way data channel between user behavior and demonstration content, it significantly improves the interactivity and memory retention of medical knowledge dissemination.

[0101] In some embodiments, as described in step 101, receiving voice information input by a user, the voice information including symptom description information of abnormal skin characteristics of the user, parsing the voice information to generate semantic feature data, and matching the semantic feature data with a corresponding disease feature database, and sending affected area location guidance information including medical guidance to the user terminal according to the matching result, includes: 701. Receive voice information input by a user, and extract a keyword set describing skin abnormality characteristics from the voice information, wherein the keyword set includes abnormal symptom description words such as redness, swelling, desquamation, and itching, and corresponding symptom location description words; In step 701, voice information refers to the voice content describing skin problems input by the user through the terminal device. The keyword set refers to the abnormal symptom descriptors (such as redness, swelling, desquamation, itching) and the corresponding symptom location descriptors (such as "inside of the arm", "middle of the forehead") extracted from the voice information. Abnormal symptom descriptors refer to standardized vocabulary used to characterize the clinical manifestations of skin abnormalities. Symptom location descriptors refer to text information describing the specific location of abnormal symptoms on the body surface.

[0102] In the embodiment of the present application, first, the system receives the user's voice input through the microphone of the terminal device, and calls the voice recognition engine to convert the voice into text information. Secondly, the text is parsed using the word segmentation algorithm of natural language processing to screen out keywords describing abnormal skin characteristics (such as "redness", "scaling", "itching") and corresponding location description words (such as "inside of the arm", "face"). Then, irrelevant words are removed through part-of-speech tagging and semantic disambiguation technology, and the keyword set of symptoms and locations is retained. Finally, the keyword set is classified and stored according to symptom type and location attributes to provide structured input for subsequent disease matching.

[0103] 702. Perform weight matching on the abnormal symptom description words in the keyword set and the symptom associations in the preset skin disease feature database to generate semantic feature data including symptom association weights; In step 702, symptom association weight matching refers to the process of assigning weight values ​​by calculating the semantic similarity (such as cosine similarity) between keywords and symptoms in the skin disease feature database or a probability model (such as a Bayesian network). Semantic feature data refers to structured data (such as vectors or key-value pairs) containing weight values ​​associated with keywords and disease symptoms. The skin disease feature database refers to a standardized data set that stores disease names, symptom lists, and medical guidelines.

[0104] In the embodiment of the present application, first, the symptom association data of each disease is read from the preset skin disease feature database (for example, the weight of "itching" and "erythema" associated with eczema is 0.9). Secondly, the abnormal symptom description words in the keyword set are compared with the symptom list in the database one by one, and the association score of each keyword with the disease is calculated by the weight matching algorithm (for example, the weight of "desquamation" for psoriasis is 0.8). Then, the weight scores of all keywords are accumulated to generate semantic feature data containing the disease name and the total weight. Finally, the potential disease list is screened out by sorting from high to low by weight value.

[0105] 703. Filtering target disease feature data with the highest association weight from the disease feature database according to the semantic feature data, and extracting affected area positioning rules corresponding to the target disease feature data; In step 703, the target disease feature data refers to the disease data corresponding to the highest weight selected after sorting by association weight (such as eczema, psoriasis). The affected part location rule refers to the common location and diffusion pattern of the affected part corresponding to the target disease (such as eczema often occurs on the flexion side of the joint).

[0106] In an embodiment of the present application, first, the target disease with the highest association weight (such as eczema weight value 0.95) is selected from the sorted semantic feature data. Secondly, the affected area positioning rules corresponding to the disease are extracted from the disease feature database, including typical onset locations (such as "joint flexion") and diffusion characteristics (such as "clear edges"). Then, the spatial description logic in the positioning rules (such as "symmetrical distribution" and "diffusion along hair follicles") is parsed and converted into executable coordinate calculation parameters. Finally, a positioning rule data set containing position priority and range constraints is generated to provide a basis for spatial mapping.

[0107] 704. Perform spatial mapping between the symptom location description words in the keyword set and the affected part positioning rule to generate positioning guidance information including the affected part coordinate offset and the skin surface diffusion range; In step 704, spatial mapping refers to mapping the symptom location descriptor (such as "back of hand") to a specific coordinate area in the three-dimensional model (such as X=120, Y=80, Z=0) through a coordinate conversion algorithm. The affected area coordinate offset refers to the offset value of the target affected area center point relative to the body's default coordinate system (such as 20 pixels to the right). The skin surface diffusion range refers to the maximum radius of the abnormal area extending outward from the center point (such as 5cm). Positioning guidance information refers to a set of guidance parameters including coordinate offset and diffusion range.

[0108] In an embodiment of the present application, first, the symptom location descriptor (such as "left cheek") in the keyword set is matched with the typical position in the positioning rule, and the coordinate offset between the actual affected part and the standard position is calculated by the spatial mapping algorithm (such as horizontal offset +5%, vertical offset -10%). Secondly, according to the diffusion characteristics in the positioning rule (such as "radius 3 cm") and the diffusion range described by the user (such as "diffusion to the earlobe"), the coverage area boundary of the affected part is dynamically adjusted. Then, the coordinate offset is combined with the diffusion range to generate positioning guidance information containing the center point coordinates and the coverage radius. Finally, the data is formatted into instruction parameters that can be recognized by the device and transmitted to the display control module.

[0109] 705. Generate an affected part positioning mark map adapted to the screen size according to the diffusion range in the positioning guidance information and the display device resolution of the user terminal, and send the affected part positioning guidance information including medical guidance to the user terminal.

[0110] In step 705, the display device resolution refers to the pixel size of the user terminal screen (such as 1920×1080). The affected area positioning marker image adapted to the screen size refers to a virtual marker (such as a circular highlight area) that is scaled according to the diffusion range and adapted to the screen ratio. The positioning guidance information refers to composite data including medical guidance text (such as "Please aim the device at the left arm") and visual markers.

[0111] In an embodiment of the present application, first, according to the diffusion range in the positioning guidance information (such as a radius of 200 pixels) and the resolution of the terminal display device (such as 1920×1080), the image rendering engine is called to scale and generate a positioning mark map of the affected area (such as a circular mark covering the left cheek area). Secondly, the position of the mark map on the screen is adjusted by the coordinate offset to ensure that it is aligned with the actual affected area. Then, guiding elements such as dynamic arrows or highlighted borders are superimposed to generate a positioning mark map containing visual cues. Finally, the mark map and the medical guidance text (such as "Please align with the left cheek mark area") are integrated into a positioning guidance information package, which is sent to the user through the terminal interface to complete the full process guidance.

[0112] Here is a specific example: In the respiratory disease screening scenario of a primary hospital, an intelligent doctor-patient interaction system constructs a symptom space mapping model through natural language processing technology. When the patient complains of "stinging under the left chest when inhaling, and rusty sputum when coughing", the voice recognition module extracts core symptom keywords such as "chest pain" and "rusty sputum" in real time (corresponding to step 701), and conducts multi-dimensional correlation analysis with the respiratory disease feature library, among which "rusty sputum" obtains the maximum semantic weight due to its high match with the characteristics of lobar pneumonia (step 702). After the system locks the pneumonia diagnosis path, it automatically loads the three-lobed lung positioning model unique to the disease (step 703), maps the "lower left chest" mentioned by the patient to the projection area of ​​the lower lobe of the lung, and generates a red warning box containing the costophrenic angle area in combination with the typical inflammation diffusion pattern (step 704). According to the characteristics of the mobile terminal screen, the intelligent agent intelligently scales the lesion positioning mark to the adaptation interface, generates a pulsating highlight area at the bottom of the left lung of the virtual human body model, and simultaneously pushes the voice guidance of "prioritize left lower lung percussion" (step 705). Doctors can quickly detect the dullness area based on the positioning guidance, and confirm it as lobar pneumonia based on imaging examinations. This shortens the diagnosis time by 40% compared to the traditional consultation process, and provides precise guidance with a spatial positioning error of less than 3 cm in symptom description.

[0113] In summary, steps 701 to 705 realize the intelligent fusion application of voice interaction and medical diagnosis. By constructing a weight matching model between symptom keywords and disease characteristics, the system can quickly extract core diagnostic elements from natural language descriptions. Combined with the affected area positioning guidance solution generated by the spatial mapping algorithm, the abstract symptom description is converted into specific spatial coordinate guidance, greatly improving the accuracy of patient self-positioning. This technology effectively solves the pain point problem of difficult affected area positioning in telemedicine, and by establishing a deep association between semantic features and medical knowledge, it provides reliable technical support for intelligent triage services in primary medical scenarios.

[0114] Figure 2 A schematic diagram of the structure of a doctor-patient communication agent interaction system based on natural language processing is provided for an embodiment of the present application. Figure 2 As shown, the system includes: The receiving module 21 is used to receive voice information input by the user, the voice information includes symptom description information of the user's abnormal characteristics, parse the voice information to generate semantic feature data, match the semantic feature data with the corresponding disease feature database, and send the affected part positioning guidance information including medical guidance to the user terminal according to the matching result; The comparison module 22 is used for activating the image acquisition device to synchronously acquire spectral reflection data in response to the user completing the affected part alignment operation according to the medical guidance while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area; The conversion module 23 is used to control the display device to display the marking information corresponding to the real affected part according to the position information of the target abnormal area in the abnormal distribution map, and convert the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information after the marking information reaches a preset display effect; The display module 24 is used to generate a drug action demonstration based on the spatial structural characteristics of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

[0115] Figure 2 The doctor-patient communication agent interaction system based on natural language processing can be executed Figure 1 The implementation principle and technical effects of the method for interacting with a doctor-patient communication agent based on natural language processing described in the embodiment shown are not described in detail. The specific way in which each module and unit performs operations in the above embodiment of a doctor-patient communication agent interaction system based on natural language processing has been described in detail in the embodiment of the method, and will not be described in detail here.

[0116] In one possible design, Figure 2 The doctor-patient communication agent interaction system based on natural language processing in the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0117] The processing component 32 is used for the above Figure 1 The embodiment provides a doctor-patient communication agent interaction method based on natural language processing.

[0118] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0119] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0120] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0121] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0122] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0123] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0124] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a doctor-patient communication agent interaction method based on natural language processing.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0127] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A doctor-patient communication agent interaction method based on natural language processing, characterized in that: Applied to intelligent agents, including: Receiving voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parsing the voice information to generate semantic feature data, matching the semantic feature data with a corresponding disease feature database, and sending affected area location guidance information including medical guidance to the user terminal according to the matching result; In response to the user completing the affected part alignment operation according to the medical guidance, activating the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generating an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area; According to the position information of the target abnormal area in the abnormal distribution map, the display device is controlled to display the marking information corresponding to the actual affected part, and after the marking information reaches a preset display effect, the corresponding relationship between the reflection intensity difference and the abnormal degree is converted into voice explanation information; A drug action demonstration is generated based on the spatial structural characteristics of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

2. The method according to claim 1, characterized in that The responding user completes the affected part alignment operation according to the medical guidance, activates the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generates an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area, including: After the user completes the affected area alignment operation according to the affected area positioning mark map of the medical guidance, a spectral scanning instruction of the image acquisition device is triggered to obtain spectral reflectance data of the affected area in real time; Extracting a spectral reflection intensity sequence of an abnormal area from the spectral reflection data, and simultaneously extracting a spectral reflection intensity sequence of a pre-calibrated normal area; The reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area are calculated one by one at the same wavelength point to generate a reflection intensity difference value; According to the difference value of each wavelength point in the reflection intensity difference value, a reflection intensity difference heat map of the abnormal area at different wavelengths is drawn; The wavelength points whose reflection intensity difference values ​​in the reflection intensity difference thermodynamic map exceed a preset threshold are spatially superimposed to generate an abnormal distribution map including the abnormal area position and the abnormal degree.

3. The method according to claim 1, characterized in that According to the position information of the target abnormal area in the abnormal distribution map, the display device is controlled to display the marking information corresponding to the real affected part, and after the marking information reaches a preset display effect, the corresponding relationship between the reflection intensity difference and the abnormal degree is converted into voice explanation information, including: Extracting the reflection intensity difference value of each area from the abnormal distribution map, selecting the area with the largest reflection intensity difference value as the target abnormal area, and recording the position information of the target abnormal area; According to the distribution range of the position information in the abnormal distribution map, the transparency of the marking information corresponding to the actual affected part displayed in the display device is calculated, and the maximum reflection intensity difference value of the distribution range corresponds to the minimum transparency; Controlling the marking information to use the rate of change of the reflection intensity difference value over time as the gradual rate of the transparency, and continuously reducing the transparency until the reflection intensity difference value of the target abnormal area stops increasing; Extracting the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining area in the abnormal distribution map, converting the absolute difference into a percentage level according to the distribution range, and generating the degree of abnormality; The position information of the target abnormal area is converted into a percentage position of the display device screen, and the percentage position is spliced ​​with the abnormality degree to generate voice commentary information including a screen position description of the target abnormal area and an abnormality level label corresponding to the abnormality degree.

4. The method according to claim 3, characterized in that: The calculating, according to the distribution range of the position information in the abnormal distribution map, the transparency of the marking information corresponding to the actual affected part displayed in the display device comprises: Extracting a numerical set of reflection intensity difference values ​​of all regions from the abnormal distribution map, screening a maximum reflection intensity difference value and a minimum reflection intensity difference value in the numerical set, and generating a reflection intensity difference distribution range; Calculate the relative difference between the reflection intensity difference value and the minimum reflection intensity difference value in the reflection intensity difference distribution range according to the reflection intensity difference value corresponding to the position information of the target abnormal area; The relative difference is mapped to the initial transparency of the target abnormal area, wherein the initial transparency corresponding to the maximum reflection intensity difference value in the reflection intensity difference distribution range is the lowest initial transparency, and the initial transparency corresponding to the minimum reflection intensity difference value is the highest initial transparency.

5. The method according to claim 1, characterized in that The drug action demonstration is generated based on the spatial structural features of the target abnormal area, wherein the motion state of the drug particles in the drug action demonstration is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating the affected area for treatment is displayed, including: Extracting surface depression depth parameters and edge diffusion range parameters from the spatial structural features of the target abnormal area to generate three-dimensional morphological parameters of the abnormal area; Calculating the initial movement path of the drug particles in the abnormal area according to the surface depression depth parameter in the three-dimensional morphological parameter, and adjusting the coverage density of the movement path according to the edge diffusion range parameter; Acquire direction sensor data of changes in user observation angles in real time, convert the direction sensor data into a deflection angle of a drug particle motion path, and control the drug particles to move along the deflected path; A timing device is started during the movement of the drug particle motion path, and when the accumulated time of the timing device exceeds a preset time, a three-dimensional demonstration operation of the treatment device is triggered; In the three-dimensional demonstration operation, the operation trajectory of the treatment device in the abnormal area is generated according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional morphological parameters, and the operation trajectory is controlled to overlap and display with the coverage density area of ​​the drug particle motion path.

6. The method according to claim 5, characterized in that The direction sensor data includes a horizontal rotation angle and a vertical rotation angle; The step of converting the direction sensor data into a deflection angle of the drug particle motion path and controlling the drug particles to move along the deflected path includes: Converting the horizontal rotation angle into a horizontal offset angle of the drug particle motion path, and converting the vertical rotation angle into a vertical offset angle of the drug particle motion path, to generate a deflection angle of the drug particle motion path; According to the horizontal offset angle and the vertical offset angle in the deflection angle, the moving direction of the drug particle moving path is adjusted to generate a deflected drug particle moving path, and the drug particles are controlled to move along the deflected drug particle moving path; The method further comprises: In the process of controlling the movement of the drug particles along the movement path of the deflected drug particles, the change rate of the direction sensor data is monitored in real time, and the update frequency of the deflection angle is synchronously adjusted according to the change rate; When the change rate of the direction sensor data is lower than a preset threshold, the drug particle movement path corresponding to the current deflection angle is locked, and the particle position is updated in the display device based on the deflected drug particle movement path.

7. The method according to claim 1, characterized in that The receiving of voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parsing the voice information to generate semantic feature data, matching the semantic feature data with a corresponding disease feature database, and sending affected part location guidance information including medical guidance to the user terminal according to the matching result, including: Receive voice information input by a user, and extract a keyword set describing abnormal features in the voice information, wherein the keyword set includes abnormal symptom description words such as redness, swelling, desquamation, and itching, and corresponding symptom location description words; Performing weight matching on the abnormal symptom description words in the keyword set and the symptom associations in a preset disease feature database to generate semantic feature data including symptom association weights; According to the semantic feature data, the target disease feature data with the highest association weight is screened out from the disease feature database, and the affected part positioning rule corresponding to the target disease feature data is extracted; Performing spatial mapping between the symptom location descriptors in the keyword set and the affected part positioning rules to generate positioning guidance information including the affected part coordinate offset and the surface diffusion range; According to the diffusion range in the positioning guidance information and the display device resolution of the user terminal, an affected part positioning mark map adapted to the screen size is generated, and the affected part positioning guidance information containing medical guidance is sent to the user terminal.

8. A doctor-patient communication agent interaction system based on natural language processing, characterized in that: Applied to intelligent agents, including: A receiving module, configured to receive voice information input by a user, the voice information including symptom description information of abnormal characteristics of the user, parse the voice information to generate semantic feature data, match the semantic feature data with a corresponding disease feature database, and send affected area location guidance information including medical guidance to the user terminal according to the matching result; a comparison module, configured to respond to the user completing the affected part alignment operation according to the medical guidance, activate the image acquisition device to synchronously acquire spectral reflection data while the user keeps the affected part aligned, and generate an abnormal distribution map by comparing the reflection intensity difference between the abnormal area and the normal area; a conversion module, for controlling a display device to display marking information corresponding to a real affected part according to the position information of a target abnormal area in the abnormal distribution map, and converting the corresponding relationship between the reflection intensity difference and the abnormal degree into voice explanation information after the marking information reaches a preset display effect; The display module is used to generate a drug action demonstration based on the spatial structural characteristics of the target abnormal area, in which the movement state of the drug particles is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than a preset time, a three-dimensional demonstration content including the treatment device operating and treating the affected area is displayed.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a medical-patient communication intelligent agent interaction method based on natural language processing as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a doctor-patient communication agent interaction method based on natural language processing as described in any one of claims 1 to 7 is implemented.

Images