An intelligent agent interaction method and system for doctor-patient communication based on natural language processing

Abstract

This application relates to the field of intelligent agent interaction technologies, and provides a doctor-patient communication intelligent agent interaction method and system based on natural language processing to solve the problems of inaccurate affected area positioning, low patient participation in the diagnosis and treatment process, and insufficient visualization of treatment principles. This application includes: receiving voice information describing symptoms from a user, parsing and generating semantic feature data and matching it with a disease database, and sending affected area positioning guidance; activating an image acquisition device to obtain spectral reflection data and comparing it to generate an abnormal distribution map; based on the position of the target abnormal area, controlling a display device to mark the actual affected area and triggering a voice explanation of the reflection intensity difference; generating a dynamic demonstration of drug particles in combination with spatial structure features, and automatically switching to a three-dimensional operation demonstration of treatment equipment when the viewing time exceeds a preset duration. The technical solution provided by this application realizes the full-process interaction from voice symptom analysis, spectral precise positioning to augmented reality visualization treatment, and improves the accuracy of affected area recognition and patients' treatment cognition.

Description

Technical Field

[0001] This application relates to the field of intelligent agent interaction technologies, and in particular, to a doctor-patient communication intelligent agent interaction method and system based on natural language processing. Background Art

[0002] Chronic disease patients need long-term medication management, but they face problems such as low medication compliance, insufficient dynamic monitoring of drug side effects, and medication plan conflicts caused by co-existing multiple diseases. In such scenarios, personalized medication guidance needs to be achieved through natural language interaction, requiring the intelligent agent to dynamically analyze the patient's chief complaints (such as symptom changes, drug reactions), integrate electronic medical records and real-time health monitoring data (such as blood pressure, blood sugar), and generate adaptable medication suggestions based on an evidence-based medicine knowledge base. At the same time, it provides medication explanations and risk warnings that are easy for patients to understand, so as to improve the safety and effectiveness of long-term treatment.

[0003] The current mainstream solutions adopt a dialogue system that combines a pre-trained language model (such as BERT, GPT series) and a medical knowledge graph: using NLP technology to analyze the medication problems and health status in the patient's natural language input, associating drug indications, contraindications, and interaction rules through the knowledge graph to generate preliminary suggestions; at the same time, introducing a reinforcement learning framework to optimize the response logic according to historical dialogue feedback. The system can provide standardized drug usage instructions and automatically answer common side effects.

[0004] However, the pre-trained language model has semantic understanding biases for medical long-tail problems (such as rare drug side effects, medication conflicts for multiple diseases), and is prone to generating incorrect suggestions that conflict with clinical guidelines; the knowledge graph relies on a static rule base and is difficult to dynamically integrate the patient's real-time biological indicators (such as mutated liver function data) and personalized variables (such as differences in diet and work and rest), resulting in rigid medication suggestions. In addition, the system lacks the ability to adaptively adjust to the patient's emotions and cognitive level, and cannot clarify ambiguous descriptions through multiple rounds of dialogue (such as whether "dizziness" is related to orthostatic hypotension or drug overdose), which may lead to the risk of misjudgment. Summary of the Invention

[0005] This application provides a doctor-patient communication intelligent agent interaction method and system based on natural language processing, which is used to solve the problems of inaccurate lesion location, low patient participation in the diagnosis and treatment process, and insufficient visualization of treatment principles in the prior art.

[0006] In a first aspect, this application provides a doctor-patient communication intelligent agent interaction method based on natural language processing, including:

[0007] Receive the voice information input by the user, where the voice information includes the symptom description information of the user's abnormal characteristics, parse the voice information to generate semantic feature data, and match the semantic feature data with the corresponding disease feature database, and send the affected area positioning guidance information including medical guidance to the user terminal according to the matching result;

[0008] In response to the user's completion of the affected area alignment operation according to the medical guidance, activate the image acquisition device to synchronously obtain spectral reflection data while the user keeps the affected area aligned, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area;

[0009] According to the position information of the target abnormal area in the abnormal distribution map, control the display device to display the marking information corresponding to the real affected area, and after the marking information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information;

[0010] Generate a drug action demonstration based on the spatial structure characteristics of the target abnormal area, where the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle, and when the user watches the drug action demonstration for more than the preset duration, display the three-dimensional demonstration content of the treatment device operating on the affected area.

[0011] Optionally, in response to the user's completion of the affected area alignment operation according to the medical guidance, activate the image acquisition device to synchronously obtain spectral reflection data while the user keeps the affected area aligned, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area, including:

[0012] After the user completes the affected area alignment operation according to the affected area positioning mark map of the medical guidance, trigger the spectral scanning instruction of the image acquisition device to obtain the spectral reflection data of the affected area in real time;

[0013] Extract the spectral reflection intensity sequence of the abnormal area from the spectral reflection data, and synchronously extract the spectral reflection intensity sequence of the pre-calibrated normal area;

[0014] Perform difference calculation on the reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area one by one at the same wavelength point to generate a reflection intensity difference value;

[0015] According to the difference value of each wavelength point in the reflection intensity difference value, draw a heat map of the reflection intensity difference of the abnormal area at different wavelengths;

[0016] Perform spatial superposition on the wavelength points with reflection intensity difference values exceeding the preset threshold in the reflection intensity difference heat map to generate an abnormal distribution map including the position and degree of abnormality of the abnormal area.

[0017] Optionally, according to the position information of the target abnormal area in the abnormal distribution map, control the display device to display the marker information corresponding to the real affected area, and after the marker information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information, including:

[0018] Extract the reflection intensity difference value of each area from the abnormal distribution map, screen out the area with the largest reflection intensity difference value as the target abnormal area, and record the position information of the target abnormal area;

[0019] According to the distribution range of the position information in the abnormal distribution map, calculate the transparency of the marker information corresponding to the real affected area displayed in the display device, and the maximum reflection intensity difference value of the distribution range corresponds to the lowest transparency;

[0020] Control the marker information to use the change rate of the reflection intensity difference value over time as the gradual change rate of the transparency, and continuously reduce the transparency until the reflection intensity difference value of the target abnormal area stops increasing;

[0021] Extract the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining areas in the abnormal distribution map, convert the absolute difference into a percentage level according to the distribution range, and generate the degree of abnormality;

[0022] Convert the position information of the target abnormal area into a percentage position on the screen of the display device, and splice the percentage position with the degree of abnormality to generate voice commentary information including the screen position description of the target abnormal area and the abnormal level label corresponding to the degree of abnormality.

[0023] Optionally, the calculating the transparency of the marker information corresponding to the real affected area displayed in the display device according to the distribution range of the position information in the abnormal distribution map includes:

[0024] Extract the numerical set of the reflection intensity difference values of all areas from the abnormal distribution map, screen out the maximum reflection intensity difference value and the minimum reflection intensity difference value in the numerical set, and generate the reflection intensity difference distribution range;

[0025] According to the reflection intensity difference value corresponding to the position information of the target abnormal area, calculate the relative difference amount between the reflection intensity difference value and the minimum reflection intensity difference value of the reflection intensity difference distribution range;

[0026] Map the relative difference amount to the initial transparency of the target abnormal area, where the maximum reflection intensity difference value of the reflection intensity difference distribution range corresponds to the lowest initial transparency, and the minimum reflection intensity difference value corresponds to the highest initial transparency.

[0027] Optionally, a drug action demonstration is generated based on the spatial structure characteristics of the target abnormal area. In the drug action demonstration, the motion state of drug particles is synchronized with the change of the user's viewing angle. When the user watches the drug action demonstration for more than a preset duration, three-dimensional demonstration content including the treatment device operating on the affected area is displayed, including:

[0028] Extract the surface depression depth parameter and the edge diffusion range parameter from the spatial structure characteristics of the target abnormal area to generate the three-dimensional morphological parameters of the abnormal area;

[0029] According to the surface depression depth parameter in the three-dimensional morphological parameters, calculate the initial movement path of the drug particles in the abnormal area, and adjust the coverage density of the movement path according to the edge diffusion range parameter;

[0030] Obtain the direction sensor data of the change of the user's viewing angle in real time, convert the direction sensor data into the deflection angle of the drug particle movement path, and control the drug particles to move along the deflected path;

[0031] Start a timing device during the movement of the drug particle movement path. When the cumulative duration of the timing device exceeds the preset duration, trigger the three-dimensional demonstration operation of the treatment device;

[0032] In the three-dimensional demonstration operation, according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional morphological parameters, generate the operation trajectory of the treatment device in the abnormal area, and control the operation trajectory to be displayed coincidentally with the coverage density area of the drug particle movement path.

[0033] Optionally, the direction sensor data includes the horizontal rotation angle and the vertical rotation angle;

[0034] The converting the direction sensor data into the deflection angle of the drug particle movement path and controlling the drug particles to move along the deflected path includes:

[0035] Convert the horizontal rotation angle into the horizontal offset angle of the drug particle movement path, and convert the vertical rotation angle into the vertical offset angle of the drug particle movement path to generate the deflection angle of the drug particle movement path;

[0036] According to the horizontal offset angle and the vertical offset angle in the deflection angle, adjust the moving direction of the drug particle movement path, generate the deflected drug particle movement path, and control the drug particles to move along the deflected drug particle movement path;

[0037] The method further includes:

[0038] During the process of controlling the movement of the drug particles along the deflected movement path of the drug particles, the change rate of the data of the direction sensor is monitored in real time, and the update frequency of the deflection angle is synchronously adjusted according to the change rate;

[0039] When the change rate of the data of the direction sensor is lower than a preset threshold, the movement path of the drug particles corresponding to the current deflection angle is locked, and the particle position is updated in the display device based on the deflected movement path of the drug particles.

[0040] Optionally, receive the voice information input by the user, where the voice information includes the symptom description information of the user's abnormal features, parse the voice information to generate semantic feature data, and match the semantic feature data with the corresponding disease feature database, and send the affected area positioning guidance information including medical guidance to the user terminal according to the matching result, including:

[0041] Receive the voice information input by the user, and extract the keyword set describing the abnormal features in the voice information. The keyword set includes abnormal symptom description words such as redness, desquamation, and itching and the corresponding symptom position description words;

[0042] Match the abnormal symptom description words in the keyword set with the symptom relevance in the preset disease feature database by weight to generate semantic feature data including symptom association weights;

[0043] According to the semantic feature data, screen out the target disease feature data with the highest association weight from the disease feature database, and extract the affected area positioning rules corresponding to the target disease feature data;

[0044] Perform spatial mapping on the symptom position description words in the keyword set and the affected area positioning rules to generate positioning guidance information including the affected area coordinate offset and the surface diffusion range;

[0045] Generate an affected area positioning marker map adapted to the screen size according to the diffusion range in the positioning guidance information and the resolution of the display device of the user terminal, and send the affected area positioning guidance information including medical guidance to the user terminal.

[0046] In a second aspect, the present application provides a doctor-patient communication intelligent agent interaction system based on natural language processing, including:

[0047] A receiving module, configured to receive the voice information input by the user, where the voice information includes the symptom description information of the user's abnormal features, parse the voice information to generate semantic feature data, and match the semantic feature data with the corresponding disease feature database, and send the affected area positioning guidance information including medical guidance to the user terminal according to the matching result;

[0048] A comparison module, configured to respond to the user's completion of the affected area alignment operation according to the medical guidance, activate the image acquisition device to synchronously obtain spectral reflection data while the user maintains the affected area alignment, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area;

[0049] A conversion module, configured to control the display device to display the marker information corresponding to the real affected area according to the position information of the target abnormal area in the abnormal distribution map, and after the marker information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the abnormal degree into voice commentary information;

[0050] A display module, configured to generate a drug action demonstration based on the spatial structure characteristics of the target abnormal area, wherein the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle, and when the user watches the drug action demonstration for more than the preset duration, display three-dimensional demonstration content including the treatment device operating on the affected area.

[0051] In a third aspect, an embodiment of the present application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a doctor-patient communication intelligent agent interaction method based on natural language processing as described in the first aspect above.

[0052] In a fourth aspect, an embodiment of the present application provides a computer storage medium, storing a computer program, which when executed by a computer, implements a doctor-patient communication intelligent agent interaction method based on natural language processing as described in the first aspect.

[0053] Advantages of the present application:

[0054] This application receives the voice information input by the user. The voice information includes the symptom description information of the user's abnormal characteristics. It analyzes the voice information to generate semantic feature data, and matches the semantic feature data with the corresponding disease feature database. According to the matching result, it sends the affected area positioning and guiding information including medical guidance to the user terminal, and can realize the preliminary diagnosis of the user's self-reported symptoms based on the accurate matching of symptom semantics and disease characteristics, and dynamically generate the affected area positioning and guiding instructions; by responding to the user's completion of the affected area alignment operation according to the medical guidance, it activates the image acquisition device to synchronously obtain spectral reflection data while the user keeps the affected area aligned, and generates an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area, and can improve the accuracy and positioning efficiency of abnormal area detection by combining image acquisition and spectral reflection analysis technologies; by controlling the display device to display the marker information corresponding to the real affected area according to the position information of the target abnormal area in the abnormal distribution map, and after the marker information reaches the preset display effect, converting the corresponding relationship between the reflection intensity difference and the abnormal degree into voice commentary information, it can realize the visual annotation of the abnormal area and the synchronous feedback of multi-modal information, and assist the user to intuitively understand the degree of lesion; by generating a drug action demonstration based on the spatial structure characteristics of the target abnormal area, the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's observation angle, and when the user watches the drug action demonstration for more than the preset duration, it displays the three-dimensional demonstration content of the treatment device operating on the affected area, and can enhance the user's understanding depth and treatment compliance of the treatment plan through dynamic interactive drug action simulation and treatment process demonstration.

[0055] Furthermore, through the dynamic acquisition of spectral reflection data and the multi-wavelength point difference analysis, combined with the point-by-point comparison of the reflection intensity sequences of the abnormal and normal areas, it realizes the accurate quantitative detection and spatial distribution analysis of lesion characteristics; based on the superimposed generation mechanism of the reflection intensity difference heat map, it breaks through the limitation of the fuzzy boundary of the abnormal area in traditional single-point spectral detection, synchronously fuses the multi-wavelength abnormal response characteristics, and constructs a three-dimensional distribution model with position resolution and abnormal degree grading. The technical effect is that through the dynamic calculation and spatial visualization mapping of spectral reflection differences, it eliminates the subjective error of manual visual interpretation, improves the sensitivity and positioning accuracy of lesion detection, and at the same time reveals the deep structural feature differences of lesions based on multi-wavelength collaborative analysis, provides a high-resolution objective quantitative basis for disease grading diagnosis, and optimizes the targeting of subsequent treatment plans and the scientificity of intervention parameter setting through the visual marking of abnormal area distribution and intensity threshold screening.

[0056] These aspects or other aspects of this application will be more clearly understood in the following description of the embodiments. Brief Description of the Drawings

[0057] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0058] Figure 1 It shows a flowchart of an intelligent agent interaction method for doctor-patient communication based on natural language processing provided by the present application;

[0059] Figure 2 It shows a schematic structural diagram of an intelligent agent interaction system for doctor-patient communication based on natural language processing provided by the present application;

[0060] Figure 3 It shows a schematic structural diagram of a computing device provided by the present application. Detailed implementation manners

[0061] In order to enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application.

[0062] In some processes described in the specification and claims of the present application and the above drawings, there are multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this text or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0063] Researchers have found that existing remote skin abnormality diagnosis systems generally have problems such as the fragmentation of voice interaction and optical detection data, insufficient accuracy of affected area positioning guidance, and static treatment demonstration content leading to user understanding deviation. Based on this, a dynamic diagnosis and treatment interaction method for skin abnormalities is provided. This method can integrate voice semantic parsing, spectral reflection difference analysis, and three-dimensional dynamic demonstration technology to achieve a closed-loop interaction of diagnosis and treatment guidance and treatment feedback. The technical solution of the present application is applicable to scenarios such as remote dermatological auxiliary diagnosis and treatment, home health management devices, etc. that require high-precision affected area positioning and visualization treatment guidance.

[0064] The entire R & D process embodies the technical route of multimodal data fusion and dynamic guidance collaboration, aiming to overcome the defects of isolated parsing of voice-optical data and the disconnection between treatment demonstration and user operation in existing solutions. Through the closed-loop linkage mechanism of voice guidance, optical feedback, and treatment demonstration, the user operation compliance and treatment awareness are significantly improved, providing a dynamic interaction solution with both accuracy and immersion for the diagnosis and treatment of skin abnormalities.

[0065] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0066] Figure 1 The following is a flowchart of an intelligent agent interaction method for doctor-patient communication based on natural language processing provided by an embodiment of the present application. As Figure 1 shown, the following content of the present application mainly takes skin disease patients as examples for illustration, which does not mean that the solution of the present application is only applicable to the intelligent agent interaction method for doctor-patient communication of skin disease patients, and the intelligent agent interaction method for doctor-patient communication under other diseases can also be applicable.

[0067] The method includes:

[0068] 101. Receive the voice information input by the user. The voice information includes the symptom description information of the user's abnormal features. Parse the voice information to generate semantic feature data, and match the semantic feature data with the corresponding disease feature database. According to the matching result, send the affected area positioning guidance information including medical guidance to the user terminal; in this step, the voice information refers to the oral description content of the skin abnormal features (such as redness, desquamation, itching) input by the user through the terminal device. The symptom description information refers to the keywords or phrases extracted from the voice information (such as "continuous desquamation", "local burning sensation") used to characterize the clinical manifestations of the user's skin problems. The semantic feature data refers to the vector or label set generated by structuring and parsing the symptom description information through natural language processing technologies (such as entity recognition, sentiment analysis). The disease feature database refers to the standardized data set including skin disease names, typical symptoms, affected area locations, and medical guidelines. The matching result refers to the correlation degree ranking after comparing the user's semantic feature data with the entries in the disease feature database through semantic similarity calculation (such as cosine similarity, knowledge graph retrieval). The medical guidance refers to the text or voice suggestions generated according to the matching result (such as "suspected eczema, it is recommended to check the elbow"). The affected area positioning guidance information refers to the operation instruction that guides the user to align the affected area with the detection device through visual arrows, highlighted areas, or voice prompts.

[0069] In the embodiments of the present application, first, the system receives the voice information input by the user through the terminal device, converts the voice into text data using speech recognition technology, and extracts keywords (such as "erythema", "itching") through natural language processing algorithms to generate structured semantic feature data. Subsequently, the semantic feature data is matched with the pre-set disease feature database, and the fuzzy query algorithm is used to screen out potential disease types (such as eczema, dermatitis), and medical guidance content is generated according to the matching result. Finally, graphic and text information including guidance on positioning the affected area (such as "Please align the camera with the erythema area on the inner side of the arm") is sent to the user terminal to guide the user to adjust the device to align with the affected area.

[0070] A user found irregular erythema on the face and described the symptoms by voice through the smart device: "There are red patches on the cheeks, slight desquamation, and occasional itching". After receiving the voice information, the system analyzes the semantic feature data, identifies keywords such as "erythema", "desquamation", "itching", and matches them with the skin disease features in the disease feature database. After comparison, it is found that this symptom combination has the highest similarity with seborrheic dermatitis and contact dermatitis. Immediately, medical guidance is sent to the user terminal: "Please align the mobile phone camera with the affected area, keep a distance of ten centimeters, and ensure sufficient light". At the same time, a voice prompt is given to adjust the shooting angle: "Slowly move the device so that the red area is completely displayed within the dotted line frame on the screen", guiding the user to complete the preliminary positioning of the affected area.

[0071] 102. In response to the user completing the operation of aligning the affected area according to the medical guidance, activate the image acquisition device to synchronously obtain spectral reflection data while the user keeps the affected area aligned, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area.

[0072] In this step, the operation of aligning the affected area refers to the action of the user adjusting the body position or the device angle according to the guidance information to make the abnormal skin area enter the field of view of the image acquisition device. The image acquisition device refers to a hardware device integrated with a multi-spectral camera or an infrared sensor, which is used to capture the optical characteristics of the skin surface. The spectral reflection data refers to the matrix of reflection intensity values recorded after irradiating the skin with light of different wavelengths (such as visible light, near-infrared). The reflection intensity difference refers to the difference in reflectance between the abnormal skin area and the surrounding normal skin at the same wavelength, which is used to quantify the change in the light absorption characteristics of the lesion area. The abnormal distribution map refers to the visual result that shows the spatial distribution of the reflection intensity difference in the form of a heat map or a contour line, marking the potential lesion range.

[0073] In the embodiments of the present application, first, when the user aligns the affected area with the camera according to the guidance, the system triggers the image acquisition device to start multispectral scanning, and synchronously obtains visible light images and near-infrared spectral reflection data. Secondly, the abnormal skin area is separated from the visible light image through an image segmentation algorithm, and the spectral reflection values at the corresponding positions are extracted. Then, the difference in reflection intensity between the abnormal area and the adjacent normal area is calculated, and the difference value is mapped to a color gradient (for example, red indicates a high difference, and blue indicates a low difference). Finally, an abnormal distribution map superimposed on the visible light image is generated, dynamically marking the lesion range and severity, providing a spatial positioning basis for subsequent marking and display.

[0074] The user aligns the facial erythema area with the image acquisition device according to the guidance, and the system activates the multispectral scanning function. In the stable focusing state, by detecting the difference in the reflection intensity of the skin surface for light of different wavelengths, it is found that the abnormal area shows a sudden drop in reflectivity in a specific band. The system automatically delimits the erythema boundary and generates an abnormal distribution map that gradually changes from dark red to light yellow, where the difference in reflection intensity in the core area reaches the warning threshold. At this time, the device vibrates to indicate that the scanning is completed, and the screen displays "Three suspected inflammatory areas have been locked", laying a data foundation for subsequent precise marking.

[0075] 103. According to the position information of the target abnormal area in the abnormal distribution map, control the display device to display the marking information corresponding to the real affected area, and after the marking information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the abnormal degree into voice commentary information;

[0076] In this step, the position information of the target abnormal area refers to the pixel coordinates or area contours selected from the abnormal distribution map where the difference in reflection intensity exceeds the threshold. The marking information refers to the virtual identifiers (such as red borders, flashing light points) superimposed on the user's skin surface through augmented reality (AR) technology, which are used to accurately indicate the position of the real affected area. The preset display effect refers to the visual stability standard that the marking information needs to achieve (such as continuous display for 3 seconds without deviation). The corresponding relationship between the reflection intensity difference and the skin abnormal degree refers to the mapping rule between the calibrated reflectance difference range and the disease severity (such as mild, moderate, severe) through clinical data. The voice commentary information refers to the explanatory voice content generated according to the above corresponding relationship (such as "The inflammatory reaction in the current area is relatively strong, it is recommended to take medicine in time").

[0077] In the embodiments of the present application, first, based on the coordinate information of the target abnormal area in the abnormal distribution map, the augmented reality engine is called to render a dynamic marking frame (such as a flashing red border) in real time on the user's screen, and the image tracking algorithm is used to make it fit the edge of the affected area. Secondly, the display stability (such as the position offset rate being lower than the threshold) and transparency of the marking frame are continuously detected. When the preset effect (such as the transparency gradually changing to 50%) is reached, the voice conversion module is triggered. Then, the reflection intensity difference value is matched with the preset abnormal degree grading rule (such as a difference value of 0.3 corresponding to moderate inflammation) to generate a natural language description text. Finally, the text is converted into a voice commentary (such as "The inflammation degree of the current area is moderate") through voice synthesis technology and played through the terminal speaker.

[0078] Based on the positioning data of the abnormal distribution map, the system superimposes augmented reality markers on the user's face in real time: three flashing circular cursors precisely frame the core area of the erythema, and a semi-transparent highlighting layer at the edge shows the inflammation diffusion trend. When the user keeps the head still for more than three seconds, the marker automatically turns into a continuously displayed amber contour. At the same time, the voice commentary is started: "There is abnormal keratin accumulation in the epidermal layer of the current marked area, and obvious capillary dilation in the dermis layer. It is recommended to prioritize the treatment of the central area". The commentary content is dynamically adjusted according to the gradient change of the reflection intensity difference, highlighting the pathological differences between the central area and the surrounding skin.

[0079] 104. Generate a drug action demonstration based on the spatial structure characteristics of the target abnormal area. In the drug action demonstration, the movement state of the drug particles is synchronized with the change of the user's viewing angle. When the user watches the drug action demonstration for more than a preset duration, three-dimensional demonstration content including the operation and treatment of the affected area by the treatment device is displayed.

[0080] In this step, the spatial structure characteristics refer to the volume, surface curvature, and depth information of the target abnormal area extracted by three-dimensional reconstruction technology. The drug action demonstration refers to a virtual animation that simulates the penetration and diffusion of drug components (such as anti-inflammatory molecules) in the skin layer through dynamic particle effects. The movement state of the drug particles refers to the movement path, speed, and visual effect of the interaction between the drug particles and skin cells in the animation. The synchronization of the viewing angle change refers to the augmented reality technology in which when the user moves the terminal device, the movement perspective of the drug particles is adjusted in real time according to the device pose. The preset duration refers to the time threshold for the user to continuously watch the drug demonstration to trigger the three-dimensional demonstration content (such as 30 seconds). The three-dimensional demonstration content of the operation and treatment of the treatment device refers to an interactive animation that shows the virtual treatment of the affected area by medical devices (such as laser devices, microneedles) through modeling, which is used to illustrate the treatment process and expected effects.

[0081] In an embodiment of the present application, first, a drug diffusion model is constructed based on the three-dimensional contour data (such as depth and area) of the target abnormal area, and a particle system is used to simulate the motion trajectory of drug particles on the surface of the skin. Secondly, the gyroscope is used to obtain real-time data on changes in the user device's viewing angle, and the movement direction of the drug particles is dynamically adjusted (for example, when the device is tilted to the left, the particles diffuse to the left). Then, if the user continuously watches the demonstration for more than a preset time (such as 30 seconds), the three-dimensional treatment scene loading module is triggered, and the preset laser or microneedle device model is called to generate a three-dimensional animation of the treatment device operating on the affected area (such as focusing a laser beam to ablate diseased tissue). Finally, the dynamic demonstration screen is rendered in real time to the user terminal and synchronously superimposed on the image of the affected area, completing the visual guidance of the entire process from drug action to treatment.

[0082] For the marked core area, the system generates a 3D drug action demonstration: virtual drug particles, in the form of green dots, penetrate from the edge of the marked area and diffuse deeper along the skin's texture. As the user turns their face to change the viewing angle, the particle trajectory is rendered in real time with a perspective effect, clearly demonstrating the accumulation of drug within the hair follicle tract. Flashing reminders interspersed throughout the demonstration: "The white highlighted area is the targeted treatment area," guide the user through the key areas of drug action. After two minutes of continuous viewing, a treatment progress bar automatically pops up, preparing the interface for the next step. When the drug action demonstration reaches the preset duration, the system switches to a phototherapy device operation demonstration: a 3D model of a virtual photorejuvenation device slides in from the edge of the screen, with the blue treatment spot perfectly overlapping the previously marked area. The device's pulsed light beams form a dynamic grid on the skin's surface, with red areas indicating the focus of thermal energy and purple ripples indicating the progress of cellular repair. As the user follows the guidance to rotate the device to observe the angle of the treatment head, the system displays a real-time "optimal angle" indicator and provides a voice prompt: "Maintaining this angle ensures even coverage of the abnormal area." After completing the full process demonstration, the interface generates a customized plan including treatment frequency and nursing points, forming a closed loop of diagnosis and treatment.

[0083] In summary, steps 101 to 104 achieve intelligent and precise management of the entire doctor-patient interaction process. By receiving patient voice information and parsing semantic features, the system can quickly understand skin symptom descriptions, automatically match the disease database to generate personalized guidance plans for locating the affected area, and significantly improve the accuracy of patient self-examination. Combined with the image acquisition device, spectral data is synchronously acquired and abnormal distribution maps are generated, which enables quantitative analysis of skin lesion areas and provides a visual basis for subsequent treatment. The coordinated output of dynamic marking and voice explanation enhances the intuitiveness of patients' understanding of the disease. At the same time, the three-dimensional drug action demonstration based on spatial structural features helps patients deeply understand the treatment plan, forming a complete closed-loop service from symptom identification to treatment display.

[0084] In order to accurately identify abnormal skin areas through dynamic comparison of spectral reflection data, after the user completes the positioning and marking of the affected area, the spectral scan is triggered to obtain real-time reflection data. The reflection intensity sequences of abnormal and normal areas are extracted for wavelength-level difference calculation to quantify the spectral differences. Based on the difference values, a multi-wavelength heat map is generated and the over-standard frequency bands are screened. Finally, through spatial superposition and fusion, a visualization map including the distribution of abnormal positions and the degree of lesions is formed, realizing the extraction of spectral features and spatial mapping of the boundaries and abnormal degrees of skin lesions, providing an objective optical basis and quantitative indicators for pathological diagnosis of skin diseases.

[0085] In some embodiments, as described in step 102, in response to the user completing the alignment operation of the affected area according to the medical guidance, the image acquisition device is activated to synchronously obtain spectral reflection data while the user keeps the affected area aligned. By comparing the reflection intensity differences between the abnormal skin area and the normal skin area, an abnormal distribution map is generated, including:

[0086] 201. After the user completes the alignment operation of the affected area according to the positioning and marking map of the affected area in the medical guidance, the spectral scan instruction of the image acquisition device is triggered to obtain the spectral reflection data of the affected area in real time;

[0087] In step 201, the spectral scan instruction refers to the control signal that triggers the image acquisition device to start point-by-point scanning of the skin area of the affected area with multi-wavelength light (such as visible light, near-infrared). The spectral reflection data refers to the set of reflection intensity values of the skin surface at different wavelengths obtained by scanning, and the reflectivity of each pixel point is recorded in matrix form.

[0088] In the embodiment of the present application, first, after the user completes the alignment operation of the affected area according to the positioning and marking map of the affected area in the medical guidance, the system detects that the device is stable through the camera attitude sensor and triggers the image acquisition device to start multi-spectral scanning. Second, the device continuously emits light sources of different bands within a preset wavelength range (such as visible light to near-infrared) and synchronously receives the reflection signals from the skin surface. Then, the optical signals are converted into digital spectral reflection data by the analog-to-digital converter and stored as a time-series data stream classified by wavelength points. Finally, the original spectral data is transmitted into the data processing module in real time to provide input for subsequent extraction of abnormal areas.

[0089] 202. Extract the spectral reflection intensity sequence of the abnormal area from the spectral reflection data, and synchronously extract the spectral reflection intensity sequence of the pre-calibrated normal area;

[0090] In step 202, the spectral reflection intensity sequence of the abnormal skin area refers to a one-dimensional array arranged in the order of wavelengths of the reflection intensities of the lesion area extracted from the spectral reflection data at continuous wavelengths (such as 400 nm to 1000 nm). The spectral reflection intensity sequence of the normal skin area refers to a reference sequence of reflection intensities established by pre-collecting the spectral data of healthy users or the non-lesioned skin area of the same user for comparative analysis.

[0091] In the embodiments of the present application, first, from the spectral reflection data obtained in step 201, an image segmentation algorithm is called to identify the boundary range of the abnormal skin area, and the spectral reflection intensity sequence (such as the reflectivity value at each wavelength point) of all pixel points in this area is extracted. Secondly, an area not marked as abnormal within the same field of view of the user's skin is selected as the normal reference area, and its spectral reflection intensity sequence is synchronously extracted. Then, the spectral data of the abnormal and normal areas are aligned according to the spatial coordinates to ensure that the wavelength points for subsequent difference calculation correspond one by one. Finally, a data set of the reflection intensity sequences of the abnormal area and the normal area is generated, and its position and wavelength attributes are marked.

[0092] 203. Calculate the difference between the reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area one by one at the same wavelength points to generate a reflection intensity difference value;

[0093] In step 203, the difference calculation refers to performing a numerical subtraction operation on the reflection intensity values of the abnormal skin area and the normal skin area at the same wavelength point to obtain the reflection intensity difference value at this wavelength. The reflection intensity difference value refers to the difference in reflectivity corresponding to a single wavelength point, which is used to quantify the deviation degree of the light absorption characteristics between the lesion area and the normal area.

[0094] In the embodiments of the present application, first, the reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area are matched at the same wavelength points, and the absolute difference of the reflectivity values is calculated for each wavelength point. For example, for the reflectivity at a wavelength of 500 nm, if the abnormal area is 65% and the normal area is 80%, the difference is 15%. Secondly, after traversing all wavelength points to complete the difference calculation, a list containing the difference values at each wavelength point is generated. Then, the difference values are normalized to eliminate the influence of the difference in the reflectivity base values at different wavelengths. Finally, a set of reflection intensity difference values with wavelength labels is output, providing a data basis for the generation of the heat map.

[0095] 204. Draw a heat map of the reflection intensity difference of the abnormal area at different wavelengths according to the difference value of each wavelength point in the reflection intensity difference value;

[0096] In step 204, the reflection intensity difference heat map refers to a two-dimensional image that uses wavelength as the horizontal axis and spatial position as the vertical axis, visualizing the difference values at each wavelength point through a color gradient (e.g., red represents high difference and blue represents low difference). The wavelength point refers to the data node corresponding to a specific wavelength (such as 550nm, 780nm) selected in the spectral scan.

[0097] In the embodiments of the present application, first, according to the set of reflection intensity difference values generated in step 203, the difference values are mapped into color codes in the order of wavelengths (e.g., the larger the difference value, the closer the color is to red, and the smaller the difference value, the closer it is to blue). Secondly, a heat map rendering engine is called to superimpose and display color blocks at different wavelength points on the corresponding affected area image on the user terminal screen to form a dynamic heat map with multi-wavelength superposition. Then, by sliding the window to control the currently displayed wavelength range (e.g., gradually switching from 400nm to 900nm), the user can observe the abnormal response in a specific wavelength band. Finally, a heat map sequence containing the full-wavelength difference distribution is generated to identify the wavelength sensitivity characteristics of the abnormal area.

[0098] 205. Superimpose the wavelength points in the reflection intensity difference heat map whose reflection intensity difference values exceed a preset threshold in space to generate an abnormal distribution map containing the position and degree of the abnormal area.

[0099] In step 205, the preset threshold refers to the critical value of the reflection intensity difference set according to clinical trial data. If the value is exceeded, it is considered that there are significant pathological characteristics at the corresponding wavelength. Spatial superposition refers to performing a logical OR operation on the spatial positions corresponding to the difference values exceeding the threshold at different wavelengths and merging them to generate a comprehensive abnormal area. The abnormal distribution map refers to a heat map layer generated through the superimposition result, containing the boundary of the abnormal area and the degree of abnormality (such as the accumulation of difference values), and is used to guide treatment positioning.

[0100] In the embodiments of the present application, first, filter out the wavelength points in the heat map sequence whose reflection intensity difference values exceed the preset threshold (such as the wavelength points with a difference value greater than 20%), and extract their corresponding spatial position coordinates. Secondly, superimpose and fuse the coordinates of different over-standard wavelength points, and count the number of over-standard wavelengths at each spatial position point. Then, map the number of over-standard wavelengths into the abnormal degree level (e.g., more than 5 over-standard wavelength points is severe abnormality), and mark the degree of abnormality on the affected area image with the depth of color. Finally, generate a comprehensive abnormal distribution map, and synchronously display the position boundary and severity grading of the abnormal area to provide a visual basis for subsequent diagnosis and treatment decisions.

[0101] The following is a specific example:

[0102] In the follow-up consultation scenario for patients with chronic obstructive pulmonary disease (COPD) in community hospitals, a certain intelligent doctor-patient interaction system constructs a semantic dynamic parsing framework through natural language processing technology. When a patient's chief complaint is "Recently, there is a whistling sound in my throat when I breathe, and I wake up coughing three times at night", the voice interaction module first triggers a semantic acquisition instruction (corresponding to step 201), converts feature words such as "whistling sound" and "waking up coughing at night" described by the patient into symptom entities, and at the same time extracts standard symptom descriptions such as "mild cough" and "dyspnea during daytime activities" of typical stable COPD patients from the knowledge base (step 202). The system conducts item-by-item semantic difference analysis between the abnormal features such as "whistling sound (wheezing)" and "frequent occurrence at night" of the current symptoms and the standard symptom library (step 203), and generates difference marks such as "wheezing intensity +3 levels" and "night symptom frequency exceeding the threshold". Based on the difference values, a multi-dimensional semantic weight distribution map is constructed (step 204), in which "night symptoms" are highlighted due to breaking through the acute exacerbation warning threshold. Finally, key difference dimensions such as abnormal breath sounds and diurnal distribution of symptoms are integrated (step 205), a decision-making map containing labels such as "high-risk acute exacerbation period" and "need to strengthen anti-inflammatory treatment" is generated, driving the system to automatically push suggestions for sputum examination and emergency referral guidelines, and at the same time generating an interactive health education animation of "avoiding cold air stimulation" to help doctors complete the entire process from symptom collection to intervention plan formulation within 8 minutes.

[0103] In summary, steps 201 to 205 achieve multi-spectral precise detection and analysis of skin lesion areas. By obtaining the reflection intensity data of different wavelengths through spectral scanning, the system uses the difference calculation method to effectively distinguish the reflection feature differences between abnormal and normal skin tissues. The abnormal distribution map generated by using the heat map overlay technology not only accurately calibrates the spatial position of the lesion, but also intuitively presents the abnormal degree distribution at different wavelengths, providing multi-dimensional quantitative indicators for the diagnosis of skin diseases. This technology breaks through the limitations of traditional image recognition in the detection of microscopic tissue changes, significantly improves the sensitivity of early lesion recognition through spectral feature comparison, and at the same time provides a reliable basis for defining the operation range of subsequent treatment equipment.

[0104] In some embodiments, as described in step 103, according to the position information of the target abnormal area in the abnormal distribution map, to control the display device to display the marker information corresponding to the real affected area, and after the marker information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the abnormal degree into voice commentary information, including:

[0105] 301. Extract the reflection intensity difference value of each area from the abnormal distribution map, screen out the area with the largest reflection intensity difference value as the target abnormal area, and record the position information of the target abnormal area;

[0106] In step 301, the reflection intensity difference value refers to the specific value by which the reflectivity corresponding to each pixel or region extracted from the abnormal distribution map deviates from the normal region, usually expressed as a percentage or an absolute value. The target abnormal region refers to the continuous pixel set with the maximum reflection intensity difference value after screening, which may correspond to severe lesion regions such as inflammation and ulceration. The position information refers to the geometric boundary (such as the upper left corner coordinate + width / height) or the center point coordinate of the target abnormal region in the image coordinate system.

[0107] In the embodiment of the present application, first, the system reads the reflection intensity difference value of each region from the abnormal distribution map, and extracts the values one by one by traversing pixel points or grid cells. Secondly, the maximum value screening algorithm is used to compare the difference values of all regions, and the region with the largest difference value is identified as the target abnormal region. Then, the image coordinate parsing module is called to obtain the vertex coordinates and boundary range of this region, and record its center point position and coverage area. Finally, the position information of the target abnormal region is stored as structured data, including the horizontal and vertical coordinate offsets, providing input for subsequent transparency calculation.

[0108] 302. Calculate the transparency of the marker information displayed on the display device corresponding to the actual affected area according to the distribution range of the position information in the abnormal distribution map, where the maximum reflection intensity difference value of the distribution range corresponds to the lowest transparency;

[0109] In step 302, the distribution range refers to the coverage area of the target abnormal region in the abnormal distribution map or the total number of pixels occupied. The maximum reflection intensity difference value refers to the peak value of the difference values of all pixels in this region. The lowest transparency refers to that the marker information on the display device is completely opaque (such as transparency 0%), which is used to most significantly mark the core lesion area during visualization.

[0110] In the embodiment of the present application, first, determine the distribution range of the target abnormal region in the abnormal distribution map according to its position information (such as covering a 100×100 pixel area). Secondly, extract the reflection intensity difference values of all pixel points within this range, and find the maximum value as the benchmark for transparency adjustment. Then, map the difference value to a transparency parameter through normalization processing (such as the maximum value corresponding to a transparency of 10% and the minimum value corresponding to 90%). Finally, generate a transparency control instruction for the display device according to the mapping result to ensure that the marker information in the region with a larger difference is more prominent.

[0111] 303. Control the marker information to use the change rate of the reflection intensity difference value over time as the gradual change rate of the transparency, and continuously reduce the transparency until the reflection intensity difference value of the target abnormal region stops increasing;

[0112] In step 303, the gradual change rate of transparency refers to how fast the transparency of the marked information changes linearly or non-linearly with time. Its value is equal to the change amount per unit time (such as per second) of the reflection intensity difference value of the target abnormal area (for example, when the difference value increases from 30% to 35%, the transparency decreases by 5% per second). Stopping growth means that the real-time monitored value of the difference value has no change for a continuous preset time (such as 5 seconds) or reaches the upper limit of the instrument range.

[0113] In the embodiments of the present application, first, the change rate of the reflection intensity difference value of the target abnormal area with time is monitored in real time, and the difference value increment per second is calculated through time series difference. Secondly, the change rate is converted into the gradual change rate of transparency (for example, the higher the rate, the faster the transparency decreases). Then, the transparency of the marked information is dynamically adjusted in the display device so that it continuously decreases at the gradual change rate. Finally, when the difference value increment approaches zero (that is, stops growing), the transparency parameter is frozen and the current display effect is maintained.

[0114] 304. Extract the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining areas in the abnormal distribution map, and convert the absolute difference into a percentage level according to the distribution range to generate the degree of abnormality;

[0115] In step 304, the absolute difference refers to the absolute value of the subtraction of the current difference value of the target abnormal area from the average value of the difference values of all pixels in the remaining area, which is used to quantify the prominence of local lesions. The percentage level refers to the discretized score after mapping the difference value to a preset interval (such as 0 - 10% is level 1, 10 - 20% is level 2), and the level threshold is calibrated by clinical data. The degree of skin abnormality refers to the finally generated level label (such as "mild abnormality", "severe abnormality").

[0116] In the embodiments of the present application, first, the absolute difference between the reflection intensity difference value of the target abnormal area and the average difference value of the remaining areas in the abnormal distribution map is calculated. Secondly, the difference value is proportionally converted according to the maximum difference value of the distribution range (for example, if the difference value is 50 and the maximum difference value is 100, then the percentage level is 50%). Then, according to the preset abnormal degree grading rules (such as 0 - 30% is mild, 31 - 70% is moderate, above 71% is severe), a skin abnormality degree label is generated. Finally, the percentage level is associated with the label, and a structured abnormality degree report is output.

[0117] 305. Convert the position information of the target abnormal area into the percentage position of the display device screen, and splice the percentage position with the degree of abnormality to generate voice commentary information including the screen position description of the target abnormal area and the abnormal level label corresponding to the degree of abnormality.

[0118] In step 305, the percentage position of the display device screen refers to converting the physical coordinates of the target abnormal area into the relative ratios of the horizontal and vertical axes of the screen (such as 50% on the X-axis and 70% on the Y-axis) so that the marked position adapts to different device resolutions. The abnormal level label refers to the semantic description generated by combining the percentage levels (such as "Highly abnormal and needs to be processed first"). The voice commentary information refers to the composite content broadcast through text-to-speech technology (such as "The abnormal area is located at 60% of the lower right of the screen, and the severity level is three").

[0119] In the embodiments of the present application, first, convert the position coordinates of the target abnormal area into the percentage position of the display device screen (such as the horizontal coordinate 300 pixels / the screen width 600 pixels = 50%). Secondly, splice the percentage position and the skin abnormality degree label in a fixed format (such as "At the 50% position in the middle of the right arm, abnormality degree: moderate"). Then, convert the splicing result into descriptive text through natural language generation technology. Finally, call the text-to-speech engine to convert the text into voice commentary information, play it through the terminal device, and simultaneously highlight the corresponding position label on the screen.

[0120] The following is a specific example:

[0121] In the scenario of diabetic foot screening in a community hospital, a certain intelligent doctor-patient interaction system realizes dynamic symptom analysis through natural language processing technology. When the patient complains that "there is a place on the sole of my right foot that always feels numb and the color looks a bit dark when soaking feet", the semantic analysis module first grabs core symptoms such as "numbness" and "local color change" from the conversation flow as semantic abnormal areas (corresponding to step 301). The system compares these symptoms with the early manifestations of typical diabetic foot, "symmetrical numbness" and "intact skin", and determines that "local color change" breaks through the baseline threshold and becomes a key semantic abnormal point, automatically increasing its display priority on the interaction interface (step 302). As the patient supplements the description "the dark red area has become larger this week", the system enhances the display intensity of this semantic point in real time, making the corresponding warning box change from opaque yellow to translucent red (step 303). By calculating the deviation degree of the semantic weight of the "local color change" symptom from the average value of the regular symptoms, a severity level of "moderate tissue hypoxia risk" is generated (step 304). Finally, map the semantic focus to the projection area of the third metatarsal bone of the virtual foot model, synthesize a voice prompt of "There is a moderate ischemia sign in the middle of the front sole of the right foot", and simultaneously flash a red light at the corresponding coordinate point on the three-dimensional model (step 305), driving the doctor to quickly perform a transcutaneous oxygen pressure test and detect the pre-ulcerative lesion in the latent stage 3 days earlier than the traditional consultation mode.

[0122] In summary, steps 301 to 305 achieve collaborative guidance for visual marking of abnormal regions and intelligent voice commentary. The system converts spectral difference data into visual marking information through dynamic transparency adjustment technology, enabling patients to intuitively perceive the location and severity changes of lesions. Based on the percentage grade evaluation system generated from the reflection intensity difference values, professional medical indicators are converted into easy-to-understand voice commentaries, realizing the natural language conversion of medical test results. This technology creatively establishes a mapping relationship between optical feature data and the user's cognitive experience, and significantly improves the patient's understanding efficiency of the condition judgment and treatment suggestions through the spatio-temporal synchronization of visual marking and voice commentary.

[0123] In some embodiments, as described in step 302, according to the distribution range of the position information in the abnormal distribution map, calculating the transparency of the display device to display the marking information corresponding to the real affected area includes:

[0124] 401. Extract the numerical set of all regional reflection intensity difference values from the abnormal distribution map, screen the maximum reflection intensity difference value and the minimum reflection intensity difference value in the numerical set, and generate a reflection intensity difference distribution range;

[0125] In step 401, the numerical set of reflection intensity difference values refers to an array or list composed of all data of the reflectance difference values extracted from all pixels or regions in the abnormal distribution map. The maximum reflection intensity difference value refers to the highest value in this set, reflecting the degree of deviation of the light absorption characteristics of the most severe abnormal region. The minimum reflection intensity difference value refers to the lowest value in the set, corresponding to the reflection characteristics close to normal skin. The reflection intensity difference distribution range refers to the interval composed of the maximum value and the minimum value, which is used to quantify the span of the overall abnormal degree.

[0126] In the embodiment of the present application, first, the system traverses all pixel points or grid cells of the marked regions in the abnormal distribution map, extracts the reflection intensity difference values one by one, and forms a set containing all the values. Secondly, the values in the set are sorted in ascending order through a sorting algorithm, and the first and last values after sorting are selected as the minimum reflection intensity difference value and the maximum reflection intensity difference value respectively. Then, these two extreme values are encapsulated into a data structure of the reflection intensity difference distribution range in a fixed format (such as "minimum value X, maximum value Y"). Finally, this distribution range is stored in the memory to provide a reference parameter for subsequent relative difference calculation.

[0127] 402. Calculate the relative difference between the reflection intensity difference value and the minimum reflection intensity difference value of the reflection intensity difference distribution range according to the reflection intensity difference value corresponding to the position information of the target abnormal region;

[0128] In step 402, the relative difference amount refers to the ratio of the difference between the reflection intensity difference value of the target abnormal area and the minimum value of the distribution range to the entire distribution range (maximum value - minimum value). The calculation formula is (target value - minimum value) / (maximum value - minimum value), which is used to normalize the abnormality degree of the target area.

[0129] In the embodiment of the present application, first, according to the position information of the target abnormal area, the corresponding reflection intensity difference value is located from the abnormal distribution map. Secondly, the reflection intensity difference distribution range generated in step 401 is called, and the minimum reflection intensity difference value therein is read. Then, the difference between the difference value of the target area and the minimum value is calculated to obtain the relative difference amount (for example, if the target value is 80 and the minimum value is 20, the relative difference amount is 60). Finally, the relative difference amount is proportionally associated with the maximum span of the distribution range (that is, the maximum value minus the minimum value) to provide input parameters for transparency mapping.

[0130] 403. Map the relative difference amount to the initial transparency of the target abnormal area, where the initial transparency corresponding to the maximum reflection intensity difference value in the reflection intensity difference distribution range is the lowest initial transparency, and the initial transparency corresponding to the minimum reflection intensity difference value is the highest initial transparency.

[0131] In step 403, the initial transparency refers to the initial visible transparency value assigned to the target abnormal area according to the relative difference amount. The lower the value (such as 0%) means the less transparent, which is used to highlight the high-abnormality area. The lowest initial transparency corresponds to the maximum reflection intensity difference value (when the relative difference amount is 1), and the marking information is completely visible. The highest initial transparency corresponds to the minimum reflection intensity difference value (when the relative difference amount is 0), and the marking information is nearly completely transparent. The mapping method usually uses linear interpolation or a non-linear function (such as an exponential function) to convert the relative difference amount into a transparency interval (such as 0% - 100%).

[0132] In the embodiment of the present application, first, according to the relative difference amount calculated in step 402 and its proportional relationship with the maximum span of the distribution range, the proportional value is converted into a transparency parameter through a linear mapping algorithm. For example, if the relative difference amount accounts for 50% of the maximum span, the initial transparency is set to the intermediate value. Secondly, set the mapping rule: when the relative difference amount reaches the maximum span of the distribution range (that is, the target value is equal to the maximum value), the initial transparency is the preset lowest value (such as 10%); when the relative difference amount is zero (that is, the target value is equal to the minimum value), the initial transparency is the highest value (such as 90%). Then, the transparency value is dynamically adjusted according to the real-time relative difference amount of the target area. Finally, the transparency parameter is bound to the display mark of the target abnormal area to complete the real-time update of the visual effect.

[0133] The following is a specific example:

[0134] In the follow-up scenario of hypertension patients in a community hospital, a smart doctor-patient interaction system constructs a semantic dynamic evaluation model through natural language processing. When a patient complains that "recently, I feel dizzy like being on a boat, my left hand is occasionally numb, and it gets better after taking medicine", the semantic engine first extracts the set of abnormal features in the symptom description (corresponding to step 401), and makes a global comparison between abnormal semantic units such as "vertigo like being on a boat" and "intermittent limb numbness" and "slight dizziness" and "limbs without abnormalities" in the baseline healthy state to determine the maximum value (vertigo) and the minimum value (slight dizziness) of the semantic abnormality intensity. The system calculates the semantic deviation degree of the core abnormal point "vertigo like being on a boat" (step 402), generates a priority parameter according to the degree of deviation of the symptom from the normal description, and drives the interactive interface to set the initial transparency of the warning box corresponding to this symptom to the lowest (i.e., the most prominent). As the patient supplements the description "the vertigo is most severe in the morning", the system dynamically increases the semantic weight of this symptom (step 403), making the warning box gradually change from semi-transparent yellow to opaque red, and simultaneously generating a pop-up reminder containing "posterior circulation ischemia needs to be excluded" at the doctor's terminal. This dynamic prompt mechanism based on the semantic intensity distribution enables doctors to lock in the risk of vertebrobasilar artery insufficiency caused by blood pressure fluctuations within 3 minutes, improving the efficiency of traditional medical interviews by 50% and ensuring timely adjustment of antihypertensive regimens to prevent stroke.

[0135] In summary, steps 401 to 403 achieve the dynamic optimization presentation of the visualization effect of medical images. By establishing a mapping model between the reflection intensity difference value and the transparency parameter, the system can automatically adjust the display effect according to the optical feature differences in the lesion area. Using the gradual change rate control technology to transform the data change process into a smooth transition of visualization parameters enables patients to clearly perceive the dynamic development process of the lesion area. This technology effectively solves the contradiction problem of lost detailed information and visual interference in medical image presentation. Through an intelligent transparency adjustment algorithm, while ensuring the prominence of key information, it maintains the complete sense of layering of the image.

[0136] In some embodiments, as described in step 104, a drug action demonstration is generated based on the spatial structure features of the target abnormal area. The movement state of drug particles in the drug action demonstration is synchronized with the change in the user's viewing angle, and when the user watches the drug action demonstration for more than a preset duration, three-dimensional demonstration content including the operation and treatment of the affected area by the treatment device is displayed, including:

[0137] 501. Extract the surface depression depth parameter and the edge diffusion range parameter from the spatial structure features of the target abnormal area to generate the three-dimensional morphological parameters of the abnormal area;

[0138] In step 501, the surface depression depth parameter refers to the vertical depression distance of the target abnormal area extracted by 3D reconstruction technology relative to the normal skin surface, usually quantifying the depth of concave lesions such as ulcers and scars in millimeters. The edge diffusion range parameter refers to the radiation distance of the boundary of the abnormal area expanding outward, used to characterize the lateral range of inflammation or infection spread. The 3D morphological parameter refers to the vector composed of the depression depth and the diffusion range, describing the spatial geometric characteristics of the abnormal area (such as bowl-shaped, crater-shaped).

[0139] In the embodiment of the present application, first, the system obtains the spatial structure data of the target abnormal area through the 3D scanning module, measures the vertical height difference of the surface depression area using a depth sensor, and extracts the surface depression depth parameter (such as a maximum depth of 2 millimeters). Secondly, call the edge detection algorithm to analyze the boundary transition data between the abnormal area and the normal skin, and calculate the diffusion range parameter (such as an outward diffusion radius of 5 millimeters). Then, integrate the depth parameter and the diffusion range parameter in a 3D coordinate system to generate a 3D morphological parameter dataset describing the three-dimensional shape of the abnormal area. Finally, store this dataset as the input benchmark for subsequent drug path planning and treatment demonstration.

[0140] 502. Calculate the initial movement path of the drug particles in the abnormal area according to the surface depression depth parameter in the 3D morphological parameter, and adjust the coverage density of the movement path according to the edge diffusion range parameter;

[0141] In step 502, the initial movement path refers to the virtual movement trajectory of the drug particles generated according to the surface depression depth (such as a spiral descent path). The greater the depth, the higher the path length and curvature, simulating the need for the drug to penetrate deeper. The coverage density refers to the distribution quantity of drug particles per unit area. The greater the edge diffusion range, the lower the density, to achieve wide-area coverage of the diffusion area.

[0142] In the embodiment of the present application, first, determine the initial movement path of the drug particles according to the surface depression depth parameter. The greater the depth, the greater the downward tilt angle of the path, and simulate the gravity settlement trajectory of the particles in the concave structure through a physics engine. Secondly, divide the area grid based on the edge diffusion range parameter, and calculate the particle coverage density of each grid unit (such as for every 1 millimeter increase in the diffusion range, the density increases by 10%). Then, inject the density parameter into the particle system to dynamically adjust the distribution quantity of the particles in the diffusion edge area. Finally, generate a movement path network with a density gradient to ensure that the distribution of the particles in the core depression area and the diffusion edge area meets the treatment requirements.

[0143] 503. Real-time obtain the direction sensor data of the user's viewing angle change, convert the direction sensor data into the deflection angle of the drug particle movement path, and control the drug particles to move along the deflected path;

[0144] In step 503, the direction sensor data refers to the real-time attitude angles (such as pitch angle, yaw angle) of the user's handheld device obtained through a gyroscope or an accelerometer. The deflection angle refers to the offset of the drug particle path dynamically calculated based on the sensor data (such as deflecting 15° to the left), ensuring that the particle movement direction is consistent with the user's perspective.

[0145] In the embodiment of the present application, first, the direction sensor data of the user's viewing angle change is collected in real time through the built-in gyroscope of the device (such as rotating 15 degrees horizontally and tilting 10 degrees vertically). Secondly, the angle data is converted into a direction vector in three-dimensional space, and the deflection angle of the drug particle movement path is calculated through a coordinate transformation algorithm (such as the horizontal rotation corresponding to the X-axis offset and the vertical tilt corresponding to the Y-axis offset). Then, in the particle system, the movement direction vector of the particle is dynamically modified to make it move along the new angle trajectory, and the particle position update animation is rendered in real time. Finally, the adjusted path coordinates are continuously output to maintain the synchronization between the user's perspective and the particle movement direction.

[0146] 504. Start a timing device during the movement of the drug particle movement path. When the cumulative duration of the timing device exceeds a preset duration, trigger a three-dimensional demonstration operation of the treatment device;

[0147] In step 504, the timing device refers to a cumulative time module that records the continuous time of the user's continuous viewing of the drug particle movement animation. The preset duration refers to the threshold time for triggering the treatment demonstration (such as 30 seconds), which is used to ensure that the user fully understands the drug effect before showing the treatment process. The three-dimensional demonstration operation refers to a virtual treatment process that simulates the treatment device (such as a laser probe) through particle effects or robotic arm animations.

[0148] In the embodiment of the present application, first, start the timing device when the particle starts to move, and accumulate the continuous duration of the user's viewing of the demonstration. When the duration exceeds the preset threshold (such as 30 seconds), trigger a three-dimensional treatment scene loading instruction. Secondly, generate the operation depth of the treatment device (such as a microneedle) according to the surface depression depth parameter, and set the treatment coverage area in combination with the edge diffusion range parameter (such as the needle movement radius matching the diffusion range). Then, generate the device operation trajectory (such as the microneedle spirally probing down in the depression area) through the animation engine, and control the trajectory to coincide with the high-density area of the particle movement path for display. Finally, superimpose the dual visualization effect of the particle path and the treatment trajectory on the user terminal to complete the collaborative demonstration of the treatment process.

[0149] 505. In the three-dimensional demonstration operation, generate an operation trajectory of the treatment device in the abnormal area according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional form parameters, and control the operation trajectory to coincide with the coverage density area of the drug particle movement path for display.

[0150] In step 505, the operation trajectory refers to the movement route planned by the treatment device in the three-dimensional model (such as a circular scanning trajectory), and its depth and radius are controlled by the surface depression depth and the edge diffusion range parameters respectively. The coincidence display of the coverage density area refers to superimposing the treatment trajectory on the high-density coverage area of the drug particles to visually demonstrate the synergistic effect of "drug targeting + device treatment".

[0151] In the embodiments of the present application, first, the operation depth level of the treatment device is determined according to the surface depression depth parameter (such as the depth value determines the penetration distance of the microneedles), and the treatment coverage radius is set in combination with the edge diffusion range parameter. Secondly, the movement path of the treatment device is planned in the three-dimensional model (such as spirally expanding outward from the center of the depression) to ensure that the path completely covers the high-density area where the particles move. Then, the dynamic collision detection algorithm is used to avoid the conflict between the device operation trajectory and the particle path, and the trajectory direction is adjusted to achieve seamless superposition. Finally, the treatment trajectory is rendered in the display device with a semi-transparent highlighting effect and played synchronously with the drug particle path to visually demonstrate the synergistic effect of treatment and drug action.

[0152] The following is a specific example:

[0153] In the follow-up visit scenario of asthma patients in primary hospitals, a certain intelligent doctor-patient interaction system constructs a dynamic semantic topology model through natural language processing technology. When the patient complains that "I can't catch my breath at night recently and it takes half an hour to recover after taking the medicine", the semantic analysis module first extracts core symptom entities such as "nocturnal attack" and "delayed drug effect" from the conversation flow (corresponding to step 501), and analyzes their semantic depth (symptom severity) and the associated diffusion range (complication risk). The system generates an initial interrogation path according to the symptom depth, focusing on the possibility of drug use compliance and inhalation technique defects (step 502). When the patient mentions that "I always feel that I haven't inhaled to the bottom when taking the medicine", the semantic tracking of the direction sensor immediately captures the potential signal of operation error and turns the conversation focus to the technical guidance module (step 503). Continuously monitoring that the patient has not mentioned the key symptom changes for more than 90 seconds in the description (step 504), the intelligent agent automatically triggers a three-dimensional demonstration instruction, generating a comparison atlas containing the standard inhalation action decomposition animation and the deviation points of the patient's self-report (step 505), in which the wrong operation nodes and the correct operation trajectory are highlighted and overlapped, helping the doctor lock in the problem of insufficient drug deposition caused by the inhalation angle deviation of the patient within 5 minutes, and synchronously generating a correction training plan with voice explanation, so that the nocturnal attack frequency of the patient is reduced by 70% in the next follow-up visit cycle.

[0154] In summary, steps 501 to 505 achieve the three-dimensional dynamic demonstration and interactive operation of the treatment process. Based on the three-dimensional morphological parameters of the lesion, the system creates a spatially immersive treatment visualization effect by collecting the user's viewing angle data in real time. The personalized demonstration plan generated by combining the surface depression depth and edge diffusion range parameters enables patients to intuitively understand the drug action mechanism. When the demonstration duration reaches the preset value, it automatically switches to the operation demonstration of the treatment device, forming a full-process visualization explanation system from drug treatment to physical treatment, significantly improving the patient's acceptance of complex treatment plans.

[0155] In some embodiments, as described in step 503, the direction sensor data includes a horizontal rotation angle and a vertical rotation angle;

[0156] The converting the direction sensor data into the deflection angle of the drug particle movement path and controlling the drug particle to move along the deflected path includes:

[0157] 601. Convert the horizontal rotation angle into the horizontal offset angle of the drug particle movement path, and convert the vertical rotation angle into the vertical offset angle of the drug particle movement path to generate the deflection angle of the drug particle movement path;

[0158] In step 601, the horizontal rotation angle refers to the angle value of the device rotating around the vertical axis (Z axis) obtained by a direction sensor (such as a gyroscope) (for example, 30° left turn or 45° right turn). The vertical rotation angle refers to the angle value of the device rotating around the horizontal axis (X axis or Y axis) (for example, 20° upward view or 15° downward view). The horizontal offset angle refers to converting the horizontal rotation angle into the offset amount of the drug particle movement path in the horizontal direction (left and right) according to a preset ratio (for example, a 30° rotation corresponds to a 6-pixel right offset of the path). The vertical offset angle refers to converting the vertical rotation angle into the offset amount of the path in the vertical direction (up and down) (for example, a 15° downward angle corresponds to a 3-pixel downward offset of the path). The deflection angle refers to a two-dimensional vector synthesized by the horizontal offset angle and the vertical offset angle (for example, [6, -3]).

[0159] In the embodiment of the present application, first, the system obtains the horizontal rotation angle and vertical rotation angle data of the device in real time through a direction sensor (such as a gyroscope). Secondly, a linear mapping algorithm is used to convert the horizontal rotation angle into the horizontal offset angle of the drug particle movement path according to a ratio (for example, a 30-degree left rotation of the device corresponds to a 15-degree left deflection of the particle path). Then, the vertical rotation angle is converted into the vertical offset angle in the same way to ensure a one-to-one correspondence between the tilting action of the device and the movement direction of the particles. Finally, the horizontal and vertical offset angles are combined into a deflection angle parameter in three-dimensional space to generate a complete direction description of the particle movement path, providing an input for path adjustment.

[0160] 602. Adjust the moving direction of the drug particle movement path according to the horizontal offset angle and the vertical offset angle in the deflection angle, generate a deflected drug particle movement path, and control the drug particles to move along the deflected drug particle movement path;

[0161] In step 602, the adjustment of the moving direction refers to modifying the initial movement direction vector of the drug particles according to the deflection angle (for example, the original direction [0,1] becomes [6, -3]). The deflected drug particle movement path refers to the updated particle movement trajectory (for example, moving 9 pixels downward to the right per second). The control of particle movement refers to the process of driving the particles to move along the new path through an animation engine or a physics engine.

[0162] In the embodiments of the present application, first, based on the deflection angle parameters generated in step 601, call a path generation algorithm to recalculate the three-dimensional movement trajectory of the drug particles. For example, the original path is straight forward, and the horizontal offset angle makes it tilt downward to the left. Secondly, dynamically modify the movement direction vector of the particles through a particle system, and inject the new path parameters into the physics engine to simulate the movement of the particles. Then, in the display device, render the animation effect of the particles moving along the new path in real time to ensure its synchronization with the device rotation action. Finally, continuously output the updated particle position coordinates to provide a data stream for subsequent dynamic monitoring.

[0163] The method further includes:

[0164] 603. During the process of controlling the drug particles to move along the deflected drug particle movement path, real-time monitor the change rate of the direction sensor data, and synchronously adjust the update frequency of the deflection angle according to the change rate;

[0165] In step 603, the change rate of the direction sensor data refers to the change amount of the device attitude angle (such as the horizontal / vertical rotation angle) per unit time (for example, deflecting 5° per second). The update frequency refers to the deflection angle calculation frequency dynamically adjusted according to the change rate (for example, updating 30 times per second at high speed changes).

[0166] In the embodiments of the present application, first, during the movement of the particles, real-time collect the change rate of the direction sensor data (such as the angle change amount per second), and calculate the rotation acceleration of the current device through a time series difference algorithm. Secondly, dynamically adjust the update frequency of the deflection angle according to the change rate, increasing the update frequency (such as 60 frames per second) when the device rotates rapidly and decreasing the frequency (such as 30 frames per second) when it rotates slowly. Then, synchronize the adjusted frequency parameters to the path generation module to ensure that the response delay between the movement direction of the particles and the device action is controllable. Finally, continuously loop the monitoring and frequency adaptation process to maintain the smoothness and accuracy of the movement demonstration.

[0167] 604. When the change rate of the direction sensor data is lower than the preset threshold, lock the movement path of the drug particles corresponding to the current deflection angle, and update the particle position in the display device based on the deflected movement path of the drug particles.

[0168] In step 604, the preset threshold refers to the critical change rate value for determining whether the user's perspective tends to be static (for example, less than 1° change per second). Locking the deflection angle means stopping responding to the change of sensor data and fixing the movement path of the particles to the current direction. Updating the particle position refers to the process of continuously refreshing the particle position in the display device based on the locked path.

[0169] In the embodiment of the present application, first, when the change rate of the direction sensor data continuously is lower than the preset threshold (such as less than 5 degrees of angle change per second), it is determined that the device enters a stable state. Second, immediately freeze the current deflection angle parameter, stop the path update calculation, and lock the movement trajectory of the particles. Then, according to the finally locked path parameters, fix the particle position in the display device through the rendering engine and highlight the path end point. Finally, overlay the prompt information of "Path locked" on the interface to complete the full process switch from dynamic adjustment to static display.

[0170] The following is a specific example:

[0171] In the scenario of guiding asthma patients' inhalation treatment in a community hospital, a certain intelligent doctor-patient interaction system constructs a dynamic semantic navigation model through natural language processing. When the patient complains that "I always feel the medicine powder stuck in my throat when inhaling", the semantic parsing engine converts the "throat choking feeling" described by the patient into a horizontal deviation parameter of drug delivery (corresponding to step 601), maps "weak inhalation" to a vertical path deviation parameter, and generates an interaction path with inhalation angle calibration as the core. The system then adjusts the three-dimensional trajectory of the virtual demonstration (step 602), and dynamically displays the orange guiding line of the standard inhalation action at a 30-degree elevation angle on the interface. When the patient repeatedly asks "How high should I raise my head when inhaling", resulting in an increased fluctuation of the semantic flow, the system immediately increases the trajectory fine-tuning frequency to 3 times per second, and helps the patient understand by flashing the green comparison paths at different elevation angles frequently (step 603). When the patient stares at the screen for more than 5 seconds without asking questions, the intelligent agent automatically locks the optimal 45-degree elevation angle path, superimposes the golden steady-state trajectory on the patient's actual medicine holder in the augmented reality glasses (step 604), and simultaneously generates a voice prompt of "Slightly raise the lower jaw when inhaling deeply". This semantic-action linkage guiding mechanism enables the patient to achieve a drug deposition rate of 80% in the lungs for the first time, and improves the operation accuracy by 3 times compared with the traditional education method.

[0172] In summary, steps 601 to 604 achieve the intelligent adaptation of three-dimensional presentation content to user interaction behaviors. By converting device orientation sensor data into particle motion path parameters in real time, the system creates an immersive interactive viewing experience. The dynamic frequency adjustment technology is adopted to ensure that the presentation content is kept in real-time synchronization with user operations. When it is detected that the user stops operating, the current viewing angle is automatically locked to ensure the stable presentation of key treatment scenarios. This technology breaks through the one-way playback mode of traditional three-dimensional presentations and significantly improves the interactivity and memory retention effect of medical knowledge dissemination by establishing a two-way data channel between user behaviors and presentation content.

[0173] In some embodiments, as described in step 101, receiving the voice information input by the user, where the voice information includes symptom description information of the user's skin abnormality features, parsing the voice information to generate semantic feature data, and matching the semantic feature data with the corresponding disease feature database, and sending a lesion location guidance information including medical guidance to the user terminal according to the matching result, includes:

[0174] 701. Receiving the voice information input by the user, and extracting a keyword set describing the skin abnormality features in the voice information, where the keyword set includes abnormal symptom description words such as redness, desquamation, and itching and corresponding symptom location description words;

[0175] In step 701, the voice information refers to the voice content describing skin problems input by the user through the terminal device. The keyword set refers to the abnormal symptom description words (such as redness, desquamation, itching) and the corresponding symptom location description words (such as "inner side of the arm", "middle of the forehead") extracted from the voice information. The abnormal symptom description words refer to the standardized words used to characterize the clinical manifestations of skin abnormalities. The symptom location description words refer to the text information describing the specific location of the abnormal symptom on the body surface.

[0176] In the embodiment of the present application, first, the system receives the user's voice input through the microphone of the terminal device, and calls the speech recognition engine to convert the speech into text information. Secondly, the text is parsed by using the word segmentation algorithm of natural language processing to screen out the keywords (such as "redness", "desquamation", "itching") describing the skin abnormality features and the corresponding location description words (such as "inner side of the arm", "face"). Then, irrelevant words are removed through part-of-speech tagging and semantic disambiguation technologies, and the keyword set of symptoms and locations is retained. Finally, the keyword set is classified and stored according to the symptom type and location attribute to provide a structured input for subsequent disease matching.

[0177] 702. Performing a weighted match on the abnormal symptom description words in the keyword set with the symptom relevance in the preset skin disease feature database to generate semantic feature data including symptom association weights;

[0178] In step 702, symptom relevance weight matching refers to the process of assigning weight values by calculating the semantic similarity (such as cosine similarity) or probability model (such as Bayesian network) between keywords and symptoms in the skin disease feature database. Semantic feature data refers to structured data (such as vectors or key-value pairs) containing the association weight values between keywords and disease symptoms. The skin disease feature database refers to a standardized data set storing disease names, symptom lists, and medical guidelines.

[0179] In the embodiments of the present application, first, read the symptom relevance data of each disease from the preset skin disease feature database (for example, the weights of "itching" and "erythema" associated with eczema are 0.9). Secondly, compare the abnormal symptom description words in the keyword set item by item with the symptom list in the database, and calculate the association score between each keyword and the disease through the weight matching algorithm (for example, the weight of "desquamation" for psoriasis is 0.8). Then, accumulate the weight scores of all keywords to generate semantic feature data containing the disease name and the total weight. Finally, sort them in descending order of the weight value and screen out the potential disease list.

[0180] 703. According to the semantic feature data, screen out the target disease feature data with the highest association weight from the disease feature database, and extract the affected area location rule corresponding to the target disease feature data;

[0181] In step 703, the target disease feature data refers to the disease data corresponding to the highest weight screened out after sorting by the association weight (such as eczema, psoriasis). The affected area location rule refers to the common location and spread pattern of the affected area corresponding to the target disease (such as eczema mostly occurs in the flexor side of joints).

[0182] In the embodiments of the present application, first, select the target disease with the highest association weight from the sorted semantic feature data (such as the weight value of eczema is 0.95). Secondly, extract the affected area location rule corresponding to this disease from the disease feature database, including the typical onset location (such as "flexor side of joints") and spread characteristics (such as "clear edge"). Then, analyze the spatial description logic in the location rule (such as "symmetric distribution", "spread along hair follicles"), and convert it into executable coordinate calculation parameters. Finally, generate a location rule data set containing position priorities and range constraints to provide a basis for spatial mapping.

[0183] 704. Perform spatial mapping on the symptom location description words in the keyword set and the affected area location rule to generate location guidance information including the coordinate offset of the affected area and the skin surface spread range;

[0184] In step 704, spatial mapping refers to mapping the symptom location descriptor (such as "back of hand") to a specific coordinate area in the three-dimensional model (such as X=120, Y=80, Z=0) through a coordinate conversion algorithm. The affected area coordinate offset refers to the offset value of the target affected area center point relative to the body's default coordinate system (such as 20 pixels to the right). The skin surface diffusion range refers to the maximum radius of the abnormal area extending outward from the center point (such as 5 cm). Positioning guidance information refers to a set of guidance parameters including coordinate offset and diffusion range.

[0185] In an embodiment of the present application, first, the symptom location descriptor (such as "left cheek") in the keyword set is matched with the typical location in the positioning rule, and the coordinate offset of the actual affected part and the standard position is calculated by the spatial mapping algorithm (such as horizontal offset +5%, vertical offset -10%). Secondly, according to the diffusion characteristics in the positioning rule (such as "radius 3 cm") and the diffusion range described by the user (such as "diffusion to the earlobe"), the coverage area boundary of the affected part is dynamically adjusted. Then, the coordinate offset is combined with the diffusion range to generate positioning guidance information containing the center point coordinates and the coverage radius. Finally, the data is formatted into instruction parameters that can be recognized by the device and transmitted to the display control module.

[0186] 705. Generate an affected area positioning mark map adapted to the screen size according to the diffusion range in the positioning guidance information and the display device resolution of the user terminal, and send the affected area positioning guidance information including medical guidance to the user terminal.

[0187] In step 705, the display device resolution refers to the pixel size of the user terminal screen (e.g., 1920×1080). The screen-adapted affected area location marker image refers to a virtual marker (e.g., a circular highlighted area) that is scaled based on the diffusion range and adapted to the screen ratio. The location guidance information refers to composite data containing medical guidance text (e.g., "Please aim the device at your left arm") and a visual marker.

[0188] In an embodiment of the present application, first, based on the diffusion range in the positioning guidance information (such as a radius of 200 pixels) and the resolution of the terminal display device (such as 1920×1080), the image rendering engine is called to generate a proportionally scaled affected area positioning marker image (such as a circular marker covering the left cheek area). Secondly, the position of the marker image on the screen is adjusted by the coordinate offset to ensure that it is aligned with the actual affected area. Then, guiding elements such as dynamic arrows or highlighted borders are superimposed to generate a positioning marker image containing visual cues. Finally, the marker image and medical guidance text (such as "Please align with the left cheek marked area") are integrated into a positioning guidance information package, which is sent to the user through the terminal interface to complete the full process guidance.

[0189] Here's a specific example:

[0190] In the scenario of respiratory disease screening in primary hospitals, an intelligent doctor-patient interaction system constructs a symptom space mapping model through natural language processing technology. When the patient complains of "stabbing pain under the left chest during inhalation and coughing up rust-colored sputum", the speech recognition module extracts core symptom keywords such as "chest pain" and "rust-colored sputum" in real time (corresponding to step 701), and conducts multi-dimensional correlation analysis with the respiratory disease feature library. Among them, "rust-colored sputum" obtains the maximum semantic weight due to its high match with the characteristics of lobar pneumonia (step 702). After the system locks the pneumonia diagnosis path, it automatically loads the unique three-lobe lung localization model of this disease (step 703), maps the "under the left chest" described by the patient to the lower lobe lung projection area, and generates a red warning box including the costophrenic angle area in combination with the typical inflammation spread pattern (step 704). According to the characteristics of the mobile terminal screen, the intelligent agent intelligently zooms the lesion location marker to fit the interface, generates a pulsating high-light area at the bottom of the left lung of the virtual human model, and synchronously pushes the voice guidance of "priority percussion of the lower left lung" (step 705). The doctor quickly detects the dullness area based on this positioning guidance, and combines imaging examinations to confirm lobar pneumonia, shortening the diagnosis time by 40% compared with the traditional interrogation process, and achieving accurate guidance with a symptom description space positioning error of less than 3 cm.

[0191] In summary, steps 701 to 705 achieve the intelligent fusion application of voice interaction and medical diagnosis. By constructing a weight matching model of symptom keywords and disease characteristics, the system can quickly extract core diagnostic elements from natural language descriptions. Combining the lesion location guidance scheme generated by the space mapping algorithm, it transforms abstract symptom descriptions into specific space coordinate guidance, greatly improving the accuracy of patients' self-positioning. This technology effectively solves the pain point problem of difficult lesion location in telemedicine, and provides reliable technical support for intelligent triage services in primary medical scenarios by establishing a deep association between semantic features and medical knowledge.

[0192] Figure 2 The following is a schematic structural diagram of an intelligent agent interaction system for doctor-patient communication based on natural language processing provided by an embodiment of the present application, as Figure 2 shown. The system includes:

[0193] A receiving module 21, configured to receive voice information input by a user, where the voice information includes symptom description information of the user's abnormal features, parse the voice information to generate semantic feature data, match the semantic feature data with a corresponding disease feature database, and send lesion location guidance information including medical guidance to a user terminal according to a matching result;

[0194] A comparison module 22, configured to respond to the user's completion of the lesion alignment operation according to the medical guidance, activate an image acquisition device to synchronously obtain spectral reflection data while the user maintains the lesion alignment state, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area;

[0195] A conversion module 23, configured to control a display device to display marker information corresponding to an actual affected area according to the position information of a target abnormal area in the abnormal distribution map, and after the marker information reaches a preset display effect, convert the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information;

[0196] A display module 24, configured to generate a drug action demonstration based on the spatial structure characteristics of the target abnormal area, where the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle, and when the user watches the drug action demonstration for more than a preset duration, display three-dimensional demonstration content including the operation and treatment of the affected area by a treatment device.

[0197] Figure 2 The described intelligent agent interaction system for doctor-patient communication based on natural language processing can execute Figure 1 The described intelligent agent interaction method for doctor-patient communication based on natural language processing in the illustrated embodiment, and its implementation principle and technical effects will not be elaborated further. For the intelligent agent interaction system for doctor-patient communication based on natural language processing in the above embodiment, the specific ways in which each module and unit perform operations have been described in detail in the embodiment related to the method, and will not be elaborated here.

[0198] In a possible design, Figure 2 The intelligent agent interaction system for doctor-patient communication based on natural language processing in the illustrated embodiment can be implemented as a computing device, such as Figure 3 shown, and the computing device can include a storage component 31 and a processing component 32;

[0199] The storage component 31 stores one or more computer instructions, where the one or more computer instructions are called and executed by the processing component 32.

[0200] The processing component 32 is used for the Figure 1 intelligent agent interaction method for doctor-patient communication based on natural language processing in the above

[0201] embodiment. Among them, the processing component 32 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above method.

[0202] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0203] Of course, the computing device may also necessarily include other components, such as an input / output interface, a display component, a communication component, etc.

[0204] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above-mentioned peripheral interface module may be an output device, an input device, etc.

[0205] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0206] Among them, the computing device may be a physical device or an elastic computing host provided by a cloud computing platform, etc. At this time, the computing device may refer to a cloud server, and the above-mentioned processing component, storage component, etc. may be basic server resources leased or purchased from a cloud computing platform.

[0207] The embodiment of the present application also provides a computer storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the above-mentioned Figure 1 a method for intelligent agent interaction in doctor-patient communication based on natural language processing shown in the above embodiment.

[0208] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0209] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0210] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An intelligent agent interaction method for doctor-patient communication based on natural language processing, characterized in that, Applied to an agent, including: Receiving voice information input by a user, where the voice information includes symptom description information of the user's abnormal features, parsing the voice information to generate semantic feature data, and matching the semantic feature data with a corresponding disease feature database, and sending lesion location guidance information including medical guidance to the user terminal according to the matching result; Responding to the user's completion of the lesion alignment operation according to the medical guidance, activating an image acquisition device to synchronously acquire spectral reflection data while the user maintains the lesion alignment state, and generating an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area; According to the position information of the target abnormal area in the abnormal distribution map, controlling a display device to display marker information corresponding to the real lesion, and after the marker information reaches a preset display effect, converting the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information; Generating a drug action demonstration based on the spatial structure features of the target abnormal area, where the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle, and when the user watches the drug action demonstration for more than a preset duration, displaying three-dimensional demonstration content of a treatment device operating on the lesion; The controlling the display device to display marker information corresponding to the real lesion according to the position information of the target abnormal area in the abnormal distribution map, and after the marker information reaches a preset display effect, converting the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information, includes: Extracting the reflection intensity difference value of each area from the abnormal distribution map, screening out the area with the largest reflection intensity difference value as the target abnormal area, and recording the position information of the target abnormal area; Calculating the transparency of the marker information displayed corresponding to the real lesion in the display device according to the distribution range of the position information in the abnormal distribution map, where the maximum reflection intensity difference value of the distribution range corresponds to the lowest transparency; Controlling the marker information to gradually decrease the transparency at a rate of change of the reflection intensity difference value over time as the transparency gradient rate until the reflection intensity difference value of the target abnormal area stops increasing; Extracting the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining areas in the abnormal distribution map, converting the absolute difference into a percentage level according to the distribution range, and generating the degree of abnormality; Converting the position information of the target abnormal area into a percentage position on the screen of the display device, and splicing the percentage position with the degree of abnormality to generate voice commentary information including the screen position description of the target abnormal area and the abnormal level label corresponding to the degree of abnormality.

2. The method according to claim 1, wherein The responding to the user's completion of the lesion alignment operation according to the medical guidance, activating an image acquisition device to synchronously acquire spectral reflection data while the user maintains the lesion alignment state, and generating an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area, includes: After the user completes the alignment operation of the affected area according to the location marking diagram of the medical guidance, a spectral scanning instruction of the image acquisition device is triggered to obtain the spectral reflection data of the affected area in real time; Extract the spectral reflection intensity sequence of the abnormal area from the spectral reflection data, and synchronously extract the spectral reflection intensity sequence of the pre-calibrated normal area; Perform difference calculation on the reflection intensity sequence of the abnormal area and the reflection intensity sequence of the normal area one by one at the same wavelength points to generate a reflection intensity difference value; Draw a heat map of the reflection intensity difference of the abnormal area at different wavelengths according to the difference value of each wavelength point in the reflection intensity difference value; Perform spatial superposition on the wavelength points with the reflection intensity difference value exceeding the preset threshold in the reflection intensity difference heat map to generate an abnormal distribution map including the position and degree of the abnormal area.

3. The method according to claim 1, wherein Calculating the transparency of the marking information corresponding to the real affected area displayed in the display device according to the distribution range of the position information in the abnormal distribution map, including: Extract the numerical set of the reflection intensity difference values of all areas from the abnormal distribution map, and screen the maximum reflection intensity difference value and the minimum reflection intensity difference value in the numerical set to generate a reflection intensity difference distribution range; Calculate the relative difference amount between the reflection intensity difference value corresponding to the position information of the target abnormal area and the minimum reflection intensity difference value of the reflection intensity difference distribution range; Map the relative difference amount to the initial transparency of the target abnormal area, where the initial transparency corresponding to the maximum reflection intensity difference value of the reflection intensity difference distribution range is the lowest initial transparency, and the initial transparency corresponding to the minimum reflection intensity difference value is the highest initial transparency.

4. The method according to claim 1, wherein Generating a drug action demonstration based on the spatial structure characteristics of the target abnormal area, where the movement state of the drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle, and when the user watches the drug action demonstration for more than the preset duration, display three-dimensional demonstration content including the treatment device operating on the affected area, including: Extract the surface depression depth parameter and the edge diffusion range parameter from the spatial structure characteristics of the target abnormal area to generate the three-dimensional morphological parameters of the abnormal area; Calculate the initial movement path of the drug particles in the abnormal area according to the surface depression depth parameter in the three-dimensional morphological parameters, and adjust the coverage density of the movement path according to the edge diffusion range parameter; Obtain the direction sensor data of the change of the user's viewing angle in real time, convert the direction sensor data into the deflection angle of the movement path of the drug particles, and control the drug particles to move along the deflected path; Start a timing device during the movement of the movement path of the drug particles. When the cumulative duration of the timing device exceeds the preset duration, trigger the three-dimensional demonstration operation of the treatment device; In the three-dimensional demonstration operation, generate the operation trajectory of the treatment device in the abnormal area according to the surface depression depth parameter and the edge diffusion range parameter in the three-dimensional morphological parameters, and control the operation trajectory to be displayed in coincidence with the coverage density area of the movement path of the drug particles.

5. The method according to claim 4, wherein The direction sensor data includes a horizontal rotation angle and a vertical rotation angle; Converting the direction sensor data into a deflection angle of the movement path of the drug particles and controlling the drug particles to move along the deflected path includes: Converting the horizontal rotation angle into a horizontal offset angle of the movement path of the drug particles, and converting the vertical rotation angle into a vertical offset angle of the movement path of the drug particles, to generate a deflection angle of the movement path of the drug particles; According to the horizontal offset angle and the vertical offset angle in the deflection angle, adjusting the movement direction of the movement path of the drug particles, generating a deflected movement path of the drug particles, and controlling the drug particles to move along the deflected movement path of the drug particles; The method further includes: During the process of controlling the drug particles to move along the deflected movement path of the drug particles, the change rate of the direction sensor data is monitored in real time, and the update frequency of the deflection angle is synchronously adjusted according to the change rate; When the change rate of the direction sensor data is lower than a preset threshold, the movement path of the drug particles corresponding to the current deflection angle is locked, and the particle position is updated based on the deflected movement path of the drug particles in the display device.

6. The method according to claim 1, wherein Receiving the voice information input by the user, the voice information includes symptom description information of the user's abnormal characteristics, parsing the voice information to generate semantic feature data, and matching the semantic feature data with the corresponding disease feature database, and sending a lesion location guidance information including medical guidance to the user terminal according to the matching result, includes: Receiving the voice information input by the user, extracting a keyword set describing abnormal characteristics in the voice information, the keyword set includes abnormal symptom description words such as redness, desquamation, and itching and corresponding symptom location description words; Performing a weight matching on the abnormal symptom description words in the keyword set with the symptom relevance in the preset disease feature database to generate semantic feature data including symptom association weights; According to the semantic feature data, screening out the target disease feature data with the highest association weight from the disease feature database, and extracting the lesion location rule corresponding to the target disease feature data; Performing a spatial mapping on the symptom location description words in the keyword set with the lesion location rule to generate a location guidance information including a lesion coordinate offset and a surface diffusion range; According to the diffusion range in the location guidance information and the display device resolution of the user terminal, generating a lesion location marker map adapted to the screen size, and sending a lesion location guidance information including medical guidance to the user terminal.

7. An intelligent agent interaction system for doctor-patient communication based on natural language processing, characterized in that, Applied to an intelligent agent, includes: A receiving module, configured to receive the voice information input by the user, the voice information includes symptom description information of the user's abnormal characteristics, parse the voice information to generate semantic feature data, and match the semantic feature data with the corresponding disease feature database, and send a lesion location guidance information including medical guidance to the user terminal according to the matching result; A comparison module, which is used to respond to the user's operation of aligning the affected area according to the medical guidance, activate the image acquisition device to synchronously obtain spectral reflection data while the user keeps the affected area aligned, and generate an abnormal distribution map by comparing the reflection intensity differences between the abnormal area and the normal area; A conversion module, which is used to control the display device to display the marker information corresponding to the real affected area according to the position information of the target abnormal area in the abnormal distribution map, and after the marker information reaches the preset display effect, convert the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information; A display module, which is used to generate a drug action demonstration based on the spatial structure characteristics of the target abnormal area, and the movement state of drug particles in the drug action demonstration is synchronized with the change of the user's viewing angle. When the user watches the drug action demonstration for more than the preset duration, display three-dimensional demonstration content of the treatment device operating on the affected area; The controlling the display device to display the marker information corresponding to the real affected area according to the position information of the target abnormal area in the abnormal distribution map, and after the marker information reaches the preset display effect, converting the corresponding relationship between the reflection intensity difference and the degree of abnormality into voice commentary information includes: Extracting the reflection intensity difference value of each area from the abnormal distribution map, screening out the area with the largest reflection intensity difference value as the target abnormal area, and recording the position information of the target abnormal area; Calculating the transparency of the marker information corresponding to the real affected area displayed on the display device according to the distribution range of the position information in the abnormal distribution map, and the maximum reflection intensity difference value of the distribution range corresponds to the lowest transparency; Controlling the marker information to continuously reduce the transparency at a gradual change rate of the reflection intensity difference value over time until the reflection intensity difference value of the target abnormal area stops increasing; Extracting the absolute difference between the reflection intensity difference value of the target abnormal area and the average reflection intensity difference value of the remaining areas in the abnormal distribution map, converting the absolute difference into a percentage level according to the distribution range, and generating the degree of abnormality; Converting the position information of the target abnormal area into a percentage position on the display device screen, and splicing the percentage position with the degree of abnormality to generate voice commentary information including the screen position description of the target abnormal area and the abnormal level label corresponding to the degree of abnormality.

8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a natural language processing-based doctor-patient communication intelligent agent interaction method as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that, There is a computer program stored, and when the computer program is executed by the computer, it implements a natural language processing-based doctor-patient communication intelligent agent interaction method as described in any one of claims 1 to 6.

Images