Medical inspection robot control method and system

Abstract

This invention discloses a control method and system for a medical inspection robot, relating to the field of virtual reality technology. The method includes: receiving medical inspection task information; collecting first-view environmental information along the inspection path in an autonomous inspection state; calculating a risk score for inspection events based on the first-view environmental information and using this score to calculate an interaction trigger score; determining whether a target is lost based on occlusion probability and overall following confidence during target tracking; and re-capturing a target by selecting an information gain viewpoint from a set of candidate viewpoints to improve visibility probability; performing remote control or two-way dialogue in a remote control or dialogue takeover state, and triggering an emergency response state when emergency stop conditions are met. The system provides a multi-state collaborative mechanism for autonomous inspection, remote control, dialogue takeover, and emergency response, maintains target identity consistency under occlusion conditions, and achieves highly reliable re-capturing after target loss.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to a control method and system for a medical inspection robot. Background Technology

[0002] In their daily operations, medical institutions need to conduct continuous inspections and rounds of wards, corridors, and public areas to obtain information on patient conditions and the environment, and to promptly address any abnormalities. With the development of mobile robot technology, some medical institutions have introduced inspection robots to assist in patrols or information collection.

[0003] First, existing medical inspection robots mostly employ remote control methods based on two-dimensional displays or ordinary videos. This makes it difficult for medical staff to obtain an immersive first-person perspective that aligns with the robot's movement direction, limiting their ability to observe and assess complex ward environments. Furthermore, existing medical inspection robots typically only support a single autonomous inspection or simple remote control mode, lacking an efficient collaborative mechanism for smoothly switching from autonomous inspection to remote takeover in the event of sudden risks. They also lack human-computer interaction capabilities during inspections, making it difficult to complete basic communication or information acquisition during the process.

[0004] Secondly, in scenarios such as medical care and inspection escort, mobile robots need to stably follow designated personnel in order to maintain human-machine collaboration efficiency and safe distance. However, existing human tracking technology is easily affected by the following factors in real environment: (1) pedestrians are blocked by others, door frames, etc., causing the target to be temporarily or continuously invisible; (2) personnel intersection / similar clothing / similar body type, etc., cause identity confusion and lead to the problem of switching people to follow; (3) after the target temporarily leaves the field of vision or turns into a corner, the mobile robot cannot effectively recapture it, causing the following task to be interrupted.

[0005] Existing technologies often employ detection + tracking + Person Re-identification (ReID), which passively waits or performs simple rotational searches when the target is occluded or leaves the field of view. These methods lack explicit modeling of the occlusion structure, quantifiable visibility / confidence constraints, and an active re-capture mechanism that combines maps and uncertainty. Consequently, in complex scenarios, it is difficult to simultaneously meet the requirements of target retention during occlusion, high-confidence recovery after occlusion removal, and minimizing the need for person switching. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a medical inspection robot control method and system, providing a multi-state collaborative mechanism for autonomous inspection, remote control, dialogue takeover, and emergency response. This solves problems such as insufficient immersion during remote control and low efficiency of collaboration between autonomous inspection and manual takeover. Furthermore, during personnel following, it can maintain the consistency of the target's identity under occlusion conditions and achieve highly reliable recapture after the target is lost.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a control method for a medical inspection robot, comprising:

[0009] Receive medical inspection task information, including inspection path and inspection mode status;

[0010] In autonomous inspection mode, first-person perspective environmental information is collected along the inspection path. Based on the first-person perspective environmental information, a risk score is calculated for the inspection event. Based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision, an interaction trigger score is calculated.

[0011] When the interaction trigger score is greater than or equal to the trigger threshold, the interaction response operation is executed;

[0012] In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint.

[0013] In remote control or dialogue takeover mode, it performs remote control according to the received motion control command, or conducts two-way dialogue and collaboration with the virtual reality display terminal according to the received dialogue takeover command, and triggers emergency response mode when the set emergency stop conditions are met.

[0014] As an alternative implementation, after encoding the first-view environment information into a media stream, it is sent to a virtual reality display terminal via a server for panoramic rendering and display, and the virtual reality display terminal performs attitude alignment compensation on the panoramic rendering viewpoint; wherein, the virtual reality display terminal is used as the target viewpoint for rendering the panoramic image. for: ; This is the initial viewpoint of the virtual reality display. For mobile robots in The heading angle at any given moment; This is the initial heading angle.

[0015] As an alternative implementation method, risk scoring The calculation is as follows: ;in, For the Sigmoid function, ; To extract feature vectors from people's facial expressions, voice, and behavior based on first-person perspective environmental information; This is the weight vector; For bias;

[0016] when Or in the duration window Internal satisfaction The cumulative duration is not less than When the time comes, a prompt message is generated and reported to the server;

[0017] when Or in the duration window Internal satisfaction The cumulative duration is not less than At that time, an alarm message is generated and reported to the server, and the server allows access to the takeover link;

[0018] in, The first risk threshold, The second risk threshold, The first time threshold, The second time threshold, , Less than or equal to .

[0019] As an alternative implementation method, interactive triggering scoring for:

[0020] ;

[0021] in, These are the weighting coefficients. For the desired interaction distance, Target stay time; Assess risk level; The distance between the mobile robot and the target person; The duration of time a target person remains within the field of vision; This is the amplitude limiting function.

[0022] As an alternative implementation, light is projected onto the line of sight between the predicted positions of the mobile robot and the target person, the occlusion ratio is statistically analyzed, and the target visibility probability is calculated. and occlusion probability : ; ; This represents the proportion of the line-of-sight sampling points that are obscured by obstacles. This is the proportionality coefficient.

[0023] As an alternative implementation, when the overall follow confidence is less than the loss threshold and the cumulative duration of continuous unreliable observation is greater than or equal to a first duration threshold; or when the occlusion probability is greater than the occlusion threshold and the cumulative duration of continuous target invisibility is greater than or equal to a second duration threshold, the target is determined to be lost and switched to the reacquisition state; wherein, when the overall follow confidence is less than the association threshold, it is determined to be unreliable observation, and when the occlusion probability is greater than the occlusion threshold, it is determined to be target invisibility.

[0024] As an optional implementation method, the following process for following the target personnel also includes: constructing a target file that includes the target personnel's long-term identity memory and short-term motion state memory; determining the joint confidence score of each candidate personnel based on the identity consistency score, geometric consistency score, and motion consistency score of the target personnel and the candidate personnel; and using the candidate personnel with the highest joint confidence score as the observation target at the current moment.

[0025] If the target visibility probability is greater than or equal to the visibility threshold, and the overall follow confidence is greater than or equal to the update threshold, then the long-term identity memory and short-term motion state in the target file are updated; wherein, the update of the long-term identity memory is as follows: ; As weight; For long-term identity memory of the target personnel at time t+1; For long-term identity memory of the target personnel at time t; For the target observed at the current time t Long-term identity memory;

[0026] When the target visibility probability is less than the visibility threshold, or the overall follow confidence is less than the update threshold, the update of long-term identity memory in the target file is frozen, and only the short-term motion state memory in the target file is updated.

[0027] As an alternative implementation method, the recapture process includes: generating a candidate viewpoint set based on the constructed target prediction distribution and map occlusion structure. Rate each viewpoint: ;

[0028] in, It is obtained by sampling the target prediction distribution, that is: obtained by sampling from the target prediction distribution. Target location samples For each target location sample, viewpoint Starting from, with To obtain the occlusion percentage, light is projected onto the occlusion structure of the map at the endpoint. And calculate the sample visibility probability: ;

[0029] but , Visibility attenuation coefficient; To reach the viewpoint The path cost is at least one or a weighted combination of path length, time consumption, or turning cost. For the weighting factor;

[0030] Select from the candidate viewpoint set and planned to Recapture the path after it is found.

[0031] As an alternative implementation method, in remote control mode, the motion control command is: , Forward / backward amount, Let be the turning radius, then at time t, the target linear velocity of the mobile robot. With the target angular velocity They are respectively:

[0032] ; ;

[0033] For target linear velocity With the target angular velocity Perform first-order smoothing filtering separately:

[0034] ; ;

[0035] in, This is the proportionality coefficient. These are the maximum linear velocity and the maximum angular velocity. This is a limiting function; For smoothing coefficients, These are the filtered target linear velocity and target angular velocity;

[0036] Let the distance to the nearest obstacle in front of the mobile robot be... The deceleration threshold is The parking threshold is ,and Then the speed scaling factor for: ;right Perform safe scaling as follows: ;

[0037] When satisfied When an emergency response state is triggered, it will revert to remote control or autonomous inspection state after the preset conditions for termination are met.

[0038] In a second aspect, the present invention provides a medical inspection robot control system, comprising:

[0039] The receiving module is configured to receive medical inspection task information, including inspection path and inspection mode status.

[0040] The evaluation module is configured to collect first-person perspective environmental information along the inspection path in autonomous inspection mode, score the risk of inspection events based on the first-person perspective environmental information, and calculate the interaction trigger score based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision.

[0041] The interaction module is configured to execute an interaction response operation when the interaction trigger score is greater than or equal to the trigger threshold.

[0042] In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint.

[0043] The remote module is configured to perform remote control based on received motion control commands in remote control or dialogue takeover mode, or to conduct two-way dialogue and collaboration with the virtual reality display terminal based on received dialogue takeover commands, and to trigger an emergency response state when the set emergency stop conditions are met.

[0044] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0045] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0046] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] To address the issues of insufficient immersion and low efficiency in the collaboration between autonomous robot inspection and manual intervention during remote control of existing medical inspection robots, this invention proposes a control method and system for a medical inspection robot that supports panoramic virtual reality remote collaboration. By capturing panoramic audio and video on the mobile robot and forwarding it in real time via a server, medical staff can view an immersive scene consistent with the mobile robot's perspective on a virtual reality display. They can also remotely control the robot, view inspection events and dialogue content at any time. Furthermore, the system incorporates multimodal perception and emotion assessment of personnel during the inspection process, and performs proactive interaction and alarm pushes when trigger conditions are met. It also supports medical staff intervention and intervention in the robot's dialogue with patients, enhancing the robot's interactive capabilities, care capabilities, and information acquisition capabilities in medical inspection scenarios, thereby improving hospital inspection efficiency and environmental awareness during remote inspections.

[0049] This invention provides a multi-state collaborative mechanism for autonomous inspection, remote control, dialogue takeover, and emergency response. Through server-side permission verification and session arbitration, it achieves multi-state switching and priority control, enabling on-demand intervention and takeover of robot movement and dialogue. This improves the efficiency of handling emergencies and human-machine collaboration, and provides unified storage and traceability of inspection logs, alarm records, and event replays. This enhances the sense of presence, anomaly detection efficiency, and emergency response capabilities of remote inspections in medical scenarios, thereby improving human-machine collaboration and interactive experience.

[0050] To address the issues of easy loss, easy switching of followers, and low recapture efficiency in human tracking scenarios with occlusion / intersection, this invention proposes an occlusion-robust human tracking target maintenance and recapture method. The method incorporates occlusion-aware visibility probability estimation, dual-layer memory for identity and trajectory maintenance, a confidence-driven state machine, and an information gain-based active recapture planning technique. The visibility probability estimation and confidence-driven state machine improve stability in occlusion / corner scenarios; dual-layer memory and a controlled update strategy reduce the probability of identity drift and follower switching during occlusion; and active recapture planning based on uncertainty and map occlusion structure improves recapture success rate and efficiency. Finally, safety constraints and interpretable output enhance the security and deployability of the tracking process. This achieves target maintenance during occlusion and high-confidence recapture after occlusion removal, reducing the probability of follower switching and improving task continuity and security.

[0051] This invention establishes a target profile on a mobile robot, including long-term identity memory and short-term motion state memory. A candidate set is generated based on images, depth information, laser point clouds, or combinations thereof. Joint confidence is calculated by fusing identity similarity, geometric consistency, and motion consistency to achieve target association. Furthermore, based on depth / point cloud and map data, occlusion is assessed on the robot's line of sight to the predicted target location. The target visibility probability is calculated, and identity memory is updated in a controlled manner under visible and high-confidence conditions. Under occlusion or low-confidence conditions, identity updates are frozen, and motion prediction is performed only to suppress driver-switching drift. When a target is lost, active recapture viewpoint / path planning is performed based on target uncertainty and map occlusion structure. High-confidence recapture and resumption of following are achieved when joint threshold conditions are met. This improves the continuity and reliability of human tracking in complex occlusion scenarios, reduces the probability of driver-switching, and enhances recapture efficiency and safety.

[0052] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 This is an overall framework diagram of the medical inspection robot control method provided in Embodiment 1 of the present invention;

[0055] Figure 2 This is an overall flowchart of the medical inspection robot control method provided in Embodiment 1 of the present invention;

[0056] Figure 3 This is a flowchart of the inspection mode state switching provided in Embodiment 1 of the present invention;

[0057] Figure 4 This is an interactive triggering flowchart provided in Embodiment 1 of the present invention;

[0058] Figure 5 This is a flowchart of the personnel following method provided in Embodiment 1 of the present invention;

[0059] Figure 6 This is a flowchart of the target file update process provided in Embodiment 1 of the present invention;

[0060] Figure 7 This is a flowchart of visibility probability estimation provided in Embodiment 1 of the present invention;

[0061] Figure 8 This is a flowchart of the recapture process provided in Embodiment 1 of the present invention;

[0062] Figure 9 This is a schematic diagram illustrating the switching between follow, keep, search, and resume states provided in Embodiment 1 of the present invention;

[0063] Figure 10 This is a data structure diagram for server-side inspection information management and traceability provided in Embodiment 1 of the present invention. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0065] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0066] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0068] Example 1

[0069] This embodiment provides a control method for a medical inspection robot that supports remote collaboration using panoramic virtual reality, applicable to scenarios such as hospital wards, corridors, and public areas. Medical staff wear virtual reality display devices to remotely view and control the mobile robot from an immersive panoramic first-person perspective, and can intervene and take over the dialogue between the mobile robot and patients when necessary. Simultaneously, the mobile robot performs multimodal perception and emotion assessment during the inspection process, and actively interacts and reports alarms when trigger conditions are met.

[0070] like Figure 1 As shown, it includes a mobile robot terminal, a server terminal, and a virtual reality display terminal, which are connected through network communication.

[0071] The mobile robot is used to perform inspection movements according to medical inspection task information, collect and encode panoramic audio and video from a first-person perspective, analyze inspection events and emotional states, execute interactive responses and dialogue outputs, and switch states and take safety measures when receiving remote control / dialogue takeover / emergency handling instructions.

[0072] The server is used to generate and distribute medical inspection task information, forward / relay panoramic media streams, manage and arbitrate inspection tasks, status, event alarms, and session takeovers, and provide real-time display and post-event traceability capabilities to the virtual reality display terminal.

[0073] The virtual reality display terminal is used to receive and render panoramic audio and video to form an immersive remote viewing screen, display inspection events and alarm information, and collect remote operation input and dialogue takeover input from medical staff to generate motion control commands and / or dialogue takeover commands.

[0074] Therefore, the medical inspection robot control method provided in this embodiment mainly includes the following steps:

[0075] Receive medical inspection task information, including inspection path and inspection mode status;

[0076] In autonomous inspection mode, first-person perspective environmental information is collected along the inspection path. Based on the first-person perspective environmental information, a risk score is calculated for the inspection event. Based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision, an interaction trigger score is calculated.

[0077] When the interaction trigger score is greater than or equal to the trigger threshold, the interaction response operation is executed;

[0078] In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint.

[0079] In remote control or dialogue takeover mode, it performs remote control according to the received motion control command, or conducts two-way dialogue and collaboration with the virtual reality display terminal according to the received dialogue takeover command, and triggers emergency response mode when the set emergency stop conditions are met.

[0080] The following is combined Figures 1-2 The method of this embodiment will be described in detail.

[0081] Step S1: Task generation and distribution.

[0082] The server generates and stores medical inspection task information, and then distributes this information to the mobile robot, storing the task or session identifier locally. Related task records.

[0083] Medical Inspection Task Information It should include at least the inspection path, inspection points, and inspection mode parameters, represented as:

[0084] (1);

[0085] in, This is a sequence of inspection points arranged according to the inspection order. In map coordinate system, the first Location of each inspection point This refers to the number of inspection points. The inspection path (which can be a sequence of line segments or a sequence of curve segments) connecting inspection points is defined by the following key points: Provided or generated by the server; This is a set of inspection mode parameters used to indicate the initial inspection mode and mode switching strategy. A set of task execution parameters, including the endpoint threshold. Maximum inspection time At least one of them; Generate a timestamp for the task; Used as a task or session identifier for message association and tracing.

[0086] Step S2: Task reception and execution preparation.

[0087] The mobile robot receives and caches medical inspection task information. After loading the task execution parameters, it enters the waiting state and reports the task reception results and the current mode status to the server.

[0088] Furthermore, mobile robots can process information related to medical inspection tasks. Decoding yields:

[0089] (2);

[0090] in, Indicates the decoding / parsing function; Includes task or session identifier Generate timestamps Inspection point sequence Inspection route Inspection mode parameter set and execution parameter set .

[0091] To ensure the task is not corrupted and the fields are complete, the mobile robot performs the following checks: Perform validity determination and define the validation result indicator. :

[0092]

[0093] in, For indicator functions; The checksum carried in the task package; the set of execution parameters. Includes point threshold Maximum inspection time At least one of them.

[0094] when At that time, the mobile robot writes the task to the local task cache / task queue. And set the current task to be executed as:

[0095] (3).

[0096] Simultaneously, the execution context state is initialized based on the task parameters. :

[0097] (4);

[0098] in, This represents the initial value of the current inspection point number. Indicates the robot's initial pose / position. Indicates the initial inspection mode status (by...) (Given or determined by its default value).

[0099] Furthermore, the mobile robot returns a task receipt to the server. Used for server task closure and tracing; receipt is represented as:

[0100] (5);

[0101] in, For the mobile robot to receive and complete the verification / caching of local timestamps, The link latency estimate is sent to the task. This represents the current mode state; when When this happens, the mobile robot discards or isolates the task and includes a failure reason code (optional) in the receipt to trigger a server-side resend or task policy.

[0102] Step S3: Initialize the inspection mode state and load switching constraints.

[0103] The mobile robot sets the initial inspection mode state according to the inspection mode parameters, and loads the state switching conditions, priority rules and rollback rules.

[0104] The inspection mode states include at least autonomous inspection state, remote control state, dialogue takeover state, and emergency response state, and their switching relationships are as follows: Figure 3 As shown, after defining the inspection mode state, it first determines whether a remote control or takeover command has been received. If so, it switches to the remote control state and the dialogue takeover collaboration state. If not, it performs autonomous inspection and multimodal data collection, and performs immersive remote presentation. It pushes alarms through event analysis, and then determines whether the interaction conditions are met. If so, it executes an active interaction response, approaches or follows, and generates dialogue output. Otherwise, it switches to the remote control state and the dialogue takeover collaboration state. Finally, it manages, archives, and traces the inspection information.

[0105] Specifically:

[0106] Autonomous Inspection Status: The mobile robot performs autonomous navigation and obstacle avoidance according to the inspection path, and performs panoramic audio and video collection and inspection event analysis.

[0107] Remote control status: After receiving remote operation input from the virtual reality display terminal, the server generates motion control commands and forwards them to the mobile robot terminal. The mobile robot terminal generates motion control signals based on the motion control commands and pauses, downweights, limits, or blocks the autonomous navigation output to avoid interference between autonomous navigation and manual control.

[0108] Dialogue takeover status: After receiving the dialogue takeover command from the virtual reality display terminal, the server forwards the voice or text input by the medical staff as dialogue input to the mobile robot terminal. The mobile robot terminal stops or restricts automatic dialogue output and outputs the takeover dialogue content to the scene. At the same time, it collects the on-site voice and sends it back to the virtual reality display terminal to realize two-way dialogue.

[0109] Emergency Response Status: When a preset risk event is detected or an emergency response instruction is received, the mobile robot executes a safety response strategy. The safety response strategy includes at least one of the following: emergency stop, stay in place, evacuate to a safe point, report an alarm, and wait for manual intervention.

[0110] Furthermore, when the server arbitrates motion control commands, dialogue takeover commands, and emergency response commands, the priorities of the four states are set as follows: It corresponds to emergency response, dialogue takeover, remote control, and autonomous inspection, and meets the following requirements:

[0111] (6).

[0112] When multiple instructions are received within the same time window, the server selects the highest priority state as the current target state and issues the corresponding state switching instruction to the mobile robot; when the high priority state is deactivated and the rollback condition is met, it rolls back to the second highest priority state or the autonomous inspection state.

[0113] Step S4: Autonomous inspection movement and panoramic multimodal acquisition.

[0114] In autonomous inspection mode, the mobile robot performs autonomous navigation and obstacle avoidance according to the inspection path, and collects first-person view environmental information through a panoramic image acquisition device mounted on the mobile robot; the first-person view environmental information includes at least panoramic video information and audio information.

[0115] Furthermore, the mobile robot along the inspection path... Each inspection point When moving, let the current position of the mobile robot be... Then the distance error to the point is:

[0116] (7).

[0117] When satisfied When the inspection point is reached, the arrival time is determined and the timestamp is recorded; among which, The threshold for reaching the specified point can be preset or issued by the server.

[0118] Step S5: Media encoding and uplink transmission.

[0119] The mobile robot encodes first-person environmental information to form a media stream and sends the media stream to the server for remote immersive presentation.

[0120] Step S6: After the media stream is sent to the server, the server forwards or relays the media stream and sends the forwarded media stream to the virtual reality display terminal for parsing and panoramic rendering to form an immersive panoramic image consistent with the first-person perspective of the mobile robot, thereby realizing immersive remote viewing.

[0121] Furthermore, to reduce remote viewing drift caused by changes in robot posture, this embodiment performs posture alignment compensation on the panoramic rendering viewpoint at the virtual reality display end:

[0122] Suppose the mobile robot is at time... The heading angle is The initial heading angle is The heading deviation is:

[0123] (8).

[0124] The target viewpoint used by the virtual reality display to render panoramic images :

[0125] (9);

[0126] in, This is the initial viewpoint of the virtual reality display. This refers to real-time pose data reported by mobile robots or pose metadata attached to media streams.

[0127] Furthermore, when there is end-to-end delay in the link At that time, the target's viewpoint can be calculated based on the attitude after delay compensation as follows:

[0128] (10);

[0129] in, It is estimated by the server based on the timestamp and arrival time of the media stream, or by the virtual reality display based on the buffer queue; for The heading angle at any given moment.

[0130] Step S7: As Figure 4 As shown, the mobile robot performs personnel detection and multimodal analysis based on first-person environmental information to obtain inspection event information. The inspection event information includes at least the occurrence of personnel, facial features, body state features, voice features, and emotion assessment results generated from the features. The inspection event information is sent to the server, which generates prompt or alarm information based on the inspection event information and pushes it to the virtual reality display terminal for display.

[0131] Furthermore, multimodal analysis involves fusing facial expressions, voice, and behavioral features to obtain a risk / emotion score:

[0132] Let the feature vectors extracted from facial expressions, speech, and behavior be... The weight vector is , bias is Then risk score The calculation is as follows:

[0133] (11);

[0134] in, For the Sigmoid function, .

[0135] Threshold rules (used for alert / alarm classification and takeover linkage):

[0136] A prompt message will be generated and reported when any of the following conditions are met: Or in the duration window Internal satisfaction The cumulative duration is not less than .

[0137] An alarm message is generated and reported when any of the following conditions are met, and the server allows access to the takeover link: Or in the duration window Internal satisfaction The cumulative duration is not less than .

[0138] in, The first risk threshold, The second risk threshold, The first time threshold, The second time threshold, , Less than or equal to To achieve a tiered strategy of rapid alerts for high-risk situations and continuous notifications for low-risk situations; threshold parameters It can be configured by the server and distributed to the mobile robot, or pre-installed in the mobile robot's local policy.

[0139] Step S8: Proactive interactive response and dialogue generation output.

[0140] When the emotion assessment result or behavioral characteristics meet the preset interaction trigger conditions, the mobile robot is controlled to perform an interactive response operation, which includes at least one of approaching, following, stopping, voice prompts, and initiating a dialogue.

[0141] During the interactive response process, the mobile robot recognizes and processes the collected voice information, generates interactive dialogue content and outputs it, and reports the dialogue text, dialogue summary or key tags to the server for recording and display.

[0142] Furthermore, the interaction trigger conditions include at least one of the following: a low mood threshold, a duration threshold, behavioral feature matching, or manual triggering. Therefore, to quantify the interaction trigger conditions, let the risk score be... The distance between the mobile robot and the target person is The duration of the target person remaining within the field of vision is Then the interaction triggers the score. for:

[0143] (12);

[0144] in, These are the weighting coefficients. This is a distance parameter, preset by the user as the desired interaction distance or the maximum effective interaction distance (e.g., the effective range benchmark for robot voice interaction, trigger prompts, follow-up visits, etc.). This is the dwell time parameter, preset by the user as the upper limit of the trigger time window or the target dwell threshold (e.g., a baseline for determining when interaction / prompts are needed after a certain number of seconds of dwell time). When the condition is met... When the interactive response is triggered, This is the trigger threshold.

[0145] In this embodiment, an occlusion-robust method for maintaining and recapturing the human target during the following process is designed, which can be applied to scenarios with occlusion and crowd intersection, such as indoor corridors and medical areas. Figure 5 As shown, the specific steps include the following:

[0146] (1) Target initialization and file creation.

[0147] Upon first entering a follow-up task, the mobile robot receives the target selected by the user from the candidate set or selects the target person according to interaction rules, and establishes a long-term identity memory for them. With short-term motor state memory Target file: ;in, For the short-time motion state uncertainty covariance, Used as a task / objective identifier.

[0148] (2) Perception acquisition and candidate generation.

[0149] The mobile robot performs human detection based on acquired images, depth information, laser point clouds, or a combination thereof, to obtain a set of candidate personnel. And extract appearance features and geometric observations for each candidate. ;in, For time t, the first The observations of each candidate relative to the mobile robot include at least the relative distance. relative orientation One of them; For time t, the first Key points / posture descriptions of each candidate; For time t, the first The physical characteristics of each candidate.

[0150] The candidate set is used to perform target association matching with the target file established in step (1) to determine the current observation target of the target personnel at time t, without changing the identity of the following target personnel determined in step (1).

[0151] (3) The joint confidence level is associated with the target, such as Figure 6 As shown.

[0152] (3-1) For each candidate, perform identity similarity assessment, geometric consistency assessment, and motion consistency assessment, that is, calculate the identity consistency score respectively. Geometric consistency score Consistency score with movement :

[0153] (13);

[0154] (14);

[0155] (15);

[0156] Among them, identity consistency score Long-term identity memory features used to characterize target individuals With time t Appearance characteristics of each candidate Similarity score; geometric consistency score Used to characterize time t The position of each candidate Predicted location of the target personnel The degree of consistency; motor consistency score Used to characterize time t The speed of each candidate Predicting speed of target personnel The degree of consistency; The scale parameter / standard deviation of the position error (in meters) is used to characterize the predicted position of the target personnel. Position of the candidate The allowable deviation range between them, and control the rate at which the geometric consistency score decays as the position error increases; The scale parameter / standard deviation of the velocity error (unit: m / s) is used to characterize the predicted velocity of the target personnel. Speed ​​of candidates The allowable deviation range between them, and control the rate at which the motion consistency score decreases as the speed error increases; and It can be determined through experimental calibration, or it can be adaptively set according to the prediction covariance when using filtering prediction.

[0157] (3-2) Identity consistency score Geometric consistency score Consistency score with movement Perform weighted fusion to obtain the first time step t. Joint confidence level of the candidates :

[0158] (16);

[0159] in, The identity consistency score is respectively Geometric consistency score Consistency score with movement The fusion weights, and satisfying .

[0160] (3-3) The highest joint confidence level among all candidates is used as the overall follow-up confidence level. ,Right now: In order to perform target association;

[0161] make ,like Then the candidate If it is used as the target for observation at the current moment, it is otherwise determined to be unreliable observation; As an association threshold, this target association is only used to determine the observation target at the current moment, representing the observation object that is most consistent with the identity of the target person at the current moment, and is used to update the prediction and control, without changing the identity of the following target person determined in step (1).

[0162] (4) Occlusion visibility estimation, such as Figure 7 As shown.

[0163] First, an obstacle set is constructed based on depth or point cloud and occupancy grid or map, and a mobile robot is generated to be in the line of sight of the target person based on the predicted position of the target person.

[0164] Then, based on depth or point cloud and occupancy grid or map, the predicted positions of the mobile robot and the target person are determined. Projecting light between lines of sight;

[0165] in, The position of the target person predicted in the current cycle by a short-time motion state model (such as Kalman filter / uniform velocity model) is used to estimate visibility even under occlusion or low confidence conditions;

[0166] When the target association is successful in step (3) and the overall following confidence meets the association threshold, the position of the observed target at the current time can be used. Substitute or with After fusion, the result is Projecting light.

[0167] Finally, the occlusion ratio is statistically analyzed, and the target visibility probability is calculated. and occlusion probability :

[0168] (17);

[0169] (18);

[0170] in, This represents the proportion of line-of-sight sampling points that are obscured by occupied grid cells or obstacle points. This is the proportionality coefficient.

[0171] (5) Controlled updates of dual-layer memory, such as Figure 6 As shown.

[0172] (5-1) If the target visibility probability Greater than or equal to the visibility threshold And the overall confidence level Greater than or equal to the update threshold Then, the long-term identity memory and short-term motion status in the target file are updated; among them, the long-term identity memory is updated using an exponential moving average: ; As weight; For the long-term identity memory of the target personnel at time t+1 in the updated target file; For candidates (i.e., the long-term identity memory of the observed target at the current time t).

[0173] (5-2) When the target visibility probability Less than the visibility threshold Or a combination of confidence levels Less than the update threshold At that time, the update of long-term identity memories in the target file is frozen: It only updates the short-term motion state memory in the target file to maintain target continuity and suppress player-switching drift.

[0174] In this process, the short-term motion state is updated according to the model (example is a uniform motion model): And when reliable observations are available, corrections and updates are performed, such as using Kalman filtering or particle filtering, to maintain target retention during occlusion. For the short-term movement state of the target personnel, The location of the target personnel in the map coordinate system. For speed; The state transition matrix corresponding to the uniform motion model (based on the sampling period) Sure); This is the process noise term, used to describe the random disturbances in the target motion. It is usually set to zero-mean noise, and its covariance is determined by calibration or configuration.

[0175] (6) Follow-up control and safety constraints.

[0176] In follow mode, motion control commands, including the target's linear velocity and angular velocity, are generated based on the target's relative position, and safety constraints are applied. These safety constraints include scaling the target's linear velocity based on the obstacle distance and applying a stopping threshold when the obstacle distance is below a certain threshold. When the braking or safety holding state is triggered.

[0177] Specifically:

[0178] Let the relative distance to the target be... The relative orientation is The expected following distance is Then the target linear velocity With the target angular velocity for:

[0179] (19);

[0180] in, and This is the proportionality coefficient. These are the maximum linear velocity and the maximum angular velocity; This represents a saturation function.

[0181] Then based on the distance to the forward obstacle Target linear velocity scaling: ;when At that time, the braking command is triggered. And enter a safe holding state. This is a clipping function, indicating that the input is clipped to... .

[0182] (7) Loss detection and state machine switching.

[0183] When the loss determination condition is met, switch to recapture state and perform active recapture based on target uncertainty and map occlusion structure.

[0184] Specifically: when the overall follow-up confidence level Less than the loss threshold And the cumulative duration of continuous unreliable observation Greater than or equal to the first duration threshold Or, when the occlusion probability Greater than the occlusion threshold And the cumulative duration during which the target remains continuously invisible. Greater than or equal to the second duration threshold When the target is lost, the system will switch to recapture mode.

[0185] The main switching conditions include the following two:

[0186] (A) Search state switching conditions: when the following conditions are met At that time, it switches from follow mode to search mode.

[0187] (B) Maintain state transition conditions: when the following conditions are met If the search state switching conditions are not met, switch from the follow state to the hold state.

[0188] (C) When both the maintenance condition and the search condition are met, the search state is switched first.

[0189] in, and The determination method is as follows:

[0190] (A) Continuously unreliable observations: By controlling the period when the continuous period is determined to be unreliable observations. The results are obtained through accumulation; if reliable observations are found, then... Reset to 0;

[0191] That is: if or ,but ,otherwise ;when The duration criterion in the condition for maintaining state transition is satisfied.

[0192] (B) Continuous occlusion and invisibility: The cumulative duration of continuous invisibility of the target. By continuously judging it as "invisible" ( The time period is based on the control cycle. The results are accumulated; if the target becomes visible again, then... Reset to 0;

[0193] That is: if ,but ,otherwise ;when Trigger the search state switching condition. Indicates the control cycle.

[0194] (8) Actively recapture viewpoint / path planning.

[0195] like Figure 8 As shown, the target prediction distribution is obtained from the short-time motion state prediction. Let the short-time motion state be... Predicted as Define the position extraction matrix Then the predicted mean and covariance of the target location are respectively Thus obtain .

[0196] Map occlusion structures are used to characterize occupancy / obstacle information in the environment that can cause line-of-sight occupancy. They can be represented by occupancy grid maps or point cloud obstacle layers and provided by prior maps obtained by SLAM (Simultaneous Localization and Mapping). They are also updated online locally by combining depth information or laser point clouds.

[0197] Based on target prediction distribution Based on the map occlusion structure, a set of candidate viewpoints is generated. Each viewpoint is scored to assess its potential for improved visibility:

[0198] (20).

[0199] in, This is a weighting factor. By sampling / integrating the target prediction distribution, we obtain: From Sampling Target location samples For each target location sample, viewpoint Starting from, with To obtain the occlusion percentage, light is projected onto the occlusion structure of the map at the endpoint. And calculate the sample visibility probability: ;

[0200] but , This is the visibility attenuation coefficient.

[0201] Used to characterize arrival at the viewpoint The path cost can be at least one of path length, time consumption, or turning cost, or a weighted combination thereof. Specifically:

[0202] Planning from the robot's current position to the viewpoint path The path length cost is ;

[0203] The time consumption cost is expressed as ,in, For the desired translation speed, The desired angular velocity;

[0204] The cost of turning is ;in, For the first Segment path direction angle, For the first Segment path direction angle, Indicates the angle difference reduction to ;

[0205] Based on the above cost term, take Or take a weighted combination Weight .

[0206] Active recapture includes selecting the optimal viewpoint from the candidate viewpoint set that has the maximum information gain and is cost-constrained, i.e., selecting... and planned to Find the shortest path, perform probing movement or gimbal scanning to improve target visibility and complete recapture; if recapture fails, return to regenerate the candidate viewpoint set.

[0207] (9) Recapture and recovery.

[0208] During the recapture process, the joint confidence of candidates is continuously calculated during the search process. The joint threshold determination is performed on the candidates; the joint threshold determination includes: the candidate's identity consistency score. Satisfy the first threshold Geometric consistency score Satisfy the second threshold And the probability of target visibility Meets the visibility threshold When the recapture is successful, the tracking function is restored.

[0209] That is, when there are candidates who simultaneously satisfy: (Identity consistency); (Geometric consistency); (Current visibility); then it is determined that the recapture was successful, the follow state is switched back, and the following is... Perform controlled updates.

[0210] The complete Follow, Hold, Search, and Recover process is as follows: Figure 9 As shown.

[0211] (10) Log and interpretable output; record various status data during the following process, and output status prompts such as visible / occluded / searching / recaptured.

[0212] Step S9: The virtual reality display terminal receives the operation input from medical staff and generates motion control instructions or dialogue takeover instructions, which are then sent to the server. The server performs permission verification and session arbitration on the instructions before forwarding them to the mobile robot.

[0213] When a motion control command is received, the mobile robot switches to remote control mode and maps the motion control command into a motion control signal. At the same time, it pauses, reduces the weight of the autonomous navigation control output, limits the amplitude, or blocks it.

[0214] When it receives a dialogue takeover command, the mobile robot switches to dialogue takeover mode, takes the input from medical staff as dialogue input and outputs it to the scene, and at the same time transmits the on-site voice back to the virtual reality display to achieve two-way dialogue collaboration.

[0215] Further (motion control command mapping): Let the normalized remote control input generated by the virtual reality display be... ,in Indicates the forward / backward amount. Indicates the turning amount, then the target linear velocity of the mobile robot. With the target angular velocity They are respectively:

[0216] (twenty one);

[0217] (twenty two);

[0218] in, This is the proportionality coefficient. These are the maximum linear velocity and the maximum angular velocity. This is the amplitude limiting function.

[0219] Furthermore (jitter suppression and network fluctuation smoothing): To reduce the impact of remote control input jitter and network latency jitter on control, the mobile robot's target linear velocity... With the target angular velocity Perform first-order smoothing filtering separately:

[0220] (twenty three);

[0221] (twenty four);

[0222] in, For smoothing coefficients, These are the filtered target linear velocity and target angular velocity.

[0223] Further (safety limit and emergency response triggering): Suppose the mobile robot obtains the distance to the nearest obstacle ahead based on the obstacle avoidance sensor as follows: The deceleration threshold is The parking threshold is ,and , It can be configured and distributed by the server, or pre-installed in the local security policy.

[0224] Then, the speed scaling factor for:

[0225] (25).

[0226] And linear velocity Perform safe scaling:

[0227] (26).

[0228] When satisfied When the emergency occurs, the mobile robot performs an emergency stop and enters an emergency response state, while simultaneously reporting the emergency response event to the server; after the preset release conditions are met, it returns to the remote control state or the autonomous inspection state.

[0229] Step S10: Inspection information management and traceability.

[0230] The server receives and manages inspection logs, alarm records, interactive dialogue summaries, or event replay indexes from the mobile robot, and provides real-time viewing or post-event tracing to the virtual reality display.

[0231] like Figure 10 As shown, the server processes inspection information from the mobile robot by task or session identifier. If you perform associated documentation, then the inspection log will be... Alarm Log Interactive Dialogue Summary With playback index They are represented as follows:

[0232] (27);

[0233] (28);

[0234] (29);

[0235] (30);

[0236] in, The inspection log type identifier field is used to distinguish the category or event name to which the log belongs. It is preferably a preset enumeration value or string identifier, such as status reporting, navigation event, location update, target following / recapture, strategy switching, anomaly diagnosis, etc. This is the log payload field, used to carry the specific data content corresponding to this log entry, preferably structured key-value pairs; for example, it may include robot pose / velocity, path points, sensor summaries, recognition results, control command parameters, resource usage information, etc. This is the alarm level field, used to characterize the urgency of the alarm. It is preferably a preset level value, such as L1 / L2 / L3, or alert / warning / critical. This is a summary field used to summarize the alarm or dialogue content in short text, facilitating quick retrieval and human understanding; the summary of the Alarm should preferably describe the cause / object / location / impact of the alarm, and the summary of the Dialogue should preferably describe the core intent or key conclusion of this round of dialogue. The dialogue role field identifies the role category of the initiator or speaker of this dialogue content, preferably including mobile robot / user / medical staff / system, etc. This is a tag field used to annotate the dialogue summary with keywords or topics to support retrieval, statistics or clustering. It is preferably a set of tags, such as battery level, navigation, target following, alarm handling, query confirmation, etc. This is the start timestamp of the replay segment; This is the timestamp indicating the end of the playback segment; This is a Uniform Resource Locator field for media resources, used to point to the storage location or access address of the playback media segment, preferably a file path, object storage address, or network URL (link address, Uniform Resource Locator).

[0237] The server creates query / tracing-oriented index functions:

[0238] (28);

[0239] in, For querying time windows, Indicates recording events Timestamp (take the interval of the playback index) (overlap determination).

[0240] Therefore, the virtual reality display terminal can be based on Enables real-time viewing or post-event tracing, and associates alarm records with corresponding playback segments for display; supports arbitrary alarm timestamps. The server selects those that meet the criteria in the replay index. Playback clips, or select to make The smallest segment is used as the corresponding alarm playback.

[0241] Example 2

[0242] This embodiment provides a medical inspection robot control system, including:

[0243] The receiving module is configured to receive medical inspection task information, including inspection path and inspection mode status.

[0244] The evaluation module is configured to collect first-person perspective environmental information along the inspection path in autonomous inspection mode, score the risk of inspection events based on the first-person perspective environmental information, and calculate the interaction trigger score based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision.

[0245] The interaction module is configured to execute an interaction response operation when the interaction trigger score is greater than or equal to the trigger threshold.

[0246] In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint.

[0247] The remote module is configured to perform remote control based on received motion control commands in remote control or dialogue takeover mode, or to conduct two-way dialogue and collaboration with the virtual reality display terminal based on received dialogue takeover commands, and to trigger an emergency response state when the set emergency stop conditions are met.

[0248] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0249] In further embodiments, the following is also provided:

[0250] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0251] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0252] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0253] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0254] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0255] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0256] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0257] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0258] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0259] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0260] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A control method for a medical inspection robot, characterized in that, include: Receive medical inspection task information, including inspection path and inspection mode status; In autonomous inspection mode, first-person perspective environmental information is collected along the inspection path. Based on the first-person perspective environmental information, a risk score is calculated for the inspection event. Based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision, an interaction trigger score is calculated. Among them, risk score The calculation is as follows: ;in, For the Sigmoid function, ; To extract feature vectors from people's facial expressions, voice, and behavior based on first-person environmental information; This is the weight vector; For bias; when Or in the duration window Internal satisfaction The cumulative duration is not less than When, generate a prompt message and report it to the server; when Or in the duration window Internal satisfaction The cumulative duration is not less than At that time, an alarm message is generated and reported to the server, and the server allows access to the takeover link; among which, The first risk threshold, The second risk threshold, The first time threshold, The second time threshold, , Less than or equal to ; When the interaction trigger score is greater than or equal to the trigger threshold, the interaction response operation is executed; In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint. The recapture process includes: generating a candidate viewpoint set based on the constructed target prediction distribution and map occlusion structure. Rate each viewpoint: ; It is obtained by sampling the target prediction distribution, that is: obtained by sampling from the target prediction distribution. Target location samples For each target location sample, viewpoint v Starting from, with Projecting light onto the occlusion structure of the map at the endpoint yields the occlusion percentage. And calculate the sample visibility probability: ; but , Visibility attenuation coefficient; To reach the viewpoint The path cost is one or a weighted combination of path length, time consumption, or turning cost. For the weighting factor; Select from the candidate viewpoint set and planned to Recapture the path after it has been taken; In remote control or dialogue takeover mode, it performs remote control according to the received motion control command, or conducts two-way dialogue and collaboration with the virtual reality display terminal according to the received dialogue takeover command, and triggers emergency response mode when the set emergency stop conditions are met.

2. The medical inspection robot control method as described in claim 1, characterized in that, After encoding the first-person perspective environmental information into a media stream, it is sent to the virtual reality display terminal via a server for panoramic rendering and display. The virtual reality display terminal also performs pose alignment compensation on the panoramic rendering viewpoint. The virtual reality display terminal is used to render the target viewpoint of the panoramic image. for: ; This is the initial viewpoint of the virtual reality display. For mobile robots in The heading angle at any given moment; This is the initial heading angle.

3. The medical inspection robot control method as described in claim 1, characterized in that, Interactive Triggered Score for: ； in, These are the weighting coefficients. For the desired interaction distance, Target stay time; Assess risk level; The distance between the mobile robot and the target person; The duration of time a target person remains within the field of vision; This is the amplitude limiting function.

4. The medical inspection robot control method as described in claim 1, characterized in that, Light is projected between the predicted positions of the mobile robot and the target person, the occlusion ratio is statistically analyzed, and the target's visibility probability is calculated. and occlusion probability : ; ; This represents the proportion of the line-of-sight sampling points that are obscured by obstacles. This is the proportionality coefficient.

5. The medical inspection robot control method as described in claim 4, characterized in that, When the overall follow confidence is less than the loss threshold and the cumulative duration of continuous unreliable observation is greater than or equal to the first duration threshold; or when the occlusion probability is greater than the occlusion threshold and the cumulative duration of continuous target invisibility is greater than or equal to the second duration threshold, the target is determined to be lost and the reacquisition state is switched. Among these, when the overall follow confidence is less than the association threshold, it is determined to be unreliable observation, and when the occlusion probability is greater than the occlusion threshold, it is determined to be target invisibility.

6. The medical inspection robot control method as described in claim 4, characterized in that, The process of following the target personnel also includes: constructing a target profile that includes the target personnel's long-term identity memory and short-term motion state memory; determining the joint confidence score of each candidate based on the identity consistency score, geometric consistency score, and motion consistency score of the target personnel and the candidate personnel; and using the candidate corresponding to the highest joint confidence score as the observation target at the current moment. If the target visibility probability is greater than or equal to the visibility threshold, and the overall follow confidence is greater than or equal to the update threshold, then the long-term identity memory and short-term motion state in the target file are updated; wherein, the update of the long-term identity memory is as follows: ; As weight; For long-term identity memory of the target personnel at time t+1; For long-term identity memory of the target personnel at time t; For the target observed at the current time t Long-term identity memory; When the target visibility probability is less than the visibility threshold, or the overall follow confidence is less than the update threshold, the update of long-term identity memory in the target file is frozen, and only the short-term motion state memory in the target file is updated.

7. The medical inspection robot control method as described in claim 1, characterized in that, In remote control mode, the motion control commands are: , Forward / backward amount, Let be the turning radius, then at time t, the target linear velocity of the mobile robot. With the target angular velocity They are respectively: ； ； For target linear velocity With the target angular velocity Perform first-order smoothing filtering separately: ； ； in, This is the proportionality coefficient. These are the maximum linear velocity and the maximum angular velocity. This is a limiting function; For smoothing coefficients, These are the filtered target linear velocity and target angular velocity; Let the distance to the nearest obstacle in front of the mobile robot be... The deceleration threshold is The parking threshold is ,and Then the speed scaling factor for: ;right Perform safe scaling as follows: ; When satisfied When an emergency response state is triggered, it will revert to remote control or autonomous inspection state after the preset conditions for termination are met.

8. A medical inspection robot control system, characterized in that, The method for controlling a medical inspection robot according to any one of claims 1-7 includes: The receiving module is configured to receive medical inspection task information, including inspection path and inspection mode status. The evaluation module is configured to collect first-person perspective environmental information along the inspection path in autonomous inspection mode, score the risk of inspection events based on the first-person perspective environmental information, and calculate the interaction trigger score based on the risk score, the distance between the mobile robot and the target person, and the duration of the target person in the field of vision. The interaction module is configured to execute an interaction response operation when the interaction trigger score is greater than or equal to the trigger threshold. In the process of following the target person, the joint confidence of each candidate person is determined based on the consistency assessment between the target person and the candidate person. The highest joint confidence is taken as the comprehensive following confidence. Occlusion judgment is performed between the mobile robot and the target person. The target is lost based on the obtained occlusion probability and comprehensive following confidence. When lost, an information gain viewpoint is selected from the candidate viewpoint set to improve the visibility probability, so as to recapture by planning a path to the information gain viewpoint. The remote module is configured to perform remote control based on received motion control commands in remote control or dialogue takeover mode, or to conduct two-way dialogue and collaboration with the virtual reality display terminal based on received dialogue takeover commands, and to trigger an emergency response state when the set emergency stop conditions are met.

9. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-7.

Images