A positioning method, vehicle, and storage medium for a portable endoscope system

Abstract

An embodiment of the present application discloses a positioning method, a vehicle, and a storage medium for a portable endoscope system. Among them, the method is applied to the portable endoscope system, and the portable endoscope system includes a visual recognition module, a robotic arm, an endoscope workstation, and an endoscope. The method includes: determining a first relative position relationship between a first object and the robotic arm according to a first image collected by the visual recognition module; controlling the robotic arm according to the first relative position relationship so that the endoscope faces the first object; when the endoscope is removed by the first object, controlling the movement of the robotic arm according to a second image collected by the visual recognition module. In this way, automatic positioning of the endoscope workstation is achieved through image acquisition, recognition, and robotic arm control, reducing the time for manual movement of the endoscope workstation to find a suitable diagnosis and treatment position and shortening the diagnosis and treatment time. Moreover, the portable endoscope system is arranged on the vehicle, enabling first aid with the endoscope and realizing endoscopic-related medical services in remote areas.

Description

Technical Field

[0001] This application relates to the field of medical devices, and more specifically, to a positioning method for a portable endoscope system, a vehicle, and a storage medium. Background Art

[0002] With the continuous improvement of endoscopes, the scope of examination and treatment using endoscopes almost covers the entire digestive tract lumen. Among them, the endoscope can enter the stomach through the oral cavity or enter the body through other natural orifices. Doctors can observe the diseased area of the human body with the help of the endoscope, formulate a suitable treatment plan accordingly, or treat the diseased area of the human body with the help of the endoscope.

[0003] Currently, due to the uneven distribution of medical resources and the concentration of high-quality medical resources in developed areas, mobile medical care will bring efficient and convenient endoscope medical services to remote areas.

[0004] However, current endoscope examination equipment occupies a large area, and there are vibration problems with the endoscope workstation during vehicle movement, which is not conducive to the operation of the endoscope. This restricts the development of mobile endoscope medical services. Moreover, when operating the endoscope, medical staff need to manually move the endoscope workstation to the corresponding position to find a suitable diagnosis and treatment position. This requires spending a lot of time moving the endoscope workstation, resulting in an increase in the diagnosis and treatment time. Summary of the Invention

[0005] In view of this, embodiments of this application disclose a positioning method for a portable endoscope system, a vehicle, and a storage medium. By setting the portable endoscope system on the vehicle, setting the endoscope workstation at the end of the robotic arm, and installing the endoscope on the endoscope workstation, the floor area of the endoscope examination equipment is reduced. By driving the endoscope workstation to move with the robotic arm, the problem of vibration of the endoscope workstation during vehicle movement can be avoided to a certain extent. Further, it is beneficial to the development of mobile endoscope medical services. Moreover, through image acquisition, recognition combined with robotic arm control, the endoscope workstation can be automatically positioned to a suitable position, reducing the time for medical staff to manually move the endoscope workstation to find a suitable diagnosis and treatment position, and shortening the diagnosis and treatment time.

[0006] The technical solutions provided by the embodiments of this application are as follows:

[0007] In a first aspect, embodiments of this application provide a positioning method for a portable endoscope system. The method is applied to the portable endoscope system, which is set on a vehicle. The portable endoscope system includes: a vision recognition module, a robotic arm, an endoscope workstation, and an endoscope. The endoscope workstation is set at the end of the robotic arm, and the endoscope is installed on the endoscope workstation. The method includes:

[0008] Determine the first relative position relationship between the first object in the carriage space and the robotic arm based on the first image of the carriage space of the vehicle collected by the visual recognition module;

[0009] Control the robotic arm according to the first relative position relationship so that the endoscope faces the first object;

[0010] When the endoscope is removed by the first object, control the movement of the robotic arm according to the second image of the carriage space collected by the visual recognition module.

[0011] In a possible implementation manner, the endoscope includes an endoscope handle and a cable. Controlling the movement of the robotic arm according to the second image of the carriage space collected by the visual recognition module includes:

[0012] Determine the second relative position relationship between the first object and the second object in the carriage space according to the second image;

[0013] Determine the spatial trajectory of the cable according to the second relative position relationship;

[0014] Set the reference curvature of the cable according to the spatial trajectory;

[0015] Adjust the distance between the robotic arm and the endoscope handle according to the reference curvature.

[0016] In a possible implementation manner, the method further includes:

[0017] Determine the first target displacement of the endoscope workstation according to the speed and acceleration of the endoscope handle and the spatial trajectory;

[0018] Adjust the pose of the robotic arm according to the first target displacement.

[0019] In a possible implementation manner, the method further includes:

[0020] Determine the first speed of the end of the robotic arm according to the speeds and accelerations of the joints of the robotic arm;

[0021] Determine the second speed of the end of the robotic arm by using the actually measured acceleration of the end of the robotic arm collected;

[0022] Determine the disturbance speed according to the difference between the first speed and the second speed;

[0023] Determine the passive movement amount of the robotic arm according to the integral result corresponding to the disturbance speed;

[0024] Compensate the passive motion amount of the robotic arm through the inverse kinematics of the robotic arm.

[0025] In a possible implementation, the visual recognition module includes a first infrared camera and a second infrared camera. The second image includes a first infrared image collected by the first infrared camera and a second infrared image collected by the second infrared camera. Controlling the movement of the robotic arm according to the second image of the carriage space collected by the visual recognition module includes:

[0026] Establish a point cloud model based on the first infrared image and the second infrared image;

[0027] Match the key information template with the point cloud model to determine the position information of the object to be measured; the object to be measured includes at least one of the following: the first object, the endoscope workstation, and the endoscope;

[0028] Control the movement of the robotic arm according to the position information.

[0029] In a possible implementation, the method further includes:

[0030] Segment and fit the point cloud model to generate multiple point cloud regions;

[0031] Determine the surface curvature of the point cloud region corresponding to the obstacle in the carriage space;

[0032] Predict the back space of the obstacle according to the surface curvature, and generate an enclosure corresponding to the obstacle according to the back space;

[0033] The controlling the movement of the robotic arm according to the position information includes:

[0034] Perform path planning on the robotic arm using the enclosure and the position information to obtain a path planning result;

[0035] Control the movement of the robotic arm according to the path planning result.

[0036] In a possible implementation, the method further includes:

[0037] Store the basic information of the first object, and the basic information includes: face recognition information and hand control instruction definition information;

[0038] The determining the first relative position relationship between the first object in the carriage space and the robotic arm according to the first image of the vehicle's carriage space collected by the visual recognition module includes:

[0039] When a start instruction initiated by the first object is recognized according to the basic information, determine a first relative position relationship between the first object in the vehicle's carriage space and the robotic arm according to a first image of the vehicle's carriage space collected by the visual recognition module.

[0040] In a second aspect, an embodiment of the present application provides a portable endoscope system, which is arranged on a vehicle. The portable endoscope system includes: a visual recognition module, a robotic arm, an endoscope workstation, an endoscope, and a robotic arm control module. The endoscope workstation is arranged at the end of the robotic arm, and the endoscope is installed on the endoscope workstation;

[0041] The visual recognition module is configured to collect a first image of the vehicle's carriage space and determine a first relative position relationship between the first object in the carriage space and the robotic arm according to the first image;

[0042] The robotic arm control module is configured to control the robotic arm according to the first relative position relationship so that the endoscope faces the first object;

[0043] The visual recognition module is further configured to collect a second image of the carriage space;

[0044] The robotic arm module is further configured to control the movement of the robotic arm according to the second image when the endoscope is removed by the first object.

[0045] In a third aspect, an embodiment of the present application provides a vehicle, which includes the portable endoscope system described in the second aspect above.

[0046] In a fourth aspect, an embodiment of the present application provides a positioning device, including:

[0047] A memory for storing instructions;

[0048] A processor for executing the instructions in the memory to execute the portable endoscope system positioning method according to any one of the first aspects above.

[0049] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute the portable endoscope system positioning method according to any one of the first aspects above.

[0050] In a sixth aspect, an embodiment of the present application provides a computer program product, which when running on a terminal device, causes the terminal device to execute the portable endoscope system positioning method according to any one of the first aspects above.

[0051] Based on the above technical solutions, the present application has the following beneficial effects:

[0052] The embodiments of the present application disclose a positioning method for a portable endoscope system, a vehicle, and a storage medium. Among them, the method is applied to a portable endoscope system, which is arranged on a vehicle. The portable endoscope system includes a visual recognition module, a robotic arm, an endoscope workstation, and an endoscope. The endoscope workstation is arranged at the end of the robotic arm, and the endoscope is installed on the endoscope workstation. The method includes: determining a first relative position relationship between a first object in the carriage space and the robotic arm according to a first image of the carriage space of the vehicle collected by the visual recognition module; controlling the robotic arm according to the first relative position relationship so that the endoscope faces the first object; when the endoscope is removed by the first object, controlling the movement of the robotic arm according to a second image of the carriage space collected by the visual recognition module. It can be seen that in the embodiments of the present application, by arranging the portable endoscope system on the vehicle, mobile endoscope medical services are realized. Moreover, by arranging the endoscope workstation at the end of the robotic arm and installing the endoscope on the endoscope workstation, the floor area occupied by the endoscope examination equipment is reduced, and the robotic arm drives the endoscope workstation to move, so that the problem of vibration of the endoscope workstation during the movement of the vehicle can be avoided to a certain extent. Further, it is beneficial to the development of mobile endoscope medical services, realizing first aid with an endoscope on the vehicle and realizing endoscope-related medical services in remote areas. Moreover, by arranging the endoscope workstation at the end of the robotic arm, before the diagnosis and treatment, according to the relative position relationship, the robotic arm can be controlled to position the endoscope workstation at a position convenient for the first object to remove the endoscope. During the diagnosis and treatment process, according to the collected second image, the robotic arm can be controlled to move to position the endoscope workstation at a position convenient for the first object to operate the endoscope. In this way, through image acquisition, recognition and robotic arm control, the endoscope workstation can be automatically positioned at a suitable position, reducing the time for medical staff to manually move the endoscope workstation to find a suitable diagnosis and treatment position, thereby shortening the diagnosis and treatment time. Description of the Drawings

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the disclosed drawings without creative efforts.

[0054] Figure 1 It is a schematic diagram of the application scenario of a positioning method for a portable endoscope system disclosed in the embodiments of the present application;

[0055] Figure 2 It is a flowchart of a positioning method for a portable endoscope system disclosed in the embodiments of the present application;

[0056] Figure 3 A schematic diagram of the relative position before diagnosis and treatment disclosed in the embodiment of the present application;

[0057] Figure 4 A schematic diagram of the relative position during the diagnosis and treatment process disclosed in the embodiment of the present application;

[0058] Figure 5 A flowchart of a robotic arm control method disclosed in the embodiment of the present application;

[0059] Figure 6 A schematic diagram of the spatial trajectory of a cable disclosed in the embodiment of the present application;

[0060] Figure 7 A schematic diagram of the robotic arm null space obstacle avoidance and the robotic arm joint axis disclosed in the embodiment of the present application;

[0061] Figure 8 A flowchart of a method for compensating the passive motion amount of a robotic arm disclosed in the embodiment of the present application;

[0062] Figure 9 A schematic diagram of the working state of a visual recognition module disclosed in the embodiment of the present application;

[0063] Figure 10 A schematic diagram of the deformation of grid lines disclosed in the embodiment of the present application;

[0064] Figure 11 A flowchart of another robotic arm control method disclosed in the embodiment of the present application;

[0065] Figure 12 A schematic diagram of an RRT algorithm disclosed in the embodiment of the present application;

[0066] Figure 13 A schematic diagram of a hand control instruction disclosed in the embodiment of the present application. Detailed implementation manners

[0067] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0068] The method provided by the embodiments of the present application can be applied to positioning a portable endoscope system in various application scenarios. For example, in an ambulance, a robotic arm provided on the side of the carriage can be used to drive the endoscope workstation to move, thereby achieving the positioning of the endoscope workstation. Another example is that in a mobile clinic, a robotic arm provided on the side wall of the clinic can be used to drive the endoscope workstation to move, thereby achieving the positioning of the endoscope workstation, etc. In the embodiments of the present application, specific application scenarios are not enumerated exhaustively.

[0069] Before describing in detail the method for positioning a portable endoscope system provided by the embodiments of the present application, an application scenario provided by the embodiments of the present application will be described first. Please refer to Figure 1 , Figure 1 which is a schematic diagram of an application scenario of the method for positioning a portable endoscope system provided by the embodiments of the present application. As Figure 1As shown in the figure, in this application scenario, there are a physician 101, a patient 102, a hospital bed 103, an ambulance 104, and a portable endoscope system 105. The portable endoscope system 105 is arranged inside the ambulance 104. The portable endoscope system 105 may include: a visual recognition module 1051, a robotic arm 1052, a display 1053, an endoscope workstation 1054, an endoscope 1055, and a robotic arm control module. The endoscope workstation 1054 is arranged at the end of the robotic arm 1052, and the endoscope 1055 is installed on the endoscope workstation 1054. The robotic arm control module may be arranged on the robotic arm 1052 or at a position independent of the robotic arm. The visual recognition module 1051 may include a projector 10511, a first camera 10512, a second camera 10513, and a processor. The robotic arm 1052 may include a robotic arm base 10521 and a robotic arm body 10522. The endoscope 1055 may include an endoscope handle 10551, a cable 10552, and an insertion tube 10553. The visual recognition module 1051, the robotic arm 1052, and the display 1053 may be arranged on the side wall of the carriage of the ambulance 104, without occupying the ground space of the ambulance 104. The visual recognition module 1051 may collect images of the carriage space; the robotic arm 1052 may control the movement of the robotic arm 1052 according to the collected images of the carriage space to achieve the positioning of the endoscope workstation 1054; the display 1053 may display the endoscopic images collected by the endoscope 1055 and the movement information of the robotic arm 1052. In this application scenario, the patient 102 lies on the hospital bed 103. The robotic arm 1052 moves to the vicinity of the physician 101 under the control of the robotic arm control module. The physician 101 removes the endoscope 1055 from the endoscope workstation 1054 and operates the endoscope 1055 to examine or treat the patient 102. It can be seen that in this application scenario, by arranging the endoscope workstation at the end of the robotic arm, the endoscope workstation can be automatically positioned to a suitable position through image acquisition, recognition, and robotic arm control, reducing the time for medical staff to manually move the endoscope workstation to find a suitable diagnosis and treatment position, and shortening the diagnosis and treatment time. Moreover, by arranging the portable endoscope system inside the ambulance, the patient can be examined or treated with the endoscope on the ambulance, realizing timely and effective endoscopic intervention for the patient, so as to realize first aid with the endoscope or provide endoscopic-related medical services in remote areas.

[0070] Specifically, please refer to Figure 2 , the flowchart of a method for positioning a portable endoscope system disclosed in an embodiment of the present application. This method is applied to a portable endoscope system. The portable endoscope system is arranged on a vehicle. The portable endoscope system includes a visual recognition module, a robotic arm, an endoscope workstation, and an endoscope. The endoscope workstation is arranged at the end of the robotic arm, and the endoscope is installed on the endoscope workstation. This method includes:

[0071] S201. Determine a first relative position relationship between a first object in the vehicle's carriage space and the robotic arm based on a first image of the carriage space of the vehicle collected by the vision recognition module.

[0072] In the embodiments of the present application, the vision recognition module can be started first to enable the robotic arm; then the first image can be collected by the vision recognition module; and then S201 is executed. Among them, the first image can be an infrared image.

[0073] It should be noted that the first object in the embodiments of the present application can be a physician, etc., and the endoscope can be a portable digestive endoscope, etc. The installation of the endoscope on the endoscope workstation means that the endoscope can be removed from the endoscope workstation during use and placed on the endoscope workstation when not in use. Among them, the endoscope can include an endoscope handle, a cable, an insertion tube, etc., and the endoscope handle can be hung on the endoscope workstation. The vehicle can be an ambulance, a first aid vehicle, etc.

[0074] Among them, the first relative position relationship between the first object and the robotic arm refers to the relative spatial position between the first object and the robotic arm in the carriage space, such as the relative distance and relative spatial angle between the first object and the first robotic arm.

[0075] S202. Control the robotic arm according to the first relative position relationship so that the endoscope faces the first object.

[0076] It should be noted that S202 is executed before the operation or examination. An installation structure for the endoscope handle is provided on one side of the endoscope workstation. The robotic arm is controlled according to the first relative position relationship, so that the endoscope workstation and the endoscope move together with the robotic arm, so that the endoscope handle faces the first object, so that the first object can remove the endoscope handle from the endoscope workstation. When the endoscope is in a non-working state, the height of the endoscope workstation can be adjusted to prevent the endoscope from contacting the ground.

[0077] Please refer to Figure 3 , which is a schematic diagram of the relative position before diagnosis and treatment disclosed in the embodiments of the present application. Figure 3 shows the position of the endoscope handle before removing the endoscope handle from the endoscope workstation. At this time, there is a distance between the endoscope workstation and the first object. Figure 3Let {V} be the coordinate system corresponding to the visual recognition module, {R} be the coordinate system corresponding to the robotic arm, and {W} be the world coordinate system. The moving direction ds of the robotic arm can be determined according to the first relative position relationship, and the robotic arm can be controlled to move along the moving direction ds so that the endoscope handle faces the first object. The first object can pull the endoscope at the dropdown point of the endoscope, applying a certain force to cause the endoscope workstation to descend. The first object can also issue a descent command, and the robotic arm can lower the endoscope workstation according to the descent command. After the endoscope workstation descends, the first object can remove the endoscope from the endoscope workstation.

[0078] S203. After the endoscope is removed by the first object, control the movement of the robotic arm according to the second image of the carriage space collected by the visual recognition module.

[0079] Please refer to Figure 4 , which is a schematic diagram of the relative positions of a diagnosis and treatment process disclosed in an embodiment of the present application. Figure 4 In the figure, the endoscope handle has been removed from the endoscope workstation, and the first object holds the endoscope handle and operates the endoscope. During the diagnosis and treatment process, the visual recognition module will collect a second image, and the robotic arm will control its movement according to the second image so that the endoscope workstation can be positioned at a location convenient for the doctor to operate the endoscope.

[0080] It should be noted that the second image is an image of the carriage space collected in real time by the visual recognition module during the diagnosis and treatment process of the first object on the second object.

[0081] It can be seen that in the embodiment of the present application, by setting the portable endoscope system on a vehicle, mobile endoscope medical services are realized. Moreover, by setting the endoscope workstation at the end of the robotic arm and installing the endoscope on the endoscope workstation, the floor area occupied by the endoscope examination equipment is reduced. The robotic arm drives the endoscope workstation to move, thus to a certain extent avoiding the problem of vibration of the endoscope workstation during vehicle movement. Further, it is beneficial to the development of mobile endoscope medical services, realizing first aid using an endoscope on a vehicle and providing endoscope-related medical services in remote areas. Moreover, by setting the endoscope workstation at the end of the robotic arm, before the diagnosis and treatment, the robotic arm can be controlled to move according to the relative position relationship, and the endoscope workstation can be positioned at a location convenient for the first object to remove the endoscope. During the diagnosis and treatment process, the robotic arm can be controlled according to the collected second image, and the endoscope workstation can be positioned at a location convenient for the first object to operate the endoscope. Thus, through image acquisition, recognition and robotic arm control, the endoscope workstation can be automatically positioned at a suitable position, reducing the time for medical staff to manually move the endoscope workstation to find a suitable diagnosis and treatment position, thereby shortening the diagnosis and treatment time.

[0082] Please refer to Figure 5, is a flowchart of a robotic arm control method provided by an embodiment of the present application. Controlling the robotic arm to move according to the second image of the carriage space collected by the vision recognition module may include:

[0083] S501. Determine a second relative position relationship between a first object and a second object in the carriage space according to the second image.

[0084] It should be noted that the second object may be a patient or the like.

[0085] Among them, the second relative position relationship between the first object and the second object refers to the relative spatial position between the first object and the second object in the carriage space, such as the relative distance and relative spatial angle between the first object and the second object.

[0086] S502. Determine the spatial trajectory of the cable according to the second relative position relationship.

[0087] The endoscope in the embodiment of the present application may include an endoscope handle and a cable.

[0088] Among them, the spatial trajectory is the trajectory information calculated by the vision recognition module using a parabola function according to the second relative position relationship.

[0089] S503. Set a reference curvature of the cable according to the spatial trajectory.

[0090] Please refer to Figure 6 , which is a schematic diagram of the spatial trajectory of a cable disclosed by an embodiment of the present application. The workstation access point is the position where the endoscope workstation connects the cable, and the handle position is the position where the endoscope handle connects the cable. Figure 6 In which V E represents the speed of the endoscope workstation, and V H represents the speed of the endoscope handle. Figure 6 The minimum value in Figure 6 corresponds to the spatial trajectory with the minimum curvature, and the maximum value in

[0091] corresponds to the spatial trajectory with the maximum curvature. The reference curvature of the cable can be set according to the spatial trajectory with the maximum curvature.

[0092] It should be noted that during the diagnosis and treatment process, the first object (such as a physician) will operate the endoscope handle. There is a cable with a fixed length between the endoscope handle and the endoscope workstation. According to the relative position relationship between the first object and the second object (such as a patient), the spatial trajectory of the cable can be quickly calculated using a parabola function. The reference curvature is set according to the spatial trajectory. The reference curvature refers to the maximum curvature of the calculated spatial trajectory. Then, the distance between the robotic arm and the endoscope handle is adjusted to ensure that the curvature of the cable is less than or equal to half of the maximum curvature.

[0093] In a possible implementation, the method for positioning a portable endoscope system provided in the embodiments of the present application may further include: determining a first target displacement of the endoscope workstation according to the speed and acceleration of the endoscope handle and the spatial trajectory; adjusting the pose of the robotic arm according to the first target displacement. Among them, since the endoscope workstation and the endoscope handle are connected by a cable, in order to avoid the pulling problem between the endoscope workstation and the endoscope handle, the movement directions of the endoscope workstation and the endoscope handle need to be basically the same, and the movement speeds and accelerations of the endoscope workstation and the endoscope handle need to be basically the same. In the embodiments of the present application, the movement speed and acceleration of the endoscope workstation can be mapped through the movement speed and acceleration of the endoscope handle, and further, according to the movement speed and acceleration of the endoscope workstation and in combination with the spatial trajectory of the cable, the first target displacement of the endoscope workstation is mapped. The pose may include position and attitude. In this way, the adjustment of the pose of the robotic arm is related to the relative position relationship between the doctor and the patient and the speed and acceleration of the endoscope handle, which can further avoid the situation of pulling between the operation endoscope handle and the endoscope workstation, and is convenient for the doctor to complete the endoscope diagnosis and treatment operation.

[0094] It should be noted that the second image is a continuous series of multiple images. In the embodiments of the present application, the speed and acceleration of the endoscope handle can also be determined by using the collected second images, the optimal spatial trajectory is determined from multiple spatial trajectories, and then according to the speed and acceleration of the endoscope handle and the optimal spatial trajectory, the target displacement and target speed of the endoscope workstation are determined, and the pose of the robotic arm is adjusted according to the target displacement and target speed.

[0095] In a possible implementation, the method for positioning a portable endoscope system provided in the embodiments of the present application may further include: determining a second target displacement of the endoscope workstation based on the contact force of the robotic arm and the impedance control algorithm, and adjusting the pose of the robotic arm according to the second target displacement. Among them, based on the characteristics of the redundant robotic arm, the robotic arm is adjusted with the trend of the smallest change in the pose of the endoscope workstation. In this way, adjusting the robotic arm based on the contact force and impedance control algorithm can enable the robotic arm to adapt to a complex and narrow working space, so that the endoscope workstation can adapt to a complex and narrow working space.

[0096] Among them, the formula corresponding to the impedance control algorithm can be expressed as follows:

[0097]

[0098] M represents the inertial characteristic parameter, B represents the damping characteristic parameter, K represents the stiffness characteristic parameter, F e represents the force value at the end of the robotic arm, x e is the calculated second target displacement, that is, the displacement amount, Indicates acceleration, Indicates velocity, The manipulator can be adjusted in the trend with the smallest change in the working position and posture of the endoscope.

[0099] The manipulator in the embodiment of the present application can be a manipulator such as a seven-degree-of-freedom redundant manipulator, and the embodiment of the present application does not limit this. The seven-degree-of-freedom redundant manipulator can adopt a joint module structure, and a corresponding torque sensor is configured for each degree of freedom, and the torque sensor is used to measure the torque at the manipulator joint. The seven-degree-of-freedom redundant manipulator can adjust the joint angles of the manipulator based on null space motion and keep the position and posture of the endoscope workstation set at the end of the manipulator unchanged. During the diagnosis and treatment process, in the Cartesian coordinate system, when no obstacle is recognized, the joint angular velocity of the manipulator is smoothed through the kinematic objective function. When an obstacle is recognized, the target position and posture of the endoscope workstation are preferentially guaranteed, that is, when there is an obstacle, the accuracy of the position and posture is guaranteed, and the smoothness characteristic of the speed is sacrificed. Among them, the kinematic objective function includes the operation constraint conditions for the redundant-controlled manipulator.

[0100] Please refer to Figure 7 , which is a schematic diagram of null space obstacle avoidance of a manipulator and a schematic diagram of the manipulator joint axis disclosed in the embodiment of the present application. Figure 7 In {R} is the coordinate system corresponding to the manipulator, and {E} is the coordinate system corresponding to the endoscope workstation. Figure 7 The obstacle in is the above-mentioned obstacle. Figure 7 In the left half sub-diagram, when the manipulator touches the obstacle, it will receive an external force and will adjust the joint angles of the manipulator based on null space motion. Figure 7 In the right half sub-diagram, the joints J1 to J7 between the manipulator joints are shown, and the coordinate change of the coordinate system {E} corresponding to the endoscope workstation in the manipulator coordinates can be expressed as:

[0101]

[0102] Among them, Indicates the coordinate change of {E} in the manipulator coordinates, and T i Indicates the homogeneous matrix of the pose transformation of the i-th joint relative to the previous joint coordinate system.

[0103] It should be noted that the manipulator adopts a configuration where three adjacent joint axes intersect at one point (S-W-S), that is, joints J1, J2, and J3 intersect at one point, and J5, J6, and J7 intersect at one point. The Rodriguez formula can be used to solve 8 sets of inverse solutions or a set of solutions with small joint angular acceleration changes through the analytical method, and the manipulator is further controlled through the solution results.

[0104] For the kinematic solution of the robot involved in the embodiments of the present application, it can be obtained through the calculation formula of the kinematics of a seven-degree-of-freedom redundant robotic arm, and the control process of the robotic arm in the above implementation scenario is not limited.

[0105] In the embodiments of the present application, a six-axis force sensor and a gyroscope can also be installed at the end of the robotic arm, or the installation of the six-axis force sensor and the gyroscope can be integrated inside the endoscopic workstation. Among them, the six-axis force sensor can measure three axial forces and three axial torques simultaneously, and the gyroscope can detect the angle, angular velocity or angular acceleration of an object.

[0106] In a possible implementation manner, in the embodiments of the present application, the control represents the speed of the endoscopic workstation, and the speed of the endoscopic workstation has a direction. k represents the speed mapping ratio. represents the moving speed of the endoscopic handle, and the moving speed of the endoscopic handle is used to detect the stress of the cable through the six-axis force sensor on the robotic arm, so as to adjust k, obtain the speed of the endoscopic workstation at the current moment, integrate the speed of the endoscopic workstation at the current moment to obtain the target position of the endoscopic workstation, and control the robotic arm to move to the target position.

[0107] It can be seen that in the embodiments of the present application, by correspondingly adjusting the distance between the robotic arm and the endoscopic handle according to the reference curvature of the cable, the curvature of the cable during the diagnosis and treatment process can be ensured to be within a reasonable range, so that the situation of pulling between the operating endoscopic handle and the endoscopic workstation can be avoided, maintaining sufficient surgical operating space for the physician and facilitating the physician to complete the endoscopic diagnosis and treatment operation.

[0108] Please refer to Figure 8 , which is a flowchart of a method for compensating the passive movement amount of a robotic arm provided by the embodiments of the present application. The positioning method of the portable endoscopic system provided in the above embodiments may further include:

[0109] S801. Determine the first speed of the end of the robotic arm according to the speeds and accelerations of the joints of the robotic arm.

[0110] In the embodiments of the present application, the first speed of the end of the robotic arm can be calculated through forward kinematics, that is, the first speed of the endoscopic workstation is calculated.

[0111] S802. Determine the second speed of the end of the robotic arm by using the actually measured acceleration of the end of the robotic arm.

[0112] In the embodiments of the present application, the actually measured acceleration of the end of the robotic arm can be collected through the gyroscope installed on the robotic arm, and the second speed of the end of the robotic arm is determined by integrating the actually measured acceleration.

[0113] S803. Determine the disturbance speed based on the difference between the first speed and the second speed.

[0114] It should be noted that the vehicle equipped with the robotic arm will shake during driving, thus there will be a disturbance speed.

[0115] S804. Determine the passive movement amount of the robotic arm according to the integral result corresponding to the disturbance speed.

[0116] In the embodiments of the present application, the disturbance speed can be integrated to obtain the integral result corresponding to the disturbance speed.

[0117] S805. Compensate the passive movement amount of the robotic arm through the inverse kinematics of the robotic arm.

[0118] It should be noted that the robotic arm in the embodiments of the present application is arranged on the vehicle. Thus, the movement disturbance of the robotic arm includes the shaking during the driving of the transport vehicle carrying the robotic arm. Therefore, passive movement amount compensation is required.

[0119] It can be seen that in the embodiments of the present application, by determining the passive movement amount of the robotic arm and compensating the passive movement amount of the robotic arm, it is ensured that the end of the robotic arm does not shake with the vehicle, thereby ensuring that the endoscope workstation is in a stable state, and further facilitating the physician to operate the endoscope during the diagnosis and treatment process.

[0120] Please refer to Figure 9 , which is a schematic diagram of the working state of a visual recognition module disclosed in the embodiments of the present application. Figure 9 The visual recognition module in [the figure] includes an infrared projector, a first camera, and a second camera, and h represents the height information of the surface of the object to be measured. Among them, infrared projection can be understood as using the infrared projector for corresponding projection. The infrared projector projects infrared speckles or can also project grid lines. The optical axes of the infrared projector and the first camera, and the optical axis of the second camera form a certain angle to construct a triangular relationship. Please refer to Figure 10 , which is a schematic diagram of grid line deformation disclosed in the embodiments of the present application. The image projected by the infrared projector is as shown in the left sub - figure of Figure 10 . After being modulated by the height information of the object surface into a special grid line pattern, the image captured by the first camera or the second camera is as shown in the right sub - figure of Figure 10 .

[0121] Please refer to Figure 11 , which is a flowchart of another robotic arm control method disclosed in the embodiments of the present application. The visual recognition module includes a first infrared camera and a second infrared camera. The second image may include the first infrared image captured by the first infrared camera and the second infrared image captured by the second infrared camera. The above - mentioned controlling the movement of the robotic arm according to the second image of the carriage space captured by the visual recognition module may include:

[0122] S1101. Establish a point cloud model based on the first infrared image and the second infrared image.

[0123] In the embodiments of the present application, the processor of the visual recognition module can perform phase extraction, phase unwrapping on the speckle feature points in the first infrared image and the speckle feature points in the second infrared image, and then perform matching calculations to obtain a disparity map; establish the depth information of each measured point according to the coordinate system of the visual recognition module and the disparity map; establish a point cloud model according to the depth information. Among them, phase extraction is to generate a sinusoidal fringe through computer programming, project the sinusoidal fringe onto the measured object through a projection device, and use the first infrared camera and the second infrared camera to capture the bending degree of the fringe-modulated object. Phase unwrapping is to demodulate the bent fringe to obtain the phase, and common algorithms such as the four-step phase-shifting method, etc. Matching calculation is to convert the phase into the height of the whole field, and this phase difference represents the height information of the measured object relative to the reference plane, and then substitute it into the phase-height conversion formula to obtain a three-dimensional model, that is, obtain a disparity map.

[0124] S1102. Use the key information template to match with the point cloud model to determine the position information of the measured object.

[0125] In the embodiments of the present application, the key information template refers to a template containing the key information corresponding to the measured object. For example, the key information template may include: the first object template, the endoscope workstation template, the endoscope handle template, the second object template, etc. The measured object may include at least one of the following: the first object, the endoscope workstation, the endoscope handle, the second object, etc.

[0126] In the embodiments of the present application, according to the key information template, the area corresponding to the measured object can be found from the point cloud model to realize the matching of the key information template and the point cloud model, and then the position information of the measured object can be determined based on the area corresponding to the measured object.

[0127] In the embodiments of the present application, the first relative position relationship and the second relative position relationship, and the relative position relationship between different measured objects can be determined according to the position information.

[0128] In the embodiments of the present application, the geometric position transformation of the measured object based on the coordinate system corresponding to the visual recognition module can be expressed as:

[0129]

[0130] Among them, represents the transformation relationship between the measured object and the endoscope workstation under the coordinate system {V} corresponding to the visual recognition module; represents the coordinate change of the measured object, that is, the position of the observed object in the camera's field of view; represents the matrix transformation of the endoscope workstation under the coordinate system {V} corresponding to the visual recognition module; Represents the relative position transformation of the coordinate system between the visual recognition module and the robotic arm; Represents the position transformation of the endoscopic workstation relative to the robotic arm base, which can be calculated based on the joint angles of the robotic arm

[0131] S1103. Segment and fit the point cloud model to generate multiple point cloud regions.

[0132] In the example of this application, the point cloud model can be segmented and fitted by a point cloud segmentation and fitting algorithm to distinguish the point cloud data of different parts, calculate the relatively close regions between multiple point cloud data, set the maximum connectivity distance, and quickly spread to generate multiple point cloud regions.

[0133] S1104. Determine the surface curvature of the point cloud region corresponding to the obstacle in the carriage space.

[0134] It should be noted that due to the limitations of binocular vision, the point cloud data is surface data, so the surface curvature of the corresponding point cloud region can be determined.

[0135] S1105. Predict the back space of the obstacle according to the surface curvature, and generate a package corresponding to the obstacle according to the back space.

[0136] It should be noted that what the camera sees is the protruding part of the surface, and the back of the surface is invisible, there is a field of view blind area. Estimate the back space according to the known surface curvature to form a closed space as the package.

[0137] S1106. Use the package and the position information to perform path planning for the robotic arm to obtain the path planning result.

[0138] It should be noted that the path planning result should make the movement path of the endoscopic workstation in front of the surface, avoiding passing through the invisible part of the surface. If it is necessary to pass through the invisible part of the surface, it is determined whether to pass through by the contact force between the endoscopic workstation and the uncertain obstacle. If the contact force is greater than the preset value, it stops moving to avoid damage.

[0139] Please refer to Figure 12, which is a schematic diagram of a rapidly exploring random tree (RRT) algorithm disclosed in an embodiment of the present application. In the embodiment of the present application, the RRT algorithm is used for path planning. First, initialize the space and define information such as the starting point, target position, number of sampling points, step size t between points, etc. Randomly generate a point in the space, find the point closest to this random point in the set of known tree points, take t step sizes on the line connecting the two points as a new node, and determine whether there are obstacles on the line connecting the new and old nodes. If there are obstacles, discard the node. If there are no obstacles, add the node to the tree set, and loop until there is a new node within the set neighborhood of the target position. In the embodiment of the present application, traverse the valid nodes obtained in the previous round for the second time, from the starting point directly to the last node, and determine whether there are obstacles. If there are no obstacles, pass through the obstacle-free points directly to the last node. If there are obstacles, calculate the second-to-last node, and then calculate the last one from this point. Finally, traverse the valid nodes of the previous round in sequence, and continue backward until passing through the obstacle-free points.

[0140] S1107. Control the movement of the robotic arm according to the path planning result.

[0141] It can be seen that in the embodiment of the present application, through the establishment of the point cloud model and the matching of the key information template with the point cloud model, the position information of the object to be measured can be determined more accurately. Then, based on the inclusion body and the position information, path planning is performed on the robotic arm, which can effectively control the robotic arm to avoid the obstacles recognized in the space, and protect the safety of the endoscope operation through the force feedback mechanism.

[0142] In a possible implementation manner, the method for positioning a portable endoscope system provided in the embodiment of the present application may further include: storing the basic information of the first object, where the basic information includes: face recognition information and hand control instruction definition information.

[0143] In the embodiment of the present application, the basic information of the first object can be written and stored through the touch interface of the display or the keyboard connected to the display.

[0144] Among them, S201 may include: when a start instruction initiated by the first object is recognized according to the basic information, determine the first relative position relationship between the first object in the carriage space of the vehicle and the robotic arm according to the first image of the carriage space of the vehicle collected by the visual recognition module.

[0145] Under the trigger of the start instruction, the visual recognition module can determine the first relative position relationship.

[0146] In a possible implementation, when an end instruction initiated by the first object is recognized based on the basic information, the robotic arm can be controlled to approach the first object and then descend. After the first object installs the endoscope handle at the endoscope workstation, the robotic arm is reset.

[0147] Among them, the start instruction and the end instruction can be hand control instructions issued by the first object through hand movements.

[0148] In the embodiments of the present application, different instructions such as start, end, run, and stop can be included at the same time, and the operation information of the portable endoscope system is announced by voice. During the diagnosis and treatment process, based on the eigenface-based face detection and recognition method, the face is encoded and decoded through machine learning, and the face variance in the image dataset is calculated to determine whether to execute the issued instruction. Among them, a set of eigenfaces is a collection of "standardized face components" determined through statistical analysis of a large number of face images, and then a statistical database is constructed through this collection. In the embodiments of the present application, the face pictures of the licensed operating physicians can be pre-recorded. After matching the physicians before the operation, the portable endoscope system enters the initialization to wait for the physicians to issue control instructions.

[0149] In the embodiments of the present application, different hand control instructions can be defined according to the physician's body posture. Please refer to Figure 13 , which is a schematic diagram of a hand control instruction disclosed in the embodiments of the present application. Using hand movements as control instructions, such as Figure 13 the number "5" in it means removing the endoscope, and it can also be set that the number "1" means waiting for the start of the surgical operation, etc. In the embodiments of the present application, the hand pictures can be obtained through the first camera and the second camera, the palm part is extracted, after binarization processing, the edge contour is extracted, and then the contour semantics are extracted. Among them, the contour center point can be extracted, the connection line between the convex point relative to the parallel contour and the center point is read, and the number of connection lines and the connection line features are obtained to determine the hand gesture number to determine the hand control instruction issued by the physician. Among them, Figure 13 the left sub-diagram in it is the image after binarization processing, Figure 13 the middle sub-diagram in it is the image with the contour extracted, Figure 13 the right sub-diagram in it is the image with the contour semantics extracted.

[0150] It can be seen that in the embodiments of the present application, by storing the basic information of the physician, subsequent instructions initiated by a specific physician can be recognized, avoiding misoperations of the portable endoscope system by other personnel. Moreover, corresponding control instructions can be issued through the hand to control the portable endoscope system, which is convenient for the physician to operate the portable endoscope system.

[0151] An embodiment of the present application further provides a portable endoscope system, which is installed on a vehicle. The system includes: a vision recognition module, a robotic arm, an endoscope workstation, an endoscope, and a robotic arm control module. The endoscope workstation is arranged at the end of the robotic arm, and the endoscope is installed on the endoscope workstation;

[0152] The vision recognition module is configured to collect a first image of the carriage space of the vehicle and determine a first relative position relationship between a first object in the carriage space and the robotic arm;

[0153] The robotic arm control module is configured to control the robotic arm according to the first relative position relationship, so that the endoscope faces the first object;

[0154] The vision recognition module is further configured to collect a second image of the carriage space of the vehicle;

[0155] The robotic arm control module is further configured to control the movement of the robotic arm according to the second image when the endoscope is removed by the first object.

[0156] It can be seen that in the embodiment of the present application, by installing the portable endoscope system on the vehicle, mobile endoscope medical services are realized. Moreover, by arranging the endoscope workstation at the end of the robotic arm and installing the endoscope on the endoscope workstation, the floor area occupied by the endoscope examination equipment is reduced. The robotic arm drives the endoscope workstation to move, so that the problem of vibration of the endoscope workstation during the movement of the vehicle can be avoided to a certain extent. Further, it is beneficial to the development of mobile endoscope medical services, realizing first aid with an endoscope on the vehicle and providing endoscope-related medical services in remote areas. Moreover, by arranging the endoscope workstation at the end of the robotic arm, before the diagnosis and treatment, the robotic arm can be controlled to move according to the relative position relationship, so that the endoscope workstation can be positioned at a position convenient for the first object to remove the endoscope. During the diagnosis and treatment process, the robotic arm can be controlled according to the collected second image, so that the endoscope workstation can be positioned at a position convenient for the first object to operate the endoscope. Thus, through image acquisition, recognition and robotic arm control, the endoscope workstation can be automatically positioned at a suitable position, reducing the time for medical staff to manually move the endoscope workstation to find a suitable diagnosis and treatment position, thereby shortening the diagnosis and treatment time.

[0157] In a possible implementation manner, in the portable endoscope system provided in the embodiment of the present application, the endoscope includes an endoscope handle and a cable;

[0158] The vision recognition module is further configured to determine a second relative position relationship between the first object and a second object in the carriage space according to the second image;

[0159] The visual recognition module is further configured to determine the spatial trajectory of the cable according to the second relative position relationship;

[0160] The visual recognition module is further configured to set a reference curvature of the cable according to the spatial trajectory;

[0161] The robotic arm control module is specifically configured to adjust the distance between the robotic arm and the endoscope handle according to the reference curvature.

[0162] In a possible implementation manner, in the portable endoscope system provided in the embodiments of the present application, the visual recognition module is further configured to determine a first target displacement of the endoscope workstation according to the speed and acceleration of the endoscope handle and the spatial trajectory;

[0163] The robotic arm control module is further configured to adjust the pose of the robotic arm according to the first target displacement.

[0164] In a possible implementation manner, in the portable endoscope system provided in the embodiments of the present application, the robotic arm control module is further configured to determine a first speed of the end of the robotic arm according to the speeds and accelerations of the joints of the robotic arm;

[0165] The robotic arm control module is further configured to determine a second speed of the end of the robotic arm by using the actually measured acceleration of the end of the robotic arm;

[0166] The robotic arm control module is further configured to determine a disturbance speed according to the difference between the first speed and the second speed;

[0167] The robotic arm control module is further configured to determine a passive movement amount of the robotic arm according to an integration result corresponding to the disturbance speed;

[0168] The robotic arm control module is further configured to compensate the robotic arm for the passive movement amount through inverse kinematics of the robotic arm.

[0169] In a possible implementation manner, in the portable endoscope system provided in the embodiments of the present application, the visual recognition module includes a first infrared camera, a second infrared camera, and a processor; the second image includes a first infrared image collected by the first infrared camera and a second infrared image collected by the second infrared camera;

[0170] The first infrared camera is configured to collect the first infrared image;

[0171] The second infrared camera is configured to collect the second infrared image;

[0172] The processor is configured to establish a point cloud model based on the first infrared image and the second infrared image;

[0173] The processor is further configured to match the key information template with the point cloud model to determine the position information of the object to be measured; the object to be measured includes at least one of the following: the first object, the endoscope workstation, and the endoscope;

[0174] The robotic arm control module is specifically configured to control the movement of the robotic arm according to the position information.

[0175] In a possible implementation manner, in the portable endoscope system provided in the embodiments of the present application, the processor is further configured to segment and fit the point cloud model to generate multiple point cloud regions;

[0176] The processor is further configured to determine the surface curvature of the point cloud region corresponding to the obstacle in the carriage space;

[0177] The processor is further configured to predict the back space of the obstacle according to the surface curvature and generate a package corresponding to the obstacle according to the back space;

[0178] The processor is further configured to perform path planning on the robotic arm by using the package and the position information to obtain a path planning result;

[0179] The robotic arm control module is specifically configured to control the robotic arm according to the path planning result.

[0180] In a possible implementation manner, in the portable endoscope system provided in the embodiments of the present application,

[0181] The visual recognition module is further configured to store the basic information of the first object, and the basic information includes: face recognition information and hand control instruction definition information;

[0182] The visual recognition module is further configured to, when recognizing the start instruction initiated by the first object according to the basic information, determine the first relative position relationship between the first object in the carriage space and the robotic arm according to the first image collected by the visual recognition module.

[0183] Furthermore, an embodiment of the present application further provides a vehicle, including: any one of the above portable endoscope systems.

[0184] Furthermore, an embodiment of the present application further provides a positioning device, including:

[0185] A memory for storing instructions;

[0186] A processor for executing instructions in a memory to implement any of the above-described implementation methods of the portable endoscope system positioning method.

[0187] Furthermore, an embodiment of the present application also provides a computer-readable storage medium storing instructions that, when run on a terminal device, cause the terminal device to execute any of the above-described implementation methods of the portable endoscope system positioning method.

[0188] Furthermore, an embodiment of the present application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute any of the above-described implementation methods of the portable endoscope system positioning method.

[0189] From the description of the above embodiments, those skilled in the art can clearly understand that all or part of the steps in the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment or some parts of the embodiments of the present application.

[0190] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0191] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0192] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0193] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A positioning method for a portable endoscope system, characterized in that The method is applied to a portable endoscope system, which is installed on a vehicle. The portable endoscope system includes: a visual recognition module, a robotic arm, an endoscope workstation, and an endoscope. The endoscope includes an endoscope handle and a cable. The endoscope workstation is arranged at the end of the robotic arm, and the endoscope is installed on the endoscope workstation. The method includes: Determine a first relative position relationship between a first object in the vehicle compartment space and the robotic arm according to a first image of the vehicle compartment space collected by the visual recognition module; Control the robotic arm according to the first relative position relationship so that the endoscope faces the first object; When the endoscope is removed by the first object, control the movement of the robotic arm according to a second image of the vehicle compartment space collected by the visual recognition module; The controlling the movement of the robotic arm according to the second image of the vehicle compartment space collected by the visual recognition module includes: Determine a second relative position relationship between the first object and a second object in the vehicle compartment space according to the second image; Determine the spatial trajectory of the cable according to the second relative position relationship; Calculate the maximum curvature of the spatial trajectory, and adjust the distance between the robotic arm and the endoscope handle to ensure that the curvature of the cable is less than or equal to half of the maximum curvature; The method further includes: Determine a first target displacement of the endoscope workstation according to the speed and acceleration of the endoscope handle and the spatial trajectory; Adjust the pose of the robotic arm according to the first target displacement.

2. The method according to claim 1, wherein The method further includes: Determine a first speed of the end of the robotic arm according to the speeds and accelerations of the joints of the robotic arm; Determine a second speed of the end of the robotic arm by using the actually measured acceleration of the end of the robotic arm collected; Determine a disturbance speed according to the difference between the first speed and the second speed; Determine the passive movement amount of the robotic arm according to the integral result corresponding to the disturbance speed; Compensate the robotic arm for the passive movement amount through inverse kinematics of the robotic arm.

3. The method according to claim 1, wherein The visual recognition module includes a first infrared camera and a second infrared camera. The second image includes a first infrared image collected by the first infrared camera and a second infrared image collected by the second infrared camera. The controlling the movement of the robotic arm according to the second image of the vehicle compartment space collected by the visual recognition module includes: Establish a point cloud model according to the first infrared image and the second infrared image; Match the key information template with the point cloud model to determine the position information of the object to be measured. The object to be measured includes at least one of the following: the first object, the endoscope workstation, and the endoscope; Control the movement of the robotic arm according to the position information.

4. The method according to claim 3, wherein The method further includes: Segment and fit the point cloud model to generate multiple point cloud regions; Determine the surface curvature of the point cloud region corresponding to the obstacle in the vehicle compartment space; Predict the back space of the obstacle according to the surface curvature, and generate an enclosure corresponding to the obstacle according to the back space; The controlling the movement of the robotic arm according to the position information includes: Performing path planning on the robotic arm by using the enclosure and the position information to obtain a path planning result; Controlling the movement of the robotic arm according to the path planning result.

5. The method according to claim 1, characterized in that, The method further includes: Storing the basic information of the first object, where the basic information includes: face recognition information and hand control instruction definition information; The determining the first relative position relationship between the first object in the carriage space of the vehicle and the robotic arm according to the first image collected by the visual recognition module includes: When a start instruction initiated by the first object is recognized according to the basic information, determining the first relative position relationship between the first object in the carriage space of the vehicle and the robotic arm according to the first image collected by the visual recognition module.

6. A portable endoscope system, characterized in that, The portable endoscope system is disposed on a vehicle, and the portable endoscope system includes: a visual recognition module, a robotic arm, an endoscope workstation, an endoscope, and a robotic arm control module. The endoscope includes an endoscope handle and a cable. The endoscope workstation is disposed at the end of the robotic arm, and the endoscope is mounted on the endoscope workstation; The visual recognition module is configured to collect a first image of the carriage space of the vehicle, and determine a first relative position relationship between a first object in the carriage space and the robotic arm according to the first image; The robotic arm control module is configured to control the robotic arm according to the first relative position relationship so that the endoscope faces the first object; The visual recognition module is further configured to collect a second image of the carriage space; The robotic arm control module is further configured to control the movement of the robotic arm according to the second image when the endoscope is removed by the first object; The visual recognition module is further configured to determine a second relative position relationship between the first object and a second object in the carriage space according to the second image; The visual recognition module is further configured to determine a spatial trajectory of the cable according to the second relative position relationship; The robotic arm control module is specifically configured to calculate a maximum curvature of the spatial trajectory, and adjust a distance between the robotic arm and the endoscope handle to ensure that the curvature of the cable is less than or equal to half of the maximum curvature; The visual recognition module is further configured to determine a first target displacement of the endoscope workstation according to the speed and acceleration of the endoscope handle and the spatial trajectory; The robotic arm control module is further configured to adjust the pose of the robotic arm according to the first target displacement.

7. A vehicle, characterized in that, The vehicle includes the portable endoscope system according to claim 6.

8. A computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute the portable endoscope system positioning method according to any one of claims 1 to 5.

Images