Puncture point determination method, operating device, and computer-readable storage medium

Abstract

The present specification provides a method for determining a punch point, an operating device and a computer readable storage medium. Based on the method, before a specific operation is performed, the operating device can acquire a target image containing a target of interest in a current field of view of a user, determine a key local image in the target image and a target manipulation type of the target operation, acquire coordinate parameters of a target focus point in the target image based on the key local image, determine position parameters of a target punch point for the target operation according to the coordinate parameters of the target focus point and the target manipulation type, and project the corresponding target punch point in the current field of view of the user according to the position parameters of the target punch point. Thus, the target punch point for the target operation can be intelligently determined, and the target punch point is accurately projected into the current field of view of the user, effectively simplifying the user operation, so that the user can conveniently and efficiently perform the subsequent target operation.

Description

Technical Field

[0001] This manual belongs to the field of medical device technology, and in particular relates to methods for determining drilling points, operating equipment, and computer-readable storage media. Background Technology

[0002] Before performing minimally invasive surgery, doctors usually need to manually measure and mark the puncture points on the corresponding locations on the patient's body based on the measurement results before performing the actual surgical puncture.

[0003] When implementing the above methods, the operation is relatively cumbersome and complicated for doctors, and the marking of punching points mainly relies on the doctor's personal experience, which is prone to errors.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides a method for determining punching points, an operating device, and a computer-readable storage medium. It can intelligently determine the target punching point for the target operation and accurately project the target punching point into the user's current field of view based on the operating device in a holographic projection manner, which facilitates user positioning and observation. This can effectively simplify user operation and enable users to perform subsequent target operations conveniently and efficiently.

[0006] This specification provides a method for determining punch points, applied to an operating device. The method includes: acquiring a target image in the user's current field of view; wherein the target image contains at least a target of interest; determining key local images in the target image and the target manipulation type of the target operation; acquiring coordinate parameters of a target point of interest in the target image based on the key local images; determining the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and projecting the corresponding target punch point into the user's current field of view based on the position parameters of the target punch point.

[0007] This specification also provides a method for determining punch points, applied to a server. The method includes: receiving a key local image sent by an operating device; wherein the operating device acquires a target image in the user's current field of view; and determines a key local image in the target image; processing the key local image using a preset image semantic segmentation model to identify and segment a region of interest from the key local image; calculating the centroid coordinates in the region of interest; and sending the centroid coordinates in the region of interest to the operating device; wherein the operating device determines the coordinate parameters of a target point of interest in the target image based on the centroid coordinates in the region of interest; the operating device also determines the position parameters of a target punch point for the target operation based on the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0008] This specification also provides an operating system, comprising at least an operating device and a server, wherein the operating device is used to acquire a target image in the user's current field of view, and determine key local images in the target image, as well as the target operation type of the target operation; wherein the target image at least contains a target of interest; the operating device is also used to send the key local images to the server; the server is used to process the key local images using a preset image semantic segmentation model to identify and segment the region of interest from the key local images; and calculate the centroid coordinates in the region of interest; the server is also used to send the centroid coordinates of the region of interest to the operating device; the operating device is used to determine the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and project the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0009] This specification also provides a computer device, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the relevant steps of the method for determining the punch points.

[0010] This specification also provides a computer-readable storage medium storing computer instructions that, when executed, perform the following steps: acquiring a target image in the user's current field of view; wherein the target image at least contains a target of interest; determining key local images in the target image and the target manipulation type of the target operation; acquiring coordinate parameters of a target point of interest in the target image based on the key local images; determining position parameters of a target punch point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and projecting the corresponding target punch point into the user's current field of view based on the position parameters of the target punch point.

[0011] Based on the punch point determination method, operating device, and computer-readable storage medium provided in this manual, before the user performs a specific target operation using the operating device, the operating device can first acquire a target image containing the target of interest in the user's current field of view; determine the key local images in the target image, as well as the target manipulation type of the target operation to be performed; then, based on the key local images, obtain the coordinate parameters of the target interest point in the target image; then, based on the coordinate parameters of the target interest point and the target manipulation type, automatically determine the position parameters of the target punch point for the target operation; and project the corresponding target punch point into the user's current field of view based on the position parameters of the target punch point. This allows for the intelligent and automatic determination of the target punch point for the target operation, and the accurate projection of the target punch point into the user's current field of view based on the operating device through holographic projection or other methods, facilitating user positioning and observation during operation, effectively simplifying user operation, and enabling users to perform subsequent target operations conveniently and efficiently. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a method for determining drilling points provided in one embodiment of this specification;

[0014] Figure 2 This is a schematic diagram illustrating a scenario in which an operating device, based on the method for determining the punching point provided in the embodiments of this specification, performs a surgical operation on a target patient.

[0015] Figure 3 This is a schematic diagram of the structural composition of the operating device provided in one embodiment of this specification;

[0016] Figure 4 This is a scenario example where a user triggers the acquisition of a target image by operating the main interface projected by the device.

[0017] Figure 5 This is a scene illustration where a user uses the menu interface projected by the operating device to determine key local images.

[0018] Figure 6 This is a scenario example, illustrating how a user uses the menu interface projected by the operating device to determine a target location point.

[0019] Figure 7This is a scenario example where a user uses the projection technique selection interface of an operating device to determine the type of operation for a target operation.

[0020] Figure 8 This is a scenario example illustrating how a user uses the menu interface projected onto an operating device to locate surgical drilling points.

[0021] Figure 9 This is a schematic diagram showing the location of the target drilling point determined for cholecystectomy in a scenario example;

[0022] Figure 10 This is a schematic diagram of the location of the target punch point determined for a prostatectomy procedure, presented in a scenario example.

[0023] Figure 11 This is a schematic diagram showing the location of the target punch point determined for anterior nephrectomy in a scenario example;

[0024] Figure 12 This is a flowchart illustrating a method for determining punching points provided in another embodiment of this specification;

[0025] Figure 13 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0026] Figure 14 This is a schematic diagram of the structural composition of a punching point determination device provided in one embodiment of this specification;

[0027] Figure 15 This is a schematic diagram of the structural composition of a device for determining the punching point provided in another embodiment of this specification. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0029] See Figure 1 As shown in the embodiments of this specification, a method for determining drilling points is provided, wherein the method is specifically applied to the side of the operating device. In specific implementation, the method may include the following:

[0030] S101: Acquire a target image in the user's current field of view; wherein, the target image contains at least the target of interest;

[0031] S102: Identify the key local images in the target image, and the target manipulation type of the target operation;

[0032] S103: Based on the key local image, obtain the coordinate parameters of the target interest point in the target image;

[0033] S104: Determine the position parameters of the target drilling point for the target operation based on the coordinate parameters of the target focus point and the target manipulation type;

[0034] S105: Based on the position parameters of the target punching point, project the corresponding target punching point into the user's current field of view based on the operating device.

[0035] In some embodiments, see Figure 2 As shown, the method for determining the aforementioned punching points can be specifically applied to one side of the operating device. Specifically, the operating device can be an AR-enabled device. It can also be connected to a server. Specifically, the operating device can be a surgical device assisting doctors in surgical procedures, a construction device assisting engineers in construction operations, or a maintenance device assisting repair workers in inspecting and maintaining large electrical systems, etc. It should be noted that the operating devices listed above are only illustrative. In actual implementation, depending on the specific application scenario and processing requirements, the operating device may include other types of operating devices. This specification does not limit this.

[0036] Specifically, AR (Augmented Reality) can refer to a technology that cleverly integrates virtual information with the real world. It mostly uses a variety of technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world, so that the two types of information can complement each other, thereby achieving "enhancement" of the real world.

[0037] For details, please refer to Figure 2 As shown, the aforementioned operating device can specifically be an AR head-mounted device, such as AR glasses or an AR helmet that supports user surgical operations. Specifically, for example, when a user performs a target operation on a target of interest (e.g., a surgeon performing a target surgery on a patient), they can wear the aforementioned AR head-mounted device to observe the operating environment (e.g., the surgical environment) and the target of interest (e.g., the patient to be operated on) before performing the specific target operation.

[0038] Further reading Figure 3As shown, the aforementioned operating equipment may specifically include: an observation window, a camera, a processor, and a signal transceiver, etc.

[0039] The aforementioned observation window provides users with a field of view, allowing them to observe the actual operating environment and the patient awaiting surgery. Furthermore, the observation window can also embed a display screen. Correspondingly, the observation window can display virtual elements such as the user interface, character markers, and patterns to the user.

[0040] The aforementioned camera can be positioned near the observation window to capture images within the user's field of view. The aforementioned processor can support local data processing on the operating device, involving relatively simple processes and small data volumes.

[0041] The aforementioned transceivers can connect to the server via wired or wireless means to facilitate data interaction. The server can specifically include a cloud server or a locally deployed terminal server. The server can utilize its processing resources to assist the operating equipment in completing relatively complex data processing procedures involving relatively large volumes of data.

[0042] Specifically, the aforementioned server can be a backend server capable of data transmission and data processing. Specifically, the server can be, for example, an electronic device with data processing, storage, and network interaction capabilities. Alternatively, the server can be a software program running on the electronic device that provides support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server can be a single server, several servers, or a server cluster formed by several servers.

[0043] In practice, users can wear the aforementioned operating device to perform specific procedures on target patients, such as minimally invasive abdominal surgery (which can be referred to as the target procedure). Before performing the target procedure, users can observe the actual operating environment and the target of interest (e.g., the target patient's body) through the operating device's viewing window; simultaneously, the operating device can capture a target image containing the target of interest in the user's current field of view via a camera. Furthermore, the operating device can use a processor to identify and extract a key local image containing the region of interest (e.g., the target patient's navel area) from the target image; then, this key local image is sent to the server via a signal transceiver.

[0044] After receiving the key local image, the server can process the key local image using a preset image semantic segmentation model to identify and segment the region of interest from the key local image; then calculate the centroid coordinates of the region of interest and feed them back to the operating device.

[0045] After receiving the centroid coordinates of the region of interest via a transceiver, the operating device can first convert the two-dimensional plane centroid coordinates into spatial coordinates using a processor to obtain the coordinate parameters of the target point of interest in three-dimensional space. Then, based on the target manipulation type and the coordinate parameters of the target point of interest, it automatically determines the position parameters of the target punch point for the target operation. Furthermore, based on the position parameters of the target punch point, the operating device can project the corresponding target punch point holographically into the user's field of view through the operating device's observation window, using augmented reality technology.

[0046] In this way, users can navigate and locate the target punching point in their field of vision, and conveniently and accurately punch holes in targets of interest, as well as perform other target operations.

[0047] Of course, it should be noted that the operating devices listed above are only illustrative. In practice, the method for determining the perforation points described above can also be applied to surgical robots. Specifically, for example, the surgical robot can use the above method to acquire and automatically perform corresponding surgical operations on the target patient based on the field of view projected with the target perforation points.

[0048] In some embodiments, after wearing the operating device, the user can observe the current operating environment and the target of interest (ROI) to be operated on through the field of view provided by the operating device's viewing window or display screen. Simultaneously, the user can initiate a trigger operation through the operating device. Correspondingly, the operating device can respond to the trigger operation by acquiring a target image within the user's current field of view. The target image at least includes the ROI. Specifically, for example, in a surgical setting, the ROI can be the target patient to be operated on, and the target patient can be lying supine on a fixed operating table.

[0049] For specific implementation, please refer to Figure 4As shown, after wearing the operating device, the user observes the current operating environment and the target of interest to be operated on through the field of view provided by the observation window or display screen of the operating device. Simultaneously, the observation window or display screen can also display the main interface in the user's field of view based on augmented reality technology. The main interface at least includes an icon for the auxiliary positioning system. The user can initiate a trigger operation by clicking the icon of the auxiliary positioning system in the main interface. Correspondingly, the operating device receives and responds to the user's trigger operation, acquiring an image of the target of interest within the user's current field of view as the target image.

[0050] In addition, the operating device can also listen to voice commands initiated by the user; when it hears that the user's voice command contains a corresponding trigger keyword (e.g., "get image"), it determines that the user has initiated a trigger operation; and then it can capture the target image in the user's current field of vision.

[0051] In some embodiments, the key local image can be specifically understood as a local image of the target image containing the region of interest.

[0052] The aforementioned area of ​​interest can be understood as the area used for reference and positioning during drilling. Specifically, for example, in a surgical scenario involving abdominal surgery, the area of ​​interest might include the navel region. In a repair scenario, the area of ​​interest might include the outer protective shell of a faulty area. It should be noted that the navel region listed above is merely illustrative. In practice, depending on the specific application scenario, the area of ​​interest may include other types of areas. This specification does not limit this.

[0053] In practice, users can define and select key local images in the target image, or the operating device can intelligently select key local images in the target image. This allows for subsequent fine-grained image recognition processing only on the key local images in the target image, thus avoiding wasting processing resources and time processing image areas other than key local images in the target image and improving image recognition processing efficiency.

[0054] In some embodiments, the determination of key local images in the target image as described above may include: receiving a user's region selection operation on the target image; and determining and cropping the user-selected region image in the target image as the key local image based on the region selection operation.

[0055] For details, please refer to Figure 5As shown, in a surgical setting, the aforementioned observation window or display screen can also display a menu interface in the user's field of vision based on augmented reality technology. Specifically, this menu interface can include function icons such as "Place Head," "Target Selection," "Drill Positioning," and "Clear Marks."

[0056] Specifically, for example, see Figure 5 As shown, users can click the "Target Selection" icon to trigger a user-defined selection box for key local images. Correspondingly, the operating device can respond to this action by generating a virtual selection box in the user's field of view. The user can then move this selection box to the area of ​​interest within their field of view and use it to select that area to obtain the desired key local image.

[0057] When making a selection, users can also adjust the shape and size of the selection box by using corresponding gestures, based on the specific shape and size of the area of ​​interest in the current field of view, so as to more accurately select the key local image they need.

[0058] In some embodiments, the determination of key local images in the target image may further include the following: processing the target image using a preset image recognition model to obtain the corresponding target image recognition result; and determining and cropping the corresponding region from the target image as the required key local image based on the target image recognition result.

[0059] Specifically, the aforementioned preset image recognition model can be deployed locally on the operating device. Specifically, the preset image recognition model can be a neural network model involving relatively low processing power, capable of roughly identifying image regions containing areas of interest from an image, and further automatically segmenting those image regions from the image.

[0060] In practice, the operating device can automatically determine and extract key local images from the target image by using the aforementioned preset image recognition model.

[0061] Before implementation, the above-mentioned preset image recognition model can be trained in the following way: acquire a first sample image; wherein the first sample image contains a region of interest; select the region of interest in the first sample image and label it to obtain labeled first sample data; construct a first initial model containing an initial classification network and an initial segmentation network; train the first initial model using the labeled first sample data to obtain a preset image recognition model with the required accuracy.

[0062] In some embodiments, the user can also set a positioning point in the current field of view so that the positioning point parameters can be used later to more accurately determine the position parameters of the target drilling point.

[0063] Specifically, for example, see Figure 6 As shown, the user can click the "Place Head" icon in the menu interface displayed in the field of view. Correspondingly, the operating device can respond to the user's action by generating a virtual positioning point in the user's field of view. The user can then move this positioning point to the target patient's head location within the field of view, using it as the target positioning point for subsequent determination of the target punching point. The coordinate parameters of the target positioning point are recorded for later use.

[0064] Of course, in practice, depending on the specific situation and processing needs, users can also move the virtual positioning point to the target patient's nose or neck in the field of vision, as guided by the interface, and use it as the target positioning point.

[0065] In some embodiments, the user can also set the operation type of the target operation through the operating device. Accordingly, the operating device can determine the operation type of the target operation as the target operation type.

[0066] In some embodiments, determining the target manipulation type of the target operation as described above may include: projecting a manipulation type selection interface (e.g., a technique selection interface) into the user's current field of view based on the operating device; receiving and responding to the user's manipulation type selection operation on the manipulation type selection interface; and determining the manipulation type selected by the user as the target manipulation type of the target operation.

[0067] Specifically, for example, see Figure 7 As shown, the operating device can also utilize augmented reality technology to project a surgical procedure selection interface into the user's field of vision. This interface can include multiple selectable manipulation types, such as nephrectomy, prostatectomy, cholecystectomy, and hepatectomy. The user can then select "prostatectomy" from the interface based on their surgical needs. The operating device can then receive and respond to this user's action, determining that the target manipulation type for the procedure to be performed is prostatectomy.

[0068] In some embodiments, after determining the key local images in the target image, the operating device can send the key local images to the server so that the server's processing resources can be used to efficiently and accurately determine the coordinate parameters of the target interest points in the target image.

[0069] Specifically, the aforementioned target focus can be understood as the centroid (or center) of the region of interest.

[0070] In practice, for example, refer to Figure 8As shown, the user can click the "Drill Point Location" icon in the menu interface displayed in the field of view. Correspondingly, the operating device can respond to the user's operation by sending the aforementioned key local image to the server; then, the operating device can receive the coordinate parameters of the target point of interest obtained by the server processing the key local image; and based on the coordinate parameters of the target point of interest, automatically locate the target drill point for the target operation; and display the target drill point in the user's current field of view through projection, so that the user can perform drill location. (See reference...) Figure 8 As shown.

[0071] In some embodiments, obtaining the coordinate parameters of the target interest point in the target image based on the key local image may specifically include the following:

[0072] S1: The key local image is sent to the server; wherein the server is configured with at least a preset image semantic segmentation model; the server uses the preset image semantic segmentation model to process the key local image, identify and segment the region of interest from the key local image; and calculate the centroid coordinates in the region of interest;

[0073] S2: Receive the centroid coordinates of the region of interest;

[0074] S3: Based on the preset coordinate transformation relationship, the centroid coordinates in the region of interest are converted into corresponding spatial coordinates to obtain the coordinate parameters of the target point of interest.

[0075] Specifically, the aforementioned preset image semantic segmentation model can be deployed on a server side. This preset image semantic segmentation model can be a neural network model involving relatively large processing power, capable of accurately identifying the image region of interest. The aforementioned preset coordinate transformation relationship can be obtained in advance by testing the operating device, acquiring the data, and then registering the relevant coordinate systems based on the test data.

[0076] In practice, the server can use a preset image semantic segmentation model to process key local images and obtain corresponding processing results. Based on the processing results, the contour region of the region of interest is precisely determined in the key local image. Then, combined with the contour region, the centroid of the region of interest is determined in the key local image, and the corresponding centroid coordinates are calculated.

[0077] Furthermore, the server can send the centroid coordinates of the region of interest to the operating device. Correspondingly, the operating device can obtain the centroid coordinates of the region of interest.

[0078] However, at this point, the operating device directly acquires coordinates in a two-dimensional plane, while the user's field of vision often displays a true three-dimensional image. Therefore, to match the user's actual field of vision, the operating device can also convert the centroid coordinates of the two-dimensional plane in the area of ​​interest into corresponding three-dimensional spatial coordinates based on a preset coordinate transformation relationship, thereby obtaining the coordinate parameters of the target point of interest.

[0079] Specifically, in a surgical setting, the aforementioned target focus could be, for example, the centroid or center of the patient's umbilical region within the field of vision.

[0080] It should be added that, provided the operating device's local processor performance allows, the aforementioned preset image semantic segmentation model can also be deployed locally on the operating device. Accordingly, the operating device can use the preset image semantic segmentation model locally to determine the coordinate parameters of the required target interest points.

[0081] In some embodiments, before implementation, a preset image semantic segmentation model can be trained in the following manner: acquiring a second sample image containing a region of interest; marking the region of interest along the contour line of the region of interest in the second sample image to obtain a labeled second sample image; and training a second initial model using the labeled second sample image to obtain a preset image semantic segmentation model that meets the requirements.

[0082] In some embodiments, the location parameters of the target drilling point for the target operation are determined based on the coordinate parameters of the target point of interest and the target manipulation type. In specific implementations, this may include the following:

[0083] S1: Based on the target manipulation type, determine the matching target initial drilling point parameters from the preset manipulation database;

[0084] S2: Determine the position parameters of the target punching point based on the coordinate parameters of the target focus point and the initial punching point parameters of the target.

[0085] The aforementioned preset manipulation database can store multiple initial punch point parameters, each corresponding to a manipulation type. Each initial punch point parameter contains at least one or more position parameters of punch points associated with the target point of interest.

[0086] Before implementation, multiple historical punching records can be obtained. These records contain at least the location parameters of the target interest point and the location parameters of the punching points. Based on the operation type of the historical punching records, the records are divided into multiple data groups. Each data group corresponds to one operation type. Clustering is performed on the historical punching records in each data group to determine the common relationship between the location parameters of the target interest point and the location parameters of the punching points, thus obtaining the initial punching point parameters for each operation type.

[0087] In practice, the coordinate parameters of the target point of interest can be substituted into the target initialization punching point parameters to calculate the position parameters of the target punching point for the target point of interest, thereby determining the target punching point for the target operation.

[0088] For specific implementation, please refer to Figure 9 , Figure 10 ,as well as Figure 11 As shown, for different manipulation types, the initial punch point parameters used are different, so even if the same target interest is used, the determined target punch point can be completely different. The aforementioned target interest can be a punch point within the target punch point, or it can be a reference point adjacent to the target punch point.

[0089] Specifically, in surgical settings, for example, Figure 9 The corresponding manipulation type is cholecystectomy. The blackened point 3 is the target focus point. The identified target drilling points include: point 1, point 2, point 3 and point 4. The target focus point itself is also a target drilling point.

[0090] For example, Figure 10 The corresponding manipulation type is nephrectomy. The blackened point A is the target focus. The identified target punching points include: point 1, point 2, point 3 and point 4. The target focus itself is not the target punching point.

[0091] For example, Figure 11 The corresponding manipulation type is prostatectomy. The blackened point 3 is the target focus point. The identified target punching points include: point 1, point 2, point 3 and point 4. The target focus point itself is also a target punching point.

[0092] In some embodiments, the determination of the target drilling point's position parameters based on the coordinate parameters of the target point of interest and the target initial drilling point parameters may specifically include the following:

[0093] S1: Obtain the coordinate parameters of the target location point in the current field of view;

[0094] S2: Construct the perpendicular bisector of the target based on the coordinate parameters of the target point of interest and the target location point;

[0095] S3: Determine the position parameters of the target drilling point based on the coordinate parameters of the perpendicular bisector, the target point of interest, and the target initial drilling point parameters.

[0096] For specific implementation, please refer to Figure 9 , Figure 10 ,as well as Figure 11 As shown, by connecting the target interest point and the target positioning point, a perpendicular bisector along the longitudinal direction of the target can be obtained, for example, the y-axis. This perpendicular bisector can then be used for auxiliary positioning. Finally, using the coordinate parameters of the target interest point and the initial drilling point parameters, the position parameters of the drilling point can be determined.

[0097] Based on the above embodiments, by introducing and utilizing the perpendicular bisector constructed based on the target positioning point and the target interest point, and combining the coordinate parameters of the target interest point and the target initial drilling point parameters, the position parameters of the target drilling point can be determined more accurately.

[0098] In some embodiments, after determining the position parameters of the target punching point based on the coordinate parameters of the target point of interest and the target initial punching point parameters, the method may further include the following:

[0099] S1: Obtain the feature parameters of the target of interest;

[0100] S2: Adjust the position parameters of the target drilling point according to the feature parameters of the target of interest.

[0101] In a surgical setting, the characteristic parameters of the target of interest mentioned above could be, for example, the body shape characteristics of the target patient. These body shape characteristics include at least one of the following: height, weight, waist circumference, shoulder width, body fat percentage, etc.

[0102] In practice, the type of target of interest can be determined based on the characteristic parameters of the target of interest (e.g., the body type of the target patient: thin, normal, fat, etc.). Then, the position parameters of the determined target punching point can be adjusted in a targeted manner according to the type of target of interest, so that the adjusted target punching point is more suitable for the target of interest.

[0103] Specifically, for example, for obese patients, the spacing between adjacent perforation points can be increased. For instance, the spacing between adjacent perforation points can be increased from 5 centimeters to 7 centimeters. Based on these adjusted perforation points, users can perform surgical perforations more effectively on the target patient.

[0104] In some embodiments, the above-mentioned projection of the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point may include the following: based on augmented reality technology, the corresponding target punch point is projected in the user's current field of view based on the position parameters of the target punch point through holographic projection.

[0105] Based on the above embodiments, the advantages of AR technology can be fully utilized to project the determined target punching points into the user's field of vision in a holographic manner. At this time, the virtual target punching points can be directly displayed in the user's real observation field of vision obtained based on the operating device. This allows the user to conveniently and intuitively use the virtual target punching points directly displayed in the field of vision for punching location. At the same time, there is no need to make additional markings on the target of interest, reducing damage to the target of interest.

[0106] In some embodiments, after projecting the corresponding target punching point into the user's current field of view based on the position parameters of the target punching point, the method may further include the following:

[0107] S1: Receives user adjustments for the target punching points;

[0108] S2: Adjust the position of the target punch point in the current field of view according to the adjustment operation.

[0109] For details, please refer to Figure 8 As shown, for the target punch point projected in the current field of view, the user can also move the position of the target punch point within a preset range based on the specific situation and personal experience, so as to better match the punching of the target of interest.

[0110] Furthermore, when the operating device detects that the movement of the target punch point exceeds a preset range, it can determine that there is an error in the currently determined position of the target punch point. At this time, it can obtain the correction data input by the user, and based on the correction data, re-determine and display the target punch point.

[0111] In some embodiments, when a user completes the punching operation and no longer needs to use the target punch point, the user can click the "Clear Mark" icon in the menu interface displayed in the user's field of view. Correspondingly, the operating device can respond to the user's operation by eliminating the target punch point from the user's current field of view, thus avoiding interference with the user's subsequent target operations.

[0112] As can be seen from the above, based on the punch point determination method provided in the embodiments of this specification, before the user performs a target operation using the operating device, the operating device can first acquire a target image of the target patient containing the target operation to be performed in the user's current field of view; and determine the key local images in the target image, as well as the target manipulation type of the target operation; then, based on the key local images, obtain the coordinate parameters of the target interest points in the target image; then, based on the coordinate parameters of the target interest points and the target manipulation type, automatically determine the position parameters of the target punch point for the target operation; and based on the position parameters of the target punch point, project the corresponding target punch point into the user's current field of view based on the operating device. Thus, the target punch point for the target operation can be intelligently determined, and the aforementioned target punch point can be accurately projected into the user's current field of view, facilitating user positioning and observation, effectively simplifying user operation, and enabling the user to perform subsequent target operations conveniently and efficiently.

[0113] See Figure 12 As shown in the embodiments of this specification, another method for determining the punching points is also provided, applied to the server side. Specifically, this method may include the following:

[0114] S1201: Receive a key local image sent by the operating device; wherein the operating device acquires a target image in the user's current field of view; and determines a key local image in the target image;

[0115] S1202: Process key local images using a preset image semantic segmentation model to identify and segment regions of interest from key local images;

[0116] S1203: Calculate the centroid coordinates in the region of interest;

[0117] S1204: Send the centroid coordinates of the region of interest to the operating device; wherein, the operating device determines the coordinate parameters of the target point of interest in the target image based on the centroid coordinates of the region of interest; the operating device also determines the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0118] This specification also provides a computer device, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: acquiring a target image in the user's current field of view; wherein the target image at least contains a target of interest; determining key local images in the target image and the target manipulation type of the target operation; acquiring coordinate parameters of a target point of interest in the target image based on the key local images; determining the position parameters of a target punch point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and projecting the corresponding target punch point into the user's current field of view based on the position parameters of the target punch point.

[0119] To execute the above instructions more accurately, please refer to... Figure 13 As shown in the embodiments of this specification, a server is also provided, wherein the server includes a network communication port 1301, a processor 1302 and a memory 1303, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0120] Specifically, the network communication port 1301 can be used to receive key local images sent by the operating device; wherein the operating device acquires the target image in the user's current field of view; and determines the key local images in the target image.

[0121] The processor 1302 can specifically be used to process key local images using a preset image semantic segmentation model to identify and segment regions of interest from the key local images; calculate the centroid coordinates of the regions of interest; and send the centroid coordinates of the regions of interest to the operating device; wherein, the operating device determines the coordinate parameters of the target point of interest in the target image based on the centroid coordinates of the regions of interest; the operating device also determines the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0122] The memory 1303 can be used to store the corresponding instruction program.

[0123] In this embodiment, the network communication port 1301 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0124] In this embodiment, the processor 1302 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0125] In this embodiment, the memory 1303 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0126] See Figure 2 As shown in the illustration, this specification also provides an operating system, which may include at least: an operating device and a server, wherein,

[0127] The operating device can be used to acquire a target image in the user's current field of view, determine key local images in the target image, and identify the target operation type; wherein the target image at least contains a target patient for whom the target operation is to be performed; the operating device is also used to send the key local images to a server;

[0128] Specifically, the server can be used to process key local images using a preset image semantic segmentation model to identify and segment regions of interest from the key local images; and calculate the centroid coordinates of the regions of interest; the server is also used to send the centroid coordinates of the regions of interest to the operating device;

[0129] Specifically, the operating device can be used to determine the position parameters of the target punching point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and to project the corresponding target punching point into the user's current field of view based on the position parameters of the target punching point.

[0130] This specification also provides a computer device, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the relevant steps of the method for determining the punch points. Specifically, the computer device may include operating devices and / or servers, etc.

[0131] This specification also provides a computer-readable storage medium based on the above-described method for determining punch points. The computer-readable storage medium stores computer program instructions that, when executed, implement the following steps: acquiring a target image in the user's current field of view; wherein the target image at least contains a target of interest; determining key local images in the target image and the target manipulation type of the target operation; acquiring coordinate parameters of a target point of interest in the target image based on the key local images; determining the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest and the target manipulation type; and projecting the corresponding target punch point into the user's current field of view based on the position parameters of the target punch point.

[0132] This specification also provides another computer-readable storage medium based on the above-described method for determining punch points. The computer-readable storage medium stores computer program instructions that, when executed, implement the following steps: receiving a key local image sent by an operating device; wherein the operating device acquires a target image in the user's current field of view; and determines a key local image in the target image; processes the key local image using a preset image semantic segmentation model to identify and segment a region of interest from the key local image; calculates the centroid coordinates in the region of interest; and sends the centroid coordinates in the region of interest to the operating device; wherein the operating device determines the coordinate parameters of a target point of interest in the target image based on the centroid coordinates in the region of interest; the operating device also determines the position parameters of a target punch point for the target operation based on the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0133] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0134] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0135] See Figure 14 As shown, at the software level, this specification also provides a device for determining the punching point, which may specifically include the following structural modules:

[0136] The first acquisition module 1401 is specifically used to acquire a target image in the user's current field of view; wherein, the target image contains at least a target of interest;

[0137] The first determining module 1402 can be used to determine the key local images in the target image and the target manipulation type of the target operation;

[0138] The second acquisition module 1403 can be specifically used to acquire the coordinate parameters of the target interest point in the target image based on the key local image;

[0139] The second determining module 1404 can be used to determine the position parameters of the target drilling point for the target operation based on the coordinate parameters of the target interest point and the target manipulation type.

[0140] The projection module 1405 can be used to project the corresponding target punching point into the user's current field of view based on the position parameters of the target punching point.

[0141] In some embodiments, when the first determining module 1402 is specifically implemented, the key local image in the target image can be determined in the following manner: receiving a user's region selection operation on the target image; determining and cropping the region image selected by the user in the target image as the key local image based on the region selection operation.

[0142] In some embodiments, when the first determining module 1402 is specifically implemented, the key local image in the target image can also be determined in the following way: the target image is processed using a preset image recognition model to obtain the corresponding target image recognition result; based on the target image recognition result, the corresponding region of the image is determined and cropped from the target image as the key local image.

[0143] In some embodiments, when the first determining module 1402 is specifically implemented, the target operation type of the target operation can be determined in the following manner: projecting an operation type selection interface into the user's current field of view based on the operating device; receiving and responding to the user's operation type selection operation on the operation type selection interface, and determining the operation type selected by the user as the target operation type of the target operation.

[0144] In some embodiments, when the second acquisition module 1403 is specifically implemented, it can acquire the coordinate parameters of the target interest point in the target image based on the key local image in the following manner: sending the key local image to a server; wherein the server is configured with at least a preset image semantic segmentation model; the server processes the key local image using the preset image semantic segmentation model, identifies and segments the interest region from the key local image; and calculates the centroid coordinates in the interest region; receives the centroid coordinates in the interest region; and converts the centroid coordinates in the interest region into corresponding spatial coordinates according to a preset coordinate transformation relationship to obtain the coordinate parameters of the target interest point.

[0145] In some embodiments, when the second determining module 1404 is specifically implemented, the position parameters of the target punch point for the target operation can be determined according to the coordinate parameters of the target interest point and the target operation type in the following manner: according to the target operation type, the matching target initial punch point parameters are determined from the preset operation database; the position parameters of the target punch point are determined according to the coordinate parameters of the target interest point and the target initial punch point parameters.

[0146] In some embodiments, when the second determining module 1404 is specifically implemented, the position parameters of the target punching point can be determined according to the coordinate parameters of the target interest point and the target initial punching point parameters in the following manner: obtaining the coordinate parameters of the target positioning point in the current field of view; constructing a perpendicular bisector for the target of interest according to the coordinate parameters of the target interest point and the target positioning point; and determining the position parameters of the target punching point according to the perpendicular bisector, the coordinate parameters of the target interest point, and the target initial punching point parameters.

[0147] In some embodiments, after determining the position parameters of the target punching point based on the coordinate parameters of the target point of interest and the target initial punching point parameters, the device can also be used to obtain the feature parameters of the target of interest and adjust the position parameters of the target punching point based on the feature parameters of the target of interest.

[0148] In some embodiments, when the projection module 1405 is specifically implemented, it can project the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point in the following manner: based on augmented reality technology, the corresponding target punch point is projected in the user's current field of view based on the position parameters of the target punch point through holographic projection.

[0149] In some embodiments, after the target punch point is projected into the user's current field of view based on the position parameters of the target punch point, the device can also be used to receive the user's adjustment operation on the target punch point; and adjust the position of the target punch point in the current field of view according to the adjustment operation.

[0150] See Figure 15 As shown, at the software level, this specification also provides another device for determining the punching point, which may specifically include the following structural modules:

[0151] The receiving module 1501 is specifically used to receive key local images sent by the operating device; wherein the operating device acquires the target image in the user's current field of view; and determines the key local images in the target image;

[0152] The processing module 1502 can be used to process key local images using a preset image semantic segmentation model in order to identify and segment the region of interest from the key local images.

[0153] The calculation module 1503 can be used to calculate the centroid coordinates in the region of interest.

[0154] The sending module 1504 is specifically used to send the centroid coordinates of the region of interest to the operating device; wherein, the operating device determines the coordinate parameters of the target point of interest in the target image based on the centroid coordinates of the region of interest; the operating device also determines the position parameters of the target punch point for the target operation based on the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

[0155] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0156] As can be seen from the above, based on the punching point determination device provided in the embodiments of this specification, before the user performs a surgical operation using the operating device, the operating device can acquire a target image of the target patient containing the target operation to be performed in the user's current field of vision; determine the key local images in the target image and the target manipulation type of the target operation; then, based on the key local images, obtain the coordinate parameters of the target interest point in the target image; then, based on the coordinate parameters of the target interest point and the target manipulation type, determine the position parameters of the target punching point for the target operation; and project the corresponding target punching point into the user's current field of vision according to the position parameters of the target punching point. Thus, the target punching point for the target operation can be intelligently determined, and the aforementioned target punching point can be accurately projected into the user's current field of vision, effectively simplifying the user's operation and enabling the user to perform subsequent target operations conveniently and efficiently.

[0157] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0158] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0159] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0160] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0161] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0162] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A method for determining punching points, characterized in that, Applied to operating equipment, the method includes: Acquire a target image within the user's current field of view; wherein the target image contains at least the target of interest; The key local images in the target image and the target manipulation type of the target operation are identified; the key local images are local images in the target image that contain a region of interest; the region of interest is the area used for reference positioning in the punching process. Based on the key local image, the coordinate parameters of the target interest point in the target image are obtained; wherein, the coordinate parameters of the target interest point are obtained in the following manner: the key local image is processed using a preset image semantic segmentation model to obtain the corresponding processing result; according to the processing result, the contour region of the interest region is determined in the key local image; and according to the contour region of the interest region, the centroid coordinates in the interest region are determined; according to a preset coordinate transformation relationship, the centroid coordinates are converted into the corresponding three-dimensional spatial coordinates to obtain the coordinate parameters of the target interest point. Based on the coordinate parameters of the target focus point and the target manipulation type, determine the position parameters of the target drilling point for the target operation; Based on the position parameters of the target punching point, the corresponding target punching point is projected into the user's current field of view based on the operating device.

2. The method for determining the punching point according to claim 1, characterized in that, Key local images in the target image were identified, including: Receives user selection of a region from a target image; Based on the region selection operation, the region image selected by the user is determined and cropped from the target image as the key local image.

3. The method for determining the punching point according to claim 1, characterized in that, Identifying key local images in the target image also includes: The target image is processed using a pre-defined image recognition model to obtain the corresponding target image recognition result; Based on the target image recognition results, the corresponding region of the image is determined and cropped from the target image as the key local image.

4. The method for determining the punching point according to claim 1, characterized in that, Determine the target manipulation type of the target operation, including: The operation type selection interface is projected onto the user's current field of view based on the operating device; Receive and respond to the user's operation type selection operation on the operation type selection interface, and determine the operation type selected by the user as the target operation type.

5. The method for determining the punching point according to claim 1, characterized in that, Based on the key local image, the coordinate parameters of the target interest point in the target image are obtained, including: The key local image is sent to the server; wherein the server is configured with at least a preset image semantic segmentation model; the server uses the preset image semantic segmentation model to process the key local image, identify and segment the region of interest from the key local image; and calculate the centroid coordinates of the region of interest. Receive the centroid coordinates of the region of interest; According to the preset coordinate transformation relationship, the centroid coordinates in the region of interest are converted into corresponding spatial coordinates to obtain the coordinate parameters of the target point of interest.

6. The method for determining the punching point according to claim 1, characterized in that, Based on the coordinate parameters of the target point of interest and the target manipulation type, the position parameters of the target drilling point for the target manipulation type are determined, including: Based on the target manipulation type, the matching target initial drilling point parameters are determined from the preset manipulation type database; Based on the coordinate parameters of the target point of interest and the initial drilling point parameters, the position parameters of the target drilling point are determined.

7. The method for determining the punching point according to claim 6, characterized in that, Based on the coordinate parameters of the target point of interest and the initial drilling point parameters, the position parameters of the target drilling point are determined, including: Obtain the coordinate parameters of the target location point in the current field of view; Based on the coordinate parameters of the target's point of interest and the target's location point, construct a perpendicular line for the target of interest; Based on the coordinate parameters of the perpendicular bisector, the target point of interest, and the target initial drilling point parameters, the position parameters of the target drilling point are determined.

8. The method for determining the punching point according to claim 6, characterized in that, After determining the position parameters of the target drilling point based on the coordinate parameters of the target point of interest and the initial drilling point parameters, the method further includes: Obtain the feature parameters of the target of interest; Adjust the position parameters of the target drilling point based on the aforementioned feature parameters.

9. The method for determining the punching point according to claim 1, characterized in that, Based on the position parameters of the target drilling point, the corresponding target drilling point is projected into the user's current field of view based on the operating device, including: Based on augmented reality technology, the corresponding target punching point is projected into the user's current field of view based on the position parameters of the target punching point through holographic projection.

10. The method for determining the punching point according to claim 1, characterized in that, After projecting the corresponding target drilling point into the user's current field of view based on the position parameters of the target drilling point, the method further includes: Receive user adjustments for the target punching points; Adjust the position of the target punch point in the current field of view based on the adjustment operation.

11. A method for determining punching points, characterized in that, Applied to a server, the method includes: The system receives a key local image sent by an operating device; wherein the operating device acquires a target image in the user's current field of view; and determines a key local image in the target image; the key local image is a local image in the target image containing a region of interest; the region of interest is the area used for reference positioning in the punching process; The key local image is processed using a pre-defined image semantic segmentation model. Based on the processing results, the contour region of the region of interest is determined in the key local image. Based on the contour region of the region of interest, the region of interest is identified and segmented from the key local image. Calculate the centroid coordinates in the region of interest; The centroid coordinates of the region of interest are sent to the operating device; wherein, the operating device converts the centroid coordinates of the region of interest into corresponding three-dimensional spatial coordinates according to the centroid coordinates of the region of interest and a preset coordinate transformation relationship, and determines the coordinate parameters of the target point of interest in the target image; the operating device also determines the position parameters of the target punch point for the target operation according to the coordinate parameters of the target point of interest; and projects the corresponding target punch point in the user's current field of view based on the position parameters of the target punch point.

12. An operating system, characterized in that, At least including: Operating equipment and servers, among which, The operating device is used to acquire a target image in the user's current field of view, and to determine key local images in the target image, as well as the target operation type of the target operation; wherein, the target image at least contains a target patient to be targeted; the operating device is also used to send the key local image to a server; the key local image is a local image of the target image containing a region of interest; the region of interest is the area used for reference positioning in the punching process; The server is used to process key local images using a preset image semantic segmentation model, and based on the processing results, to determine the contour region of the region of interest in the key local image; and to identify and segment the region of interest from the key local image based on the contour region of the region of interest; and to calculate the centroid coordinates of the region of interest; the server is also used to send the centroid coordinates of the region of interest to the operating device. The operating device is used to convert the centroid coordinates into corresponding three-dimensional spatial coordinates according to a preset coordinate transformation relationship to obtain the coordinate parameters of the target point of interest; determine the position parameters of the target punching point for the target operation according to the coordinate parameters of the target point of interest and the target manipulation type; and project the corresponding target punching point into the user's current field of view based on the position parameters of the target punching point.

13. A computer device, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the relevant steps of the method for determining the punch point as described in any one of claims 1 to 10, or 11.

14. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed, implement the relevant steps of the method for determining the punching point as described in any one of claims 1 to 10, or 11.

Images