Image generation method, apparatus, device, medium, and program product

Abstract

The application discloses an image generation method, device, equipment, medium and program product, and belongs to the image processing field. The method comprises the following steps: acquiring a medical image of a lesion tissue, wherein the medical image comprises a lesion contour of the lesion tissue; identifying a pathological sampling area corresponding to the lesion contour; generating a sampling grid for grid division of the pathological sampling area, wherein the sampling grid comprises at least two sub-grids which are arrayed and cover the pathological sampling area; each sub-grid covers the pathological sampling area and contains the lesion tissue; and displaying the medical image covered with the sampling grid. According to the above scheme, the sampling grid can be automatically generated without manual delineation by a doctor, errors and randomness of manual planning of the sampling grid can be minimized, a unified and homogeneous division scheme of the sampling grid is provided, the doctor can be assisted in pathological sampling, errors in sampling can be avoided, and the doctor can be further assisted in subsequent pathological analysis and formulation of a treatment plan.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an image generation method, apparatus, device, medium, and program product. Background Technology

[0002] Mohs micrographic surgery (NMS) is a common surgical resection method. In Mohs surgery, the visible tumor is first removed. Then, a Mohs grid is drawn at the base and boundaries of the tumor, and tissue samples are taken. The samples are examined under a microscope to check for any remaining tumor. If tumor remnants are found, the corresponding sampling location is located, and a second resection is performed. This process is repeated until no tumor remnants remain.

[0003] In related technologies, the Mohs grating in the Mohs procedure is based on the doctor manually drawing it on medical images, which then guides the doctor in subsequent tissue sampling. However, the shape and size of the Mohs grating obtained by these technologies vary considerably, which can easily lead to errors during tissue sampling. Summary of the Invention

[0004] This application provides an image generation method, apparatus, device, medium, and program product. The technical solution is as follows:

[0005] According to one aspect of this application, an image generation method is provided, the method comprising:

[0006] Acquire medical images of the lesion tissue, the medical images including the lesion outline of the lesion tissue;

[0007] Identify the pathological sampling area corresponding to the outline of the lesion;

[0008] A sampling grid is generated for dividing the pathological sampling area into grids. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Each pathological sampling area covered by the sub-grid contains the lesion tissue.

[0009] The medical image is displayed with the sampling grid covering it.

[0010] According to another aspect of this application, an image generation method is provided, the method comprising:

[0011] Acquire medical images of the lesion tissue, the medical images including the lesion outline of the lesion tissue;

[0012] The medical image is displayed covered by a sampling grid, the sampling grid comprising at least two sub-grids arranged in an array covering a pathological sampling area corresponding to the lesion outline; each of the sub-grids covers a pathological sampling area containing the lesion tissue.

[0013] According to another aspect of this application, an image generation apparatus is provided, the apparatus comprising:

[0014] The acquisition module is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue;

[0015] The identification module is used to identify the pathological sampling area corresponding to the lesion outline;

[0016] A generation module is used to generate a sampling grid for dividing the pathological sampling area into grids, wherein the sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array; each of the pathological sampling areas covered by the sub-grid contains the lesion tissue;

[0017] A display module is used to display the medical image covered by the sampling grid.

[0018] According to another aspect of this application, an image generation apparatus is provided, the apparatus comprising:

[0019] The acquisition module is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue;

[0020] A display module is used to display the medical image covered by a sampling grid, wherein the sampling grid includes at least two sub-grids arranged in an array, covering the pathological sampling area corresponding to the lesion outline; each sub-grid covers the pathological sampling area containing the lesion tissue.

[0021] According to another aspect of this application, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the image generation method as described above.

[0022] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that is loaded and executed by a processor to implement the image generation method as described above.

[0023] According to another aspect of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein a processor obtains the computer instructions from the computer-readable storage medium, causing the processor to load and execute the image generation method provided in the above aspect.

[0024] The beneficial effects of the technical solutions provided in this application include at least the following:

[0025] By acquiring medical images of lesion tissue, including the lesion outline, and then identifying the pathological sampling area corresponding to the lesion outline, the automatic identification of the pathological sampling area is achieved, thereby improving the processing efficiency of medical images. Next, a sampling grid is generated to divide the pathological sampling area. The sampling grid includes at least two sub-grids arranged in an array, each covering a pathological sampling area containing lesion tissue. This allows for automatic generation of the sampling grid without manual drawing by the doctor, minimizing errors and the randomness of manual grid planning, providing a unified and homogeneous sampling grid division scheme, and ensuring that lesion tissue can be sampled within each sub-grid, thus improving the efficiency of pathological sampling. Finally, by displaying the medical image covered by the sampling grid, doctors can be assisted in pathological sampling, avoiding errors during sampling, and also assisting in subsequent pathological analysis and treatment planning. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A block diagram of a computer system provided in an exemplary embodiment is shown;

[0028] Figure 2 This illustration shows an application scenario diagram of an image generation method provided by an exemplary embodiment;

[0029] Figure 3 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0030] Figure 4 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0031] Figure 5 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0032] Figure 6 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0033] Figure 7 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0034] Figure 8 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0035] Figure 9 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0036] Figure 10 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0037] Figure 11 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0038] Figure 12 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0039] Figure 13 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0040] Figure 14 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0041] Figure 15 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0042] Figure 16 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0043] Figure 17 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0044] Figure 18 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0045] Figure 19 A schematic diagram of an image generation method provided by an exemplary embodiment is shown;

[0046] Figure 20 A flowchart of an image generation method provided by an exemplary embodiment is shown;

[0047] Figure 21A structural block diagram of an image generation apparatus provided in an exemplary embodiment is shown;

[0048] Figure 22 A structural block diagram of an image generation apparatus provided in an exemplary embodiment is shown;

[0049] Figure 23 A structural block diagram of a computer device provided in an exemplary embodiment is shown. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0053] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0054] Figure 1 A structural block diagram of a computer system provided in an exemplary embodiment of this application is shown. This computer system can implement a system architecture for an image generation method. The computer system includes a terminal 120 and a server 140.

[0055] Terminal 120 can be an electronic device such as a mobile phone, tablet computer, vehicle-mounted terminal (vehicle system), wearable device, PC (Personal Computer), or unmanned reservation terminal. A client application for the target application can be installed and run on terminal 120. This target application can be a specialized application for image processing of medical images, or other applications that provide image processing functions for medical images; this application does not limit the specific application. Furthermore, this application does not limit the form of the target application, including but not limited to Apps (Applications), applets, etc., installed on terminal 120, and can also be in web page form.

[0056] Server 140 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud servers, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 140 can be a backend server for the aforementioned target application, used to provide backend services to the clients of the target application.

[0057] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0058] In some embodiments, the server 140 described above can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0059] Terminal 120 and server 140 can communicate via a network, such as a wired or wireless network.

[0060] The image generation method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Figure 1 Taking the implementation environment of the scheme shown as an example, the image generation method can be executed by the terminal 120, such as by the client of the target application installed and running in the terminal 120, or by the server 140, or by the interaction and cooperation between the terminal 120 and the server 140. This application does not limit this.

[0061] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0062] The embodiments of this application relate to artificial intelligence and computer vision technologies.

[0063] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0064] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0065] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0066] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0067] The image generation method of this application embodiment can be used as a medical aid to assist doctors in taking pathological samples from lesions. It is applicable to various medical and healthcare-related technical fields, including but not limited to Mohs Micrographic Surgery (NMS), biopsy, aspiration biopsy, and puncture biopsy pathological examination. It can be used to generate a sampling grid corresponding to a medical image of the lesion and display a medical image covered by the sampling grid to assist doctors in the next step.

[0068] For example, Figure 2 The illustration shows an application scenario diagram of the image generation method provided by an exemplary embodiment of this application. The application scenario mainly involves a doctor 200, a patient 201, a shooting component 202, a projection component 203, a terminal 206, and a (back-end) server 205.

[0069] Specifically, during the Morse code procedure performed on patient 201 by doctor 200, imaging component 202 can capture images of the lesion tissue of patient 201, obtaining medical image 204, which includes the lesion outline of the lesion tissue. Server 205 acquires the medical image of the lesion tissue, identifies the pathological sampling area corresponding to the lesion outline, and generates a sampling grid for dividing the pathological sampling area. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array, and each sub-grid contains lesion tissue within its covered pathological sampling area. Then, server 205 generates medical image 207 covered by the sampling grid and sends it to terminal 206, where it is displayed. Thus, doctor 200 can refer to medical image 207 covered by the sampling grid to perform subsequent pathological sampling of the pathological sampling area.

[0070] Optionally, the projection component 203 can also project the sampling grid onto the pathological sampling area of ​​the patient 201, displaying the sampling grid on the pathological sampling area. Thus, the physician 200 can refer to the sampling grid to perform subsequent pathological sampling in the pathological sampling area.

[0071] Optionally, the shooting component 202 is a device with image acquisition function, which mainly includes a lens and an autofocus component, and has an autofocus function, including at least one of the following: the shooting component 202 has its own autofocus function (such as a mechanical component inside the camera moving the image sensor up and down to focus) and is equipped with a normal lens without autofocus function; the shooting component 202 uses an adapter to drive an autofocus lens with an autofocus component; the shooting component 202 has its own focusing lens; the shooting component 202 is a normal camera and is equipped with a normal lens, and focuses by adding an external liquid lens.

[0072] Optionally, the projection component 203 is a device with screen projection function, used to project the sampling grid onto the pathological sampling area of ​​the patient 201, and can be at least one of the following: a digital projector, an LCD projector, or a laser projector.

[0073] Common surgical resection methods include Wide Local Excision (WLE) and Mohs Micrographic Surgery (NMS). WLE involves removing an additional portion of the surrounding tissue during tumor removal to prevent tumor residue. However, for skin tumors with significant infiltration and spread, NMS offers a significant advantage because the tumor's boundaries are difficult to precisely define. The NMS procedure can be summarized as follows: After the initial tumor resection, samples are taken from the basal and border areas of the tumor. These samples are rapidly prepared into paraffin sections for pathological examination, which determines whether any tumor residue remains. If residue is found, a secondary resection is performed at the corresponding sampling location, and sampling and pathological examination are repeated until no tumor residue is found. Compared to WLE, NMS minimizes damage to healthy tissue and reduces the tumor recurrence rate. One of the more challenging aspects of the NMS procedure is the sampling process, particularly determining the sampling area. Due to the limitations of microscope slide size, tumor samples should not be too large or too small, generally around 20mm in length. For larger tumors, the number of sampling areas and samples may be numerous, potentially reaching dozens or even hundreds. Manually delineating different sampling areas by the physician can easily introduce errors, potentially making subsequent pathological examinations difficult. Sampling areas may overlap or exhibit significant inconsistencies in sample size, making it challenging to achieve uniform, standardized manual planning.

[0074] Based on this, taking its application in the Mozart procedure as an example, Figure 3 A flowchart of an image generation method provided in an exemplary embodiment of this application is shown, in which the method is applied... Figure 1 The method, illustrated by the terminal 120 or the image processing-enabled client or server 140 installed on the terminal 120, includes:

[0075] Step 320: Obtain a medical image of the lesion tissue, which includes the lesion outline of the lesion tissue.

[0076] Lesion tissue refers to tissue containing pathogenic microorganisms. Any tissue or organ can become a lesion, and lesion tissue can be tumor tissue. For example, a lesion outline refers to a continuous contour curve of the lesion tissue. The lesion outline can be a regular shape or an irregular shape. A medical image refers to an image taken after the lesion tissue has been removed using the Mohr's procedure.

[0077] For example, a medical image of the lesion tissue is obtained by taking a picture of the lesion tissue using a camera component.

[0078] Optionally, the imaging component can be a first imaging component, including at least one of an industrial camera, a color (RGB) camera, and a two-dimensional (2D) camera. The imaging component then captures images of the lesion tissue, resulting in a two-dimensional planar image containing two-dimensional information about the lesion tissue.

[0079] Optionally, the imaging component can be a second imaging component, including at least one of a depth camera, a stereo color (RGBD) camera, a point cloud camera, and a three-dimensional (3D) camera. The imaging component then captures images of the lesion tissue, resulting in a two-dimensional planar image containing two-dimensional information of the lesion tissue and / or a three-dimensional point cloud image containing three-dimensional information of the lesion tissue.

[0080] Optionally, the type of imaging component can be determined based on the location of the lesion tissue.

[0081] For example, when the lesion is located in a relatively flat area or with minimal curvature, such as on the limbs or trunk, a first imaging component that is geared towards acquiring two-dimensional planar images can be selected. When the lesion is located in an uneven area or with significant curvature, such as on the nose, head, or joints, a second imaging component that is geared towards acquiring three-dimensional point cloud images can be selected to further incorporate the three-dimensional information of the lesion and improve the accuracy of subsequent medical image processing.

[0082] Step 340: Identify the pathological sampling area corresponding to the lesion outline.

[0083] The pathological sampling area is the area where tissue samples are taken after tumor resection, and the sampled tissue is quickly prepared into paraffin slides for pathological examination.

[0084] Optionally, the pathological sampling area can be the region corresponding to the lesion outline of the lesion tissue, in which case the pathological sampling area is the region where the lesion tissue is located, and the pathological sampling area only contains the lesion tissue. Alternatively, the pathological sampling area can be the region defined by extending the lesion outline of the lesion tissue outward, in which case the pathological sampling area includes the lesion tissue and at least a portion of healthy tissue.

[0085] In some embodiments, the lesion outline of the lesion tissue is first identified, and then the pathological sampling area corresponding to the lesion outline is identified.

[0086] Optionally, the lesion contour of the lesion tissue can be automatically identified. An artificial intelligence (AI) model is used to identify the lesion contour in the medical image, and then the corresponding pathological sampling area is identified. The AI ​​model is a pre-trained neural network model, which can be at least one of the following types: the Unet image segmentation model, or the Deeplab V3+ semantic segmentation model.

[0087] Optionally, the lesion outline of the lesion tissue can also be manually drawn by the doctor. In response to the outline drawing operation, the lesion outline of the lesion tissue in the medical image is identified. Then, the pathological sampling area corresponding to the lesion outline is identified.

[0088] Optionally, the lesion outline of the lesion tissue can also be determined by a combination of automatic recognition and manual delineation by the doctor. Using an artificial intelligence (AI) model, the initial lesion outline of the lesion tissue in the medical image is identified. Then, in response to the outline delineation operation, the initial lesion outline is adjusted to determine the final lesion outline. This prevents errors caused by automatic lesion outline recognition, allowing doctors to manually modify or redraw the automatically recognized outline to obtain a more accurate lesion outline.

[0089] Step 360: Generate a sampling grid for dividing the pathological sampling area into grids. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Each sub-grid covers a pathological sampling area containing lesion tissue.

[0090] A sampling grid is a grid created by dividing the pathological sampling area into multiple smaller sampling areas. At least two sub-grids within the sampling grid can be used to help locate the specific sampling position within the pathological sampling area, assisting the physician in subsequent pathological sampling.

[0091] For example, when applying a sampling grid in the Mohs procedure, the sampling grid can also be referred to as a Mohs grid.

[0092] Optionally, the sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Array arrangement means that at least two sub-grids in the sampling grid are arranged in a regular row and column pattern.

[0093] For example, each subgrid covers a pathological sampling area containing lesion tissue. That is, even when the lesion tissue is irregular, the sampling grid is also irregular. Accordingly, during subsequent pathological sampling based on the sampling grid, lesion tissue can be sampled from each subgrid, which can help improve the efficiency of pathological sampling.

[0094] Optionally, based on actual clinical experience, at least two subgrids in the sampling grid refer to at least four subgrids.

[0095] Optionally, at least two subgrids of the sampling grid have the same subgrid parameters, which include at least one of the following: subgrid size, shape, area, and interior angle size.

[0096] For example, at least two sub-grids in the sampling grid can be triangles, squares, matrices, parallelograms, rhombuses, sectors, etc., specifically determined automatically based on the lesion type of the lesion tissue, or specified by the doctor based on clinical experience. For example, when the sub-grid is a square, the side length of the square can be set to 20mm-30mm.

[0097] In some embodiments, the subgrid parameters of at least two subgrids in the sampling grid can be fixed by default, and the same subgrid parameters are used for all grid divisions of the pathological sampling area corresponding to the lesion tissue; or, the subgrid parameters are preset and can be set based on the specific type of lesion tissue; or, the subgrid parameters are dynamically changed and can be dynamically changed according to the relevant parameters of the medical image capturing component.

[0098] In an optional embodiment, after generating a sampling grid for dividing the pathological sampling area, the method further includes: numbering at least two sub-grids in the sampling grid to identify at least two sub-grids in the sampling grid.

[0099] Optionally, at least two subgrids in the sampling grid may be numbered using one or more combinations of special characters, letters, and numbers to identify at least two subgrids in the sampling grid.

[0100] Optionally, the numbering method can be clockwise or counterclockwise, sequentially numbering from the outermost to the innermost grid of the sampling grid, or sequentially numbering from the innermost to the outermost grid of the sampling grid. Alternatively, it can be S-shaped, sequentially numbering from the topmost to the bottommost sub-grid of the sampling grid, or sequentially numbering from the bottommost to the topmost sub-grid, or sequentially numbering from the leftmost to the rightmost sub-grid, or sequentially numbering from the rightmost to the leftmost sub-grid. In this embodiment, the numbering method is not limited, as long as it uniquely identifies at least two sub-grids in the sampling grid.

[0101] Step 380: Display the medical image covered with the sampling grid.

[0102] For example, after generating a sampling grid for dividing the pathological sampling area, a medical image covered with the sampling grid is displayed to assist the doctor in subsequent pathological sampling.

[0103] In some embodiments, the method further includes displaying the numbers of at least two sub-grids in the sampling grid. This assists the physician in knowing the specific sampling location of the pathological sampling area, facilitating secondary surgery and secondary sampling when residual lesion tissue exists.

[0104] In summary, the method provided in this application acquires a medical image of the lesion tissue, including the lesion outline, and then identifies the pathological sampling area corresponding to the lesion outline, thereby achieving automatic identification of the pathological sampling area and improving the processing efficiency of medical images. Next, a sampling grid is generated for dividing the pathological sampling area. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Each sub-grid covers a pathological sampling area containing lesion tissue. Accordingly, the sampling grid can be automatically generated without manual drawing by the doctor, minimizing errors and the randomness of manual grid planning, providing a unified and homogeneous sampling grid division scheme, and ensuring that lesion tissue can be sampled within each sub-grid, thus improving the efficiency of pathological sampling. Finally, by displaying the medical image covered by the sampling grid, the method assists doctors in performing pathological sampling, avoiding errors during sampling, and also assists doctors in subsequent pathological analysis and treatment planning.

[0105] In an optional embodiment of this application, step 360 generates a sampling grid for dividing the pathological sampling area into grids, including: generating a sampling grid for dividing the pathological sampling area into grids based on at least one reference point in the medical image.

[0106] For example, a reference point is a pixel used as a reference during the generation of the sampling mesh, and the reference point may include at least one. Optionally, the at least one reference point may be predetermined before generating the sampling mesh, or it may be determined iteratively in real time during the generation of the sampling mesh.

[0107] Optionally, if there is only one reference point, the reference point can be the center of the entire sampling grid, and a sampling grid centered on the reference point can be expanded to generate a sampling grid for dividing the pathological sampling area.

[0108] For example, when there is only one reference point, based on this reference point in the medical image, the first, second, and nth rings of grids are sequentially expanded around the reference point, where n is greater than 0 and is an integer. The i-th ring of grids does not overlap with the (i+1)-th ring of grids, where i is greater than 0 and less than or equal to n and is an integer.

[0109] Optionally, when there are multiple reference points, the reference points can be used as vertices of sub-grids, midpoints of sub-grid edges, or points that divide sub-grid edges into thirds or quarters. Based on the reference points in each iteration, at least two sub-grids are iteratively expanded and arranged in an array, ultimately generating a sampling grid for dividing the pathological sampling area. The expansion and generation of at least two sub-grids can be either generating a ring of grids or generating a grid-like sub-grid surrounding the reference points.

[0110] For example, when there are multiple reference points, the first reference point is used as the sub-mesh vertex of the first iteration, and multiple sub-mesh arrays are generated in a grid pattern. Then, based on the sub-mesh generated in the first iteration, the second reference point for the next iteration is determined, and multiple sub-mesh arrays in a grid pattern around each second reference point are generated.

[0111] Optionally, when there are multiple reference points, all reference points in the medical image can be predetermined, and the reference points can be connected sequentially in a certain order to generate a sampling grid for dividing the pathological sampling area.

[0112] For example, Figure 4 A flowchart illustrating an exemplary embodiment of this application shows an image generation method that generates a sampling grid for dividing a pathological sampling region based on at least one reference point in a medical image. This can be implemented through the following steps:

[0113] Step 362: Determine the center point of the lesion outline.

[0114] Optionally, the center point of the lesion outline can be automatically identified using an image detection algorithm, or it can be manually drawn by the doctor. The image detection algorithm is implemented using the open-source computer vision library OpenCV.

[0115] Step 363: Using the center point as a reference point, expand to generate at least two sub-grids for the array arrangement.

[0116] For example, using the center point of the lesion contour as a reference point, at least two sub-grids are expanded to generate an array. Optionally, the reference point can be the center of the entire sampling grid, or a vertex of a sub-grid, or the midpoint of a sub-grid edge, or a point that divides a sub-grid edge into thirds or quarters.

[0117] For example, when the reference point is the center of the entire sampling grid and the subgrids are squares, the center point can be used as the reference point to expand and generate a ring of grids arranged in an array around the reference point. The ring of grids includes at least two subgrids.

[0118] For example, when the reference point is used as the vertex of the sub-mesh and the sub-mesh is a square, the center point can be used as the reference point to expand and generate four sub-mesh arrays. The common vertex of the four sub-mesh arrays is the center point.

[0119] For example, if the reference point is the midpoint of the sub-grid edge and the sub-grid is a square, the center point can be used as the reference point to expand and generate two sub-grids arranged in an array. The midpoint of the common edge of the two sub-grids is the center point.

[0120] Optionally, the method further includes: using the center point as a reference point, and generating a coordinate system for the medical image based on the reference point. Accordingly, the coordinate positions of the reference point and at least two sub-grids of the expanded array can be determined in the coordinate system of the medical image, improving the accuracy of the expansion process.

[0121] Step 365: If at least two sub-grids intersect with the entire contour of the lesion, end the expansion to generate at least two sub-grids, and obtain a sampling grid for dividing the pathological sampling area.

[0122] For example, if at least two sub-grids intersect with the entire contour of the lesion, it means that the sampling grid has completely exceeded the entire contour of the lesion. At this point, the expansion is stopped and at least two sub-grids are generated to obtain the sampling grid used for meshing the pathological sampling area.

[0123] Step 367: If at least two sub-mesh do not intersect with at least a portion of the lesion contour, use the newly added sub-mesh vertices of the at least two sub-mesh generated in this round of expansion as the reference point for the next round, and continue to expand to generate at least two sub-mesh arrays using the reference point of the next round as the expansion point.

[0124] For example, a newly added submesh vertex refers to each submesh vertex of the at least two submesh cells generated in this round of expansion. If a reference point is used as a submesh vertex during the generation of the sampling mesh, then a newly added submesh vertex is any submesh vertex other than the reference point used in this round, among the submesh vertices of the at least two submesh cells generated during expansion.

[0125] For example, if at least two sub-mesh grids do not intersect with at least a portion of the lesion contour, it indicates that the sampling mesh has not completely exceeded the entire lesion contour, and at least a portion of the lesion contour is still outside the sampling mesh. In this case, if there are multiple reference points, the newly added sub-mesh vertices of the at least two sub-mesh grids generated in this round of expansion are used as reference points for the next round, and the expansion continues with the next round of reference points as expansion points to generate at least two sub-mesh grids arranged in an array. If there is only one reference point, step 363 continues with the center point as the reference point to expand and generate at least two sub-mesh grids arranged in an array.

[0126] Optionally, the next reference point is used as the expansion point to expand and generate at least two sub-grids in the array arrangement. This can be to generate a ring of grids or to generate sub-grids in a grid pattern around the expansion point.

[0127] In some embodiments, step 367 above can be implemented as follows: when at least two sub-mesh do not intersect with at least a portion of the lesion contour, the newly added sub-mesh vertices of the at least two sub-mesh generated in this round of expansion are used as reference points for the next round, and the next round reference points are used as expansion points to expand and generate at least two sub-mesh arranged in a grid pattern around the expansion points.

[0128] In this embodiment, by iteratively expanding and generating at least two sub-grids in an array, it is ensured that at least two sub-grids in the generated sampling grid completely cover the pathological sampling area, avoiding omissions. By expanding and generating a ring of grids in each iteration, complete coverage of the pathological sampling area can be ensured to the greatest extent, improving the accuracy of the generated sampling grid. In each iteration, a grid of sub-grids in a crisscross pattern around the expansion point is generated. If an expansion point extends beyond the edge of the pathological sampling area in the current iteration's crisscross pattern, that expansion point does not need to proceed to the next iteration, and it will not affect the next iteration of other expansion points. This approach, while ensuring complete coverage of the pathological sampling area to the greatest extent, also improves the processing efficiency of medical images.

[0129] As an example, Figure 5 A schematic diagram of an image generation method provided by an exemplary embodiment of this application is shown. In this embodiment, the example is illustrated by taking a sample grid where each sub-grid is a square, a reference point is used as the vertex of the sub-grid, and at least two sub-grids are generated around the expansion point in a crisscross pattern.

[0130] like Figure 5-1 The image shown is a medical image of the lesion tissue. The image includes the outline of the lesion tissue, with point O as the center point of the lesion outline. Figure 5-2 As shown, a coordinate system for the medical image is defined with point O as the origin. The first quadrant of the coordinate system is denoted as Q1, the second quadrant as Q2, the third quadrant as Q3, and the fourth quadrant as Q4. Using the center point O as the reference point, four sub-grids are generated in an array arrangement, denoted as square sub-grids a, b, c, and d. The vertices of each of the four square sub-grids, excluding vertex O, are called newly added sub-grid vertices. There are a total of eight newly added sub-grid vertices: A, B, C, D, E, F, G, and H. Figure 5-2 It is evident that the four square sub-grids do not intersect with the lesion outline. Therefore, the newly added sub-grid vertices of at least two square sub-grids generated in this round of expansion are used as reference points for the next round. The expansion continues, using the next round's reference points as expansion points, to generate at least two square sub-grids arranged in a tic-tac-toe pattern around the expansion points. For example, as... Figure 5-3As shown, with sub-mesh vertex A as the expansion point, square sub-mesh a, f, e, h are generated around sub-mesh vertex A in a tic-tac-toe pattern; with sub-mesh vertex B as the expansion point, square sub-mesh a, b, f, g are generated around sub-mesh vertex B in a tic-tac-toe pattern; with sub-mesh vertex D as the expansion point, square sub-mesh a, c, h, i are generated around sub-mesh vertex D in a tic-tac-toe pattern. Figure 5-4 As shown, the expansion process continues until at least two square sub-grids intersect with the entire outline of the lesion, at which point the expansion ends and at least two square sub-grids are generated, resulting in a sampling grid for dividing the pathological sampling area.

[0131] In some embodiments, please continue reading Figure 4 Generating a sampling mesh for dividing the pathological sampling area based on at least one reference point in the medical image can be achieved through the following steps:

[0132] Step 362: Determine the center point of the lesion outline.

[0133] Optionally, the center point of the lesion contour can be automatically identified using an image detection algorithm, or it can be manually drawn by the doctor. The image detection algorithm can be any image detection algorithm from the open-source computer vision library OpenCV.

[0134] Step 364: Determine the center point as the reference point and determine the coordinate system based on the reference point.

[0135] For example, the center point is determined as the reference point, and a coordinate system is determined based on the reference point. Optionally, the coordinate system is a planar coordinate system, with the horizontal axis of the coordinate system parallel to the length of the medical image and the vertical axis of the coordinate system parallel to the width of the medical image.

[0136] Step 366: In the coordinate system, calculate the vertex coordinates of at least two subgrids in the array arrangement.

[0137] For example, based on the subgrid parameters of at least two subgrids, the subgrid vertex coordinates of at least two subgrids in the array are calculated in the coordinate system, so that the subgrid vertex coordinates can be connected to generate the sampling mesh.

[0138] In some embodiments, the submesh parameters of at least two submesh include at least one of the following: submesh size, shape, area, and interior angle size; wherein the submesh parameters are fixed by default; or, the submesh parameters are preset; or, the submesh parameters are dynamically changed.

[0139] Optionally, if the submesh shape is not rectangular or square, the submesh parameters include the interior angle sizes of the submesh. For example, if the submesh is a triangle, the interior angle size refers to the size of each interior angle of the triangle. If the submesh is a rhombus, the interior angle size refers to the size of each interior angle of the rhombus.

[0140] Step 368: Based on the vertex coordinates of each sub-grid, generate a sampling grid for dividing the pathological sampling area into grids.

[0141] Optionally, based on the vertex coordinates of each sub-grid, the vertex coordinates of each sub-grid are connected sequentially in a certain order to generate a sampling grid for dividing the pathological sampling area.

[0142] For example, Figure 6 A flowchart of an image generation method provided by an exemplary embodiment of this application is shown. Step 368 can be further implemented as the following steps:

[0143] Step 410: Generate a sub-mesh based on the coordinates of at least three sub-mesh vertices; at least one edge of the sub-mesh is parallel to the target axis in the coordinate system.

[0144] For example, at least two sub-grids in the sampling grid can be one of triangles, rectangles, squares, rhombuses, or parallelograms. Based on the vertex coordinates of at least three sub-grids, a corresponding sub-grid is generated, and at least one edge of the sub-grid is parallel to the target axis in the coordinate system, which can be the horizontal axis or the vertical axis.

[0145] As an example, Figure 7 A schematic diagram of an image generation method provided in an exemplary embodiment of this application is shown. For example... Figure 7-1 The image shows the calculated vertex coordinates of at least two sub-grids in the array arrangement. For example... Figure 7-2 As shown, when the submesh is triangular, a triangular submesh is generated based on the coordinates of the three submesh vertices. At least one edge of the triangular submesh is parallel to the horizontal axis or parallel to the vertical axis. Figure 7-3 As shown, when the submesh is a parallelogram, a parallelogram submesh is generated based on the coordinates of the four submesh vertices, and at least one edge of the parallelogram submesh is parallel to the horizontal axis. Figure 7-4 As shown, when the subgrid is square, a square subgrid is generated based on the coordinates of the four subgrid vertices. At least one edge of the square subgrid is parallel to the horizontal axis or parallel to the vertical axis.

[0146] It should be noted that, Figure 7 This illustration demonstrates an exemplary method for generating a submesh based on the coordinates of at least three submesh vertices. Depending on specific technical requirements, the orientation of the submesh can also be changed, for example... Figure 7-3 The parallelogram subgrids divided in the middle can also be set to be uniformly tilted to the right, or other subgrid generation methods can be set, such as generating a triangular subgrid based on the coordinates of the four subgrid vertices. This embodiment does not limit this.

[0147] Step 411: Determine the target sub-mesh from at least two sub-mesh as a sub-mesh belonging to the sampling mesh, wherein at least one sub-mesh vertex of the target sub-mesh is located within the lesion contour.

[0148] For example, a target subgrid refers to a subgrid belonging to the sampling grid among at least two subgrids. Optionally, the target subgrid among at least two subgrids is determined to be a subgrid belonging to the sampling grid, and at least one vertex of the target subgrid is located within the lesion contour. This ensures that the sampling grid can completely extend beyond the entire lesion contour.

[0149] As an example, with Figure 7-4 For example, the top right subgrid of the first quadrant Q1, the top left subgrid of the second quadrant Q2, and the bottom left subgrid of the third quadrant Q3 are subgrids that do not belong to the sampling grid and need to be removed later.

[0150] In this embodiment, the sampling mesh is generated by directly calculating the vertex coordinates of at least two sub-meshes and connecting them. This eliminates the need for iteration and effectively avoids the impact of errors from the previous iteration on the next iteration or on the entire sampling mesh, thereby improving the accuracy of the generated sampling mesh.

[0151] In an optional embodiment of this application, the method further includes: removing duplicate subgrids from at least two subgrids; or, removing subgrids from at least two subgrids that do not intersect with the lesion outline and do not contain lesion tissue.

[0152] Optionally, when generating the sampling grid based on the above steps, at least a portion of the pathological sampling area may be repeatedly divided into sub-grids, or healthy tissue outside the lesion outline may also be divided into sub-grids. Therefore, it is necessary to remove duplicate sub-grids in at least two sub-grids, or remove sub-grids in at least two sub-grids that do not intersect with the lesion outline and do not contain lesion tissue, that is, remove sub-grids that only cover healthy tissue, to ensure that the pathological sampling area covered by each sub-grid contains lesion tissue.

[0153] In this embodiment, by removing duplicate subgrids or subgrids that do not intersect with the lesion outline and do not contain lesion tissue, the accuracy of the generated sampling grid can be improved, which is beneficial for subsequent pathological sampling. Moreover, since the pathological sampling area covered by each subgrid contains lesion tissue, lesion tissue can be sampled in each subgrid during subsequent pathological sampling, which can help improve the efficiency of pathological sampling.

[0154] In one optional embodiment of this application, Figure 8 A flowchart of an image generation method provided by an exemplary embodiment of this application is shown, the method further comprising the following steps:

[0155] Step 402: Obtain the binary mask image of the medical image.

[0156] For example, a binary mask image is an image obtained by converting a medical image to grayscale and then to binary. Binary mask images of medical images can assist in the analysis of lesion contours.

[0157] Optionally, in the binary mask image of the medical image, the pixel value of the lesion tissue can be set to 255, and the pixel value of the healthy tissue located outside the lesion contour can be set to 0.

[0158] Optionally, a binary mask image of the medical image can be obtained.

[0159] Step 404: Determine at least two contour points on the lesion contour in the binary mask image.

[0160] For example, at least two contour points on the lesion contour in the binary mask image are determined. Wherein, if all eight adjacent pixels of a bright spot in the binary mask image are bright spots, then that bright spot is an interior point of the lesion contour; otherwise, it is a contour point of the lesion contour.

[0161] Step 406: Calculate the distance between every two contour points of at least two contour points.

[0162] For example, after determining at least two contour points, the distance between every two contour points is calculated.

[0163] Step 408: Determine the major axis of the lesion contour based on distance. The major axis is used to help determine the clinical medical parameters of the lesion tissue.

[0164] For example, the line segment corresponding to the longest distance is defined as the major axis of the lesion contour, and the major axis is used to help determine the clinical medical parameters of the lesion tissue.

[0165] Optionally, when the lesion outline is a relatively uniform circle or ellipse, the center point of the long axis is determined as the center point of the lesion outline.

[0166] In this embodiment, the major axis of the lesion contour is determined based on the binary mask image of the medical image. This can help determine the clinical medical parameters of the lesion tissue, facilitate the generation of the sampling mesh, assist doctors in performing pathological analysis of the pathological sampling area, and provide effective guidance for subsequent treatment planning.

[0167] In one optional embodiment of this application, Figure 9 A flowchart of an image generation method provided in an exemplary embodiment of this application is shown, the method further comprising:

[0168] Step 502: Determine the perpendicular bisector of the major axis of the lesion outline.

[0169] For example, the perpendicular bisector of the major axis is a straight line perpendicular to the major axis and bisecting it in two. Alternatively, the perpendicular bisector of the major axis of the lesion contour can be determined based on the center point of the major axis of the lesion contour.

[0170] Step 504: Determine the minor axis of the lesion contour based on the intersection of the vertical bisector and the lesion contour.

[0171] For example, based on the intersection of the perpendicular bisector and the lesion contour, the line segment corresponding to the two intersection points is determined as the minor axis of the lesion contour. The minor axis is used to help determine the clinical medical parameters of the lesion tissue.

[0172] Step 506: Based on the pixel lengths of the major and minor axes in the medical image, and the spatial distance between each pixel in the medical image, determine the length of the major axis and the length of the minor axis of the lesion contour. The length of the major axis and the length of the minor axis are used to assist in the pathological analysis of the pathological sampling area.

[0173] For example, the long axis length and short axis length refer to the actual lengths of the long axis and short axis of the lesion outline. These are clinical medical parameters, mainly used to characterize the size of the lesion tissue and assist doctors in performing pathological analysis on the pathological sampling area. Pathological analysis includes, but is not limited to, analyzing the nature of the lesion tissue, whether it is inflammation or a tumor, benign or malignant, etc.

[0174] Optionally, the length of the lesion contour is determined by calculating the product of the pixel length of the major axis and the spatial distance between each pixel in the medical image, based on the pixel length of the major axis in the medical image and the spatial distance between each pixel in the medical image.

[0175] Optionally, the short axis length of the lesion contour is determined by calculating the product of the pixel length of the short axis and the spatial distance between each pixel in the medical image, based on the pixel length of the short axis in the medical image and the spatial distance between each pixel in the medical image.

[0176] In some embodiments, the spatial distance between each pixel of a medical image can be determined based on camera parameters of the medical image, which may be intrinsic parameters.

[0177] Optionally, the spatial resolution of the medical images under different camera intrinsic parameters can be measured in advance. The spatial resolution is the actual physical spatial distance represented by each pixel in the medical image. After acquiring the medical image, the intrinsic parameters of the camera parameters of the medical image are obtained, and the spatial resolution corresponding to the intrinsic parameters is determined, thereby determining the spatial distance between each pixel in the medical image.

[0178] In some embodiments, the method further includes: determining the actual area of ​​the lesion contour based on the pixel area of ​​the lesion contour in the medical image and the actual pixel area corresponding to each pixel in the medical image, wherein the actual area of ​​the lesion contour is used to assist in pathological analysis of the pathological sampling area.

[0179] Optionally, the actual area of ​​the lesion contour can be determined by multiplying the pixel area and the actual pixel area of ​​each pixel in the medical image.

[0180] In this embodiment, by determining the clinical medical parameters such as the long axis length, short axis length, and area, the size of the lesion tissue can be characterized more accurately. This can assist doctors in conducting pathological analysis of the pathological sampling area and provide effective guidance for subsequent treatment planning.

[0181] As an example, Figure 10 A schematic diagram of an image generation method provided by an exemplary embodiment of this application is shown. Figure 10 The image shown is a binary mask of a medical image. The white portion of the binary mask represents lesion tissue, and the black portion represents healthy tissue. The contour points on the lesion contour in the binary mask image are identified as A, B, C, D, E, F, G, H, I, and J. The distance between every two contour points is calculated, and the line segment AE corresponding to the maximum distance AE is determined as the major axis of the lesion contour. Next, the perpendicular bisector of the major axis AE is determined, and the line segment corresponding to the intersection of the perpendicular bisector and the lesion contour is determined as the minor axis of the lesion contour. Finally, based on the pixel lengths of the major and minor axes in the medical image, and the spatial distance between each pixel in the medical image, the lengths of the major and minor axes of the lesion contour are determined. These lengths are used to assist doctors in subsequent pathological analysis of the pathological sampling area corresponding to the lesion contour.

[0182] In one optional embodiment of this application, the sub-mesh parameters of at least two sub-mesh include at least one of the following: sub-mesh size, shape, area, and interior angle size; wherein the sub-mesh parameters are fixed by default; or, the sub-mesh parameters are preset; or, the sub-mesh parameters are dynamically changed. The specific selection can be made according to actual technical needs to improve the accuracy of the generated sampling mesh.

[0183] For example, in the case where the sub-mesh parameters change dynamically, Figure 11 A flowchart of an image generation method provided in an exemplary embodiment of this application is shown, the method further comprising:

[0184] Step 602: Obtain the camera parameters of the medical image. The camera parameters include at least one of the following: internal parameters, external parameters, and shooting distance.

[0185] For example, a medical image is obtained by capturing images of the lesion tissue using an imaging component. The parameters of the imaging component are called camera parameters, which include at least one of intrinsic parameters, extrinsic parameters, and imaging distance.

[0186] Optionally, the intrinsic parameters include at least one of focal length, pixel size, and intrinsic parameter matrix, and the extrinsic parameters include at least one of camera position, rotation matrix, and translation matrix. The shooting distance refers to the distance between the lens of the shooting component and the lesion tissue.

[0187] Step 604: Adjust the subgrid parameters based on the camera parameters of the medical image.

[0188] For example, based on the camera parameters of a medical image, the sub-mesh parameters can be adjusted in real time to make the resulting sampling mesh more suitable for the medical image.

[0189] In this embodiment, the sub-grid parameters can be adjusted in real time, making the resulting sampling grid more suitable for medical images and improving the accuracy of the generated sampling grid.

[0190] In some embodiments, after generating a sampling grid for dividing the pathological sampling area, some sub-grids may contain very little lesion tissue but a large amount of healthy tissue. Sampling from these sub-grids could lead to a waste of medical resources. In this case, the method further includes merging at least two sub-grids in the sampling grid that meet the merging criteria. After merging, the sub-grid parameters of at least some sub-grids in the sampling grid are different from the sub-grid parameters of other sub-grids; that is, the sub-grid parameters of each sub-grid in the sampling grid are not completely identical.

[0191] For example, Figure 12A flowchart illustrating an exemplary embodiment of this application shows an image generation method that merges at least two sub-grids in a sampling grid that meet merging conditions, including:

[0192] Step 701: Determine the first subgrid from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids.

[0193] For example, the first sub-mesh is the sub-mesh that needs to be merged. Generally, the first sub-mesh is the sub-mesh located at the edge region of the lesion outline, and it contains both lesion tissue and healthy tissue.

[0194] Optionally, the first subgrid is automatically identified. In this case, the first subgrid is determined from at least two subgrids based on the size of the lesion tissue contained in at least two subgrids. For example, a subgrid containing lesion tissue smaller than one-quarter of the total subgrid size is determined as the first subgrid. Alternatively, a subgrid containing lesion tissue smaller than the size of the healthy tissue it contains is determined as the first subgrid.

[0195] Optionally, the first subgrid is manually drawn by the physician. In this case, in response to the subgrid selection operation, the first subgrid is determined from at least two subgrids. The first subgrid can be determined by the physician based on actual technical needs.

[0196] Step 702: From at least two subgrids other than the first subgrid, determine the second subgrid that is closest to the first subgrid.

[0197] For example, the second subgrid is a subgrid that can be merged with the first subgrid. The second subgrid and the first subgrid share a common edge. The second subgrid includes one...

[0198] Optionally, from at least two subgrids other than the first subgrid, determine the second subgrid that is closest to the first subgrid.

[0199] Step 705: Merge the first sub-mesh with the second sub-mesh.

[0200] Optionally, the first sub-grid and the second sub-grid can be merged to optimize the sampling grid.

[0201] For example, Figure 12 A flowchart illustrating an exemplary embodiment of this application shows an image generation method that merges at least two sub-grids in a sampling grid that meet merging conditions, including:

[0202] Step 701: Determine the first subgrid from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids.

[0203] For example, the first sub-mesh is the sub-mesh that needs to be merged. Generally, the first sub-mesh is the sub-mesh located at the edge region of the lesion outline, and it contains both lesion tissue and healthy tissue.

[0204] Optionally, the first subgrid is automatically identified. In this case, the first subgrid is determined from at least two subgrids based on the size of the lesion tissue contained in at least two subgrids. For example, a subgrid containing lesion tissue smaller than one-quarter of the total subgrid size is determined as the first subgrid. Alternatively, a subgrid containing lesion tissue smaller than the size of the healthy tissue it contains is determined as the first subgrid.

[0205] Optionally, the first subgrid is manually drawn by the physician. In this case, in response to the subgrid selection operation, the first subgrid is determined from at least two subgrids. The first subgrid can be determined by the physician based on actual technical needs.

[0206] Step 703: From at least two subgrids other than the first subgrid, determine the third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue.

[0207] For example, the third subgrid is a subgrid that can be merged with the first subgrid. The third subgrid shares a common edge with the first subgrid. The third subgrid comprises one. If there is more than one second subgrid, i.e., more than one second subgrid is closest to the first subgrid, then the third subgrid can also be the subgrid containing the largest lesion tissue determined from the second subgrids.

[0208] Optionally, from at least two subgrids other than the first subgrid, a third subgrid is determined that is closest to the first subgrid and contains the largest amount of lesion tissue.

[0209] Step 706: Merge the first sub-mesh with the third sub-mesh.

[0210] Optionally, the first subgrid and the third subgrid can be merged to optimize the sampling grid.

[0211] For example, Figure 12 A flowchart illustrating an exemplary embodiment of this application shows an image generation method that merges at least two sub-grids in a sampling grid that meet merging conditions, including:

[0212] Step 701: Determine the first subgrid from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids.

[0213] For example, the first sub-mesh is the sub-mesh that needs to be merged. Generally, the first sub-mesh is the sub-mesh located at the edge region of the lesion outline, and it contains both lesion tissue and healthy tissue.

[0214] Optionally, the first subgrid is automatically identified. In this case, the first subgrid is determined from at least two subgrids based on the size of the lesion tissue contained in at least two subgrids. For example, a subgrid containing lesion tissue smaller than one-quarter of the total subgrid size is determined as the first subgrid. Alternatively, a subgrid containing lesion tissue smaller than the size of the healthy tissue it contains is determined as the first subgrid.

[0215] Optionally, the first subgrid is manually drawn by the physician. In this case, in response to the subgrid selection operation, the first subgrid is determined from at least two subgrids. The first subgrid can be determined by the physician based on actual technical needs.

[0216] Step 704: From at least two subgrids other than the first subgrid, determine the fourth subgrid that is closest to the first subgrid and closest to the target direction of the center point of the lesion contour.

[0217] For example, the fourth subgrid is a subgrid that can be merged with the first subgrid. The fourth subgrid shares a common edge with the first subgrid. The fourth subgrid includes one. If there is more than one third subgrid, that is, if there is more than one third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue, then the fourth subgrid can also be the subgrid that is closest in the target direction to the center point of the lesion outline determined from the third subgrid.

[0218] Optionally, the target direction distance refers to the horizontal or vertical component of the distance, and the target direction distance to the center point of the lesion contour refers to the horizontal or vertical component of the distance between the center point of the sub-grid and the center point of the lesion contour.

[0219] Optionally, from at least two subgrids other than the first subgrid, a fourth subgrid is determined that is closest to the first subgrid and closest in the target direction to the center point of the lesion contour.

[0220] Step 707: Merge the first sub-mesh with the fourth sub-mesh.

[0221] Optionally, the first subgrid and the fourth subgrid can be merged to optimize the sampling grid.

[0222] In some embodiments, the three merging methods described above can be selected according to actual technical needs. For example, when the first subgrid has only one nearest subgrid, merging can be performed by determining the second subgrid. When the first subgrid has more than one nearest subgrid, and the lesions contained in each nearest subgrid are of different sizes, merging can be performed by determining the third subgrid. When the first subgrid has more than one nearest subgrid, and the lesions contained in each nearest subgrid are of the same size, merging can be performed by determining the fourth subgrid.

[0223] In this embodiment, the sampling grid can be optimized by merging at least two sub-grids that meet the merging conditions. By using different merging methods, the accuracy of grid merging can be improved, the optimization effect of the sampling grid can be enhanced, and this is beneficial for subsequent pathological sampling.

[0224] In some embodiments, at least two subgrids in the sampling grid that meet the merging conditions are first merged, and then at least two subgrids in the sampling grid are numbered to identify the at least two subgrids in the sampling grid.

[0225] As an example, Figure 13 A schematic diagram of an image generation method provided in an exemplary embodiment of this application is shown. For example... Figure 13-1 The diagram shows a sampling grid used for dividing the pathological sampling area. Based on the size of the lesion tissue contained in at least two sub-grids, a first sub-grid that needs to be merged is determined from the at least two sub-grids. In this embodiment, the first sub-grid is sub-grid y.

[0226] Merging Method 1: From at least two sub-grids other than the first sub-grid, determine the second sub-grid that is closest to the first sub-grid. When the only sub-grid that is closest to the first sub-grid is sub-grid x, then the second sub-grid is sub-grid x. Merge the first sub-grid y with the second sub-grid x.

[0227] Merging Method Two: From at least two subgrids excluding the first subgrid, determine the third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue. Figure 13-1 The subgrids x and z that are closest to the first subgrid are selected. Subgrid x contains the largest amount of lesion tissue, so the third subgrid is subgrid x. The first subgrid y and the third subgrid x are then merged.

[0228] Merging Method 3: From at least two sub-mesh units other than the first sub-mesh, determine the fourth sub-mesh that is closest to the first sub-mesh and also closest in the target direction to the center point of the lesion contour. In this embodiment, the target direction distance is set as the vertical component of the distance. Figure 13-1 The distance between sub-mesh x and the center point O of the lesion contour is line segment OM, and the target direction distance is the vertical component of line segment OM. The distance between sub-mesh z and the center point O of the lesion contour is line segment ON, and the target direction distance is the vertical component of line segment ON. Among them, the vertical component of the distance to the center point of the lesion contour is the vertical component of line segment OM, so the fourth sub-mesh is sub-mesh x. The first sub-mesh y and the fourth sub-mesh x are merged.

[0229] like Figure 13-2 The image shows the sampling mesh after mesh merging and removal. Figure 13-3 The image shows a sampling grid after at least two sub-grids in the sampling grid have been numbered, which can then be used for display.

[0230] In one optional embodiment of this application, Figure 14 A flowchart illustrating an exemplary embodiment of the image generation method provided in this application is shown. After acquiring a medical image of the lesion tissue in step 320 and before identifying the pathological sampling area corresponding to the lesion contour in step 340, the method further includes:

[0231] Step 321: Using an AI model, identify the lesion outline of the lesion tissue in the medical image.

[0232] Optionally, the lesion contours of the lesion tissue in the medical image are automatically identified. Specifically, the lesion contours of the lesion tissue in the medical image are identified using an AI model. The artificial intelligence AI model is a pre-trained neural network model, which can be at least one of the following types: image segmentation Unet model, semantic segmentation model Deeplab V3+ model.

[0233] In some embodiments, the method further includes: displaying the lesion outline of the lesion tissue in a medical image.

[0234] For example, after automatically identifying the lesion outline, the lesion outline of the lesion tissue in the medical image is displayed to help doctors know whether the automatically identified lesion outline is accurate, so as to facilitate manual modification or redrawing.

[0235] In this embodiment, a pre-trained AI model can be used to automatically identify the lesion contours of lesions in medical images, thereby improving the processing efficiency of medical images.

[0236] In one optional embodiment of this application, although most lesions requiring Mohr's procedure have a relatively flat base after resection, the base in some cases may have a greater curvature, which may lead to some errors in the sampling mesh. Therefore, please refer to [further details needed]. Figure 14 After acquiring the medical image of the lesion tissue in step 320, and before identifying the pathological sampling area corresponding to the lesion contour in step 340, the procedure also includes:

[0237] Step 321: Using an AI model, identify the lesion outline of the lesion tissue in the medical image.

[0238] Optionally, the lesion contours of the lesion tissue in the medical image are automatically identified. Specifically, the lesion contours of the lesion tissue in the medical image are identified using an AI model. The artificial intelligence AI model is a pre-trained neural network model, which can be at least one of the following types: image segmentation Unet model, semantic segmentation model Deeplab V3+ model.

[0239] In some embodiments, the method further includes: displaying the lesion outline of the lesion tissue in a medical image.

[0240] For example, after automatically identifying the lesion outline, the lesion outline of the lesion tissue in the medical image is displayed to help doctors know whether the automatically identified lesion outline is accurate, so as to facilitate manual modification or redrawing.

[0241] Step 322: Obtain a three-dimensional point cloud image of the lesion tissue.

[0242] For example, when the location of the lesion tissue is uneven or has a large curvature, such as in the nose, head, or joints, 3D information of the lesion tissue can be introduced to improve the accuracy of medical image processing.

[0243] Optionally, a three-dimensional point cloud image of the lesion tissue can be acquired.

[0244] Step 323: Generate a three-dimensional model of the lesion tissue based on the three-dimensional point cloud image, flatten the three-dimensional model, and obtain a two-dimensional flattened image of the lesion tissue.

[0245] For example, a three-dimensional model (3D mesh) of the lesion tissue is generated based on a three-dimensional point cloud image, and the three-dimensional model is flattened to obtain a two-dimensional flattened image (UV Field) of the lesion tissue.

[0246] Optionally, the 3D model is generated using the rolling sphere method, and then flattened using the Boundary First Flatten (BFF) algorithm. There is a one-to-one correspondence between the point cloud of the 3D model and the pixels in the 2D flattened image, and a one-to-one correspondence between the pixels in the medical image and the pixels in the 2D flattened image.

[0247] Step 324: Determine the lesion outline of the lesion tissue on the two-dimensional flattened image.

[0248] Optionally, based on the one-to-one correspondence between pixels in the medical image and pixels in the two-dimensional flattened image, the lesion contour of the lesion tissue is determined on the two-dimensional flattened image. Therefore, the subsequent identification of the pathological sampling area corresponding to the lesion contour, and the generation of a sampling grid for dividing the pathological sampling area, are performed based on the lesion contour on the two-dimensional flattened image.

[0249] In some embodiments, after generating a three-dimensional model of the lesion tissue based on a three-dimensional point cloud image, a sampling grid for dividing the pathological sampling area can be generated directly on the three-dimensional model. The specific generation method in this case is not limited in the embodiments of this application.

[0250] In this embodiment, by further introducing three-dimensional information of the lesion tissue, the algorithm accuracy can be further improved, and the accuracy of the generated sampling mesh can be increased.

[0251] In one alternative embodiment of this application, please refer to [link / reference]. Figure 14 The method also includes:

[0252] Step 325: Expand the lesion outline equidistantly to determine the sampling outline, which is used to determine the pathological sampling area.

[0253] For example, the sampling outline refers to the outline of the pathological sampling area. Optionally, the lesion outline is expanded outward at equal intervals to determine the sampling outline. The sampling outline is used to determine the pathological sampling area so that the subsequently generated sampling mesh can fully include the edge area of ​​the lesion tissue, preventing the edge area from being missed.

[0254] Optionally, the equidistant expansion distance can be set according to actual technical needs. Based on clinical practice, it can be set to an expansion of about 2mm.

[0255] Optionally, the coordinates of at least two contour points on the lesion contour are determined, and based on the coordinates of the at least two contour points on the lesion contour, the lesion contour is expanded outward at equal intervals using the polygon scaling algorithm (Vatti's Clipping Algorithm) to determine the sampling contour.

[0256] In this embodiment, the sampling outline is determined by equidistantly expanding the lesion contour. This sampling outline is used to define the pathological sampling area. This ensures that the subsequently generated sampling mesh can fully encompass the edge area of ​​the lesion tissue, preventing the omission of lesion tissue in that edge area.

[0257] In some embodiments, the lesion contour may be expanded outward at non-equidistant intervals to determine the sampling contour. For example, the relatively smooth parts of the lesion contour may be expanded outward at equidistant intervals, while the relatively rough parts of the lesion contour may be expanded outward at non-equidistant intervals, so that the overall lesion contour is relatively smooth, in order to facilitate the division of the sampling grid.

[0258] In some embodiments, Figure 15 A flowchart illustrating an exemplary embodiment of the image generation method provided in this application is shown. After generating a sampling grid for dividing a pathological sampling area, and before displaying a medical image covered by the sampling grid, the method further includes:

[0259] Step 371: Draw the sampling mesh onto the 3D model to obtain a 3D model with the sampling mesh drawn on it.

[0260] Optionally, if the sampling mesh for dividing the pathological sampling area is generated based on the lesion outline on a two-dimensional flattened image, the sampling mesh is drawn onto a three-dimensional model to obtain a three-dimensional model with the sampling mesh drawn on it.

[0261] Optionally, based on the one-to-one correspondence between the point cloud of the 3D model and the pixels in the 2D flattened image, the sampling mesh is drawn onto the 3D model to obtain a 3D model with the sampling mesh drawn on it.

[0262] Step 372: Render the 3D model with the sampling grid to obtain a medical image covered with the sampling grid.

[0263] Optionally, a 3D rendering method from computer vision technology can be used to render the 3D model with the sampling grid drawn on it to obtain a medical image covered with the sampling grid.

[0264] Optionally, the 3D rendering method includes at least one of scanline rendering and radiosity.

[0265] In this embodiment, by rendering a 3D model with a sampling mesh, a medical image covered with the sampling mesh is obtained. This allows the sampling mesh to fit the lesion tissue area in the medical image more closely, and also addresses the problem of inaccurate sampling mesh generation in areas of uneven lesion tissue.

[0266] In some embodiments, after pathological sampling, the sampled tissue can be rapidly paraffin-embedded to create pathological sections, which are then examined under a microscope to check for any residual lesions. This process can also be called rapid pathology. If residual lesions are present, a second excision and secondary pathological sampling are performed at the corresponding location. For rapid pathological testing, the detection and processing time still requires 4 to 6 hours. During this period, the lesion tissue inevitably undergoes some changes. Therefore, it is necessary to accurately locate the pathological sampling area containing residual lesions within the lesion tissue that has undergone some movement and change.

[0267] Optionally, the method further includes: acquiring a second medical image of the lesion tissue, the second medical image having a different lesion contour from at least a portion of the medical image; displaying the second medical image having a second sampling grid, the second sampling grid having at least two sub-grids corresponding to the sampling grid, and the same sub-grid covering the same pathological sampling area in both medical images.

[0268] The second medical image refers to an image taken at a second time after the lesion tissue has been removed using the Mohr's procedure. For example, if, after rapid pathology, microscopic pathological examination determines that residual lesions still exist, a second resection and a second pathological sample are required. The second time refers to any point in time following the rapid pathology.

[0269] For example, the second medical image differs from at least a portion of the lesion contour in the medical image. The reasons for at least a portion of the lesion contour differing include at least one of the following: changes in the position of the imaging component, changes in the camera parameters of the imaging component, changes in the lesion tissue itself, and changes in the patient's position.

[0270] In some embodiments, the lesion tissue is photographed at a second time using the same imaging components as the medical image, to obtain a second medical image of the lesion tissue at a second time.

[0271] The second sampling grid is obtained by transforming the sampling grid based on the image registration parameters, and is not regenerated based on the second medical image. The image registration parameters are configured based on the medical image and the second medical image. The image registration parameters are used to transform the sampling grid, and the type of the image registration parameters also indicates the method of transforming the sampling grid. Different types of image registration parameters result in different methods of transforming the sampling grid.

[0272] For example, the second sampling grid has at least two sub-grids corresponding to the sampling grid, and the same sub-grid covers the same pathological sampling area in both medical images. The sub-grid parameters of the at least two sub-grids in the second sampling grid are the same as or different from the sub-grid parameters of the at least two sub-grids in the sampling grid. The sub-grid parameters include at least one of the following: sub-grid size, shape, area, and interior angle size.

[0273] Optionally, a second medical image with a second sampling grid is displayed. The second sampling grid has at least two sub-grids corresponding to the sampling grid, and the same sub-grid covers the same pathological sampling area in both medical images.

[0274] For example, if all subgrids in the sampling grid are squares, and the first subgrid in the sampling grid covers the central region of the lesion in the medical image, then after transforming the sampling grid, a second sampling grid is obtained. The subgrids in the second sampling grid can all be rhombuses, and the second subgrid, corresponding to the first subgrid, covers the central region of the lesion in the second medical image.

[0275] In some embodiments, the second sampling grid has at least two sub-grids corresponding to the sampling grid, and the same sub-grid has the same number in both medical images. Thus, the physician can refer to the sampling grid number to determine the same pathological sampling area represented by the same number in both medical images, facilitating secondary resection and secondary pathological sampling of the pathological sampling area when residual lesions exist there.

[0276] In some embodiments, the method further includes: displaying the numbers of at least two subgrids in the second sampling grid to assist the physician in accurately relocating the specific sampling location.

[0277] In some embodiments, the method further includes: determining a first subgrid in a sampling grid corresponding to the pathological sampling area with residual lesions, displaying a second subgrid in a second sampling grid, the second subgrid corresponding to the first subgrid, and the second subgrid covering the pathological sampling area with residual lesions.

[0278] Optionally, after rapid pathology, the pathological sampling area with residual lesions can be identified. In this case, it is not necessary to transform and reposition all the sampling grids. Only the corresponding position of the pathological sampling area with residual lesions in the lesion tissue after rapid pathology can be transformed and repositioned to assist the doctor in performing secondary excision and secondary pathological sampling at this position.

[0279] Alternatively, when applying the second sampling grid during the Mohs procedure, the second sampling grid may also be referred to as the second Mohs grid.

[0280] Figure 16 A flowchart of an image generation method provided in an exemplary embodiment of this application is shown, in which the method is applied... Figure 1 The method, illustrated by the terminal 120 or an image processing-enabled client installed on the terminal 120, includes:

[0281] Step 820: Obtain a medical image of the lesion tissue, which includes the lesion outline of the lesion tissue.

[0282] For example, lesion tissue refers to tissue containing pathogenic microorganisms; any tissue or organ can become a lesion, such as tumor tissue. A lesion outline refers to a continuous contour curve of the lesion tissue. The lesion outline can be a regular shape or an irregular shape. A medical image refers to an image taken after tumor tissue has been removed using the Mohr's procedure.

[0283] Optionally, the medical image of the lesion tissue is a two-dimensional image, and the terminal acquires the medical image of the lesion tissue, which includes the lesion outline of the lesion tissue.

[0284] Step 840: Display a medical image covered by a sampling grid, the sampling grid including at least two sub-grids arranged in an array covering the pathological sampling area corresponding to the lesion outline; each sub-grid covers a pathological sampling area containing lesion tissue.

[0285] For example, the server identifies the pathological sampling area corresponding to the lesion outline and generates a sampling grid for dividing the pathological sampling area into grids. The terminal then displays a medical image covered by the sampling grid, which includes at least two sub-grids arranged in an array covering the pathological sampling area corresponding to the lesion outline; each sub-grid covers a pathological sampling area containing lesion tissue.

[0286] In some embodiments, the method further includes displaying medical images of the lesion tissue.

[0287] For example, after acquiring a medical image of the lesion tissue, the terminal also displays the medical image of the lesion tissue to help doctors know whether the medical image is clear, whether the shooting location is accurate, and whether the medical image contains all the lesion tissue, thereby improving the accuracy of medical image processing.

[0288] In some embodiments, the method further includes: displaying the lesion outline of the lesion tissue in a medical image.

[0289] For example, after the computer device automatically identifies the lesion outline in the background, the terminal also displays the lesion outline of the lesion tissue in the medical image to help doctors know whether the automatically identified lesion outline is accurate, so as to facilitate manual modification or redrawing.

[0290] In some embodiments, the sampling grid can also be projected onto the pathological sampling area using a projection component, displaying the sampling grid on the pathological sampling area. In this case, the terminal does not need to display a medical image covered by the sampling grid.

[0291] In summary, the method provided in this application acquires a medical image of the lesion tissue, including the lesion outline, and then displays the medical image covered by a sampling grid. The sampling grid includes at least two sub-grids arranged in an array, covering the pathological sampling area corresponding to the lesion outline; each sub-grid contains lesion tissue within its covered pathological sampling area. Accordingly, the sampling grid can be automatically generated without manual drawing by the doctor, minimizing errors and the randomness of manual grid planning, providing a unified and homogeneous sampling grid division scheme, assisting doctors in pathological sampling, avoiding errors during sampling, and ensuring that lesion tissue can be sampled within each sub-grid, thus improving the efficiency of pathological sampling. Furthermore, it can assist doctors in subsequent pathological analysis and treatment planning.

[0292] As an example, the overall flow of the image generation method according to the embodiments of this application will be described next.

[0293] Please see Figure 17 and Figure 18 This embodiment provides an image generation method based on two-dimensional information. Please refer to [link / reference]. Figure 19 and Figure 20 This embodiment also provides an image generation method based on two-dimensional and three-dimensional information.

[0294] Optionally, when the lesion is located in a relatively flat area or with minimal curvature, such as on the limbs or trunk, a two-dimensional information-based image generation method can be used for medical image processing. When the lesion is located in an uneven area or with significant curvature, such as on the nose, head, or joints, three-dimensional information can be incorporated, and a two-dimensional and three-dimensional information-based image generation method can be used for medical image processing.

[0295] 1. Image generation method based on two-dimensional information

[0296] Step 10: Obtain a medical image of the lesion tissue, which includes the lesion outline of the lesion tissue.

[0297] Optionally, the lesion tissue is photographed using an industrial camera or an RGB camera to obtain a medical image of the lesion tissue. The medical image includes the outline of the lesion tissue, such as... Figure 17-1 As shown. Among them, Figure 17-1 The medical images shown can be displayed on the terminal.

[0298] Step 11: Using an AI model, identify the lesion outline of the lesion tissue in the medical image.

[0299] Optionally, the lesion contours of lesion tissue in medical images can be identified using a pre-trained image segmentation Unet model and / or a semantic segmentation model Deeplab V3+, such as... Figure 17-2 As shown. Among them, Figure 17-2 The outline of the lesion shown can be displayed on the terminal.

[0300] Step 12: Expand the lesion outline equidistantly to determine the sampling outline. The sampling outline is used to determine the pathological sampling area and identify the pathological sampling area.

[0301] Optionally, Vatti's Clipping Algorithm is used to equidistantly expand the lesion outline by 2mm to determine the sampling outline. This sampling outline is used to define and identify the pathological sampling area. Figure 17-3 As shown. Among them, Figure 17-3 The sample outline shown can be displayed on the terminal.

[0302] Step 13: Generate a sampling grid for dividing the pathological sampling area into grids. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Each sub-grid covers a pathological sampling area containing lesion tissue.

[0303] Optionally, after generating the sampling grid, at least two sub-grids within the sampling grid are numbered, resulting in a final sampling grid as follows: Figure 17-4 As shown. Among them, Figure 17-4 The material sampling grid, four quadrants, coordinate axes, and numbers shown can all be displayed on the terminal.

[0304] In some embodiments, in actual clinical applications, the sampling grid includes at least four subgrids, that is, at least one subgrid in each of the four quadrants. After the subgrids in the sampling grid are merged, the number of subgrids may be less than four. In this case, the user is generally required to reset the subgrid parameters.

[0305] Step 14: Display the medical image covered with the sampling grid.

[0306] Optionally, a medical image covered with a sampling grid can be displayed on a terminal screen, virtual reality (VR) glasses, or the sampling grid can be projected onto the pathological sampling area using a projection component.

[0307] 2. Image generation methods based on two-dimensional and three-dimensional information

[0308] Step 20: Obtain a medical image of the lesion tissue, which includes the lesion outline of the lesion tissue.

[0309] Optionally, the lesion tissue is photographed using an RGBD camera to obtain a medical image of the lesion tissue, which includes the outline of the lesion tissue, such as... Figure 19-1 As shown. Among them, Figure 19-1 The medical images shown can be displayed on the terminal.

[0310] Step 22: Using an AI model, identify the lesion outline of the lesion tissue in the medical image.

[0311] Optionally, the lesion contours of lesion tissue in medical images can be identified using a pre-trained image segmentation Unet model and / or a semantic segmentation model Deeplab V3+, such as... Figure 19-2 As shown. Among them, Figure 19-2 The outline of the lesion shown can be displayed on the terminal.

[0312] Step 21: Obtain a three-dimensional point cloud image of the lesion tissue.

[0313] Optionally, the lesion tissue can be photographed using an RGBD camera to obtain a three-dimensional point cloud image of the lesion tissue, such as... Figure 19-4 As shown.

[0314] Step 23: Generate a three-dimensional model of the lesion tissue based on the three-dimensional point cloud image, flatten the three-dimensional model, and obtain a two-dimensional flattened image of the lesion tissue.

[0315] Optionally, based on the 3D point cloud image, a 3D model (3D mesh) of the lesion tissue is generated using the rolling sphere method, such as... Figure 19-5 As shown, the three-dimensional model was flattened using a Boundary First Flatten (BFF) algorithm to obtain a two-dimensional flattened image (UV field) of the lesion tissue, as shown. Figure 19-6 As shown. There is a one-to-one correspondence between the point cloud of the 3D model and the pixels in the 2D flattened image, and a one-to-one correspondence between the pixels in the medical image and the pixels in the 2D flattened image. Figure 19-6 The two-dimensional flattened image shown can be displayed on the terminal.

[0316] Optionally, steps 20 and 22 above can be performed in parallel with steps 21 and 23.

[0317] Step 24: Determine the lesion outline of the lesion tissue on the two-dimensional flattened image.

[0318] Optionally, based on the one-to-one correspondence between pixels in the medical image and pixels in the two-dimensional flattened image, the lesion outline of the lesion tissue is determined on the two-dimensional flattened image, such as... Figure 19-7 As shown. Among them, Figure 19-7 The outline of the lesion on the two-dimensional flattened image shown can be displayed on the terminal.

[0319] Step 25: Expand the lesion outline equidistantly to determine the sampling outline. The sampling outline is used to determine the pathological sampling area and identify the pathological sampling area.

[0320] Optionally, Vatti's Clipping Algorithm is used to expand the lesion outline outwards at equal intervals, setting the equal interval expansion to 2mm to determine the sampling outline. This sampling outline is used to determine and identify the pathological sampling area. This sampling outline can be displayed on the terminal.

[0321] Optionally, the above steps can also involve first identifying the lesion outline of the lesion tissue in the medical image, such as... Figure 19-2 As shown, the lesion outline is then expanded outwards at equal intervals to determine the sampling outline, as follows. Figure 19-3 As shown, the sampling outline is then determined on the two-dimensional flattened image, as follows. Figure 19-7 As shown.

[0322] Step 26: Generate a sampling grid for dividing the pathological sampling area into grids. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. Each sub-grid covers a pathological sampling area containing lesion tissue.

[0323] Optionally, a sampling mesh is generated for dividing the pathological sampling area, such as... Figure 19-8 As shown. After generating the sampling grid, at least two sub-grids within the sampling grid are also numbered.

[0324] Step 27: Draw the sampling mesh onto the 3D model to obtain a 3D model with the sampling mesh drawn on it.

[0325] Optionally, based on the one-to-one correspondence between the point cloud of the 3D model and the pixels in the 2D flattened image, the sampling mesh is drawn onto the 3D model to obtain a 3D model with the sampling mesh drawn on it.

[0326] Step 28: Render the 3D model with the sampling mesh to obtain a medical image covered with the sampling mesh.

[0327] Optionally, 3D rendering techniques from computer vision are used to render the 3D model with the sampling mesh to obtain a medical image covered by the sampling mesh, such as... Figure 19-9 As shown. 3D rendering methods include at least one of scanline rendering and radiosity. Among them, Figure 19-9 The two medical images shown, covered with sampling grids, can be displayed on the terminal.

[0328] Step 29: Display the medical image covered with the sampling grid.

[0329] Optionally, a medical image covered with a sampling grid can be displayed on a terminal screen, virtual reality (VR) glasses, or the sampling grid can be projected onto the pathological sampling area using a projection component.

[0330] Figure 21 This application shows a structural block diagram of an image generation apparatus provided in an exemplary embodiment, the apparatus comprising:

[0331] The acquisition module 910 is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue.

[0332] The identification module 920 is used to identify the pathological sampling area corresponding to the lesion outline.

[0333] The generation module 930 is used to generate a sampling grid for dividing the pathological sampling area into grids, the sampling grid including at least two sub-grids that cover the pathological sampling area and are arranged in an array; each of the sub-grids covers the pathological sampling area containing the lesion tissue.

[0334] Display module 940 is used to display the medical image covered by the sampling grid.

[0335] In one example, the generation module 930 is used for:

[0336] Based on at least one reference point in the medical image, a sampling grid is generated for dividing the pathological sampling area into grids.

[0337] In one example, the generation module 930 is used for:

[0338] Determine the center point of the lesion outline;

[0339] Using the center point as the reference point, expand to generate at least two sub-grids arranged in an array;

[0340] If at least two sub-grids intersect with the entire contour of the lesion, the expansion process ends, generating the at least two sub-grids to obtain a sampling grid for dividing the pathological sampling area.

[0341] If at least two sub-grids do not intersect with at least a portion of the lesion contour, the newly added sub-grid vertices of the at least two sub-grids generated in this round of expansion are used as reference points for the next round, and the at least two sub-grids of the array arrangement are expanded using the reference points for the next round as expansion points.

[0342] In one example, the generation module 930 is used for:

[0343] If the at least two sub-mesh do not intersect with at least a portion of the lesion contour, the newly added sub-mesh vertices of the at least two sub-mesh generated in this round of expansion are used as reference points for the next round. The at least two sub-mesh arranged in a grid pattern around the expansion points are then expanded using the next round reference points as expansion points.

[0344] In one example, the generation module 930 is used for:

[0345] Determine the center point of the lesion outline;

[0346] The center point is determined as the reference point, and a coordinate system is determined based on the reference point;

[0347] In the coordinate system, calculate the subgrid vertex coordinates of at least two subgrids in the array arrangement;

[0348] Based on the coordinates of each of the sub-grid vertices, a sampling grid is generated for dividing the pathological sampling area into grids.

[0349] In one example, the generation module 930 is used for:

[0350] A sub-mesh is generated based on the coordinates of at least three sub-mesh vertices; at least one edge of the sub-mesh is parallel to the target axis in the coordinate system.

[0351] The target sub-mesh in at least two of the sub-mesh is determined to be a sub-mesh belonging to the sampling mesh, and at least one sub-mesh vertex of the target sub-mesh is located within the lesion contour.

[0352] In one example, the device further includes: a removal module;

[0353] In one example, the removal module is used for:

[0354] Remove duplicate subgrids from the at least two subgrids;

[0355] or,

[0356] Remove the subgrids from the at least two subgrids that do not intersect with the lesion outline and do not contain the lesion tissue.

[0357] In one example, the device further includes: a parameter determination module;

[0358] In one example, the parameter determination module is used for:

[0359] Obtain the binary mask image of the medical image;

[0360] Determine at least two contour points on the lesion contour in the binary mask image;

[0361] Calculate the distance between every two contour points of the at least two contour points;

[0362] The major axis of the lesion contour is determined based on the distance, and the major axis is used to help determine the clinical medical parameters of the lesion tissue.

[0363] In one example, the parameter determination module is used for:

[0364] Determine the perpendicular bisector of the major axis of the lesion contour;

[0365] The minor axis of the lesion contour is determined based on the intersection of the vertical bisector and the lesion contour.

[0366] Based on the pixel lengths of the major axis and the minor axis on the medical image, and the spatial distance between each pixel in the medical image, the lengths of the major axis and the minor axis of the lesion contour are determined respectively. The lengths of the major axis and the minor axis are used to assist in the pathological analysis of the pathological sampling area.

[0367] In one example, the submesh parameters of the at least two submesh include at least one of the submesh size, shape, area, and interior angle size;

[0368] Wherein, the sub-mesh parameters are fixed by default; or, the sub-mesh parameters are preset; or, the sub-mesh parameters are dynamically changed.

[0369] In one example, the device further includes: an adjustment module;

[0370] In one example, the adjustment module is used for:

[0371] The camera parameters of the medical image are obtained, and the camera parameters include at least one of internal parameters, external parameters, and shooting distance;

[0372] The subgrid parameters are adjusted based on the camera parameters of the medical image.

[0373] In one example, the device further includes: a merging module;

[0374] In one example, the merging module is used for:

[0375] Merge at least two subgrids in the sampling grid that meet the merging conditions.

[0376] In one example, the merging module is used for:

[0377] A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids;

[0378] From the at least two subgrids other than the first subgrid, determine the second subgrid that is closest to the first subgrid;

[0379] Merge the first sub-mesh with the second sub-mesh.

[0380] In one example, the merging module is used for:

[0381] A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids;

[0382] From the at least two subgrids other than the first subgrid, determine the third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue;

[0383] Merge the first sub-mesh with the third sub-mesh.

[0384] In one example, the merging module is used for:

[0385] A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids;

[0386] From the at least two subgrids other than the first subgrid, determine the fourth subgrid that is closest to the first subgrid and closest in the target direction to the center point of the lesion contour;

[0387] Merge the first sub-grid with the fourth sub-grid.

[0388] In one example, the identification module 920 is used for:

[0389] The AI ​​model identifies the lesion outline of the lesion tissue in the medical image.

[0390] In one example, the identification module 920 is used for:

[0391] The AI ​​model is used to identify the lesion outline of the lesion tissue in the medical image;

[0392] Obtain a three-dimensional point cloud image of the lesion tissue;

[0393] A three-dimensional model of the lesion tissue is generated based on the three-dimensional point cloud image, and the three-dimensional model is flattened to obtain a two-dimensional flattened image of the lesion tissue.

[0394] The lesion outline of the lesion tissue is determined on the two-dimensional flattened image.

[0395] In one example, the identification module 920 is used for:

[0396] The lesion outline is expanded outward at equal intervals to determine the sampling outline, which is used to determine the pathological sampling area.

[0397] In one example, the generation module 930 is used for:

[0398] The sampling mesh is drawn onto the three-dimensional model to obtain the three-dimensional model with the sampling mesh drawn on it.

[0399] The three-dimensional model with the sampling grid drawn on it is rendered to obtain the medical image covered by the sampling grid.

[0400] Figure 22 This application shows a structural block diagram of an image generation apparatus provided in an exemplary embodiment, the apparatus comprising:

[0401] The acquisition module 950 is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue.

[0402] Display module 960 is used to display the medical image covered by a sampling grid, the sampling grid including at least two sub-grids arranged in an array covering the pathological sampling area corresponding to the lesion outline; each of the sub-grids covers the pathological sampling area containing the lesion tissue.

[0403] This application also provides a computer device, which includes: a processor and a memory, wherein the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the image generation method provided in the above-described method embodiments.

[0404] Alternatively, the computer device is a server. For example, Figure 23 This is a structural block diagram of a server provided in an exemplary embodiment of this application.

[0405] Typically, server 1000 includes a processor 1001 and memory 1002.

[0406] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a central processing unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1001 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0407] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the image generation method provided in the method embodiments of this application.

[0408] In some embodiments, the server 1000 may optionally include an input interface 1003 and an output interface 1004. The processor 1001, memory 1002, and input interfaces 1003 and 1004 can be connected via a bus or signal lines. Various peripheral devices can be connected to the input interfaces 1003 and 1004 via a bus, signal lines, or a circuit board. The input interfaces 1003 and 1004 can be used to connect at least one input / output (I / O) related peripheral device to the processor 1001 and memory 1002. In some embodiments, the processor 1001, memory 1002, and input interfaces 1003 and 1004 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, memory 1002, and input interfaces 1003 and 1004 can be implemented on separate chips or circuit boards, and this application does not limit this aspect.

[0409] Those skilled in the art will understand that Figure 23 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0410] In an exemplary embodiment, this application also provides a chip including programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement the image generation method described above.

[0411] This application provides a computer-readable storage medium storing a computer program that is loaded and executed by a processor to implement the image generation method provided in the above-described method embodiments.

[0412] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device retrieves the computer instructions from the computer-readable storage medium, causing the processor to load and execute the image generation method provided in the above-described method embodiments.

[0413] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0414] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0415] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0416] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image generation method, characterized in that, The method includes: Acquire medical images of the lesion tissue, the medical images including the lesion outline of the lesion tissue; Identify the pathological sampling area corresponding to the outline of the lesion; Determine the center point of the lesion outline, and use the center point as a reference point to expand and generate at least two sub-grids arranged in an array; If at least two sub-grids intersect with the entire contour of the lesion, the expansion process ends, generating the at least two sub-grids to obtain a sampling grid for dividing the pathological sampling area. If at least two sub-grids do not intersect with at least a portion of the lesion contour, the newly added sub-grid vertices of the at least two sub-grids generated in this round of expansion are used as reference points for the next round. The next round reference points are used as expansion points to continue expanding and generating the at least two sub-grids arranged in an array. The sampling grid includes at least two sub-grids that cover the pathological sampling area and are arranged in an array. The pathological sampling area covered by each sub-grid contains the lesion tissue. The medical image is displayed with the sampling grid covering it.

2. The method according to claim 1, characterized in that, When at least two sub-mesh grids do not intersect with at least a portion of the lesion contour, the newly added sub-mesh vertices of the at least two sub-mesh grids generated in this round of expansion are used as reference points for the next round, and the expansion continues using the next round reference points as expansion points to generate the at least two sub-mesh grids arranged in the array, including: If the at least two sub-mesh do not intersect with at least a portion of the lesion contour, the newly added sub-mesh vertices of the at least two sub-mesh generated in this round of expansion are used as reference points for the next round. The at least two sub-mesh arranged in a grid pattern around the expansion points are then expanded using the next round reference points as expansion points.

3. The method according to claim 2, characterized in that, The method further includes: Remove duplicate subgrids from the at least two subgrids; or, Remove the subgrids from the at least two subgrids that do not intersect with the lesion outline and do not contain the lesion tissue.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the binary mask image of the medical image; Determine at least two contour points on the lesion contour in the binary mask image; Calculate the distance between every two contour points of the at least two contour points; The major axis of the lesion contour is determined based on the distance, and the major axis is used to help determine the clinical medical parameters of the lesion tissue.

5. The method according to claim 4, characterized in that, The method further includes: Determine the perpendicular bisector of the major axis of the lesion contour; The minor axis of the lesion contour is determined based on the intersection of the vertical bisector and the lesion contour. Based on the pixel lengths of the major axis and the minor axis on the medical image, and the spatial distance between each pixel in the medical image, the lengths of the major axis and the minor axis of the lesion contour are determined respectively. The lengths of the major axis and the minor axis are used to assist in the pathological analysis of the pathological sampling area.

6. The method according to any one of claims 1 to 3, characterized in that, The sub-mesh parameters of the at least two sub-mesh include at least one of the following: sub-mesh size, shape, area, and interior angle size; Wherein, the sub-mesh parameters are fixed by default; or, the sub-mesh parameters are preset; or, the sub-mesh parameters are dynamically changed.

7. The method according to claim 6, characterized in that, The method further includes: The camera parameters of the medical image are obtained, and the camera parameters include at least one of internal parameters, external parameters, and shooting distance; The subgrid parameters are adjusted based on the camera parameters of the medical image.

8. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Merge at least two subgrids in the sampling grid that meet the merging conditions.

9. The method according to claim 8, characterized in that, The step of merging at least two sub-grids in the sampling grid that meet the merging conditions includes: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the second subgrid that is closest to the first subgrid; Merge the first sub-mesh with the second sub-mesh.

10. The method according to claim 8, characterized in that, The step of merging at least two sub-grids in the sampling grid that meet the merging conditions includes: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue; Merge the first sub-mesh with the third sub-mesh.

11. The method according to claim 8, characterized in that, The step of merging at least two sub-grids in the sampling grid that meet the merging conditions includes: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the fourth subgrid that is closest to the first subgrid and closest in the target direction to the center point of the lesion contour; Merge the first sub-grid with the fourth sub-grid.

12. The method according to any one of claims 1 to 3, characterized in that, After acquiring the medical image of the lesion tissue, and before identifying the pathological sampling area corresponding to the lesion contour, the method further includes: The AI ​​model identifies the lesion outline of the lesion tissue in the medical image.

13. The method according to any one of claims 1 to 3, characterized in that, After acquiring the medical image of the lesion tissue, and before identifying the pathological sampling area corresponding to the lesion contour, the method further includes: The AI ​​model is used to identify the lesion outline of the lesion tissue in the medical image; Obtain a three-dimensional point cloud image of the lesion tissue; A three-dimensional model of the lesion tissue is generated based on the three-dimensional point cloud image, and the three-dimensional model is flattened to obtain a two-dimensional flattened image of the lesion tissue. The lesion outline of the lesion tissue is determined on the two-dimensional flattened image.

14. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The lesion outline is expanded outward at equal intervals to determine the sampling outline, which is used to determine the pathological sampling area.

15. The method according to claim 13, characterized in that, Prior to displaying the medical image covered by the sampling grid, the method further includes: The sampling mesh is drawn onto the three-dimensional model to obtain the three-dimensional model with the sampling mesh drawn on it. The three-dimensional model with the sampling grid drawn on it is rendered to obtain the medical image covered by the sampling grid.

16. An image generation method, characterized in that, The method includes: Acquire medical images of the lesion tissue, the medical images including the lesion outline of the lesion tissue; The medical image is displayed covered by a sampling grid, the sampling grid comprising at least two sub-grids arranged in an array covering a pathological sampling area corresponding to the lesion outline; each of the sub-grids covers a pathological sampling area containing the lesion tissue; The sampling grid is generated through the following steps: determining the center point of the lesion contour, using the center point as a reference point, and expanding to generate at least two sub-grids arranged in an array; when the at least two sub-grids intersect with the entire contour of the lesion, the expansion to generate the at least two sub-grids ends, resulting in a sampling grid used for meshing the pathological sampling area; when the at least two sub-grids do not intersect with at least a part of the contour of the lesion, the newly added sub-grid vertices of the at least two sub-grids generated in this round of expansion are used as the reference point for the next round, and the at least two sub-grids arranged in an array are continued to be expanded using the reference point for the next round.

17. An image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue; The identification module is used to identify the pathological sampling area corresponding to the lesion outline; A generation module is used to determine the center point of the lesion outline, and using the center point as a reference point, expand and generate at least two sub-grids arranged in an array; if the at least two sub-grids intersect with the entire outline of the lesion outline, the expansion and generation of the at least two sub-grids ends, resulting in a sampling grid for dividing the pathological sampling area; if the at least two sub-grids do not intersect with at least a part of the outline of the lesion outline, the newly added sub-grid vertices of the at least two sub-grids generated in this round of expansion are used as the reference point for the next round, and the expansion and generation of the at least two sub-grids arranged in an array continues, using the next round reference point as the expansion point. The sampling grid includes at least two sub-grids arranged in an array that cover the pathological sampling area; each of the sub-grids covers the pathological sampling area and contains the lesion tissue. A display module is used to display the medical image covered by the sampling grid.

18. The apparatus according to claim 17, characterized in that, The generation module is used for: If the at least two sub-mesh do not intersect with at least a portion of the lesion contour, the newly added sub-mesh vertices of the at least two sub-mesh generated in this round of expansion are used as reference points for the next round. The at least two sub-mesh arranged in a grid pattern around the expansion points are then expanded using the next round reference points as expansion points.

19. The apparatus according to claim 18, characterized in that, The device also includes a removal module; The removal module is used to remove duplicate subgrids from the at least two subgrids; or, Remove the subgrids from the at least two subgrids that do not intersect with the lesion outline and do not contain the lesion tissue.

20. The apparatus according to any one of claims 17 to 19, characterized in that, The device further includes a parameter determination module; the parameter determination module is used for: Obtain the binary mask image of the medical image; Determine at least two contour points on the lesion contour in the binary mask image; Calculate the distance between every two contour points of the at least two contour points; The major axis of the lesion contour is determined based on the distance, and the major axis is used to help determine the clinical medical parameters of the lesion tissue.

21. The apparatus according to claim 20, characterized in that, The parameter determination module is used for: Determine the perpendicular bisector of the major axis of the lesion contour; The minor axis of the lesion contour is determined based on the intersection of the vertical bisector and the lesion contour. Based on the pixel lengths of the major axis and the minor axis on the medical image, and the spatial distance between each pixel in the medical image, the lengths of the major axis and the minor axis of the lesion contour are determined respectively. The lengths of the major axis and the minor axis are used to assist in the pathological analysis of the pathological sampling area.

22. The apparatus according to any one of claims 17 to 19, characterized in that, The sub-mesh parameters of the at least two sub-mesh include at least one of the following: sub-mesh size, shape, area, and interior angle size; Wherein, the sub-mesh parameters are fixed by default; or, the sub-mesh parameters are preset; or, the sub-mesh parameters are dynamically changed.

23. The apparatus according to claim 22, characterized in that, The device further includes an adjustment module, the adjustment module being used for: The camera parameters of the medical image are obtained, and the camera parameters include at least one of internal parameters, external parameters, and shooting distance; The subgrid parameters are adjusted based on the camera parameters of the medical image.

24. The apparatus according to any one of claims 17 to 19, characterized in that, The device further includes a merging module, the merging module being configured to: Merge at least two subgrids in the sampling grid that meet the merging conditions.

25. The apparatus according to claim 24, characterized in that, The merging module is used for: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the second subgrid that is closest to the first subgrid; Merge the first sub-mesh with the second sub-mesh.

26. The apparatus according to claim 24, characterized in that, The merging module is used for: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the third subgrid that is closest to the first subgrid and contains the largest amount of lesion tissue; Merge the first sub-mesh with the third sub-mesh.

27. The apparatus according to claim 24, characterized in that, The merging module is used for: A first subgrid is determined from the at least two subgrids based on the size of the lesion tissue contained in the at least two subgrids; From the at least two subgrids other than the first subgrid, determine the fourth subgrid that is closest to the first subgrid and closest in the target direction to the center point of the lesion contour; Merge the first sub-grid with the fourth sub-grid.

28. The apparatus according to any one of claims 17 to 19, characterized in that, The identification module is used for: The AI ​​model identifies the lesion outline of the lesion tissue in the medical image.

29. The apparatus according to any one of claims 17 to 19, characterized in that, The identification module is used for: The AI ​​model is used to identify the lesion outline of the lesion tissue in the medical image; Obtain a three-dimensional point cloud image of the lesion tissue; A three-dimensional model of the lesion tissue is generated based on the three-dimensional point cloud image, and the three-dimensional model is flattened to obtain a two-dimensional flattened image of the lesion tissue. The lesion outline of the lesion tissue is determined on the two-dimensional flattened image.

30. The apparatus according to any one of claims 17 to 19, characterized in that, The identification module is used for: The lesion outline is expanded outward at equal intervals to determine the sampling outline, which is used to determine the pathological sampling area.

31. The apparatus according to claim 29, characterized in that, The generation module is used for: The sampling mesh is drawn onto the three-dimensional model to obtain the three-dimensional model with the sampling mesh drawn on it. The three-dimensional model with the sampling grid drawn on it is rendered to obtain the medical image covered by the sampling grid.

32. An image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire medical images of lesion tissue, the medical images including the lesion outline of the lesion tissue; A display module is used to display the medical image covered by a sampling grid, wherein the sampling grid includes at least two sub-grids arranged in an array, covering the pathological sampling area corresponding to the lesion outline; each of the sub-grids covers the pathological sampling area containing the lesion tissue; The sampling grid is generated through the following steps: determining the center point of the lesion contour, using the center point as a reference point, and expanding to generate at least two sub-grids arranged in an array; when the at least two sub-grids intersect with the entire contour of the lesion, the expansion to generate the at least two sub-grids ends, resulting in a sampling grid used for meshing the pathological sampling area; when the at least two sub-grids do not intersect with at least a part of the contour of the lesion, the newly added sub-grid vertices of the at least two sub-grids generated in this round of expansion are used as the reference point for the next round, and the at least two sub-grids arranged in an array are continued to be expanded using the reference point for the next round.

33. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the image generation method as described in any one of claims 1 to 16.

34. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the image generation method as described in any one of claims 1 to 16.

35. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, from which a processor retrieves the computer instructions, causing the processor to load and execute them to implement the image generation method as described in any one of claims 1 to 16.

Images