Method, device, computer-readable medium, and electronic device for generating training samples

Abstract

The embodiments of the present application provide a method, apparatus, computer-readable medium, and electronic device for generating training samples. By obtaining a set of medical images to be labeled, a first sampling ratio and a second sampling ratio are determined for the image layer thickness in the early image set and the late image in the medical image set, respectively, to sample and obtain early sampling images and late sampling images; the image information of the target area in the early sampling image is compared with the image information of the target area in the late sampling image, and a label representing the change trend of the target area in the early sampling image is determined based on the generated comparison result. Under the premise of ensuring the image quality, the number of images to be labeled is reduced, and the label of the early sampling image is generated based on the comparison result between the corresponding labeled image information. On the basis of ensuring the accuracy of medical image labeling, the efficiency of medical image labeling is improved and the cost of labeling is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer-readable medium, and electronic device for generating training samples. Background Art

[0002] As artificial intelligence (AI) technology becomes increasingly widespread and sophisticated, it is being used in a growing number of applications to solve technical problems, such as 3D object reconstruction and facial recognition. These problems are typically addressed by building models. However, model construction relies on a large number of data samples, and each sample must have a corresponding label to ensure the accuracy of the trained model.

[0003] In practical applications, each sample is basically labeled manually, which relies too much on human resources and consumes costs. Especially when the number of samples required for training the model is large, this sample labeling method often consumes a lot of costs and is too inefficient, which in turn affects the efficiency and cost of building the entire model. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, computer-readable medium, and electronic device for generating training samples, thereby improving the efficiency of medical image labeling and reducing the cost of labeling while ensuring the accuracy of medical image labeling at least to a certain extent.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a method for generating training samples is provided, comprising: obtaining a set of medical images to be labeled, wherein the medical image set comprises an early image set taken for a target area at a first time, and a later image set taken for the target area at a second time; sampling images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain early sampled images; sampling images in the later image set based on a second sampling ratio determined according to the layer thickness of the images in the later image set to obtain later sampled images; obtaining image information of the target areas corresponding to the early sampled images and the later sampled images respectively; and generating a label representing a change trend of the target area in the early sampled images based on a comparison result between the image information of the target areas.

[0007] According to one aspect of an embodiment of the present application, a device for generating training samples is provided, comprising: a first acquisition unit for acquiring a set of medical images to be labeled, wherein the medical image set comprises an early image set taken for a target area at a first time, and a later image set taken for the target area at a second time; a first sampling unit for sampling images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set, to obtain early sampling images; a second sampling unit for sampling images in the later image set based on a second sampling ratio determined according to the layer thickness of the images in the later image set, to obtain later sampling images; a second acquisition unit for acquiring image information of the target areas corresponding to the early sampling images and the later sampling images respectively; and a label generation unit for generating a label representing a change trend of the target area in the early sampling image based on a comparison result between the image information of the target area.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the first sampling unit includes: a first ratio unit, used to determine a first sampling ratio based on the layer thickness of the images in the early image set if the layer thickness of the images in the early image set is less than or equal to a preset first threshold; a first quantity unit, used to determine a target number of early sampling images based on the number of images in the early image set and the quotient of the first sampling ratio; and an early sampling unit, used to sample the target number of images from the images in the early image set as the early sampling images.

[0009] In some embodiments of the present application, based on the aforementioned scheme, the first ratio unit is configured to: determine the image features corresponding to the images based on the images of the early image set; compare the image features corresponding to the images of the early image set to determine the similarity between each of the images; determine the first sampling ratio based on the layer thickness of the images of the early image set and the similarity between each of the images, wherein the first sampling ratio is proportional to the similarity, and the first sampling ratio is inversely proportional to the layer thickness.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the early sampling unit includes: a quality unit, used to detect image quality parameters corresponding to the images in the early image set based on the pixel values of the target area in the image; and a quality sampling unit, used to sample the images in the early image set based on the image quality parameters to obtain early sampling images corresponding to the target number.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the quality unit is configured to: detect the image edge corresponding to the target area based on the pixel value of the target area of the image in the early image set; and determine the image quality parameter corresponding to the image based on the pixel value corresponding to the image edge corresponding to the target area.

[0012] In some embodiments of the present application, based on the aforementioned scheme, the second sampling unit includes: a second ratio unit, which is used to determine the second sampling ratio based on the layer thickness of the image in the later image set if the layer thickness of the image in the later image set is less than or equal to a preset second threshold; a second parameter unit, which is used to detect the image quality parameters corresponding to the images in the later image set based on the target area in the image; and a later sampling unit, which is used to sample from the later image set based on the image quality parameters to obtain a number of later sampling images corresponding to the second sampling ratio.

[0013] In some embodiments of the present application, based on the aforementioned scheme, the device for generating training samples is also used to: if the layer thickness of the image in the later image set is greater than the second threshold and less than a preset third threshold, then determine a third sampling ratio based on the layer thickness of the image in the later image set; wherein, the third sampling ratio is less than the second sampling ratio.

[0014] In some embodiments of the present application, based on the aforementioned scheme, the second acquisition unit is configured to: perform target area edge detection on the early sampling image to obtain the target area edge corresponding to the early sampling image; perform target area edge detection on the late sampling image to obtain the target area edge corresponding to the late sampling image; display the target area edges corresponding to the early sampling image and the late sampling image on the interface, so that the user can mark them based on the target area edges; obtain image information of the target area generated by the user based on the target area edge marking of the early sampling image, and image information of the target area generated based on the target area edge marking of the late sampling image.

[0015] In some embodiments of the present application, based on the aforementioned scheme, the image information of the target area includes edge information and volume of the target area; the label generation unit is configured to: determine the volume corresponding to the target area in the early sampling image based on the edge information corresponding to the early sampling image; determine the volume corresponding to the target area in the later sampling image based on the edge information corresponding to the later sampling image; if the volume corresponding to the target area in the early sampling image is smaller than the volume corresponding to the target area in the later sampling image, then generate a label indicating that the target area in the early sampling image has become larger.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the device for generating training samples is further used to: predict a first volume corresponding to the target area based on the target area in the early sampling image; determine a second volume corresponding to the target area based on the target area in the image in the early image set; calculate the difference between the first volume and the second volume as a sampling error; calculate the mean of the sampling errors based on the sampling errors corresponding to multiple early image sets of the same layer thickness; and update the sampling ratio corresponding to the layer thickness based on the mean of the sampling errors.

[0017] In some embodiments of the present application, based on the aforementioned solution, the device for generating training samples is further configured to: if the mean of the sampling errors is greater than a set threshold, reduce the sampling ratio corresponding to the layer thickness.

[0018] In some embodiments of the present application, based on the aforementioned scheme, the device for generating training samples is further used to: display the images in the medical image set in an interface; obtain at least two sampling ratios selected by the user, and sample the images in the medical image set based on the at least two sampling ratios to obtain sampling images corresponding to the at least two sampling ratios; display the sampling images corresponding to the at least two sampling ratios in the interface respectively; obtain the user's selection information, the selection information being information corresponding to the target sampling ratio selected by the user based on the sampling image displayed in the interface; and display the target sampling ratio in the selection information and the sampling image corresponding to the target sampling ratio in the interface respectively.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating training samples as described in the above embodiment is implemented.

[0020] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method for generating training samples as described in the above embodiments.

[0021] According to one aspect of an embodiment of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for generating training samples provided in the various optional implementations described above.

[0022] In the technical solutions provided in some embodiments of the present application, a medical image set to be labeled is obtained, and a first sampling ratio and a second sampling ratio are determined for the image layer thicknesses of the early image set and the later image in the medical image set, respectively. The early image set is sampled based on the first sampling ratio to obtain an early sampling image, and the later image set is sampled based on the second sampling ratio to obtain a later sampling image. Thereafter, the image information of the target area in the early sampling image is compared with the image information of the target area in the later sampling image. Based on the generated comparison result, a label representing the change trend of the target area in the early sampling image is determined. Sampling is performed by determining a corresponding sampling ratio based on the image layer thicknesses of the early image set and the later image set in the medical image set. Under the premise of ensuring image quality, the number of images to be labeled is reduced, and then a smaller number of sampled images are labeled, and labels for the early sampling images are generated based on the comparison results between the labeled image information. On the basis of ensuring the accuracy of medical image labeling, the efficiency of medical image labeling is improved and the cost of labeling is reduced.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, explaining the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0025] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0026] Figure 2 A schematic diagram schematically illustrates an exemplary system architecture based on a medical imaging application scenario according to an embodiment of the present application;

[0027] Figure 3 The following schematically shows a flow chart of a method for generating training samples according to an embodiment of the present application;

[0028] Figure 4 A schematic diagram schematically shows an early sampling image according to an embodiment of the present application;

[0029] Figure 5 A schematic diagram schematically illustrates edge information corresponding to a target area in an early sampling image according to an embodiment of the present application;

[0030] Figure 6 A flowchart of sampling an image in an early image set to obtain an early sampled image according to an embodiment of the present application is schematically shown;

[0031] Figure 7 Schematically shows a flow chart for determining a sampling ratio according to an embodiment of the present application;

[0032] Figures 8 to 10 Schematically shows brain images corresponding to different slice thicknesses according to one embodiment of the present application;

[0033] Figure 11 A schematic diagram schematically illustrates the process of obtaining an early sampling image by random sampling according to an embodiment of the present application;

[0034] Figure 12 A schematic diagram illustrating the process of obtaining an early sampling image by interval sampling according to an embodiment of the present application is shown;

[0035] Figure 13 Schematically shows a flow chart of sampling based on a set threshold according to an embodiment of the present application;

[0036] Figure 14 A schematic diagram schematically illustrates generating labels corresponding to early sampling images based on edges of a target area according to an embodiment of the present application;

[0037] Figure 15 Schematically shows a flow chart of updating the sampling ratio according to an embodiment of the present application;

[0038] Figure 16 A schematic diagram schematically illustrates a sampling result corresponding to a target sampling ratio selected according to an embodiment of the present application;

[0039] Figure 17 The following schematically shows a block diagram of an apparatus for generating training samples according to an embodiment of the present application;

[0040] Figure 18 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0042] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0043] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0044] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0045] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide or local area network (WAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0046] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to resources over the internet. Broadly speaking, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. Driven by the growth of the internet, real-time data streams, and the diversification of connected devices, as well as the demand for search services, social networking, mobile commerce, and open collaboration, cloud computing has rapidly developed. Unlike previous parallel and distributed computing approaches, the emergence of cloud computing will revolutionize the entire internet model and enterprise management model.

[0047] Artificial intelligence cloud services, also known as AIaaS (AI as a Service), are a mainstream AI platform offering. Specifically, AIaaS platforms break down several common AI services and provide them as standalone or packaged services in the cloud. This service model is similar to an AI-themed marketplace: all developers can access one or more of the platform's AI services through APIs. Experienced developers can also use the platform's AI framework and infrastructure to deploy and maintain their own cloud AI services.

[0048] In the embodiment of the present application, medical images can be annotated based on artificial intelligence cloud services to generate labels corresponding to the medical images, thereby achieving the purpose of generating training samples based on medical images, thereby improving the efficiency and accuracy of generating training samples.

[0049] Big data refers to collections of data that cannot be captured, managed, and processed within a specific timeframe using conventional software tools. These massive, rapidly growing, and diverse information assets require new processing models to enhance decision-making, insight discovery, and process optimization. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to efficiently process large amounts of time-sensitive data. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

[0050] Cloud storage is a new concept derived and developed from the concept of cloud computing. A distributed cloud storage system (hereafter referred to as a storage system) is a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (also known as storage nodes) on a network through application software or application interfaces to work together and provide data storage and service access. Currently, storage systems create logical volumes. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may consist of disks on a single storage device or several storage devices. When a client stores data on a logical volume, it stores the data on a file system. The file system divides the data into multiple parts, each of which is an object. An object contains not only the data but also additional information such as the data identifier (ID). The file system writes each object to the physical storage space of the logical volume and records the storage location of each object. When a client requests data access, the file system can provide access based on the storage location information of each object. The storage system allocates physical storage space to the logical volume in the following ways: according to the estimated capacity of the objects stored in the logical volume (this estimate is often relatively large compared to the actual capacity of the objects to be stored) and the Redundant Array of Independent Disks (RAID) The physical storage space is divided into stripes in advance by using the group of Independent Disks. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0051] In practical applications, since medical images contain a large amount of data information, they will occupy a large amount of storage space. In the embodiments of the present application, by storing the acquired medical images in the cloud, the storage efficiency and management efficiency of the medical images can be improved.

[0052] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0053] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying, tracking, and measuring objects. This involves further processing the images, transforming them into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0054] In the embodiment of the present application, medical images are identified and annotated based on computer vision technology to complete the labeling of medical images and provide a data sample basis for subsequent model training.

[0055] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental path to computer intelligence. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning. With the advancement of AI technology, research and application are expanding across a wide range of fields, including smart homes, smart wearables, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robotics, smart healthcare, and smart customer service. With the advancement of technology, AI will be applied in even more areas and play an increasingly important role.

[0056] The solutions provided in the embodiments of this application involve artificial intelligence computer vision technology and machine learning technology, and are specifically described through the following embodiments:

[0057] Figure 1 A schematic diagram shows an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

[0058] like Figure 1 As shown, the system architecture may include terminal devices (such as Figure 1 101, tablet computer 102, and portable computer 103, which may also be a desktop computer, etc.), network 104, and server 105. Network 104 is a medium for providing a communication link between the terminal device and server 105. Network 104 can include various connection types, such as wired communication links, wireless communication links, etc.

[0059] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as needed. For example, the server 105 may be a server cluster consisting of multiple servers.

[0060] A user can use a terminal device to interact with server 105 via network 104 to send medical images to the server, etc. Server 105 can be a server that provides various services. For example, a user uploads a set of medical images to be labeled to server 105 using terminal device 103 (or terminal device 101 or 102), where the medical image set includes an early image set taken at a first time for a target area and a later image set taken at a second time for the target area. Server 105 obtains the set of medical images to be labeled, samples the images in the early image set based on a first sampling ratio determined according to the slice thickness of the images in the early image set, and obtains early sampled images. It then samples the images in the later image set based on a second sampling ratio determined according to the slice thickness of the images in the later image set, and obtains later sampled images. It then obtains image information of the target area corresponding to the early sampled images and the later sampled images, respectively. Based on the comparison results of the image information of the target area, it generates a label representing the change trend of the target area in the early sampled image.

[0061] The above scheme obtains a medical image set to be labeled, determines a first sampling ratio and a second sampling ratio for the image layer thickness in the early image set and the later image in the medical image set respectively, samples the early image set based on the first sampling ratio to obtain an early sampling image, and samples the later image set based on the second sampling ratio to obtain a later sampling image, then compares the image information of the target area in the early sampling image with the image information of the target area in the later sampling image, and determines a label representing the change trend of the target area in the early sampling image based on the generated comparison result, and performs sampling by determining the corresponding sampling ratio based on the image layer thickness of the early image set and the later image set in the medical image set. Under the premise of ensuring image quality, the number of images to be labeled is reduced, and then a smaller number of sampled images are labeled, and labels for the early sampling images are generated based on the comparison results between the labeled image information. On the basis of ensuring the accuracy of medical image labeling, the efficiency of medical image labeling is improved and the cost of labeling is reduced.

[0062] It should be noted that the method for generating training samples provided in the embodiments of the present application is generally executed by the server 105, and accordingly, the device for generating training samples is generally set in the server 105. However, in other embodiments of the present application, the terminal device may also have similar functions to the server, thereby executing the solution for generating training samples provided in the embodiments of the present application.

[0063] In one embodiment of the present application, Figure 2As shown, the system architecture may include a medical image acquisition device 201, a network 202, a server 203, and a terminal device 204. In this embodiment, the acquisition device 201 is used to acquire medical images of the aorta, and may be a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, etc., without limitation herein. The network 202 in this embodiment is used to provide a communication link between the terminal device and the server 203, and may include various connection types, such as a wired communication link, a wireless communication link, or a Bluetooth, 5G network, etc., without limitation herein, for transmitting the acquired medical images to the vascular detection device. In this embodiment, the terminal device 204 may be one or more of a smartphone, a tablet computer, and a portable computer 204, and may also be a desktop computer, etc., without limitation herein.

[0064] It should be understood that Figure 2 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as needed. For example, server 203 may be a server cluster consisting of multiple servers.

[0065] It should be noted that in this embodiment, the server 203 and the terminal device 204 can have the same function, namely, to implement sample labeling. Specifically, when performing sample labeling, the server 203 or the terminal device 204 obtains a medical image set to be labeled, samples images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain early sampled images; samples images in the later image set based on a second sampling ratio determined according to the layer thickness of the images in the later image set to obtain later sampled images; obtains image information of the target areas corresponding to the early sampled images and the later sampled images respectively; and generates a label representing the change trend of the target area in the early sampled images based on the comparison result between the image information of the target area.

[0066] Through the sample annotation method of the above system, medical images can be acquired in real time for annotation processing, thereby improving the acquisition efficiency and resource utilization of medical images.

[0067] The following is a detailed description of the implementation details of the technical solution of the embodiment of the present application:

[0068] Figure 3 A flowchart of a method for generating a training sample according to an embodiment of the present application is shown. The method for generating a training sample can be executed by a server, which can be Figure 1 Refer to the server shown in . Figure 3As shown, the method for generating training samples includes at least steps S310 to S350, which are described in detail as follows:

[0069] In step S310 , a medical image set to be labeled is obtained, wherein the medical image set includes an early image set taken for a target area at a first time, and a late image set taken for the target area at a second time.

[0070] In one embodiment of the present application, a medical image set to be labeled is first obtained. The medical image set in this embodiment includes an early image set taken for a target area at a first time and a late image set taken for the target area at a second time. Specifically, the target area in this embodiment can be a human body area such as the brain, a hemorrhage area in the brain, or the heart. In the embodiment of the present application, the first time is earlier than the second time, and is used to reflect the state of the target area at the first time through the early image set, and to reflect the state of the target area at the second time through the late image set.

[0071] Optionally, in this embodiment, the medical image set can be stored in a storage device, and when the image needs to be annotated, it can be directly obtained through the storage device; in addition, the captured medical image set can be obtained in real time, and medical images can be obtained from the blockchain network or the cloud.

[0072] For example, the medical image in this embodiment may be a brain CT scan, and the medical image obtained in this embodiment is an image from a single CT scan. For example, a brain CT scan with a slice thickness of 1 mm would contain nearly 200 transverse slices for a single patient. A brain CT scan with a slice thickness of 5 mm would typically contain only about 40 transverse slices.

[0073] In step S320, the images in the early image set are sampled based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain early sampled images.

[0074] In one embodiment of the present application, the acquired medical images have corresponding layer thicknesses, which are used to represent the thickness of the scanned object corresponding to a layer of image. The thicker the image, the more information it contains, and more sampled images are required to represent the image information in the image set.

[0075] In this embodiment, a first sampling ratio is first determined based on the layer thickness of the images in the early image set, and the images in the early image set are sampled based on the first sampling ratio to obtain early sampling images, so that the early sampling images are used to represent the images in the early image set, thereby reducing the number of images to be processed.

[0076] Specifically, the first sampling ratio in this embodiment is the ratio of the number of images in the early image set to the number of samples. When determining the first sampling ratio, it can be determined based on the thickness of the image. For example, the thicker the image, the lower the first sampling ratio.

[0077] In the task of predicting the expansion of cerebral hemorrhage, the data is labeled and used in the form of CT pairs. A CT pair contains two CT scans of a single patient, namely CT1 and CT2. CT1 is an early scan of the patient, while CT2 is a later scan. In actual use, the roles of the two CTs are also different: first, the task uses CT1 as the main input of the neural network to predict whether the current cerebral hemorrhage will expand through early images and obtain a prediction result; secondly, it is necessary to simultaneously calculate the volume of cerebral hemorrhage in CT1 and CT2 to obtain a label of whether the hemorrhage has expanded, which is used as a standard judgment result to compare with the prediction result and then adjust the parameters of the model. Therefore, when constructing the data set, the task has different requirements for CT1 and CT2: CT1 requires low distortion, rich inter-layer information, and accurate measurement of hemorrhage volume; while CT2 only requires accurate measurement of hemorrhage volume. Therefore, in the embodiment of the present application, more refined image parameters are required for the images in the early image set, and a smaller sampling ratio is set relative to the later image set.

[0078] After determining the first sampling ratio, random sampling is performed from the early image set based on the first sampling ratio to obtain early sample images. By sampling based on the first sampling ratio to obtain early sample images, the number of images to be annotated can be reduced, thereby improving the efficiency of image annotation.

[0079] In step S330 , the images in the later image set are sampled based on a second sampling ratio determined according to the layer thickness of the images in the later image set to obtain a later sampled image.

[0080] In one embodiment of the present application, a second sampling ratio is first determined based on the layer thickness of the images in the late image set, and the images in the late image set are sampled based on the second sampling ratio to obtain late sampling images, so that the images in the late image set are represented by the late sampling images, thereby reducing the number of images to be processed.

[0081] Specifically, the second sampling ratio in this embodiment is the ratio of the number of images in the later image set to the number of samples. When determining the second sampling ratio, it can be determined based on the thickness of the image. For example, the thicker the image, the lower the second sampling ratio.

[0082] After determining the second sampling ratio, random sampling is performed from the later image set based on the second sampling ratio to obtain later sampled images. By sampling based on the second sampling ratio to obtain later sampled images, the number of images to be annotated can be reduced, thereby improving the efficiency of image annotation.

[0083] In step S340 , image information of the target area corresponding to the early sampling image and the late sampling image is obtained.

[0084] In one embodiment of the present application, after sampling to obtain an early sampling image and a late sampling image, image information of the target area is obtained based on the target area in the early sampling image and the late sampling image. The target area in the embodiment of the present application may include, for example, a brain hemorrhage area, and the image information of the target area may include edge information, regional information, and the like of the brain hemorrhage area.

[0085] like Figure 4 and Figure 5 As shown, in the application scenario of brain CT, Figure 4 The early sampling image obtained by sampling in the embodiment of the present application includes the target area 410, that is, the bleeding area. After the early sampling image is obtained by sampling, the edge information of the target area can be detected by image recognition, that is, Figure 5 510 in the figure can also be manually delineated to mark edge information 510 or region information of the target region, such as region area or volume, etc.

[0086] In step S350 , based on the comparison result between the image information of the target area, a label representing the change trend of the target area in the early sampling image is generated.

[0087] In one embodiment of the present application, after determining the image information of the target area corresponding to the early sampling image and the later sampling image respectively, the image information of the two times is compared to generate a comparison result, so as to determine the change trend of the target area in the early image based on the comparison result, and represent a label of the change trend of the target area.

[0088] For example, when the image information in the embodiment of the present application is the area of the target area, this embodiment compares the area of the target area. If the area of the target area in the later sampling image is larger than the area of the target area in the earlier sampling image, it means that the target area has expanded during this period, that is, a label indicating that the target area has expanded is generated.

[0089] In the above scheme, sampling is performed by determining the corresponding sampling ratio based on the image layer thickness of the early image set and the later image set in the medical image set. Under the premise of ensuring image quality, the number of images to be labeled is reduced, and then a smaller number of sampled images are labeled, and labels for the early sampled images are generated based on the comparison results between the corresponding labeled image information. On the basis of ensuring the accuracy of medical image labeling, the efficiency of medical image labeling is improved and the cost of labeling is reduced.

[0090] In one embodiment of the present application, Figure 6 As shown, in step S320, based on the first sampling ratio determined according to the slice thickness of the image in the early image set, the image in the early image set is sampled to obtain the early sampled image, including steps S321 to S323, which are described in detail as follows:

[0091] In step S321, if the slice thickness of the image in the early image set is less than or equal to a preset first threshold, a first sampling ratio is determined based on the slice thickness of the image in the early image set;

[0092] In one embodiment of the present application, a first threshold is set to measure the thickness of images in the early image set. Specifically, if the thickness of the images in the early image set is less than or equal to the preset first threshold, a first sampling ratio is determined based on the thickness of the images in the early image set. If the thickness of the images in the early image set is greater than the first threshold, it indicates that the images in the early image set have a higher thickness and contain more information. If sampling is performed, the image information in the early image set will be omitted. Therefore, when the thickness of the images in the early image set is greater than the set threshold, no sampling is performed.

[0093] like Figure 7 As shown, the process of determining the first sampling ratio based on the layer thickness of the image in the early image set in step S321 specifically includes the following steps S3211 to S3223, which are described in detail as follows:

[0094] Step S3211, based on the images in the early image set, determining image features corresponding to the images;

[0095] Step S3212, comparing the image features corresponding to the images in the early image set to determine the similarity between the images;

[0096] Step S3213: determining a first sampling ratio based on the layer thickness of the images in the early image set and the similarity between the images, wherein the first sampling ratio is proportional to the similarity and inversely proportional to the layer thickness.

[0097] In one embodiment of the present application, image features corresponding to each image in the early image set are first extracted, where the image features may include grayscale features, edge features, and the like. The image features corresponding to the images in the image set are then compared to determine the similarity between the images. In this embodiment, the image similarity may be calculated by calculating the cosine distance between the pixel values of each pixel. After determining the similarity, a first sampling ratio is determined based on the slice thickness of the images in the early image set and the similarity between the images.

[0098] Specifically, such as Figures 8 to 10 As shown, Figures 8 to 10 These are adjacent cross-sectional images scanned when the layer thickness is 1.0mm, 2.4mm and 5.0mm respectively, provided in the embodiment of the present application. As can be seen from the figure, in the embodiment of the present application, when the layer thickness of the image is low, the features corresponding to the target area are relatively similar, while when the layer thickness of the image is high, the differences between the features corresponding to the target area are relatively large. Therefore, in the embodiment of the present application, the image features in the image are first extracted based on the images in the early image set, wherein the image features can be grayscale features, edge features, etc.; after the image features are extracted, the image features corresponding to each image in the early image set are compared to determine the similarity between the images, wherein the similarity can be in the form of cosine distance between each feature, etc.; after the similarity is calculated, the first sampling ratio is determined based on the similarity and the layer thickness, wherein the higher the similarity and the smaller the layer thickness, the higher the first sampling ratio obtained.

[0099] When the first sampling ratio is determined by the similarity and the layer thickness, the similarity or the layer thickness may be used as a weight to determine the value of the first sampling ratio.

[0100] Similarly, when calculating the second sampling ratio, it can also be calculated based on the same method, which will not be described here.

[0101] In step S322 , a target number of early sampling images is determined based on a quotient between the number of images in the early image set and the first sampling ratio.

[0102] After the first sampling ratio is calculated, the first sampling ratio is used to represent the ratio of the original number of images to the sampling number. The target number of sampled images is calculated based on the quotient between the number of images in the early image set and the first sampling ratio.

[0103] Furthermore, in this embodiment, the target number can be ensured to be an integer by rounding off or rounding up.

[0104] In step S323, a target number of images are sampled from the images in the early image set to serve as early sampled images.

[0105] In one embodiment of the present application, after the target number of images in the early image set is calculated, sampling is performed from the images in the early image set to obtain early sampled images corresponding to the target number.

[0106] Specifically, such as Figure 11 As shown, the sampling method in this embodiment can be carried out by random sampling to improve the efficiency of sampling. Figure 12 As shown, the sampling interval can also be determined based on the target quantity, and sampling can be performed at a certain interval to obtain an early sampling image with uniform distribution to ensure the comprehensiveness of the image features.

[0107] The process of sampling a target number of images from the early image set in step S323 as early sampled images includes steps S3231 to S3232, which are described in detail as follows:

[0108] Step S3231, detecting image quality parameters corresponding to images in the early image set based on pixel values of the target area in the image;

[0109] Step S3232: based on the image quality parameter, sampling the images in the early image set to obtain the target number of early sampled images.

[0110] In one embodiment of the present application, in the process of sampling images in an early image set, in order to ensure the image quality of the early sampled images, so as to facilitate the subsequent detection and identification of the early sampled images, in this embodiment, image quality parameters are determined based on the pixel values of the target area in each image, and then the quality of each image is determined based on the image quality parameters to extract images with higher quality, thereby improving the accuracy of image labeling.

[0111] In one embodiment of the present application, in step S3231, the image quality parameters corresponding to the images in the early image set are detected based on the pixel values of the target area in the image, including: detecting the image edge corresponding to the target area based on the pixel values of the target area of the images in the early image set; and determining the image quality parameters corresponding to the image based on the pixel values corresponding to the image edge corresponding to the target area.

[0112] In the embodiment of the present application, during the process of determining the image quality parameters, the image edge corresponding to the target area is detected based on the pixel values of the target area in the image of the early image set, wherein the detection method can be image binarization detection. Subsequently, the image quality parameters corresponding to the image are determined based on the pixel values of the image edge corresponding to the target area. The specific method of determination can be to determine the size of the difference between the pixel values. The larger the difference, the greater the change between the pixels, the clearer the edge of the target area of the image, and the easier it is to mark the target area. Therefore, in this embodiment, the corresponding image quality is higher.

[0113] In one embodiment of the present application, the process of sampling the images in the later image set based on the second sampling ratio determined according to the slice thickness of the images in the later image set in step S330 to obtain the later sampled images includes steps S331 to S333, which are described in detail as follows:

[0114] Step S331 , if the layer thickness of the image in the later image set is less than or equal to a preset second threshold, determining a second sampling ratio based on the layer thickness of the image in the later image set;

[0115] Step S332 , detecting image quality parameters corresponding to the images in the later image set based on the target area in the image;

[0116] Step S333 : sampling from the late image set based on the image quality parameter to obtain late sampled images of a number corresponding to the second sampling ratio.

[0117] In step S330, step S334 is also included: if the layer thickness of the image in the later image set is greater than the second threshold and less than the preset third threshold, a third sampling ratio is determined based on the layer thickness of the image in the later image set; wherein the third sampling ratio is less than the second sampling ratio.

[0118] In one embodiment of the present application, unlike the requirements of the early image set CT1 for small distortion, rich inter-layer information, and accurate measurement of bleeding volume, the late image set CT2 only requires accurate measurement of bleeding volume. Therefore, in the embodiment of the present application, a second sampling ratio corresponding to the second threshold and a third sampling ratio corresponding to the third threshold are set; wherein, the third sampling ratio is less than the second sampling ratio. If the layer thickness of the image in the late image set is less than or equal to the preset second threshold, the second sampling ratio is determined based on the layer thickness of the image in the late image set; if the layer thickness of the image in the late image set is greater than the second threshold and less than the preset third threshold, the third sampling ratio is determined based on the layer thickness of the image in the late image set. Thereafter, sampling is performed from the late image set based on the image quality parameter and the sampling ratio to obtain a number of late sampling images corresponding to the sampling ratio.

[0119] like Figure 13 As shown, for example, Figure 13 In the example, the first threshold is 1.0 mm, corresponding to a first sampling ratio of 2:1; the second threshold is 1.5 mm, corresponding to a second sampling ratio of 5:1; and the third threshold is 3.5 mm, corresponding to a third sampling ratio of 2:1. Images in the early image set are sampled based on the first threshold and its corresponding first sampling ratio, and the sampled CT1 is output. For the later image set, samples are performed using the second threshold and its corresponding second sampling ratio, and the third threshold and its corresponding third sampling ratio, and the sampled CT2 is output.

[0120] The above scheme determines the corresponding sampling method based on the different requirements between the early image set and the later image set. While ensuring the sampling effect, it not only ensures the quality of the sampled image, but also reduces the image calculation amount and reduces the cost of image annotation.

[0121] In one embodiment of the present application, step S340 obtains image information of the target areas corresponding to the early sampling image and the late sampling image, including: performing target area edge detection on the early sampling image to obtain the target area edge corresponding to the early sampling image; performing target area edge detection on the late sampling image to obtain the target area edge corresponding to the late sampling image; displaying the target area edges corresponding to the early sampling image and the late sampling image on the interface so that the user can mark them based on the target area edges; obtaining image information of the target area generated by the user based on the target area edge marking of the early sampling image, and image information of the target area generated based on the target area edge marking of the late sampling image.

[0122] like Figure 14 As shown, in the embodiment of the present application, based on the early stage, the target area edge detection is performed on the early sampling image 1410 to obtain the target area edge corresponding to the early sampling image; the target area edge detection is performed on the late sampling image 1420 to obtain the target area edge corresponding to the late sampling image; and then the detected target area edge is displayed on the interface, as shown in FIG. Figure 14 The dotted area in the image is used to facilitate manual annotation by the user. The image information of the target area generated by the user based on the target area edge annotation of the early sampling image and the image information of the target area generated based on the target area edge annotation of the later sampling image are obtained. Figure 14 The solid line portions 1430 and 1440 in FIG.

[0123] Through the above method, manual labeling is performed again on the basis of computational detection, which can improve the accuracy of labeling based on the accuracy of computer recognition and manual subjective judgment.

[0124] In one embodiment of the present application, the image information of the target area includes edge information and volume of the target area; in step S350, based on the comparison result between the image information of the target area, a label representing the change trend of the target area in the early sampling image is generated, including: determining the volume corresponding to the target area in the early sampling image based on the edge information corresponding to the early sampling image; determining the volume corresponding to the target area in the later sampling image based on the edge information corresponding to the later sampling image; if the volume corresponding to the target area in the early sampling image is smaller than the volume corresponding to the target area in the later sampling image, generating a label indicating that the target area in the early sampling image has become larger.

[0125] Please continue to refer to Figure 14 As shown, in the embodiment of the present application, after the early sampling image and the late sampling image are labeled, the volume corresponding to the target area in the early sampling image is determined based on the edge information corresponding to the early sampling image; the volume corresponding to the target area in the late sampling image is determined based on the edge information corresponding to the late sampling image; then, the sizes of the two volumes are compared. If the volume corresponding to the target area in the early sampling image is smaller than the volume corresponding to the target area in the late sampling image, a label is generated indicating that the target area in the early sampling image has become larger; if the volume corresponding to the target area in the early sampling image is equal to the volume corresponding to the target area in the late sampling image, a label is generated indicating that the target area in the early sampling image remains unchanged; if the volume corresponding to the target area in the early sampling image is larger than the volume corresponding to the target area in the late sampling image, a label is generated indicating that the target area in the early sampling image has become smaller.

[0126] In one embodiment of the present application, Figure 15 As shown, the method in this embodiment further includes steps S1510 to S1550, which are described in detail as follows:

[0127] Step S1510, predicting a first volume corresponding to the target region based on the target region in the early sampling image;

[0128] Step S1520 , determining a second volume corresponding to the target area based on the target area in the image in the early image set;

[0129] Step S1530, calculating the difference between the first volume and the second volume as a sampling error;

[0130] Step S1540, calculating a mean of the sampling errors based on the sampling errors corresponding to the plurality of early image sets of the same slice thickness;

[0131] Step S1550: Based on the mean of the sampling error, update the sampling ratio corresponding to the layer thickness.

[0132] In the embodiment of the present application, the first volume corresponding to the target area is predicted based on the target area obtained by sampling, and then the second volume corresponding to the target area is calculated based on the target area in the early image set. The difference between the first volume and the second volume is that the first volume is predicted based on the sampled image obtained by sampling, and the second volume is calculated based on the actual image. Therefore, there is often a certain error between the first volume and the second volume. In the embodiment of the present application, the difference between the first volume and the second volume is calculated as the sampling error, and then the mean of the sampling errors is calculated based on the sampling errors corresponding to multiple early image sets of the same layer thickness. Finally, based on the mean of the sampling errors, the sampling ratio corresponding to the layer thickness is updated. By optimizing the sampling ratio in the above manner, the accuracy of the sampling quantity can be improved, thereby providing a reliable data basis for subsequent sample labeling.

[0133] Specifically, in the process of updating the sampling ratio, if the mean of the sampling error is greater than the set threshold, it means that the error of the sampled image is large, and the number of sampled images needs to be increased to improve the sampling effect, so the sampling ratio corresponding to the layer thickness is reduced.

[0134] Table 1 shows the mean and (±) standard deviation of the changes in bleeding volume before and after sampling under different layer thicknesses and different sampling ratios provided in the examples of this application.

[0135]

[0136]

[0137] Table 1

[0138] As shown in Table 1, 626 marked CT scans were randomly selected in the present application embodiment as sampling objects for testing, and the absolute value of the CT hemorrhage volume change before and after sampling was calculated respectively. The specific results are shown in Table 1. As can be seen from the data in the table, as the sampling ratio and CT slice thickness increase, the change in hemorrhage volume also gradually becomes larger. However, when the slice thickness is thinner or the sampling ratio is lower, the error change of hemorrhage volume is within a smaller range. For example, when the slice thickness S≤1.0mm and the sampling ratio is 2:1, its corresponding mean ± standard deviation is 0.128±0.128, and the image effect obtained by sampling is more accurate than the sampling effect in other cases. By contrast, it can be concluded that the sampling scheme adopted in the present application embodiment does not introduce excessive hemorrhage volume measurement error, and ensures the stability of hemorrhage volume measurement to a certain extent.

[0139] The above analysis proves that the technical solution provided by the embodiment of the present application can effectively help construct an expanded prediction dataset for cerebral hemorrhage. In other similar three-dimensional medical image annotation tasks, the embodiment of the present application is also universal and can adopt flexible and changeable sampling schemes based on the characteristics of the dataset.

[0140] In one embodiment of the present application, the method also includes: displaying images in a medical image set in an interface; obtaining at least two sampling ratios selected by a user, and sampling the images in the medical image set based on the at least two sampling ratios to obtain sampling images corresponding to at least two sampling ratios; displaying the sampling images corresponding to the at least two sampling ratios in the interface respectively; obtaining user selection information, the selection information being information corresponding to the target sampling ratio selected by the user based on the sampling images displayed in the interface; and displaying the target sampling ratio in the selection information and the sampling images corresponding to the target sampling ratio in the interface respectively.

[0141] like Figure 16 As shown, in the embodiment of the present application, the images in the medical image collection are displayed in the interface; at least two sampling ratios selected by the user are obtained, such as Figure 16 4:3 and 2:1 in the sample ratio; based on at least two sampling ratios, the images in the medical image collection are sampled to obtain at least two sampling ratios corresponding sampling images, and the sampling images corresponding to the at least two sampling ratios are displayed in the interface respectively. Then, the user makes a selection through the sampling images displayed in the interface, and obtains the user's selection information, which is the information corresponding to the target sampling ratio selected by the user based on the sampling images displayed in the interface; the target sampling ratio in the selection information and the sampling images corresponding to the target sampling ratio are displayed in the interface respectively. Through the above method, the sampling results can be more completely displayed to the user, so that the user can manually select sampling images with better effects and a relatively dominant number, thereby improving the sampling efficiency while ensuring the sampling effect, and reducing the sampling cost and the cost of sample labeling.

[0142] The following describes an embodiment of the device of the present application, which can be used to execute the method for generating training samples in the above-mentioned embodiment of the present application. It is understood that the device can be a computer program (including program code) running on a computer device, for example, the device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method for generating training samples described above in the present application.

[0143] Figure 17 A block diagram of an apparatus for generating training samples according to an embodiment of the present application is shown.

[0144] Reference Figure 17As shown, an apparatus 1700 for generating training samples according to an embodiment of the present application includes: a first acquisition unit 1710, for acquiring a medical image set to be labeled, wherein the medical image set includes an early image set taken for a target area at a first time, and a late image set taken for the target area at a second time; a first sampling unit 1720, for sampling images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set, to obtain early sampling images; a second sampling unit 1730, for sampling images in the late image set based on a second sampling ratio determined according to the layer thickness of the images in the late image set, to obtain late sampling images; a second acquisition unit 1740, for acquiring image information of the target areas corresponding to the early sampling images and the late sampling images respectively; a label generation unit 1750, for generating a label representing a change trend of the target area in the early sampling image based on a comparison result between the image information of the target area.

[0145] In some embodiments of the present application, based on the aforementioned scheme, the first sampling unit 1720 includes: a first ratio unit, used to determine a first sampling ratio based on the layer thickness of the image in the early image set if the layer thickness of the image in the early image set is less than or equal to a preset first threshold; a first quantity unit, used to determine a target number of early sampling images based on the number of images in the early image set and the quotient of the first sampling ratio; and an early sampling unit, used to sample the target number of images from the images in the early image set as the early sampling images.

[0146] In some embodiments of the present application, based on the aforementioned scheme, the first ratio unit is configured to: determine the image features corresponding to the images based on the images of the early image set; compare the image features corresponding to the images of the early image set to determine the similarity between each of the images; determine the first sampling ratio based on the layer thickness of the images of the early image set and the similarity between each of the images, wherein the first sampling ratio is proportional to the similarity, and the first sampling ratio is inversely proportional to the layer thickness.

[0147] In some embodiments of the present application, based on the aforementioned scheme, the early sampling unit includes: a quality unit, used to detect image quality parameters corresponding to the images in the early image set based on the pixel values of the target area in the image; and a quality sampling unit, used to sample the images in the early image set based on the image quality parameters to obtain early sampling images corresponding to the target number.

[0148] In some embodiments of the present application, based on the aforementioned scheme, the quality unit is configured to: detect the image edge corresponding to the target area based on the pixel value of the target area of the image in the early image set; and determine the image quality parameter corresponding to the image based on the pixel value corresponding to the image edge corresponding to the target area.

[0149] In some embodiments of the present application, based on the aforementioned scheme, the second sampling unit includes: a second ratio unit, which is used to determine the second sampling ratio based on the layer thickness of the image in the later image set if the layer thickness of the image in the later image set is less than or equal to a preset second threshold; a second parameter unit, which is used to detect the image quality parameters corresponding to the images in the later image set based on the target area in the image; and a later sampling unit, which is used to sample from the later image set based on the image quality parameters to obtain a number of later sampling images corresponding to the second sampling ratio.

[0150] In some embodiments of the present application, based on the aforementioned scheme, the device 1700 for generating training samples is also used to: if the layer thickness of the image in the later image set is greater than the second threshold and less than a preset third threshold, then determine a third sampling ratio based on the layer thickness of the image in the later image set; wherein the third sampling ratio is less than the second sampling ratio.

[0151] In some embodiments of the present application, based on the aforementioned scheme, the second acquisition unit 1740 is configured to: perform target area edge detection on the early sampling image to obtain the target area edge corresponding to the early sampling image; perform target area edge detection on the late sampling image to obtain the target area edge corresponding to the late sampling image; display the target area edges corresponding to the early sampling image and the late sampling image on the interface, so that the user can mark based on the target area edge; obtain image information of the target area generated by the user based on the target area edge marking of the early sampling image, and image information of the target area generated based on the target area edge marking of the late sampling image.

[0152] In some embodiments of the present application, based on the aforementioned scheme, the image information of the target area includes edge information and volume of the target area; the label generation unit 1750 is configured to: determine the volume corresponding to the target area in the early sampling image based on the edge information corresponding to the early sampling image; determine the volume corresponding to the target area in the later sampling image based on the edge information corresponding to the later sampling image; if the volume corresponding to the target area in the early sampling image is smaller than the volume corresponding to the target area in the later sampling image, generate a label indicating that the target area in the early sampling image has become larger.

[0153] In some embodiments of the present application, based on the aforementioned scheme, the device 1700 for generating training samples is further used to: predict a first volume corresponding to the target area based on the target area in the early sampling image; determine a second volume corresponding to the target area based on the target area in the image in the early image set; calculate the difference between the first volume and the second volume as a sampling error; calculate the mean of the sampling errors based on the sampling errors corresponding to multiple early image sets of the same layer thickness; and update the sampling ratio corresponding to the layer thickness based on the mean of the sampling errors.

[0154] In some embodiments of the present application, based on the aforementioned solution, the device 1700 for generating training samples is further configured to: if the mean of the sampling errors is greater than a set threshold, reduce the sampling ratio corresponding to the layer thickness.

[0155] In some embodiments of the present application, based on the aforementioned scheme, the device 1700 for generating training samples is further used to: display the images in the medical image set in an interface; obtain at least two sampling ratios selected by a user, and sample the images in the medical image set based on the at least two sampling ratios to obtain sampling images corresponding to the at least two sampling ratios; display the sampling images corresponding to the at least two sampling ratios in the interface respectively; obtain user selection information, the selection information being information corresponding to the target sampling ratio selected by the user based on the sampling image displayed in the interface; and display the target sampling ratio in the selection information and the sampling image corresponding to the target sampling ratio in the interface respectively.

[0156] Figure 18 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown.

[0157] It should be noted that Figure 18 The computer system 1800 of the electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0158] like Figure 18As shown, computer system 1800 includes a central processing unit (CPU) 1801, which can perform various appropriate actions and processes according to the program stored in read-only memory (ROM) 1802 or the program loaded from storage portion 1808 into random access memory (RAM) 1803, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in RAM 1803. CPU 1801, ROM 1802 and RAM 1803 are connected to each other via bus 1804. Input / output (I / O) interface 1805 is also connected to bus 1804.

[0159] The following components are connected to the I / O interface 1805: an input section 1806 including a keyboard, a mouse, and the like; an output section 1807 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1808 including a hard disk; and a communication section 1809 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the I / O interface 1805 as needed. Removable media 1811, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1810 as needed, so that computer programs read from the removable media can be installed in the storage section 1808 as needed.

[0160] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1809, and / or installed from a removable medium 1811. When the computer program is executed by the central processing unit (CPU) 1801, the various functions defined in the system of the present application are executed.

[0161] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0164] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0165] As another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the method described in the above embodiments.

[0166] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0167] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0168] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0169] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating training samples, characterized in that: include: Acquire a medical image set to be labeled, wherein the medical image set includes an early image set taken for a target area at a first time, and a late image set taken for the target area at a second time; Sampling the images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain an early sampled image; Sampling the images in the late image set based on a second sampling ratio determined according to the layer thickness of the images in the late image set to obtain a late sampled image; Acquire image information of the target area corresponding to the early sampling image and the late sampling image respectively; Based on the comparison result between the image information of the target area, a label representing the change trend of the target area in the early sampling image is generated.

2. The method according to claim 1, characterized in that Sampling the images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain early sampled images, including: If the slice thickness of the image in the early image set is less than or equal to a preset first threshold, determining a first sampling ratio based on the slice thickness of the image in the early image set, where the first sampling ratio is a ratio of the number of images in the early image set to the number of samples; determining a target number of early sampling images based on a quotient between the number of images in the early image set and the first sampling ratio; The target number of images are sampled from the images in the early image set to serve as the early sampled images.

3. The method according to claim 2, characterized in that Determining a first sampling ratio based on the layer thickness of the image in the early image set includes: determining, based on an image in the early image set, image features corresponding to the image; Comparing image features corresponding to the images in the early image set to determine similarities between the images; The first sampling ratio is determined based on the layer thickness of the images in the early image set and the similarity between the images, wherein the first sampling ratio is proportional to the similarity and inversely proportional to the layer thickness.

4. The method according to claim 2, characterized in that Sampling the target number of images from the images in the early image set as the early sampled images includes: detecting image quality parameters corresponding to images in the early image set based on pixel values of a target area in the image; Based on the image quality parameter, images in the early image set are sampled to obtain early sampled images corresponding to the target number.

5. The method according to claim 4, characterized in that Detecting image quality parameters corresponding to images in the early image set based on pixel values of a target area in the image includes: detecting an image edge corresponding to a target area of an image in the early image set based on pixel values of the target area; An image quality parameter corresponding to the image is determined based on pixel values corresponding to an image edge corresponding to the target area.

6. The method according to claim 1, characterized in that Sampling the images in the later image set based on a second sampling ratio determined according to the layer thickness of the images in the later image set to obtain a later sampled image, comprising: If the layer thickness of the image in the later image set is less than or equal to a preset second threshold, determining a second sampling ratio based on the layer thickness of the image in the later image set; detecting image quality parameters corresponding to images in the later image set based on a target area in the image; Based on the image quality parameter, sampling is performed from the later image set to obtain later sampled images of a number corresponding to the second sampling ratio.

7. The method according to claim 6, characterized in that The method further comprises: If the layer thickness of the image in the later image set is greater than the second threshold and less than a preset third threshold, a third sampling ratio is determined based on the layer thickness of the image in the later image set; wherein the third sampling ratio is less than the second sampling ratio.

8. The method according to claim 1, characterized in that Acquiring image information of the target area corresponding to the early sampling image and the late sampling image, respectively, includes: Performing target region edge detection on the early sampling image to obtain a target region edge corresponding to the early sampling image; Performing target area edge detection on the late sampling image to obtain the target area edge corresponding to the late sampling image; Displaying the target area edges corresponding to the early sampling image and the late sampling image on the interface, so that the user can mark based on the target area edges; Image information of the target area generated by the user based on the target area edge annotation of the early sampling image and image information of the target area generated based on the target area edge annotation of the late sampling image are obtained.

9. The method according to claim 1, characterized in that The image information of the target area includes edge information and volume of the target area; Generating a label representing a change trend of the target region in the early sampling image based on a comparison result between the image information of the target region includes: determining a volume corresponding to a target area in the early sampling image based on edge information corresponding to the early sampling image; determining a volume corresponding to a target area in the late sampling image based on edge information corresponding to the late sampling image; If the volume corresponding to the target area in the early sampling image is smaller than the volume corresponding to the target area in the late sampling image, a label indicating that the target area in the early sampling image becomes larger is generated.

10. The method according to claim 1, characterized in that The method further comprises: Predicting a first volume corresponding to a target region in an early sampling image based on the target region; determining, based on a target region in an image in the early image set, a second volume corresponding to the target region; calculating a difference between the first volume and the second volume as a sampling error; Calculating a mean value of the sampling errors based on sampling errors corresponding to a plurality of early image sets of the same slice thickness; Based on the mean of the sampling errors, the sampling ratio corresponding to the layer thickness is updated.

11. The method according to claim 10, characterized in that Based on the mean of the sampling errors, updating the sampling ratio corresponding to the layer thickness includes: If the mean of the sampling errors is greater than a set threshold, the sampling ratio corresponding to the layer thickness is reduced.

12. The method according to claim 1, characterized in that The method further comprises: Displaying the images in the medical imaging collection in an interface; Obtaining at least two sampling ratios selected by a user, and sampling images in the medical image set based on the at least two sampling ratios to obtain sampled images corresponding to the at least two sampling ratios; Displaying the sampling images corresponding to the at least two sampling ratios respectively in the interface; Acquiring user selection information, where the selection information is information corresponding to a target sampling ratio selected by the user based on the sampling image displayed in the interface; The target sampling ratio in the selection information and the sampling image corresponding to the target sampling ratio are displayed in the interface respectively.

13. A device for generating training samples, characterized in that: include: A first acquisition unit is configured to acquire a medical image set to be labeled, wherein the medical image set includes an early image set taken for a target area at a first time, and a late image set taken for the target area at a second time; a first sampling unit configured to sample the images in the early image set based on a first sampling ratio determined according to the layer thickness of the images in the early image set to obtain an early sampled image; a second sampling unit configured to sample the images in the late image set based on a second sampling ratio determined according to the layer thickness of the images in the late image set to obtain a late sampled image; A second acquiring unit, configured to acquire image information of a target area corresponding to the early sampling image and the late sampling image respectively; The label generating unit is configured to generate a label representing a change trend of the target region in the early sampling image based on a comparison result between the image information of the target region.

14. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating training samples according to any one of claims 1 to 12 is implemented.

15. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method for generating training samples according to any one of claims 1 to 12.

Images