Image detection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an image detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: taking at least two adjacent graph nodes and a graph connection relationship in a graph structure of an initial connection relationship graph corresponding to a to-be-detected image as new graph nodes, and obtaining a target connection relationship graph; and detecting the to-be-detected image according to the target connection relation graph to obtain a detection result of the to-be-detected image. The method can improve the accuracy of an image detection result.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an image detection method, apparatus, computer equipment, storage medium and program product. Background Art

[0002] With the continuous development of image detection technology, its application in the medical field is becoming more and more extensive. The detection results of medical images can assist in disease diagnosis and treatment planning.

[0003] In the related art, the medical image to be detected is mainly detected based on the initial connection relationship graph corresponding to the medical image to be detected, so as to obtain the detection result of the medical image to be detected.

[0004] However, the related art methods have the problem of inaccurate image detection results. Summary of the invention

[0005] Based on this, it is necessary to provide an image detection method, device, computer equipment, storage medium and program product that can improve the accuracy of image detection results in response to the above technical problems.

[0006] In a first aspect, the present application provides an image detection method, comprising:

[0007] At least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected are used as new graph nodes to obtain a target connection relationship graph;

[0008] The image to be detected is detected according to the target connection relationship diagram to obtain the detection result of the image to be detected.

[0009] In one embodiment, the image to be detected is detected according to the target connection relationship graph to obtain a detection result of the image to be detected, including:

[0010] Based on the target connection relationship graph, determine the constraints of the image detection process;

[0011] The detection process of the image to be detected is constrained by using the constraint conditions to obtain the detection result of the image to be detected.

[0012] In one embodiment, based on the target connection relationship graph, determining the constraint conditions of the image detection process includes:

[0013] Obtain the topological structure matrix corresponding to the target connection relationship graph;

[0014] Based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix, the constraint conditions of the image detection process are generated.

[0015] In one embodiment, obtaining a topological structure matrix corresponding to a target connection relationship graph includes:

[0016] Obtaining graph node information and graph connection relationship information of a target connection relationship graph; the graph node information represents information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents information of connection relationships between all graph nodes in the target connection relationship graph;

[0017] Based on the graph node information and graph connection relationship information, a topological structure matrix corresponding to the target connection relationship graph is generated.

[0018] In one embodiment, at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected are used as new graph nodes to obtain a target connection relationship graph, including:

[0019] Acquire multiple topological units in the initial connection relationship graph; each topological unit includes at least two adjacent graph nodes and at least node edges between the two adjacent graph nodes;

[0020] Each topological unit is used as a new graph node, and the connection between adjacent new graph nodes is used as a new connection relationship to generate a target connection relationship graph.

[0021] In one embodiment, the method further comprises:

[0022] Obtain an image detection model, and use the image detection model to perform the above-mentioned image detection method;

[0023] Among them, the construction process of the image detection model includes:

[0024] Obtaining a sample medical image and a corresponding sample initial connection relationship graph;

[0025] Inputting the sample medical image and the sample initial connection relationship graph into the initial image detection model to obtain the target sample connection relationship graph;

[0026] The sample medical image is detected according to the target sample connection relationship diagram until the detection result meets the preset conditions to obtain the image detection model.

[0027] In a second aspect, the present application also provides an image detection device, comprising:

[0028] An acquisition module, used to use at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes to acquire a target connection relationship graph;

[0029] The detection module is used to detect the image to be detected according to the target connection relationship diagram to obtain the detection result of the image to be detected.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the content of any one embodiment of the image detection method in the first aspect above is implemented.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the content of any one embodiment of the image detection method in the first aspect above.

[0032] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the content of any one embodiment of the image detection method in the first aspect above.

[0033] The above-mentioned image detection method, apparatus, computer equipment, storage medium and program product use at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes to obtain a target connection relationship graph; and detect the image to be detected according to the target connection relationship graph to obtain a detection result of the image to be detected. This method redefines the initial connection relationship graph of the image to be detected, and uses two adjacent graph nodes and graph connection relationships as new graph nodes. The generated target connection relationship graph can accurately estimate the detection target in the image to be detected. In this way, the computer device can accurately detect the image to be detected using the target connection relationship graph, and can accurately obtain the detection result of the image to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] FIG1 is a diagram showing an application environment of an image detection method according to an embodiment;

[0036] FIG2 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0037] FIG3 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0038] FIG4 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0039] FIG5 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0040] FIG6 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0041] FIG7 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0042] FIG8 is a schematic diagram of a loss function during key point detection model training in one embodiment;

[0043] FIG9 is a schematic diagram of a key point detection method in one embodiment;

[0044] FIG10 is a schematic diagram of an image to be detected in one embodiment;

[0045] FIG11 is a schematic diagram of the initial connection relationship of the lower limbs in one embodiment;

[0046] FIG12 is a schematic diagram of a target connection relationship subject to constraint conditions in one embodiment;

[0047] FIG13 is a schematic diagram of a flow chart of an image detection method in one embodiment;

[0048] FIG. 14 is a structural block diagram of an image detection device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Before introducing the technical solution of the present application in detail, a brief introduction to the background technology of the present application is first given.

[0051] Medical image analysis is an interdisciplinary field that combines digital image processing with artificial intelligence technology. It provides an important auxiliary reference for the clinical treatment of various diseases by performing automated or semi-automated analysis of medical image data. This technology has greatly improved the efficiency of clinical diagnosis and reduced the risk of misdiagnosis to a certain extent. In the process of medical image analysis, tasks such as image segmentation, image detection, and image key point detection are crucial. These tasks require very high accuracy in image recognition, so various improved methods are constantly proposed to meet this demand.

[0052] As an emerging technology, Graph Neural Networks (GNNs) have shown great potential in the field of medical image analysis. GNNs can effectively process the graph structure data corresponding to medical images, and enhance the interaction between image pixels or features by transmitting information between nodes in the network, thereby improving the performance of the entire network. This interaction mechanism makes GNNs particularly suitable for tasks that require understanding complex relationships within images.

[0053] In the design of graph neural networks, one of the most critical links is to define the connection relationship between each node in the graph. The quality of this connection relationship directly determines the ability of network modeling. A good connection model not only needs to reflect the actual relationship between nodes, but also needs to take into account computational efficiency and scalability.

[0054] In the related art, the graph structure corresponding to the medical image is mainly designed based on empirical knowledge, and then the medical image is detected based on the graph structure to obtain the detection result of the medical image. However, the method of the related art has the problem of inaccurate image detection results.

[0055] Therefore, the present application provides an image detection method, device, computer equipment, storage medium and program product, which can improve the accuracy of image detection results. Of course, the technical solution provided in the embodiments of the present application is not limited to solving the above-mentioned problems, and there are other technical effects, which can be specifically described in the following embodiments. Next, the technical solution of the present application is described in detail.

[0056] The image detection method provided in the embodiment of the present application can be applied in the application environment as shown in FIG. 1 .

[0057] The computer device in the application environment can be a server, a personal computer, a laptop computer, a smart phone, a tablet computer, a smart mobile phone, etc. The computer device may include a processor, a memory and a network interface connected via a system bus or wirelessly. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in the image detection process. The network interface of the computer device is used to communicate with an external terminal through a network connection, and the computer program is executed by the processor to implement an image detection method.

[0058] In an exemplary embodiment, as shown in FIG2 , an image detection method is provided, which is described by taking the method applied to the computer device in FIG1 as an example, and includes the following steps 201 to 203. Among them:

[0059] S201, taking at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes, and obtaining a target connection relationship graph.

[0060] Among them, the image to be detected can be a whole-body scan image of the subject, or a scan image of a certain body part of the subject. For example, the certain body part of the subject can be a body part such as the head, abdomen, or lower limbs. In addition, the embodiment of the present application does not limit the modality of the image to be detected. For example, the image to be detected can be a computed tomography (CT) image, a magnetic resonance (MRI) image, a positron emission tomography (PET) image, and the like.

[0061] The initial connection relationship graph is determined based on the detection task of the image to be detected. If the detection task is an image segmentation task, the image segmentation pixels can be used as graph nodes, the connection relationship between pixels can be used as graph edges, and the image composed of graph nodes and graph edges can be used as the initial connection relationship graph. If the detection task is an image detection task, the detection target can be used as a graph node, the connection relationship between different targets can be used as graph edges, and the image composed of graph nodes and graph edges can be used as the initial connection relationship graph. If the detection task is an image key point detection task, each key point in the image can be used as a graph node, the connection relationship between different key points can be used as graph edges, and the image composed of graph nodes and graph edges can be used as the initial connection relationship graph.

[0062] In an embodiment of the present application, if the image to be detected has been collected and is stored in an image database. Then, the computer device can search for the image to be detected from the image database. Alternatively, if the image to be detected has not been collected, the computer device can send a collection instruction to the medical scanning device according to the collection requirements. After receiving the collection instruction, the medical scanning device can scan the subject and send the collected image of the subject to the computer device, and the computer device can use the image of the subject as the image to be detected.

[0063] After acquiring the image to be detected, the computer device can select an image that matches the image to be detected from the historical images, and divide the matching historical images into nodes and edge relationships according to the detection task to obtain the initial connection relationship diagram of the image to be detected. Alternatively, different scanning parts correspond to reference connection relationship diagrams, which are designed by the doctor based on the reference body shape. Therefore, the computer device can select a reference connection relationship diagram that matches the image to be detected from multiple reference connection relationship diagrams according to the scanning part of the image to be detected, and use the matching reference connection relationship diagram as the initial connection relationship diagram of the image to be detected. The embodiment of the present application does not limit the method for obtaining the image to be detected and the initial connection relationship diagram corresponding to the image to be detected.

[0064] The initial connection relationship diagram of the above-mentioned image to be detected can be represented by G, G=(C K ,C L ,C W ), where C K represents K graph nodes; C L represents L graph edges; C W It represents the weight of the graph edge, that is, the closeness of the relationship between two graph nodes. The closer the relationship, the greater the weight value; the looser the relationship, the smaller the weight value.

[0065] Furthermore, since the graph connection relationship in the initial connection relationship graph is either 0 or 1, the pulse function cannot be derived, and therefore the graph nodes and graph connection relationships cannot be estimated based on the initial connection relationship graph. Therefore, after obtaining the image to be detected and the corresponding initial connection relationship graph, the computer device can simulate the initial connection relationship graph corresponding to the image to be detected in a simulation manner, and on the basis of the simulated relationship graph, at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph are used as new graph nodes to determine the target connection relationship graph, and then the target connection relationship graph is used to detect the image to be detected.

[0066] Alternatively, the computer device can also use the image detection model to process the graph structure in the initial connection relationship to obtain the target connection relationship graph, and then detect the image to be detected based on the target connection relationship graph. Specifically, the computer device can use the image to be detected and the corresponding initial connection relationship graph as input signals and input them into the image detection model. The image detection model can abstract the graph nodes and graph edges in the initial connection relationship graph G, that is, take the connection relationship between two adjacent graph nodes and the two adjacent graph nodes as the minimum structural unit, preliminarily define the minimum structural unit as a new graph node, and obtain the connection relationship between the new graph nodes, and use the image composed of the connection relationship between the new graph nodes and the graph nodes as the target connection relationship graph. The image detection model can learn the connection relationship between the detection targets in the initial connection relationship graph, and it can be derived everywhere, so that the accuracy of image detection is higher.

[0067] The above target connection relationship diagram can be obtained by , =( , , ),in, Represents K new graph nodes; Represents L new graph edges; Represents the weight of the new graph edge. The target connection graph includes A new link.

[0068] Exemplarily, assuming that the initial connection relationship graph includes 10 graph nodes, any two graph nodes and the connection relationship between the two graph nodes can be divided into a group, and each group can be used as a new graph node to obtain a target connection relationship graph. Alternatively, the first connection relationship graph corresponding to the new graph node based on two adjacent graph nodes and the graph connection relationship, the second connection relationship graph corresponding to the new graph node based on three adjacent graph nodes and the graph connection relationship, and the N-1th connection relationship graph corresponding to the new graph node based on N adjacent graph nodes and the graph connection relationship can be fused to obtain the target connection relationship graph. The target connection relationship graph obtained by fusion has more nodes, which can improve the accuracy of the image detection process.

[0069] S202, detecting the image to be detected according to the target connection relationship graph to obtain a detection result of the image to be detected.

[0070] In an embodiment of the present application, after obtaining the target connection relationship graph, the computer device can use the image detection model to extract the feature information of the image to be detected, and estimate the detection target in the image to be detected based on the target connection relationship graph and the feature information to obtain the detection result of the image to be detected. For example, when the detection task is an image segmentation task, the detection result of the image to be detected can be the lesion area in the image; when the detection task is an image key point detection task, the detection result of the image to be detected can be the key point information in the image. Among them, the image detection algorithm can be selected based on the detection task.

[0071] In the above-mentioned image detection method, at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected are used as new graph nodes to obtain a target connection relationship graph; and the image to be detected is detected according to the target connection relationship graph to obtain a detection result of the image to be detected. This method redefines the initial connection relationship graph of the image to be detected, and uses two adjacent graph nodes and graph connection relationships as new graph nodes. The generated target connection relationship graph can accurately estimate the detection target in the image to be detected. In this way, the computer device can accurately detect the image to be detected using the target connection relationship graph, and can accurately obtain the detection result of the image to be detected.

[0072] The target connection relationship graph can accurately estimate the detection target in the image to be detected, so that the detection result is more accurate. Then, in one embodiment, as shown in FIG3 , the specific content of the detection result of the image to be detected obtained by detecting the image to be detected according to the target connection relationship graph is introduced, and the specific content includes the following steps:

[0073] S301, determining the constraint conditions of the image detection process based on the target connection relationship graph.

[0074] In the embodiment of the present application, since the graph connectivity corresponding to the target connection relationship graph has an upper bound and a lower bound, there will be a constraint process in the detection process based on the target connection relationship graph. Can be defined as:

[0075]

[0076] Among them, S represents A part of the child nodes in; Indicates that the child node belongs to all nodes; represents the number of nodes in S; Indicates that the number of nodes in S is less than or equal to half of the total number of nodes, that is, S is a small subgraph; i, j represent any two nodes in graph S.

[0077] The upper and lower bounds of graph connectivity can be expressed as:

[0078]

[0079] in, Laplacian matrix representing the target connection graph The second smallest eigenvalue after eigendecomposition; The degree representing the maximum weight of a vertex in the target connection graph.

[0080] During the optimization process, the goal is to find the minimum graph connectivity , It can be obtained by eigenvectors and Laplacian matrices To obtain.

[0081] Specifically, the computer device can analyze the target connection relationship graph using the constraint condition calculation formula, determine the upper bound and the lower bound of the graph connection relationship in the target connection relationship graph, and determine the constraint condition corresponding to the target connection relationship graph.

[0082] S302, constraining the detection process of the image to be detected by using the constraint condition to obtain the detection result of the image to be detected.

[0083] In the embodiment of the present application, the computer device can extract features of the image to be detected through the image detection model to obtain feature information of the image to be detected. Then, the feature information of the image to be detected, the graph structure information in the target connection relationship graph, and the constraint conditions are multiplied, and the detection result of the image to be detected is obtained through multiple nonlinear convolution operations. The detection result E l It can be expressed as:

[0084]

[0085] in, Represents a graph convolution operation; Represents the feature information of the image to be detected; Indicates constraints; Represents the graph structure information in the target connection relationship graph.

[0086] In the above-mentioned image detection method, based on the target connection relationship graph, the constraint conditions of the image detection process are determined; the detection process of the image to be detected is constrained by the constraint conditions to obtain the detection result of the image to be detected. This method can accurately obtain the constraint conditions of the image detection process by analyzing the target connection graph, so that the detection process can be constrained based on the constraint conditions, and more accurate detection results can be obtained.

[0087] By constraining the detection process through constraint conditions, the accuracy of the detection results can be improved. Next, the specific content of the constraint conditions for determining the image detection process based on the target connection relationship graph is introduced through an embodiment, as shown in Figure 4, the specific content includes:

[0088] S401, obtaining a topological structure matrix corresponding to a target connection relationship graph.

[0089] The topological structure matrix refers to the matrix composed of the topological structure of the graph nodes and graph edges in the target connection relationship graph. The topological structure matrix can also be called the Laplace matrix. The topological matrix can be obtained by express.

[0090] In an embodiment of the present application, the computer device can use the Laplace algorithm to analyze the target connection relationship graph, obtain the Laplace matrix corresponding to the target connection relationship graph, and use the Laplace matrix as the topological structure matrix. Alternatively, the computer device can also input the target connection relationship graph into a preset topological structure analysis model, analyze the target connection relationship graph through the topological structure analysis model, and obtain the topological structure matrix corresponding to the target connection relationship graph. The embodiment of the present application does not limit the method for obtaining the topological structure matrix corresponding to the target connection relationship graph.

[0091] S402, based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix, generating the constraint conditions of the image detection process.

[0092] In the embodiment of the present application, the mapping relationship between the topological structure matrix and the constraint condition can be expressed by the Federer canonical constraint relationship, which can be expressed as:

[0093]

[0094] Where F represents the Federer regularization constraint; A feature vector representing the target connection graph; Represents the topological structure matrix corresponding to the target connection relationship graph.

[0095] In the above-mentioned image detection method, the topological structure matrix corresponding to the target connection relationship diagram is obtained; based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix, the constraint conditions of the image detection process are generated. This method can accurately map the topological matrix by obtaining the topological matrix corresponding to the target connection relationship diagram, and then accurately obtain the constraint conditions based on the mapping relationship between the topological structure matrix and the constraint conditions. Then, the detection process can be constrained based on the constraint conditions, and the obtained detection results will be more accurate.

[0096] In one embodiment, as shown in FIG5 , the specific contents of the topological structure matrix corresponding to the target connection relationship graph obtained above include:

[0097] S501, obtaining graph node information and graph connection relationship information of a target connection relationship graph; the graph node information represents information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents information of connection relationships between all graph nodes in the target connection relationship graph.

[0098] The graph node information can be called the degree matrix, which is a diagonal matrix D. The elements on the diagonal are the degrees of the corresponding vertices, that is, the number of edges connected to the vertex. Assuming that the target connection graph is represented by a weighted method, the degree is the sum of the weights of all the edges.

[0099] The graph connection relationship information can be called the adjacency matrix. The adjacency matrix is ​​a square matrix A, where A(i, j) represents the weight of the graph edge in the target connection relationship graph. If vertices i and j are not adjacent, then A(i, j) = 0.

[0100] In an embodiment of the present application, the computer device can analyze the information of all graph nodes in the target connection relationship graph based on the degree matrix solution algorithm to obtain the graph node information. In addition, the computer device can also analyze the connection relationship between all graph nodes in the target connection relationship graph based on the adjacency matrix solution algorithm to obtain the graph connection relationship information.

[0101] S502: Generate a topological structure matrix corresponding to the target connection relationship graph based on the graph node information and the graph connection relationship information.

[0102] In the embodiment of the present application, the Laplacian matrix is ​​the difference between the degree matrix and the adjacency matrix. The Laplacian matrix can be expressed as: ,in, Represents the degree matrix of the target connection graph (i.e., graph node information); Represents the adjacency matrix of the target connection graph (i.e., graph connection information).

[0103] Therefore, after obtaining the graph node information and graph connection relationship information corresponding to the target connection relationship graph, the computer device can calculate the difference between the graph node information and the graph connection relationship information, which is presented in the form of a matrix, and use the difference matrix as the topological structure matrix corresponding to the target connection relationship graph.

[0104] In the above-mentioned image detection method, the graph node information and graph connection relationship information of the target connection relationship graph are obtained; the graph node information represents the information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents the information of the connection relationship between all graph nodes in the target connection relationship graph; based on the graph node information and the graph connection relationship information, a topological structure matrix corresponding to the target connection relationship graph is generated. This method can accurately obtain the information of all graph nodes and the information of the connection relationship between all graph nodes by analyzing the target connection relationship graph, and then based on the graph node information and the graph connection relationship information, a topological structure matrix can be accurately generated to facilitate the determination of subsequent constraint conditions.

[0105] Next, a specific process of obtaining a target connection relationship graph by using at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes is introduced through an embodiment. As shown in FIG6 , the specific process includes:

[0106] S601, obtaining multiple topological units in an initial connection relationship graph; each topological unit includes at least two adjacent graph nodes and at least node edges between the two adjacent graph nodes.

[0107] In an embodiment of the present application, the computer device can analyze the initial connection relationship graph, determine multiple graph nodes in the initial connection relationship graph, and then analyze the edges between at least two adjacent graph nodes, and use the at least two adjacent graph nodes and the edges between at least two adjacent graph nodes as a topological unit. Multiple topological units are obtained through the above analysis method.

[0108] S602: Take each topological unit as a new graph node, and the connection between adjacent new graph nodes as a new connection relationship to generate a target connection relationship graph.

[0109] In the embodiment of the present application, after obtaining multiple topological units, the computer device can take each topological unit as a new graph node, and then determine the connection between adjacent new graph nodes to obtain a new connection relationship. And the relationship graph composed of all new graph nodes and new connection relationships is used as the target connection relationship graph.

[0110] In the above-mentioned image detection method, multiple topological units in the initial connection relationship graph are obtained; each topological unit includes at least two adjacent graph nodes and at least two node edges between adjacent graph nodes; each topological unit is used as a new graph node, and the connection between adjacent new graph nodes is used as a new connection relationship to generate a target connection relationship graph. In the process of redefinition, this method can accurately obtain the connection of the new graph nodes by obtaining multiple topological units in the initial connection relationship graph and then using the multiple topological units as new graph nodes, so that the target connection relationship graph can be accurately generated based on the connection between the new nodes and the graph nodes.

[0111] The above embodiments all provide a detailed introduction to the image detection process, which can be implemented by an image detection model, that is, obtaining an image detection model and using the image detection model to perform the above image detection method. Next, the training process of the image detection model is introduced. As shown in FIG7 , the construction process of the image detection model includes:

[0112] S701, obtaining a sample medical image and a corresponding sample initial connection relationship graph.

[0113] In the embodiment of the present application, the computer device can obtain a sample medical image with the same identification information as the target part from the PACS according to the identification information of the target part, and obtain a detection result corresponding to the sample medical image. In addition, the computer device can search for a sample initial connection relationship diagram corresponding to the identification information from a connection relationship database based on the identification information of the target part.

[0114] S702, inputting the sample medical image and the sample initial connection relationship graph into the initial image detection model to obtain the target sample connection relationship graph.

[0115] In an embodiment of the present application, a computer device can simultaneously input a sample medical image and a sample initial connection relationship graph into an initial image detection model. The initial image detection model can redefine the graph nodes in the sample initial connection relationship graph, i.e., use adjacent graph nodes and graph edges between adjacent graph nodes as new nodes to generate a target sample connection relationship graph.

[0116] S703, detecting the sample medical image according to the target sample connection relationship diagram until the detection result meets the preset conditions, thereby obtaining an image detection model.

[0117] In an embodiment of the present application, after obtaining the target sample connection relationship diagram, the computer device can assist the detection process of the sample medical image based on the target sample connection relationship diagram to obtain a predicted detection result. Then subtract the predicted detection result from the actual detection result to obtain the difference in the predicted process. And continue to compare the difference in the predicted process with the preset difference. If the difference in the predicted process is greater than the preset difference, the model parameters are adjusted. If the difference in the predicted process is less than the preset difference, it means that the training process is completed and a trained image detection model is obtained.

[0118] It should be noted that, taking the image detection process as the key point detection process and the image detection model as the key point detection model as an example, in the training process of the key point detection model, after extracting the feature vector of the key point and the connection relationship between the key points through the neural network model, the pixel probability map of the key point and the connection relationship map of the key point can be obtained. Among them, the pixel probability map of the key point can be defined as , the connection relationship graph of key points can be defined as .

[0119]

[0120] in, and represents the size of the sample medical image; K represents the number of key points; A collection of pixel probability maps representing key points; A collection of connection graphs representing key points; Pixel probability map representing key points; Represents the feature vector obtained by feature extraction of the key point detection model; represents the weight obtained by model training; D Ld represents the key point decoder; H M A connection diagram showing key points; D Li represents the keypoint decoder.

[0121] The loss function is constructed by the pixel probability map of the key points, the connection relationship map of the key points and the constraint relationship. The loss function L can be expressed as:

[0122]

[0123] in, express The loss function of Indicates H M The loss function of Represents the loss function of the constraint condition. In the above training process, the initial learning rate can be set to 0.001, the number of training times can be set to 200, and the number of samples for one training is 3; Refers to hyperparameters, which represent the coefficients of the loss function of the pixel probability map, the loss function of the connection relationship map, and the loss function of the constraint relationship. Its main purpose is to reduce the differences between the three loss functions, balance the proportions among the three loss functions, and make each loss function make a certain contribution to the loss function L. The three loss functions constrain each other, and each loss function has a certain contribution to the loss function L, which makes the loss function L more accurate and can improve the accuracy of model training.

[0124] It should be emphasized that It can be set manually, or the optimal value of each coefficient can be automatically calculated by dynamic grid search or random search. Regardless of how the coefficients of the loss function are defined, the three loss functions need to be converted to similar orders of magnitude, or even to the same order of magnitude.

[0125] Among them, different training samples correspond to different coefficients of the loss function. For example, for the first training sample, It can be 1, 1, 0.01. For the second training sample, It can be 1,2,0.01.

[0126] FIG8 is a schematic diagram of the loss function during the training process of the key point detection model. In the figure, after the sample medical image and the sample connection relationship diagram are input into the key point detection model, the key point decoder determines the pixel probability map of the key point by analyzing the sample medical image, and determines the loss function of the key point based on the pixel probability map of the key point. At the same time, the key point decoder is redefined through the sample connection relationship graph to determine the Laplacian matrix of the key point , and based on the Laplace matrix Identify constraints And, the redefined sample connection relationship graph is analyzed by the key point connection relationship decoder to obtain the key point connection relationship graph H M , and based on the connection relationship diagram H of the key points M Determine the loss function of the connection relationship graph of key points .

[0127] Figure 9 is a flow chart of the key point detection method, which includes: S801, analyzing the medical image to be detected through the key point detection model to obtain the pixel probability map of the key points, and redefining the initial connection relationship map to obtain the target connection relationship map of the key points; S802, performing feature extraction on the medical image to be detected to obtain the feature information description of the image to be detected; S803, using constraint conditions to sparse the target connection relationship map; S804, using the sparse target connection relationship map to perform key point detection on the image to be detected to obtain the coordinate information of the key points.

[0128] FIG10 is a schematic diagram of an image to be detected, from which it can be seen that the part of the image to be detected is the lower limbs. FIG11 is a schematic diagram of the initial connection relationship of the lower limbs, and the three initial connection relationships in the figure are all connection relationships of the lower limb scenic spots, namely E32, E40 and E48. FIG12 is a schematic diagram of the target connection relationship after the constraint condition, in which the connection relationship between each key point can be observed from a three-dimensional perspective.

[0129] In the above-mentioned image detection method, a sample medical image and a corresponding sample initial connection relationship graph are obtained; the sample medical image and the sample initial connection relationship graph are input into the initial image detection model to obtain a target sample connection relationship graph; the sample medical image is detected according to the target sample connection relationship graph until the detection result meets the preset conditions, thereby obtaining an image detection model. In the process of training the image detection model, this method simultaneously trains the sample medical image and the sample initial connection relationship graph, that is, in the training process, the initial graph connection relationship graph is redefined, and image detection can be accurately completed based on the target graph connection relationship. In this way, through multiple iterative training processes, end-to-end training can be achieved directly, which can increase the network's interactive information between pixels and feature vectors and improve the performance of the image detection model.

[0130] As a specific embodiment of the present application, the image detection process is described in detail below. In one embodiment, as shown in FIG13 , the image detection method includes:

[0131] S901, obtaining an image to be detected and an initial connection relationship graph corresponding to the image to be detected;

[0132] S902, obtaining a plurality of topological units in the initial connection relationship graph; each topological unit includes at least two adjacent graph nodes and at least a node edge between the two adjacent graph nodes;

[0133] S903, taking each topological unit as a new graph node and the connection between adjacent new graph nodes as a new connection relationship, to generate a target connection relationship graph;

[0134] S904, obtaining graph node information and graph connection relationship information of the target connection relationship graph; the graph node information represents information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents information of connection relationships between all graph nodes in the target connection relationship graph;

[0135] S905, generating a topological structure matrix corresponding to the target connection relationship graph based on the graph node information and the graph connection relationship information;

[0136] S906, generating constraint conditions for the image detection process based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix;

[0137] S907, using the constraint condition to constrain the detection process of the image to be detected, and obtaining the detection result of the image to be detected.

[0138] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides an image detection device for implementing the above-mentioned image detection method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more image detection device embodiments provided below can refer to the limitations on the image detection method above, and will not be repeated here.

[0140] In an exemplary embodiment, as shown in FIG14 , an image detection device is provided, including: an acquisition module 11 and a detection module 12, wherein:

[0141] An acquisition module 11 is used to use at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes to acquire a target connection relationship graph;

[0142] The detection module 12 is used to detect the image to be detected according to the target connection relationship diagram to obtain a detection result of the image to be detected.

[0143] In one embodiment, the detection module includes: a condition determination unit and a constraint unit, wherein:

[0144] A condition determination unit, used for determining the constraint conditions of the image detection process based on the target connection relationship graph;

[0145] The constraint unit is used to constrain the detection process of the image to be detected by using the constraint conditions to obtain the detection result of the image to be detected.

[0146] In one embodiment, the condition determination unit is further used to obtain a topological structure matrix corresponding to the target connection relationship diagram; based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix, the constraint conditions of the image detection process are generated.

[0147] In one embodiment, the above-mentioned condition determination unit is also used to obtain graph node information and graph connection relationship information of the target connection relationship graph; the graph node information represents information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents information of connection relationships between all graph nodes in the target connection relationship graph; based on the graph node information and the graph connection relationship information, a topological structure matrix corresponding to the target connection relationship graph is generated.

[0148] In one embodiment, the input module includes: an acquisition unit and a generation unit, wherein:

[0149] An acquisition unit, used to acquire multiple topological units in the initial connection relationship graph; each topological unit includes at least two adjacent graph nodes and at least a node edge between the two adjacent graph nodes;

[0150] The generation unit is used to use each topological unit as a new graph node and the connection between adjacent new graph nodes as a new connection relationship to generate a target connection relationship graph.

[0151] In one embodiment, the above-mentioned image detection device further includes: a model acquisition module, a sample acquisition module, a sample input module and a training module, wherein:

[0152] A model acquisition module is used to acquire an image detection model and execute an image detection method using the image detection model;

[0153] A sample acquisition module, used to acquire a sample medical image and a corresponding sample initial connection relationship graph;

[0154] A sample input module, used to input the sample medical image and the sample initial connection relationship diagram into the initial image detection model to obtain the target sample connection relationship diagram;

[0155] The training module is used to detect sample medical images according to the target sample connection relationship diagram until the detection result meets the preset conditions to obtain the image detection model.

[0156] Each module in the above-mentioned image detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0157] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the content of any one embodiment of the above-mentioned image detection method is implemented.

[0158] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the content of any embodiment of the above-mentioned image detection method is implemented.

[0159] In one embodiment, a computer program product is provided, including a computer program, which implements the content of any one embodiment of the above-mentioned image detection method when executed by a processor.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0163] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image detection method, characterized in that: The method comprises: At least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected are used as new graph nodes to obtain a target connection relationship graph; And the image to be detected is detected according to the target connection relationship diagram to obtain a detection result of the image to be detected.

2. The method according to claim 1, characterized in that The detecting the image to be detected according to the target connection relationship graph to obtain the detection result of the image to be detected includes: Determining the constraint conditions of the image detection process based on the target connection relationship graph; The constraint condition is used to constrain the detection process of the image to be detected, so as to obtain the detection result of the image to be detected.

3. The method according to claim 2, characterized in that Determining the constraint conditions of the image detection process based on the target connection relationship graph includes: Obtaining a topological structure matrix corresponding to the target connection relationship graph; Based on the mapping relationship between the topological structure matrix and the constraint conditions and the topological structure matrix, the constraint conditions of the image detection process are generated.

4. The method according to claim 3, characterized in that The obtaining of a topological structure matrix corresponding to the target connection relationship graph includes: Obtaining graph node information and graph connection relationship information of the target connection relationship graph; the graph node information represents information of all graph nodes in the target connection relationship graph; the graph connection relationship information represents information of connection relationships between all graph nodes in the target connection relationship graph; Based on the graph node information and the graph connection relationship information, a topological structure matrix corresponding to the target connection relationship graph is generated.

5. The method according to any one of claims 1 to 4, characterized in that: The step of taking at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes to obtain a target connection relationship graph includes: Acquire multiple topological units in the initial connection relationship graph; each topological unit includes at least two adjacent graph nodes and a node edge between the at least two adjacent graph nodes; Each of the topological units is used as a new graph node, and the connection between adjacent new graph nodes is used as a new connection relationship to generate the target connection relationship graph.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining an image detection model, and using the image detection model to perform the above-mentioned image detection method; The construction process of the image detection model includes: Obtaining a sample medical image and a corresponding sample initial connection relationship graph; Inputting the sample medical image and the sample initial connection relationship graph into an initial image detection model to obtain a target sample connection relationship graph; The sample medical image is detected according to the target sample connection relationship diagram until the detection result meets the preset conditions, thereby obtaining the image detection model.

7. An image detection device, characterized in that: The device comprises: An acquisition module, used to use at least two adjacent graph nodes and graph connection relationships in the graph structure of the initial connection relationship graph corresponding to the image to be detected as new graph nodes to acquire a target connection relationship graph; The detection module is used to detect the image to be detected according to the target connection relationship diagram to obtain a detection result of the image to be detected.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images