Image processing method and device

Abstract

The invention discloses an image processing method and device, and relates to the technical field of medical image processing. One specific embodiment of the method comprises the following steps: extracting medical image data transmitted by a preset target image source, identifying a target region of interest and / or a target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target region of interest and / or the target object of interest; extracting display data transmitted by preset target display equipment, positioning a display position of the target attention area and / or the target attention object, and generating a target display position of the medical auxiliary information; determining a display mode according to the target display position and the medical auxiliary information; and transmitting the medical auxiliary information to a target display device so as to present the medical auxiliary information at the target display position in the display mode. According to the embodiment, at least high-utilization-rate, accurate and efficient image diagnosis of AI auxiliary diagnosis can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an image processing method and device. Background Art

[0002] At present, AI-assisted diagnosis has become an indispensable diagnostic method in the doctor's diagnosis and treatment process, and can provide doctors with more medical information and effectively improve the efficiency of doctors' diagnosis and treatment.

[0003] However, existing diseases are highly complex and require the combination of multiple examination information. Doctors have to scan and view medical images while diagnosing patients and retrieve information manually, resulting in low utilization rate of AI-assisted diagnosis, increasing the operating burden of doctors and affecting the accuracy and efficiency of diagnosis. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an image processing method and device, which can at least achieve high utilization of AI-assisted diagnosis, and accurate and efficient image diagnosis.

[0005] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: extracting medical image data transmitted by a preset target image source, identifying a target area of ​​interest and / or a target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest; extracting display data transmitted by a preset target display device, locating the display position of the target area of ​​interest and / or the target object of interest, and generating a target display position of the medical auxiliary information; determining a display mode according to the target display position and the medical auxiliary information; transmitting the medical auxiliary information to the target display device, so as to present the medical auxiliary information at the target display position through the display mode.

[0006] Optionally, identifying the target region of interest and / or the target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target region of interest and / or the target object of interest, includes:

[0007] In response to an image processing event for the medical image data, a target area of ​​interest and / or a target object of interest corresponding to the image processing event is located; the category of the target area of ​​interest and / or the target object of interest is identified, and a processing model is obtained by matching the category with the image processing event; and the processing model is called to calculate medical auxiliary information based on the medical image data associated with the target area of ​​interest and / or the target object of interest.

[0008] Optionally, locating the display position of the target area of ​​interest and / or the target object of interest and generating a target display position of the medical auxiliary information includes:

[0009] Acquire characteristic attribute information of the target area of ​​interest and / or the target object of interest, and determine the displayed boundary information; acquire position information of the key area and / or key object of the current page, determine the relationship between the boundary information and the position information, and generate the target display position of the medical auxiliary information according to the relationship.

[0010] Optionally, determining the relationship between the boundary information and the position information, and generating a target display position of the medical auxiliary information according to the relationship, includes:

[0011] In response to selecting a relationship determination dimension in a set, the relationship between the boundary information and the position information is calculated according to the relationship determination dimension; an evaluation model corresponding to the relationship determination dimension is called to process the relationship to obtain an evaluation grade; and a target display position of the medical auxiliary information is determined according to the evaluation grade.

[0012] Optionally, after determining the target display position of the medical auxiliary information, the method further comprises:

[0013] In response to the target display position belonging to a preset specific position area, a relationship determination dimension is reselected in the set.

[0014] Optionally, determining a display mode according to the target display position and the medical auxiliary information includes:

[0015] Obtain the category of the medical auxiliary information and match it to obtain a display mode; determine whether the display mode meets the display conditions of the target display position, and if so, use the display mode as the final display mode; if not, adjust the parameter values ​​corresponding to the display mode so that the adjusted display mode meets the display conditions of the target display position.

[0016] Optionally, it also includes: in response to the display mode not meeting the display condition of the target display position; obtaining the display mode priority corresponding to the category, matching the display mode with the highest priority other than the current display mode as the final display mode.

[0017] Optionally, after the medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest is generated, the following steps are included:

[0018] The objects in the medical auxiliary information are taken as nodes, and the relationships between the objects in the medical auxiliary information are taken as edges to construct a corresponding graph of the medical auxiliary information; interactive controls are configured for objects with associated relationships according to the graph, and then the objects with associated relationships are displayed in response to the triggering of the current object interactive controls.

[0019] To achieve the above object, according to another aspect of an embodiment of the present invention, there is provided an image processing device, comprising:

[0020] An acquisition module is configured to extract medical image data transmitted by a preset target image source and to extract display data transmitted by a preset target display device;

[0021] A generating module is configured to identify a target region of interest and / or a target object of interest in the medical image data, generate medical auxiliary information corresponding to the target region of interest and / or the target object of interest; and locate a display position of the target region of interest and / or the target object of interest, and generate a target display position of the medical auxiliary information;

[0022] an adaptation module, configured to determine a display mode according to the target display position and the medical auxiliary information;

[0023] The display module is configured to transmit the medical auxiliary information to a target display device so as to present the medical auxiliary information at the display position in the display manner.

[0024] Optionally, the generation module is further configured to: in response to an image processing event for the medical image data, locate a target area of ​​interest and / or target object of interest corresponding to the image processing event; identify a category of the target area of ​​interest and / or target object of interest, and obtain a processing model based on the category and the image processing event; and call the processing model to calculate medical auxiliary information based on the medical image data associated with the target area of ​​interest and / or target object of interest.

[0025] Optionally, the generation module is further configured to: obtain characteristic attribute information of the target area of ​​interest and / or the target object of interest, and determine the displayed boundary information; obtain position information of the key area and / or key object of the current page, determine the relationship between the boundary information and the position information, and generate the target display position of the medical auxiliary information based on the relationship.

[0026] Optionally, the generating module determines the relationship between the boundary information and the position information, and generates the target display position of the medical auxiliary information according to the relationship, including:

[0027] In response to selecting a relationship determination dimension in a set, the relationship between the boundary information and the position information is calculated according to the relationship determination dimension; an evaluation model corresponding to the relationship determination dimension is called to process the relationship to obtain an evaluation grade; and a target display position of the medical auxiliary information is determined according to the evaluation grade.

[0028] Optionally, after the generating module determines the target display position of the medical auxiliary information, the generating module further comprises: in response to the target display position belonging to a preset specific position area, reselecting a relationship determination dimension in the set.

[0029] Optionally, the adaptation module is further configured to: obtain the category of the medical auxiliary information and match it to obtain a display mode; determine whether the display mode meets the display conditions of the target display position, and if so, use the display mode as the final display mode; otherwise, adjust the parameter values ​​corresponding to the display mode so that the adjusted display mode meets the display conditions of the target display position.

[0030] Optionally, the adaptation module is further configured to: in response to the display mode not meeting the display conditions of the target display position; obtain the display mode priority corresponding to the category, and match the display mode with the highest priority other than the current display mode as the final display mode.

[0031] Optionally, after the generating module generates the medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest, the generating module further comprises:

[0032] The objects in the medical auxiliary information are taken as nodes, and the relationships between the objects in the medical auxiliary information are taken as edges to construct a corresponding graph of the medical auxiliary information; interactive controls are configured for objects with associated relationships according to the graph, and then the objects with associated relationships are displayed in response to the triggering of the current object interactive controls.

[0033] To achieve the above objective, according to another aspect of an embodiment of the present invention, an image processing device is provided.

[0034] The image processing device of an embodiment of the present invention includes: one or more processors, a memory, and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of any of the above-mentioned image processing methods.

[0035] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is loaded by a processor to execute the steps of any of the above-mentioned image processing methods.

[0036] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: it can solve the technical problems of low utilization rate of existing AI-assisted diagnosis, increased operating burden of doctors, and low diagnostic accuracy and efficiency, thereby improving the information richness and interactivity of medical image display, and greatly improving the accuracy and efficiency of auxiliary diagnosis.

[0037] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0039] Figure 1 is a scene schematic diagram of an image processing method according to an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of the main flow of an image processing method according to an embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of an optional process of generating medical auxiliary information according to an embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of an optional process of generating a target display position of medical auxiliary information according to an embodiment of the present invention;

[0043] Figure 5 is a schematic diagram of an optional flow chart of determining a display mode according to an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of main modules of an image processing device according to an embodiment of the present invention;

[0045] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0046] Figure 8 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0048] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0049] In the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The acquisition, transmission, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations, and are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities.

[0050] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In this application, the word "example" is used to mean "used as an example, illustration or description". Any embodiment described as "example" in this application is not necessarily interpreted as being more preferred or more advantageous than other embodiments. In order to enable any technician in the field to implement and use the present invention, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present invention obscure. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in this application.

[0051] See also Figure 1, which shows a scene schematic diagram of an image processing method provided by an embodiment of the present invention. In the embodiment of the present invention, the image interaction scene includes an image processing host 100, a target image source 200 and a target display device 300. The image processing method is applied to an image processing device, and the image processing device is arranged in the image processing host 100. The image processing host 100 is provided with one or more processors, memories, and one or more applications, wherein the one or more applications are stored in the memories and are configured to be executed by the processor to implement the image processing method. The image processing device 100 may also include a memory for storing medical auxiliary information and other data. For example, the image processing device may be an image processing system of an AI (artificial intelligence) black box, and a computer-readable storage medium corresponding to the image processing method is run in the image processing host 100 to execute the steps of the image processing method. The target image source 200 can be connected to the image processing host 100 and the target display device 300 for communication, and can scan the target object to obtain medical image data and transmit it to the medical scanning device in the target display device 300 for displaying the target medical image, or can scan the target object to obtain medical image data and transmit it to the image processing host 100. For example, the target image source 200 can be any imaging system such as computed tomography (CT), 4D ultrasound and magnetic resonance (MR), angiography scanning, DR scanning and ultrasound scanning, ultrasound host, etc. The target display device 300 can be connected to the image processing host 100 and the target image source 200 for communication, and can not only receive the display data of the image processing host 100 and the target image source 200, but also transmit the display data to the image processing host 100. For example, the target display device 300 can be any display device such as an ultrasound display, angiography display, etc.

[0052] Understandably, Figure 1 The image processing device in the image processing method scenario shown, or the devices included in the image processing device, does not constitute a limitation on the embodiments of the present invention, that is, the number and type of image processing devices included in the image processing method scenario, or the number and type of devices included in each device, do not affect the overall implementation of the technical solution in the embodiments of the present invention, and can be regarded as an equivalent replacement or derivative of the technical solution claimed for protection in the embodiments of the present invention. In addition, Figure 1 The application environment shown in is only one of the application scenarios related to the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1The more or fewer image processing devices shown, or the image processing network connection relationship, are not specifically limited here.

[0053] Based on the above-mentioned scenario of the image processing method, various embodiments of the image processing method disclosed in the present invention are proposed.

[0054] See also Figure 2 , which shows a main flow chart of an image processing method provided by an embodiment of the present invention, comprising the following steps:

[0055] S201: extracting medical image data transmitted by a preset target image source, identifying a target region of interest and / or a target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target region of interest and / or the target object of interest.

[0056] In an embodiment, the target image source 200 transmits medical image data to the image processing host 100, and the image processing host 100 can identify and determine the target area of ​​interest and / or the target object of interest in the medical image data according to the needs, wherein the target object of interest is the physiological tissue or lesion tissue specified in the medical image data, and the target area of ​​interest is the specific image area specified in the medical image data. Then, the image processing host 100 generates corresponding medical auxiliary information based on the target area of ​​interest and / or the target object of interest, and the medical auxiliary information is any one or more of the auxiliary prompt information, associated medical images, enhanced medical information, and supplementary medical information corresponding to the target area of ​​interest and / or the target object of interest.

[0057] In a further embodiment, step S201 can respond to an image processing event for medical image data, locate the target area of ​​interest and / or target object of interest corresponding to the image processing event, then identify the category of the target area of ​​interest and / or target object of interest, obtain a processing model based on the category and the image processing event, and finally call the processing model to calculate medical auxiliary information based on the medical image data associated with the target area of ​​interest and / or target object of interest.

[0058] Furthermore, by responding to an image processing event for medical image data, an image selection operation triggered by a user for medical image data is obtained, the selection position information of the image selection operation is read, and the target area of ​​interest and / or target object of interest corresponding to the image processing event in the medical image data is located. The image selection operation may be an operation instruction for a user to select medical image data in a display area of ​​the medical image data. In addition, the image processing event may not only be an image selection operation, but may also be an automatically triggered preset recognition model, for example: any one or more image recognition models in a lesion recognition model, a medical information association table, a medical reconstruction model, etc., to automatically locate the target area of ​​interest and / or target object of interest in the medical image data where lesions may exist through a preset recognition strategy in the recognition model.

[0059] After locating the target region of interest and / or the target object of interest, the category of the target region of interest and / or the target object of interest can be identified, for example: the category of physiological tissue or lesion tissue to which the target object of interest belongs, the category of the specific image region to which the target region of interest belongs, and then the processing model (for example: image processing model, object analysis processing model, etc.) is matched in combination with the image processing event (for example: image selection operation or automatically triggered recognition model). It can be seen that different categories of target regions of interest and / or target objects of interest can correspond to different processing models, and different image processing events can correspond to different processing models. The processing model is determined by two dimensions (category of target region of interest and / or target object of interest, image processing event), which satisfies efficient and refined medical image data processing.

[0060] Finally, the matched processing model is called to calculate the medical imaging data associated with the target area of ​​interest and / or the target object of interest to obtain medical auxiliary information. For example: the matched image processing model is called to perform image segmentation on the medical imaging data associated with the target area of ​​interest and / or the target object of interest, and the segmented medical image data is reprocessed (enlarged, adjusted, annotated, etc.); for example: the matched object analysis processing model is called to perform preset dimension (time dimension, lesion type dimension, etc.) analysis on the medical imaging data associated with the target area of ​​interest and / or the target object of interest.

[0061] It is worth noting that after step S201 generates the medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest, the objects in the medical auxiliary information can be used as nodes, and the relationships between the objects in the medical auxiliary information can be used as edges to construct a corresponding graph of the medical auxiliary information, and then interactive controls are configured for objects with associated relationships according to the graph, and then the objects with associated relationships are displayed in response to the triggering of the current object interactive control. In other words, the present invention can not only display the medical auxiliary information directly, but also re-analyze and re-display the medical auxiliary information in the form of a graph through interactive controls. Of course, the display in the form of a graph through interactive controls can be on the image processing host 100, and the graph information can also be transmitted to the target display device 300 for display through interactive controls.

[0062] S202: extracting display data transmitted by a preset target display device, locating the display position of the target area of ​​interest and / or the target object of interest, and generating a target display position of the medical auxiliary information.

[0063] In an embodiment, the target display device 300 transmits display data to the image processing host 100. The image processing host 100 can locate the display position of the target area of ​​interest and / or the target object of interest according to the display data, and generate the target display position of the medical auxiliary information.

[0064] It should be noted that display data can be transmitted between the target display device 300 and the image processing host 100, and the target display device 300 can also be recorded to obtain display data so as to transmit the display data to the image processing host 100. In addition, whether it is the transmission of display data or the triggering of screen recording to obtain display signals, the target display device 300 can actively perform the transmission of display data or trigger the screen recording to obtain display signals and transmit them, or the image processing host 100 can initiate a display data transmission request or a screen recording to obtain display signals to the target display device 300, and the target display device 300 responds to the display data transmission request or the screen recording to obtain display signals.

[0065] In a further embodiment, step S202 can obtain characteristic attribute information of the target focus area and / or the target focus object, determine the displayed boundary information, then obtain the position information of the key area and / or the key object of the current page, determine the relationship between the boundary information and the position information, and generate the target display position of the medical auxiliary information according to the relationship. It can be seen that the display position of the target focus area and / or the target focus object can be determined by the characteristic attribute information of the target focus area and / or the target focus object to locate the boundary information displayed. For example, the extracted display data can be pre-processed first, such as binarization, noise reduction, enhancement, etc., and then the pre-processed display data is calculated using a feature extraction algorithm such as HOG (Histogram of Oriented Gradients), LBP (Local Binary Pattern), etc. to obtain the characteristic attribute information of the target focus area and / or the target focus object, and finally the boundary information is determined according to the characteristic attribute information using methods such as Sobel operator, Canny operator, etc.

[0066] Furthermore, the target display position of the medical auxiliary information can be determined by the relationship between the display position of the target focus area and / or the target focus object (i.e., boundary information) and the position information of the key area and / or key object of the page. For example, SIFT (scale-invariant feature transform), SURF (scale-space extreme value detection) algorithm, etc. can be used to obtain the position information of the key area and / or key object, calculate the distance between the boundary information and the position information, and judge the relationship between the boundary information and the position information based on the distance (for example: overlap, proximity, distance, etc.). After determining the relationship, the target display position of the medical auxiliary information can be generated. For example, if the relationship is overlap, the target display position of the medical auxiliary information is set to a position at a preset distance from the boundary information. If the relationship is distance, the display position of the current target focus area and / or target focus object is used as the target display position of the medical auxiliary information, and so on.

[0067] It is worth noting that the relationship between the display position (i.e., boundary information) of the target focus area and / or target focus object and the position information of the key area and / or key object of the page is not limited to the distance calculation in the above example, but can also be the relationship calculation between the corresponding features of the display position of the target focus area and / or target focus object and the corresponding features of the position information of the key area and / or key object, for example: the difference in brightness features corresponding to the two position information, the difference in density corresponding to the two position information, etc.

[0068] In a further embodiment, step S202 determines the relationship between the boundary information and the position information, and generates the target display position of the medical auxiliary information according to the relationship, and may also include: in response to selecting a relationship determination dimension in a set, calculating the relationship between the boundary information and the position information according to the relationship determination dimension, and then calling the evaluation model corresponding to the relationship determination dimension, processing the relationship to obtain an evaluation grade, and finally determining the target display position of the medical auxiliary information according to the evaluation grade. It can be seen that the present invention can dynamically select a relationship determination dimension in a preset set when determining the relationship between the boundary information and the position information, and there is a mapping relationship between the relationship determination dimension and the evaluation model, and then the relationship is determined by the evaluation model to obtain an evaluation grade, and then the target display position of the medical auxiliary information is determined according to the evaluation grade.

[0069] For example: a distance difference level evaluation dimension can be selected from a preset set, and the corresponding evaluation model is to classify the distance difference into levels, calculate the distance difference between the boundary information of the target focus area and / or the target focus object and the position information of the key area and / or the key object, and different levels correspond to the determination of the target display position of different medical auxiliary information. For example, if the level is one (the distance difference is 0), the target display position of the medical auxiliary information is set to a position at a preset first distance from the boundary information.

[0070] Also as an example: a density evaluation dimension can be selected from a preset set, and the corresponding evaluation model is to divide the levels into levels based on density differences. The density of the target focus area and / or target focus object corresponding to the boundary information and the density of the key area and / or key object corresponding to the position information are calculated respectively, and then the difference between the two densities is obtained and classified into corresponding levels. Different levels correspond to the determination of target display positions of different medical auxiliary information. For example, if the level is level one (density difference is 0), the target display position of the medical auxiliary information is set to a position at a preset second distance from the boundary information, thereby greatly reducing the visual impact on the key areas and / or key objects when displaying the medical auxiliary information.

[0071] In a preferred embodiment, after determining the target display position of the medical auxiliary information, the target display position can be reprocessed to prevent deviation and inappropriate display of the target display position, specifically including: in response to the target display position belonging to a preset specific position area, reselecting the relationship determination dimension in the set. That is to say, the present invention sets a specific position area for the page, such as the edge four corner area, the frame area, etc. If the target display position falls into (the overlap degree is greater than the prediction threshold), the relationship determination dimension can be reselected in the set to perform relationship calculation.

[0072] S203: Determine a display mode according to the target display position and the medical auxiliary information.

[0073] In an embodiment, the image processing host 100 needs to determine the display mode of the medical auxiliary information not only based on the target display position, but also based on the data content of the medical auxiliary information itself. The specific implementation process includes: the category of the medical auxiliary information can be obtained, the display mode can be matched, and then it is determined whether the display mode meets the display conditions of the target display position. If so, the display mode is used as the final display mode. If not, the parameter values ​​corresponding to the display mode are adjusted so that the adjusted display mode meets the display conditions of the target display position. Example: The category of medical auxiliary information is multiple lesion change images, and the display mode is parallel display. It is necessary to determine whether the size of the parallel display area is larger than the size of the target display position. If so, the length, width and other parameter values ​​of the parallel display area are adjusted. Otherwise, no adjustment is made.

[0074] In a further embodiment, in response to the display mode not meeting the display conditions of the target display position, the image processing host 100 may not adjust the parameter values ​​of the current display mode, but select other display modes from the multiple display modes corresponding to the category of medical auxiliary information, specifically including: obtaining the display mode priority corresponding to the category, matching the display mode with the highest priority other than the current display mode as the final display mode.

[0075] It should be noted that the display conditions of the target display position are not limited to the parameters involved in the display range in the above examples, but may also involve display conditions of various other types of parameters, such as resolution, scanning frequency, refresh speed, power consumption, etc. The various parameters involved in the display mode are adjusted to meet the display conditions of the target display position. If the adjustment cannot meet the display conditions of the target display position, the relationship determination dimension can be re-selected in the set to perform relationship calculation or match the highest priority display mode other than the current display mode.

[0076] S204: Transmitting the medical auxiliary information to a target display device, so as to present the medical auxiliary information at the target display position in the display manner.

[0077] In an embodiment, the image processing host 100 transmits data such as medical auxiliary information, target display position and display mode to the target display device 300, so that the target display device 300 presents the medical auxiliary information at the target display position by displaying.

[0078] See also Figure 3 , shows a schematic diagram of an optional process of generating medical auxiliary information according to an embodiment of the present invention, comprising the following steps:

[0079] S301: extracting medical image data transmitted by a preset target image source.

[0080] S302: In response to an image processing event for the medical image data, locate a target area of ​​interest and / or a target object of interest corresponding to the image processing event.

[0081] S303: In response to an image processing event for the medical image data, locate a target area of ​​interest and / or a target object of interest corresponding to the image processing event.

[0082] S304: Calling the processing model to calculate and obtain medical auxiliary information based on the medical image data associated with the target area of ​​interest and / or the target object of interest.

[0083] S305: Using objects in the medical auxiliary information as nodes and relationships between objects in the medical auxiliary information as edges to construct a corresponding graph of the medical auxiliary information.

[0084] S306: configuring interactive controls for objects with association relationships according to the graph, and then displaying objects with association relationships in response to triggering of the current object interactive control.

[0085] See also Figure 4 , shows a schematic diagram of an optional process of generating a target display position of medical auxiliary information according to an embodiment of the present invention, comprising the following steps:

[0086] S401: extracting display data transmitted by a preset target display device.

[0087] S402: Acquire characteristic attribute information of the target attention area and / or the target attention object, and determine displayed boundary information.

[0088] S403: Acquire the location information of the key area and / or key object of the current page.

[0089] S404: In response to selecting a relationship determination dimension in the set, calculating the relationship between the boundary information and the position information according to the relationship determination dimension.

[0090] S405: calling the evaluation model corresponding to the relationship determination dimension, and processing the relationship to obtain an evaluation grade.

[0091] S406: Determine a target display position of the medical auxiliary information according to the evaluation level.

[0092] S407: Determine whether the target display position belongs to a preset specific position area. If so, return to S404 to reselect a relationship determination dimension in the set. If not, execute step S408.

[0093] S408: Obtain the final target display position.

[0094] See also Figure 5 , shows a schematic diagram of an optional process of determining a display mode according to an embodiment of the present invention, comprising the following steps:

[0095] S501: Acquire the category of the medical auxiliary information, and match it to obtain a display mode.

[0096] S502: Determine whether the display mode meets the display condition of the target display position, if yes, execute step S503, if no, execute step S504.

[0097] S503: adjusting the values ​​of the parameters corresponding to the display mode so that the adjusted display mode meets the display condition of the target display position.

[0098] S504: Obtain the display mode priority corresponding to the category, and match the display mode with the highest priority other than the current display mode as the final display mode.

[0099] See also Figure 6 , which shows a schematic diagram of main modules of an image processing device 600 provided by an embodiment of the present invention, including:

[0100] The acquisition module 601 is configured to extract medical image data transmitted by a preset target image source and to extract display data transmitted by a preset target display device;

[0101] The generating module 602 is configured to identify the target region of interest and / or the target object of interest in the medical image data, generate medical auxiliary information corresponding to the target region of interest and / or the target object of interest; and locate the display position of the target region of interest and / or the target object of interest, and generate a target display position of the medical auxiliary information;

[0102] An adaptation module 603 is configured to determine a display mode according to the target display position and the medical auxiliary information;

[0103] The display module 604 is configured to transmit the medical auxiliary information to a target display device so as to present the medical auxiliary information at the display position in the display manner.

[0104] In the implementation device of the present invention, the generation module 602 is also configured to: in response to an image processing event for the medical image data, locate the target area of ​​interest and / or target object of interest corresponding to the image processing event; identify the category of the target area of ​​interest and / or target object of interest, and obtain a processing model based on the category and the image processing event; call the processing model to calculate medical auxiliary information based on the medical image data associated with the target area of ​​interest and / or target object of interest.

[0105] In the implementation device of the present invention, the generation module 602 is also configured to: obtain characteristic attribute information of the target area of ​​interest and / or the target object of interest, and determine the displayed boundary information; obtain the position information of the key area and / or key object of the current page, determine the relationship between the boundary information and the position information, and generate the target display position of the medical auxiliary information based on the relationship.

[0106] In the implementation device of the present invention, the generating module 602 determines the relationship between the boundary information and the position information, and generates the target display position of the medical auxiliary information according to the relationship, including:

[0107] In response to selecting a relationship determination dimension in a set, the relationship between the boundary information and the position information is calculated according to the relationship determination dimension; an evaluation model corresponding to the relationship determination dimension is called to process the relationship to obtain an evaluation grade; and a target display position of the medical auxiliary information is determined according to the evaluation grade.

[0108] In the implementation device of the present invention, after the generation module 602 determines the target display position of the medical auxiliary information, it includes: in response to the target display position belonging to a preset specific position area, reselecting the relationship determination dimension in the set.

[0109] In the implementation device of the present invention, the adaptation module 603 is also configured to: obtain the category of the medical auxiliary information and match it to obtain a display mode; determine whether the display mode meets the display conditions of the target display position, and if so, use the display mode as the final display mode; otherwise, adjust the parameter values ​​corresponding to the display mode so that the adjusted display mode meets the display conditions of the target display position.

[0110] In the implementation device of the present invention, the adaptation module 603 is also configured to: in response to the display mode not meeting the display conditions of the target display position; obtain the display mode priority corresponding to the category, and match the display mode with the highest priority other than the current display mode as the final display mode.

[0111] In the implementation device of the present invention, after the generating module 602 generates the medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest, the following steps are included:

[0112] The objects in the medical auxiliary information are taken as nodes, and the relationships between the objects in the medical auxiliary information are taken as edges to construct a corresponding graph of the medical auxiliary information; interactive controls are configured for objects with associated relationships according to the graph, and then the objects with associated relationships are displayed in response to the triggering of the current object interactive controls.

[0113] In addition, the specific implementation content of the device described in the embodiment of the present invention has been described in detail in the method described above, so the repeated content will not be described again here.

[0114] Figure 7 An exemplary system architecture 700 to which embodiments of the present invention may be applied is shown, including terminal devices 701 , 702 , 703 , a network 704 and a server 705 (only an example).

[0115] Terminal devices 701, 702, 703 can be various electronic devices with display screens and supporting web browsing, and various communication client applications are installed. Users can use terminal devices 701, 702, 703 to interact with server 705 through network 704 to receive or send messages, etc.

[0116] The network 704 is used to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0117] The server 705 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using the terminal devices 701, 702, and 703 (only as an example). The background management server may analyze and process the received information query request and other data, and feed back the processing results (such as target push information, product information - only as an example) to the terminal device. It should be noted that the method provided in the embodiment of the present invention is generally executed by the server 705, and accordingly, the device is generally set in the server 705.

[0118] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0119] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0120] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0121] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0122] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0123] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0124] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes an acquisition module, a generation module, an adaptation module, and a display module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the generation module may also be described as a determination module or a processing module, etc.

[0126] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes any of the above image processing methods.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Extracting medical image data transmitted by a preset target image source, identifying a target region of interest and / or a target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target region of interest and / or the target object of interest; Extracting display data transmitted by a preset target display device, locating the display position of the target area of ​​interest and / or the target object of interest, and generating a target display position of the medical auxiliary information; Determining a display mode according to the target display position and the medical auxiliary information; The medical auxiliary information is transmitted to a target display device so as to present the medical auxiliary information at the target display position through the display mode.

2. The method according to claim 1, characterized in that Identifying the target region of interest and / or the target object of interest in the medical image data, and generating medical auxiliary information corresponding to the target region of interest and / or the target object of interest, including: In response to an image processing event for the medical image data, locating a target area of ​​interest and / or a target object of interest corresponding to the image processing event; Identify the category of the target focus area and / or the target focus object, and obtain a processing model according to the category and the image processing event matching; The processing model is called to calculate and obtain medical auxiliary information based on the medical image data associated with the target region of interest and / or the target object of interest.

3. The method according to claim 1, characterized in that Locating the display position of the target area of ​​interest and / or the target object of interest, and generating the target display position of the medical auxiliary information, comprises: Acquire characteristic attribute information of the target concerned area and / or the target concerned object, and determine displayed boundary information; The position information of the key area and / or key object of the current page is obtained, the relationship between the boundary information and the position information is determined, and the target display position of the medical auxiliary information is generated according to the relationship.

4. The method according to claim 3, characterized in that Determining the relationship between the boundary information and the position information, and generating a target display position of the medical auxiliary information according to the relationship, comprises: In response to selecting a relationship determination dimension in the set, calculating a relationship between the boundary information and the position information according to the relationship determination dimension; Calling the evaluation model corresponding to the relationship determination dimension, processing the relationship to obtain an evaluation grade; According to the evaluation level, a target display position of the medical auxiliary information is determined.

5. The method according to claim 4, characterized in that After determining the target display position of the medical auxiliary information, the method includes: In response to the target display position belonging to a preset specific position area, a relationship determination dimension is reselected in the set.

6. The method according to claim 1, characterized in that Determining a display mode according to the target display position and the medical auxiliary information includes: Acquire the category of the medical auxiliary information, and match it to obtain a display mode; Determine whether the display mode meets the display conditions of the target display position. If so, use the display mode as the final display mode. If not, adjust the parameter values ​​corresponding to the display mode so that the adjusted display mode meets the display conditions of the target display position.

7. The method according to claim 6, characterized in that Also includes: In response to the display mode not meeting the display condition of the target display position; The display mode priority corresponding to the category is obtained, and the display mode with the highest priority other than the current display mode is matched and obtained as the final display mode.

8. The method according to any one of claims 1 to 7, characterized in that: After the medical auxiliary information corresponding to the target area of ​​interest and / or the target object of interest is generated, the method includes: Using objects in the medical auxiliary information as nodes and relationships between objects in the medical auxiliary information as edges to construct a corresponding graph of the medical auxiliary information; Interactive controls are configured for objects with association relationships according to the graph, and then in response to triggering of the current object interactive control, the objects with association relationships are displayed.

9. An image processing device, characterized in that: include: An acquisition module is configured to extract medical image data transmitted by a preset target image source and to extract display data transmitted by a preset target display device; A generating module is configured to identify a target region of interest and / or a target object of interest in the medical image data, and generate medical auxiliary information corresponding to the target region of interest and / or the target object of interest; and locating the display position of the target area of ​​interest and / or the target object of interest, and generating a target display position of the medical auxiliary information; an adaptation module, configured to determine a display mode according to the target display position and the medical auxiliary information; The display module is configured to transmit the medical auxiliary information to a target display device so as to present the medical auxiliary information at the display position in the display manner.

10. An image processing device, characterized in that: The image processing device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the steps of the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the image processing method according to any one of claims 1 to 8.

Images