Image processing method and device

Abstract

The invention discloses an image processing method and device, and relates to the technical field of medical image processing. A specific embodiment of the method comprises the following steps: extracting video stream data transmitted by preset target display equipment, positioning a current frame, and determining a target attention area and / or a target attention object in medical image data according to the medical image data corresponding to the current frame; configuring a query dimension of a target attention area and / or a target attention object, calling a historical frame corresponding to the target attention area and / or the target attention object according to the query dimension, and obtaining medical image data corresponding to the historical frame; calling a preset analysis model, and generating medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame; and transmitting the medical auxiliary information to a target display device and presenting the medical auxiliary information. According to the embodiment, at least more medical information can be provided for doctors, and the diagnosis efficiency of the doctors is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an image processing method and device. Background Art

[0002] At present, existing diseases are highly complex and require the combination of multiple examination information. Doctors need to retrieve a large amount of examination information during the diagnosis of patients, and then process and analyze it to obtain the final diagnosis result. Therefore, doctors have low diagnostic efficiency, heavy operation burden, and low diagnostic accuracy. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides an image processing method and device, which can at least provide doctors with more medical information and effectively improve the diagnosis efficiency of doctors.

[0004] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: extracting video stream data transmitted by a preset target display device, locating a current frame, and determining a target area of ​​interest and / or a target object of interest in the medical image data according to the medical image data corresponding to the current frame; configuring a query dimension of the target area of ​​interest and / or the target object of interest, and retrieving historical frames corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension to obtain the medical image data corresponding to the historical frame; calling a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame; and transmitting the medical auxiliary information to the target display device and presenting it.

[0005] Optionally, extracting video stream data transmitted by a preset target display device includes: receiving video stream data transmitted by a preset target display device, and determining whether the type of the video stream data is fixed frame; if so, taking the video stream data as the current frame, and otherwise locating the current frame in the video stream data.

[0006] Optionally, based on the medical image data corresponding to the current frame, a target area of ​​interest and / or a target object of interest in the medical image data is determined, including: obtaining an image identifier of the current frame, and determining the corresponding medical image data in the medical image data transmitted by a preset target image source; and in response to an image processing event for the medical image data, locating the target area of ​​interest and / or the target object of interest corresponding to the image processing event.

[0007] Optionally, configuring the query dimensions of the target area of ​​interest and / or the target object of interest includes: identifying the type of the target area of ​​interest and / or the target object of interest, and obtaining a query dimension list corresponding to the type; in response to a selection of a query dimension in the query dimension list, calling a parameter array of the query dimension, and using the parameter array as query dimension configuration information for the target area of ​​interest and / or the target object of interest.

[0008] Optionally, historical frames corresponding to the target area of ​​interest and / or target object of interest are retrieved according to the query dimension to obtain medical image data corresponding to the historical frames, including: retrieving historical frames of the target area of ​​interest and / or target object of interest, and determining the target historical frame by matching the query dimension with feature attribute information of the historical frame; and obtaining medical image data corresponding to the target historical frame according to the image identifier of the target historical frame.

[0009] Optionally, according to the image identifier of the target historical frame, the medical image data corresponding to the target historical frame is obtained, including: according to the image identifier of the target historical frame, determining whether the image identifier exists in the local download list; if so, obtaining the medical image data corresponding to the target historical frame; if not, generating a data request according to the image identifier of the target historical frame and sending it to a preset target image source, thereby receiving the medical image data corresponding to the target historical frame.

[0010] Optionally, a preset analysis model is called to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, including: determining the association between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model; calling the analysis model to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0011] Optionally, determining the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model, includes: respectively extracting the attribute information and corresponding numerical values ​​of the medical image data corresponding to the current frame, and the attribute information and corresponding numerical values ​​of the medical image data corresponding to the historical frame; calling a preset relationship calculation model to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model.

[0012] Optionally, matching the corresponding analysis model includes: matching to obtain the corresponding analysis model class according to the association relationship; and determining the corresponding analysis model in the analysis model class based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0013] Optionally, before transmitting the medical auxiliary information to a target display device, the method includes: determining a target display position of the medical auxiliary information in response to determining a data type of a video stream for displaying the medical auxiliary information.

[0014] Optionally, it also includes: determining whether the data type of the video stream displaying the medical auxiliary information is fixed frame; if so, locating the display position of the target area of ​​interest and / or the target object of interest in the current frame, and determining the target display position of the medical auxiliary information in combination with the display mode of the medical auxiliary information; if otherwise, configuring the target display position of the medical auxiliary information according to the display mode of the medical auxiliary information.

[0015] To achieve the above object, according to another aspect of an embodiment of the present invention, there is provided an image processing device, comprising: an acquisition module, configured to extract video stream data transmitted by a preset target display device, locate a current frame, and determine a target area of ​​interest and / or a target object of interest in the medical image data according to medical image data corresponding to the current frame;

[0016] A query module is configured to configure a query dimension of the target area of ​​interest and / or the target object of interest, retrieve a historical frame corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and obtain medical image data corresponding to the historical frame;

[0017] A generating module is configured to call a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame;

[0018] The display module is configured to transmit the medical auxiliary information to a target display device for presentation.

[0019] Optionally, the acquisition module extracts video stream data transmitted by a preset target display device, including: receiving video stream data transmitted by a preset target display device, and determining whether the type of the video stream data is fixed frame; if so, taking the video stream data as the current frame, otherwise locating the current frame in the video stream data.

[0020] Optionally, the acquisition module determines the target area of ​​interest and / or target object of interest in the medical image data based on the medical image data corresponding to the current frame, including: acquiring the image identifier of the current frame, determining the corresponding medical image data in the medical image data transmitted by a preset target image source; and locating the target area of ​​interest and / or target object of interest corresponding to the image processing event in response to an image processing event for the medical image data.

[0021] Optionally, the query module configures the query dimensions of the target focus area and / or target focus object, including: identifying the type of the target focus area and / or target focus object, and obtaining a query dimension list corresponding to the type; in response to a selection of a query dimension in the query dimension list, calling a parameter array of the query dimension, and using the parameter array as query dimension configuration information for the target focus area and / or target focus object.

[0022] Optionally, the query module retrieves historical frames corresponding to the target area of ​​interest and / or target object of interest according to the query dimension, and obtains medical image data corresponding to the historical frames, including: retrieving historical frames of the target area of ​​interest and / or target object of interest, and determining the target historical frame by matching the query dimension with feature attribute information of the historical frame; and obtaining medical image data corresponding to the target historical frame according to the image identifier of the target historical frame.

[0023] Optionally, the query module obtains medical image data corresponding to the target historical frame based on the image identifier of the target historical frame, including: judging whether the image identifier exists in the local download list based on the image identifier of the target historical frame; if so, obtaining the medical image data corresponding to the target historical frame; if not, generating a data request based on the image identifier of the target historical frame and sending it to a preset target image source, thereby receiving the medical image data corresponding to the target historical frame.

[0024] Optionally, the generation module calls a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, including: determining the association between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model; calling the analysis model to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0025] Optionally, the generation module determines the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matches the corresponding analysis model, including: respectively extracting the attribute information and corresponding numerical values ​​of the medical image data corresponding to the current frame, and the attribute information and corresponding numerical values ​​of the medical image data corresponding to the historical frame; calling a preset relationship calculation model to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model.

[0026] Optionally, the generation module matches the corresponding analysis model, including: matching the corresponding analysis model class according to the association relationship; and determining the corresponding analysis model in the analysis model class based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0027] Optionally, before the display module transmits the medical auxiliary information to the target display device, the display module includes: determining a target display position of the medical auxiliary information in response to determining a data type of a video stream for displaying the medical auxiliary information.

[0028] Optionally, the display module is further configured to determine whether the data type of the video stream displaying the medical auxiliary information is fixed frame; if so, locate the display position of the target area of ​​interest and / or the target object of interest in the current frame, and determine the target display position of the medical auxiliary information in combination with the display mode of the medical auxiliary information; if otherwise, configure the target display position of the medical auxiliary information according to the display mode of the medical auxiliary information.

[0029] To achieve the above objective, according to another aspect of an embodiment of the present invention, an image processing device is provided.

[0030] The image processing device of an embodiment of the present invention includes: one or more processors, a memory, and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of any of the above-mentioned image processing methods.

[0031] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is loaded by a processor to execute the steps of any of the above-mentioned image processing methods.

[0032] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: it can solve the technical problems of low utilization rate of existing AI-assisted diagnosis, low diagnostic accuracy and efficiency, thereby improving the richness and utilization rate of medical image display information, and achieving accurate and efficient assisted diagnosis.

[0033] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0035] Figure 1 is a scene schematic diagram of an image processing method according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of the main flow of an image processing method according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of an optional process of determining a target area of ​​interest and / or a target object of interest according to an embodiment of the present invention;

[0038] Figure 4 is a schematic diagram of an optional process of obtaining historical frame medical image data according to an embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of an optional process of generating and displaying medical auxiliary information according to an embodiment of the present invention;

[0040] Figure 6 is a schematic diagram of main modules of an image processing device according to an embodiment of the present invention;

[0041] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0042] Figure 8 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0044] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0045] In the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The acquisition, transmission, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations, and are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities.

[0046] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In this application, the word "example" is used to mean "used as an example, illustration or description". Any embodiment described as "example" in this application is not necessarily interpreted as being more preferred or more advantageous than other embodiments. In order to enable any technician in the field to implement and use the present invention, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present invention obscure. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in this application.

[0047] See also Figure 1, which shows a scene schematic diagram of an image processing method provided by an embodiment of the present invention. In the embodiment of the present invention, the image interaction scene includes an image processing host 100, a target image source 200 and a target display device 300. The image processing method is applied to an image processing device, and the image processing device is arranged in the image processing host 100. The image processing host 100 is provided with one or more processors, memories, and one or more applications, wherein the one or more applications are stored in the memories and are configured to be executed by the processor to implement the image processing method. The image processing device 100 may also include a memory for storing medical auxiliary information and other data. For example, the image processing device may be an image processing system of an AI (artificial intelligence) black box, and a computer-readable storage medium corresponding to the image processing method is run in the image processing host 100 to execute the steps of the image processing method. The target image source 200 can be connected to the image processing host 100 and the target display device 300 for communication, and can scan the target object to obtain medical image data and transmit it to the medical scanning device in the target display device 300 for displaying the target medical image, or can scan the target object to obtain medical image data and transmit it to the image processing host 100. For example, the target image source 200 can be any imaging system such as computed tomography (CT), 4D ultrasound and magnetic resonance (MR), angiography scanning, DR scanning and ultrasound scanning, ultrasound host, etc. The target display device 300 can be connected to the image processing host 100 and the target image source 200 for communication, and can not only receive the display data of the image processing host 100 and the target image source 200, but also transmit the display data to the image processing host 100. For example, the target display device 300 can be any display device such as an ultrasound display, angiography display, etc.

[0048] Understandably, Figure 1 The image processing device in the image processing method scenario shown, or the devices included in the image processing device, does not constitute a limitation on the embodiments of the present invention, that is, the number and type of image processing devices included in the image processing method scenario, or the number and type of devices included in each device, do not affect the overall implementation of the technical solution in the embodiments of the present invention, and can be regarded as an equivalent replacement or derivative of the technical solution claimed for protection in the embodiments of the present invention. In addition, Figure 1 The application environment shown in is only one of the application scenarios related to the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1The more or fewer image processing devices shown, or the image processing network connection relationship, are not specifically limited here.

[0049] Based on the above-mentioned scenario of the image processing method, various embodiments of the image processing method disclosed in the present invention are proposed.

[0050] See also Figure 2 , which shows a main flow chart of an image processing method provided by an embodiment of the present invention, comprising the following steps:

[0051] S201: extracting video stream data transmitted by a preset target display device, locating a current frame, and determining a target focus area and / or a target focus object in the medical image data according to medical image data corresponding to the current frame.

[0052] In an embodiment, the image processing host 100 extracts the video stream data transmitted by the target display device 300, and can locate the current frame according to the video stream data, thereby obtaining the medical image data corresponding to the current frame, wherein the corresponding medical image data can be obtained in the video stream data, or the corresponding medical image data can be matched locally in the image processing host 100, and of course, a data request can be generated according to the current frame and sent to the target image source 200 to obtain the corresponding medical image data. Afterwards, the image processing host 100 identifies and determines the target area of ​​interest and / or the target object of interest in the corresponding medical image data according to the needs, wherein the target object of interest is the physiological tissue or lesion tissue specified in the medical image data, and the target area of ​​interest is the specific image area specified in the medical image data.

[0053] In a further embodiment, step S201 can perform different positioning methods according to different types of video stream data. The specific implementation process includes: receiving video stream data transmitted by a preset target display device, determining whether the type of the video stream data is fixed frame, and if so, directly using the video stream data as the current frame, otherwise locating the current frame in the video stream data.

[0054] In some further embodiments, step S201 determines the target area of ​​interest and / or target object of interest in the medical image data based on the medical image data corresponding to the current frame. The specific implementation process includes: obtaining the image identifier of the current frame, determining the corresponding medical image data in the medical image data transmitted by the preset target image source 200, and responding to the image processing event for the medical image data, locating the target area of ​​interest and / or target object of interest corresponding to the image processing event.

[0055] Furthermore, by responding to an image processing event for medical image data, an image selection operation triggered by a user for medical image data is obtained, the selection position information of the image selection operation is read, and the target area of ​​interest and / or target object of interest corresponding to the image processing event in the medical image data is located. The image selection operation may be an operation instruction for a user to select medical image data in a display area of ​​the medical image data. In addition, the image processing event may not only be an image selection operation, but may also be an automatically triggered preset recognition model, for example: any one or more image recognition models in a lesion recognition model, a medical information association table, a medical reconstruction model, etc., to automatically locate the target area of ​​interest and / or target object of interest in the medical image data where lesions may exist through a preset recognition strategy in the recognition model.

[0056] S202: configuring a query dimension of the target region of interest and / or the target object of interest, retrieving historical frames corresponding to the target region of interest and / or the target object of interest according to the query dimension, and acquiring medical image data corresponding to the historical frames.

[0057] In an embodiment, there may be different query dimensions, and one or more query dimensions of the target area of ​​interest and / or target object of interest may be selected for configuration according to actual needs, so as to retrieve corresponding historical frames through the configured query dimensions to obtain corresponding medical imaging data.

[0058] In a further embodiment, when configuring the query dimension of the target region of interest and / or the target object of interest in step S202, the query dimension list can be matched according to the type of the target region of interest and / or the target object of interest, so as to select the query dimension in the query dimension list, wherein each query dimension can be stored in the form of a parameter array, and the specific implementation process includes: identifying the type of the target region of interest and / or the target object of interest, obtaining the query dimension list corresponding to the type, and in response to the selection of the query dimension in the query dimension list, calling the parameter array of the query dimension, and using the parameter array as the query dimension configuration information of the target region of interest and / or the target object of interest. That is, different types of target regions of interest and / or target objects of interest correspond to different query dimension lists, and each query dimension list has at least one query dimension stored in the form of a parameter array for selection, and for example: the query dimension list of the configured organ includes query dimensions of signs, degrees, measurements, etc.

[0059] In some other further embodiments, step S202 retrieves the historical frames corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and the implementation process of obtaining the medical image data corresponding to the historical frames may include: retrieving the historical frames of the target area of ​​interest and / or the target object of interest, determining the target historical frame by matching the query dimension with the characteristic attribute information of the historical frame, and then obtaining the medical image data corresponding to the target historical frame from the local image processing host 100 according to the image identifier of the target historical frame, or generating a request according to the image identifier of the target historical frame and sending it to the target image source 200 to obtain the medical image data corresponding to the target historical frame.

[0060] In a further embodiment, it is possible to determine whether the image identifier of the target historical frame exists in the download list of the local image processing host 100 based on the image identifier of the target historical frame. If so, the medical image data corresponding to the target historical frame is obtained. Otherwise, a data request is generated based on the image identifier of the target historical frame and sent to the preset target image source 200, thereby receiving the medical image data corresponding to the target historical frame.

[0061] S203: calling a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0062] In an embodiment, the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame can be input into the called analysis model for processing, so as to obtain medical auxiliary information. In a further embodiment, step S203 can determine the association between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, so as to match the corresponding analysis model, and then call the analysis model to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame. It can be seen that different associations between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame correspond to different analysis models that can generate medical auxiliary information.

[0063] In a further embodiment, the specific implementation process of determining the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame and then matching the corresponding analysis model may include: extracting the attribute information and corresponding values ​​of the medical image data corresponding to the current frame, the attribute information and corresponding values ​​of the medical image data corresponding to the historical frame, respectively, and then calling a preset relationship calculation model to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model. Example: The relationship calculation model can be a Pearson correlation coefficient model, a Spearman correlation coefficient model, a multivariate linear regression model, a cosine similarity model, etc., and the association relationship is obtained by performing a degree judgment (such as grade classification, threshold comparison, etc.) on the result of the relationship calculation model. The association relationship can be the same attribute information, completely different attribute information, similar attribute information, etc.

[0064] It is worth noting that, according to the association relationship, the corresponding analysis model class can be matched first, and then the corresponding analysis model in the analysis model class can be determined based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, so that the analysis model can be used to extract some lesion information (for example: morphology, size, boundary conditions, etc.) from the medical image data corresponding to the current frame, and generate medical auxiliary information in combination with the medical image data corresponding to the historical frame. For example: the association relationship is the same attribute information, and the reconstruction analysis model class is matched, and then the type of medical image data corresponding to the current frame and the type of medical image data corresponding to the historical frame can be adapted to the 3D reconstruction model, surface unfolding reconstruction model, etc. in the reconstruction analysis model class, that is, the 3D image or unfolded image is reconstructed according to the current frame value and the historical frame value corresponding to the same attribute information. Also exemplified: the association relationship is that the attribute information is completely different, and the additional analysis model class is obtained by matching. Then, the type of medical imaging data corresponding to the current frame and the type of medical imaging data corresponding to the historical frame can be adapted to the attribute coordinate axis model and the attribute dimension column model in the additional analysis model class, that is, the coordinate axis and its value or the attribute dimension column and its value of the medical imaging data of the historical frame are added to the current frame medical imaging data display diagram or display table.

[0065] S204: Transmitting the medical auxiliary information to a target display device and presenting it.

[0066] In an embodiment, the image processing host 100 transmits the medical auxiliary information to the target display device 300 so that the target display device 300 presents the medical auxiliary information.

[0067] In a further embodiment, before step S204, the target display position of the medical auxiliary information may be determined in response to the determination of the data type of the video stream displaying the medical auxiliary information. In a further embodiment, it may be determined whether the data type of the video stream displaying the medical auxiliary information is fixed frame, and if so, the display position of the target focus area and / or the target focus object is located in the current frame, and the target display position of the medical auxiliary information is determined in combination with the display mode of the medical auxiliary information, and if otherwise, the target display position of the medical auxiliary information is configured according to the display mode of the medical auxiliary information.

[0068] Preferred embodiment: If the data type of the video stream displaying the medical auxiliary information is fixed frame, the display area of ​​the medical auxiliary information is determined by the display position of the target area of ​​interest and / or the target object of interest, and then the target display position is determined in the display area in combination with the display mode of the medical auxiliary information. Example: The display area of ​​the medical auxiliary information is determined according to a preset radius and centered on the display position of the target area of ​​interest and / or the target object of interest, and the display mode of the medical auxiliary information is an analysis curve graph, then the middle position of the right half of the display area can be used as the target display position. Also example: The boundary of the target area of ​​interest and / or the target object of interest is used as the boundary of the display area, and the display mode of the medical auxiliary information is a 3D image, then the display area can be used as the target display position to present a 3D image.

[0069] Some other preferred embodiments: If the data type of the video stream displaying the medical auxiliary information is non-fixed frame, the target display position can be directly configured for the medical auxiliary information. For example: a specific position area is set for the page, such as the four corner areas, the border area, etc. If the display mode of the medical auxiliary information is a data list, the border area is used as the target display position; if the display mode of the medical auxiliary information is an image, the four corner areas are used as the target display position.

[0070] See also Figure 3 , shows a schematic diagram of an optional process of determining a target region of interest and / or a target object of interest in medical image data according to an embodiment of the present invention, comprising the following steps:

[0071] S301: receiving video stream data transmitted by a preset target display device, and determining whether the type of the video stream data is fixed frame, if so, proceeding to step S302, otherwise, proceeding to step S303.

[0072] S302: Take the video stream data as the current frame and proceed to step S304.

[0073] S303: Locate the current frame in the video stream data, and proceed to step S304.

[0074] S304: Obtain an image identifier of the current frame, and determine corresponding medical image data in the medical image data transmitted by a preset target image source.

[0075] S305: In response to an image processing event for the medical image data, locate a target area of ​​interest and / or a target object of interest corresponding to the image processing event.

[0076] See also Figure 4 , shows a schematic diagram of an optional process of obtaining historical frame medical image data according to an embodiment of the present invention, including the following steps:

[0077] S401: Identify the type of the target focus area and / or the target focus object, and obtain a query dimension list corresponding to the type.

[0078] S402: In response to a selection of a query dimension in the query dimension list, a parameter array of the query dimension is called, and the parameter array is used as query dimension configuration information of a target focus area and / or a target focus object.

[0079] S403: Retrieve the historical frames of the target focus area and / or the target focus object, and determine the target historical frame by matching the query dimension with the characteristic attribute information of the historical frame.

[0080] S404: According to the image identifier of the target historical frame, determine whether the image identifier exists in the local download list, if yes, proceed to step S405, if not, proceed to step S406.

[0081] S405: Acquire medical image data corresponding to the target historical frame.

[0082] S406: Generate a data request according to the image identifier of the target historical frame and send it to a preset target image source, thereby receiving the medical image data corresponding to the target historical frame.

[0083] See also Figure 5 , shows a schematic diagram of an optional process of generating and displaying medical auxiliary information according to an embodiment of the present invention, comprising the following steps:

[0084] S501: extracting attribute information and corresponding values ​​of the medical image data corresponding to the current frame, and attribute information and corresponding values ​​of the medical image data corresponding to the historical frame respectively.

[0085] S502: calling a preset relationship calculation model to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0086] S503: According to the association relationship, a corresponding analysis model class is matched.

[0087] S504: Determine a corresponding analysis model in the analysis model class based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0088] S505: calling the analysis model to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frames.

[0089] S506: In response to the determination of the data type of the video stream displaying the medical auxiliary information, determine whether the data type of the video stream displaying the medical auxiliary information is fixed frame, if so, proceed to step S507, if not, proceed to step S508.

[0090] S507: Locate the display position of the target area of ​​interest and / or the target object of interest in the current frame, determine the target display position of the medical auxiliary information in combination with the display mode of the medical auxiliary information, and proceed to step S509.

[0091] S508: According to the display mode of the medical auxiliary information, configure the target display position of the medical auxiliary information, and proceed to step S509.

[0092] S509: Transmit the medical auxiliary information to the target display device to be presented at the target display position.

[0093] See also Figure 6 , which shows a schematic diagram of main modules of an image processing device 600 provided by an embodiment of the present invention, including:

[0094] The acquisition module 601 is configured to extract the video stream data transmitted by the preset target display device, locate the current frame, and determine the target focus area and / or target focus object in the medical image data according to the medical image data corresponding to the current frame;

[0095] The query module 602 is configured to configure a query dimension of the target region of interest and / or the target object of interest, retrieve a historical frame corresponding to the target region of interest and / or the target object of interest according to the query dimension, and obtain medical image data corresponding to the historical frame;

[0096] A generating module 603 is configured to call a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame;

[0097] The display module 604 is configured to transmit the medical auxiliary information to a target display device for presentation.

[0098] In the implementation device of the present invention, the acquisition module 601 extracts the video stream data transmitted by the preset target display device, including: receiving the video stream data transmitted by the preset target display device, and determining whether the type of the video stream data is fixed frame; if so, taking the video stream data as the current frame, otherwise locating the current frame in the video stream data.

[0099] In the implementation device of the present invention, the acquisition module 601 determines the target area of ​​interest and / or target object of interest in the medical image data based on the medical image data corresponding to the current frame, including: acquiring the image identifier of the current frame, determining the corresponding medical image data in the medical image data transmitted by the preset target image source; and locating the target area of ​​interest and / or target object of interest corresponding to the image processing event in response to the image processing event for the medical image data.

[0100] In the implementation device of the present invention, the query module 602 configures the query dimensions of the target focus area and / or the target focus object, including: identifying the type of the target focus area and / or the target focus object, and obtaining a query dimension list corresponding to the type; in response to the selection of the query dimension in the query dimension list, calling the parameter array of the query dimension, and using the parameter array as the query dimension configuration information of the target focus area and / or the target focus object.

[0101] In the implementation device of the present invention, the query module 602 retrieves the historical frames corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and obtains the medical image data corresponding to the historical frames, including: retrieving the historical frames of the target area of ​​interest and / or the target object of interest, and determining the target historical frame by matching the query dimension with the feature attribute information of the historical frame; and obtaining the medical image data corresponding to the target historical frame according to the image identifier of the target historical frame.

[0102] In the implementation device of the present invention, the query module 602 obtains the medical image data corresponding to the target historical frame according to the image identifier of the target historical frame, including: judging whether the image identifier exists in the local download list according to the image identifier of the target historical frame; if so, obtaining the medical image data corresponding to the target historical frame; if not, generating a data request according to the image identifier of the target historical frame and sending it to a preset target image source, and then receiving the medical image data corresponding to the target historical frame.

[0103] In the implementation device of the present invention, the generation module 603 calls a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, including: determining the association between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model; calling the analysis model to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0104] In the implementation device of the present invention, the generation module 603 determines the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matches the corresponding analysis model, including: extracting the attribute information and corresponding values ​​of the medical image data corresponding to the current frame, and the attribute information and corresponding values ​​of the medical image data corresponding to the historical frame; calling a preset relationship calculation model to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis model.

[0105] In the implementation device of the present invention, the generation module 603 matches the corresponding analysis model, including: matching to obtain the corresponding analysis model class according to the association relationship; determining the corresponding analysis model in the analysis model class based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame.

[0106] In the implementation device of the present invention, before the display module 604 transmits the medical auxiliary information to the target display device, it includes: determining the target display position of the medical auxiliary information in response to determining the data type of the video stream displaying the medical auxiliary information.

[0107] In the implementation device of the present invention, the display module 604 is further configured to determine whether the data type of the video stream displaying the medical auxiliary information is fixed frame; if so, locate the display position of the target focus area and / or the target focus object in the current frame, and determine the target display position of the medical auxiliary information in combination with the display mode of the medical auxiliary information; if not, configure the target display position of the medical auxiliary information according to the display mode of the medical auxiliary information.

[0108] In addition, the specific implementation content of the device described in the embodiment of the present invention has been described in detail in the method described above, so the repeated content will not be described again here.

[0109] Figure 7 An exemplary system architecture 700 to which embodiments of the present invention may be applied is shown, including terminal devices 701 , 702 , 703 , a network 704 and a server 705 (only an example).

[0110] Terminal devices 701, 702, 703 can be various electronic devices with display screens and supporting web browsing, and various communication client applications are installed. Users can use terminal devices 701, 702, 703 to interact with server 705 through network 704 to receive or send messages, etc.

[0111] The network 704 is used to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0112] The server 705 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using the terminal devices 701, 702, and 703 (only as an example). The background management server may analyze and process the received information query request and other data, and feed back the processing results (such as target push information, product information - only as an example) to the terminal device. It should be noted that the method provided in the embodiment of the present invention is generally executed by the server 705, and accordingly, the device is generally set in the server 705.

[0113] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0114] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0115] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0116] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0117] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0118] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0119] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be arranged in a processor, for example, it may be described as: a processor includes an acquisition module, a query module, a generation module and a display module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the generation module may also be described as a determination module or a processing module, etc.

[0121] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes any of the above image processing methods.

[0122] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Extracting video stream data transmitted by a preset target display device, locating a current frame, and determining a target area of ​​interest and / or a target object of interest in the medical image data according to medical image data corresponding to the current frame; Configuring a query dimension for the target area of ​​interest and / or the target object of interest, retrieving a historical frame corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and acquiring medical image data corresponding to the historical frame; Calling a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame; The medical auxiliary information is transmitted to a target display device and presented.

2. The method according to claim 1, characterized in that Extract the video stream data transmitted by the preset target display device, including: Receiving video stream data transmitted by a preset target display device, and determining whether the type of the video stream data is fixed frame; If so, the video stream data is used as the current frame, if not, the current frame is located in the video stream data.

3. The method according to claim 1, characterized in that Determining a target area of ​​interest and / or a target object of interest in the medical image data according to the medical image data corresponding to the current frame includes: Obtaining an image identifier of the current frame, and determining corresponding medical image data in the medical image data transmitted by a preset target image source; In response to an image processing event for the medical image data, a target area of ​​interest and / or a target object of interest corresponding to the image processing event is located.

4. The method according to claim 1, characterized in that: Configure the query dimensions of the target area of ​​interest and / or target object of interest, including: Identify the type of the target area of ​​interest and / or the target object of interest, and obtain a query dimension list corresponding to the type; In response to selection of a query dimension in the query dimension list, a parameter array of the query dimension is called, and the parameter array is used as query dimension configuration information of a target focus area and / or a target focus object.

5. The method according to claim 1, characterized in that Retrieving historical frames corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and acquiring medical image data corresponding to the historical frames, including: Retrieving a historical frame of a target focused area and / or a target focused object, and determining a target historical frame by matching the query dimension with feature attribute information of the historical frame; According to the image identifier of the target historical frame, medical image data corresponding to the target historical frame is obtained.

6. The method according to claim 5, characterized in that Acquiring medical image data corresponding to the target historical frame according to the image identifier of the target historical frame includes: According to the image identifier of the target historical frame, determining whether the image identifier exists in the local download list; If yes, the medical image data corresponding to the target historical frame is obtained; if no, a data request is generated according to the image identifier of the target historical frame and sent to a preset target image source, thereby receiving the medical image data corresponding to the target historical frame.

7. The method according to claim 1, characterized in that Calling a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, including: Determine the correlation between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and match the corresponding analysis model; The analysis model is called to generate medical auxiliary information by calculating the medical image data corresponding to the current frame and the medical image data corresponding to the historical frames.

8. The method according to claim 7, characterized in that Determining the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and matching the corresponding analysis models, including: Respectively extracting the attribute information and corresponding values ​​of the medical image data corresponding to the current frame, and the attribute information and corresponding values ​​of the medical image data corresponding to the historical frame; The preset relationship calculation model is called to obtain the association relationship between the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, and the corresponding analysis model is matched.

9. The method according to claim 8, characterized in that Match the corresponding analysis model, including: According to the association relationship, a corresponding analysis model class is matched; Based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame, a corresponding analysis model in the analysis model class is determined.

10. The method according to any one of claims 1 to 9, characterized in that: Before transmitting the medical auxiliary information to the target display device, the method includes: In response to determining the data type of the video stream displaying the medical auxiliary information, a target display position of the medical auxiliary information is determined.

11. The method according to claim 10, characterized in that Also includes: Determine whether the data type of the video stream displaying the medical auxiliary information is fixed frame; If yes, locating the display position of the target attention area and / or the target attention object in the current frame, and determining the target display position of the medical auxiliary information in combination with the display mode of the medical auxiliary information; If not, the target display position of the medical auxiliary information is configured according to the display mode of the medical auxiliary information.

12. An image processing device, characterized in that: include: An acquisition module is configured to extract video stream data transmitted by a preset target display device, locate a current frame, and determine a target area of ​​interest and / or a target object of interest in the medical image data according to the medical image data corresponding to the current frame; A query module is configured to configure a query dimension of the target area of ​​interest and / or the target object of interest, retrieve a historical frame corresponding to the target area of ​​interest and / or the target object of interest according to the query dimension, and obtain medical image data corresponding to the historical frame; A generating module is configured to call a preset analysis model to generate medical auxiliary information based on the medical image data corresponding to the current frame and the medical image data corresponding to the historical frame; The display module is configured to transmit the medical auxiliary information to a target display device for presentation.

13. An image processing device, characterized in that: The image processing device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the steps of the image processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the image processing method according to any one of claims 1 to 11.

Images