Method, device, electronic device and storage medium for determining image information

Abstract

The embodiments of the present invention disclose a method, device, electronic device and storage medium for determining image information. The method includes: obtaining a medical image to be processed, extracting the corresponding time series image data in the medical image; inputting the time series image data into a pre-trained image recognition model to determine the time series image information of the time series image data; wherein the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network. In the technical solution of the embodiments of the present invention, the convolutional neural network can determine the graphic features of the time series image data, and the recurrent neural network can determine the time series features of the time series image data, thereby comprehensively determining the time series image information by combining the graphic features and the time series features, thereby improving the accuracy and effectiveness of the determined time series image information.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular, to a method, device, electronic device and storage medium for determining image information. Background Art

[0002] With the rapid development of computer science and technology, analyzing and classifying image information has become a common operation in various fields. Taking the medical field as an example, it is often necessary to use imaging equipment to obtain images (such as ultrasound imaging, electronic computer tomography imaging, magnetic resonance imaging, etc.), analyze and interpret the acquired images to assist medical staff in making accurate judgments.

[0003] In the prior art, a convolutional neural network model is usually used for image recognition to determine the image information expressed in the image. However, the convolutional neural network model can extract the graphic feature information in the image, and only determines the information reflected by the image in practical applications through the graphic feature information, which cannot fully and richly reflect the image information expressed by the image, thereby reducing the accuracy and effectiveness of the determined image information. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for determining image information, so as to comprehensively determine time-series image information by combining graphic features and time-series features, thereby improving the accuracy and effectiveness of the determined time-series image information.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining image information, including:

[0006] Acquire a medical image to be processed, and extract corresponding time-series image data from the medical image;

[0007] Inputting the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data;

[0008] Among them, the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network.

[0009] In a second aspect, an embodiment of the present invention further provides a device for determining image information, the device comprising:

[0010] A medical image acquisition module is used to acquire the medical image to be processed and extract the corresponding time series image data in the medical image;

[0011] A time-series image information determination module is used to input the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data;

[0012] Among them, the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining image information provided by any embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining image information provided by any embodiment of the present invention.

[0018] A method for determining image information provided by an embodiment of the present invention obtains a medical image to be processed and extracts corresponding time series image data from the medical image; the time series image data is input into a pre-trained image recognition model to determine the time series image information of the time series image data, wherein the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network, the convolutional neural network can determine the graphic features of the time series image data, and the recurrent neural network can determine the time series features of the time series image data, thereby comprehensively determining the time series image information by combining the graphic features and the time series features, thereby improving the accuracy and effectiveness of the determined time series image information.

[0019] In addition, an image information determination device, electronic device and storage medium provided by the present invention correspond to the above method and have the same beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a method for determining image information provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of another method for determining image information provided by an embodiment of the present invention;

[0023] Figure 3A structural diagram of a device for determining image information provided by an embodiment of the present invention;

[0024] Figure 4 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0026] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0028] Embodiment 1

[0029] Figure 1 A flowchart of a method for determining image information provided in an embodiment of the present invention. The method can be executed by an image information determination device, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the method for determining image information in an embodiment of the present invention.

[0030] like Figure 1 As shown, the method of this embodiment may specifically include:

[0031] S101, obtaining a medical image to be processed, and extracting corresponding time series image data in the medical image.

[0032] In a specific implementation, a communication connection can be established with a medical imaging device to obtain medical images collected by the medical imaging device. Medical imaging devices include ultrasound imaging devices, electronic computer tomography devices, and nuclear magnetic resonance devices. Medical images are images collected by corresponding medical imaging devices, such as electrocardiograms, chest X-rays, etc.

[0033] Specifically, medical images can be acquired from medical imaging devices at regular intervals according to a preset transmission cycle; or, according to actual application requirements, acquisition requests can be sent to connected medical imaging devices in real time to acquire medical images captured by the medical imaging devices, which is not limited to this embodiment of the present invention.

[0034] Further, after obtaining the medical image to be processed, the medical image can be processed. It should be noted that the medical image is usually a time-series image, and the corresponding time-series image data in the time-series image can be extracted. Exemplarily, the medical image can be an electrocardiogram, and the time-series image data corresponding to the electrocardiogram is extracted. The time-series image data may include the heartbeat amplitude value at each moment in the electrocardiogram.

[0035] S102: Input the time series image data into a pre-trained image recognition model to determine the time series image information of the time series image data.

[0036] The image recognition model is based on a combination of a convolutional neural network and a recurrent neural network. The convolutional neural network part of the image recognition model can determine the graphic features of the time series image data, and the recurrent neural network part of the image recognition model can determine the time series features of the time series image data. Combining the graphic features and the time series features, the image recognition model determines the time series image information corresponding to the time series image data.

[0037] Exemplarily, the image recognition model can be trained based on a model composed of a partial structure in a convolutional neural network in front and a recurrent neural network in connection, so that the trained image recognition model can first determine the graphic features and then determine the time series features; the image recognition model can also be trained based on a model composed of a partial structure in a convolutional neural network in front and a recurrent neural network in connection, so that the trained image recognition model can optionally determine the time series features and then determine the graphic features; the time series image information corresponding to the time series image data is determined by both time series features and graphic features.

[0038] A method for determining image information provided by an embodiment of the present invention obtains a medical image to be processed and extracts corresponding time series image data from the medical image; the time series image data is input into a pre-trained image recognition model to determine the time series image information of the time series image data, wherein the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network, the convolutional neural network can determine the graphic features of the time series image data, and the recurrent neural network can determine the time series features of the time series image data, thereby comprehensively determining the time series image information by combining the graphic features and the time series features, thereby improving the accuracy and effectiveness of the determined time series image information.

[0039] Embodiment 2

[0040] Figure 2A flowchart of another method for determining image information provided by an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. Optionally, before inputting the time series image data into a pre-trained image recognition model, it also includes: for each first time series sample data, inputting the first time series sample data into the convolutional neural network to be trained, and correcting the first network parameter in the convolutional neural network to be trained based on the first output result to generate a target convolutional neural network; determining the second time series sample data based on the first time series sample data and the target convolutional neural network; inputting the second time series sample data into the recurrent neural network to be trained, and correcting the second network parameter in the recurrent neural network to be trained based on the second output result of the recurrent neural network to be trained to generate a target recurrent neural network; generating an image recognition model based on the target convolutional neural network and the target recurrent neural network. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0041] like Figure 2 As shown, the method of this embodiment may specifically include:

[0042] S201. For each first time series sample data, the first time series sample data is input into the convolutional neural network to be trained, and the first network parameter in the convolutional neural network to be trained is corrected based on the first output result to generate a target convolutional neural network.

[0043] In a specific implementation, multiple time series sample data may be acquired in advance. Exemplarily, when the image recognition model identifies the image information used to determine the electrocardiogram, the electrocardiogram sample data may be acquired, and the electrocardiogram sample data may consist of two parts, one part representing the data of the heartbeat amplitude, and the other part being the annotated heartbeat type label. For example, the electrocardiogram sample data is divided into 188 fields, the first 187 fields represent the heartbeat amplitude at each moment, and the last field represents the heartbeat type label.

[0044] Optionally, before inputting the first time series sample data into the convolutional neural network to be trained for each first time series sample data, the method further includes: performing data preprocessing on each first time series sample data according to a preset length requirement.

[0045] It should be noted that when training a convolutional neural network, the data lengths of multiple first time series sample data may be different. In order to improve the accuracy and effectiveness of the training results, each first time series training sample data may be preprocessed.

[0046] Exemplarily, the data length requirement may be preset, the minimum length value in each first time series sample data may be determined, and the length requirement is set such that the data length of each first time series sample data must be equal to the minimum length value. The maximum length value in each first time series sample data may also be determined, and the length requirement is set such that the data length of each first time series sample data must be equal to the maximum length value; or, the average value of the data length of each first time series sample data may be determined, and the length requirement is set such that the data length of each first time series sample data must be equal to the average value. Those skilled in the art may preset the length requirement according to the actual application situation, and the embodiments of the present invention are not limited to this.

[0047] Optionally, each first time series sample data is subjected to data preprocessing according to a preset length requirement, including: when the data length of the first time series image data is greater than the required length in the preset length requirement, determining the first length quantity of the first time series image data exceeding the required length, selecting first data points with the same number as the first length in the time series image data in an evenly spaced manner, and deleting the first data points; or, when the data length of the first time series image data is less than the required length in the preset length requirement, determining the second length quantity of the first time series image data less than the required length; selecting second data points with the second length quantity in the time series image data, and inserting the second data points into the time series image data.

[0048] Specifically, in the case where the data length of the first time-series image data is greater than the required length in the preset length requirement, the data length of the first time-series image data can be made to meet the preset length requirement by deleting data in the first time-series image data. Specifically, the first data point can be determined in the first time-series image data in an evenly spaced manner, or the first data point of the first length in the first time-series image data can be deleted as the first data point, or the last data point of the first length in the first time-series image data can be deleted as the first data point, which is not limited in the embodiment of the present invention.

[0049] Furthermore, when the data length of the first time-series image data is less than the required length in the preset length requirement, it is necessary to insert a second data point into the first time-series image data. The first second-length number of data points in the first time-series image data can be used as the second data points, or the last second-length number of data points in the first time-series image data can be used as the second data points, and the second-length number of second data points can also be selected from the first time-series image data.

[0050] In a specific implementation, the convolutional neural network to be trained consists of 1 convolutional layer, 5 residual network units, 1 flattening layer and 3 fully connected layers. The number of filters in the convolutional layer is 32, and the convolution kernel size is 5. The first time series sample data can be input into the convolutional neural network to be trained, and the convolutional neural network to be trained outputs a first output result. Exemplarily, if the first time series image data is electrocardiogram data, the first output result is the training heartbeat type reflected by the first time series image data, and the training heartbeat type is compared with the heartbeat type identifier marked in the first time series sample data, and the first network parameter of the convolutional neural network to be trained is adjusted based on the comparison result; each first time series sample data is traversed, and finally a target convolutional neural network is generated.

[0051] S202. Determine second time series sample data based on the first time series sample data and the target convolutional neural network.

[0052] In a specific implementation, the second time series sample data is determined based on the first time series sample data and the target convolutional neural network, including: inputting the first time series sample data into the target convolutional neural network, extracting the output data obtained after the first time series sample data passes through the first fully connected layer of the target convolutional neural network; and determining the output data as the second time series sample data.

[0053] Specifically, the output data of the first fully connected layers of the target convolutional neural network can be determined as the second time series sample data for training the recurrent neural network to be trained. It should be noted that those skilled in the art can also select the second time series sample data according to the actual application situation, and this embodiment of the present invention is not limited to this.

[0054] S203, inputting the second time series sample data into the recurrent neural network to be trained, and modifying the second network parameter in the recurrent neural network to be trained based on the second output result of the recurrent neural network to be trained, so as to generate a target recurrent neural network.

[0055] In this embodiment, the recurrent neural network to be trained includes 2 recurrent network units and 2 fully connected layers, wherein the recurrent network unit is a bidirectional recurrent network layer composed of a gated recurrent unit layer. Exemplarily, if the first time series image data is electrocardiogram data, there is a field representing a heartbeat type identifier in the second time series sample data, and the second output result can be compared with the heartbeat type identifier, and the second network parameters can be adjusted based on the comparison result, and each second time series image data is traversed, and the second network parameters are corrected based on each second time series image data, and finally a target recurrent neural network is generated.

[0056] S204: Generate an image recognition model based on the target convolutional neural network and the target recurrent neural network.

[0057] In a specific implementation, the network structure of the target convolutional neural network can be spliced ​​with the target recurrent neural network to generate an image recognition model. Optionally, based on the target convolutional neural network and the target recurrent neural network, an image recognition model is generated, including: extracting the convolutional layer, five residual network units, flattening layer and the first fully connected layer in the target convolutional neural network to form a target convolutional neural sub-network; connecting the target convolutional neural sub-network with the target recurrent neural network to form an image recognition model.

[0058] Specifically, in an image recognition model composed of a target convolutional neural sub-network and a target recurrent neural network, after the time series image data is first input into the target convolutional neural sub-network, the graphic features of the time series image data can be determined. The output result of the fully connected layer of the target convolutional neural sub-network contains graphic feature information of the time series image data. The fully connected layer is connected to the target recurrent neural network, and the output result of the fully connected layer can be used as input data of the target recurrent neural network. The target recurrent neural network determines the time series features of the input data, and finally outputs time series image information that can reflect both graphic features and time series features.

[0059] Optionally, it also includes: obtaining each time series test data in a pre-established test set, inputting each of the time series test data into a trained image recognition model, and outputting test image information; based on the test image information and the image type identifier of the pre-stored time series test data, determining the recognition accuracy of the image recognition model, and feeding back the recognition accuracy to the user terminal.

[0060] In a specific implementation, a test set can be constructed to complete the testing of the image recognition model. Exemplarily, historical time series image data can be acquired in advance, and the historical time series image data can be divided into time series sample data and time series test data according to a preset ratio, and the preset ratio can be 8:2. The test set is composed of time series test data, and each of the time series test data is input into the trained image recognition model, and the test image information is output. Furthermore, the time series test data contains an image type identifier, and the output test image information is compared with the image type identifier. When the two are consistent, it means that the image recognition model recognizes correctly. If the two are inconsistent, it means that the image recognition model recognizes incorrectly. Based on the comparison results of each time, the recognition accuracy of the image recognition model is determined, and the recognition accuracy is fed back to the user terminal, so that the user can understand the accuracy of the image recognition model.

[0061] S205: Acquire a medical image to be processed, and extract corresponding time series image data from the medical image.

[0062] S206: Input the time series image data into a pre-trained image recognition model to determine the time series image information of the time series image data.

[0063] In the embodiment of the present invention, the target convolutional neural subnetwork is connected to the target recurrent neural network to form an image recognition model. The target convolutional neural subnetwork obtains the graphic features of the time series image data without destroying the time series relationship of the time series image data, and the target recurrent neural network extracts the time series features, thereby comprehensively determining the time series image information by combining the graphic features and the time series features. In addition, in the process of training the image recognition model, the recognition accuracy of the image recognition model is tested, thereby improving the accuracy and effectiveness of the image recognition model in determining the time series image information.

[0064] Embodiment 3

[0065] Figure 3 This is a structural diagram of a device for determining image information provided in an embodiment of the present invention, and the device is used to execute the method for determining image information provided in any of the above embodiments. The device and the method for determining image information in the above embodiments belong to the same inventive concept, and the details not fully described in the embodiment of the device for determining image information can refer to the embodiment of the method for determining image information. The device may specifically include:

[0066] A medical image acquisition module 10 is used to acquire a medical image to be processed and extract corresponding time series image data from the medical image;

[0067] A time-series image information determination module 11 is used to input the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data;

[0068] Among them, the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network.

[0069] Based on any optional technical solution in the embodiments of the present invention, optionally, the following further includes:

[0070] Generate an image recognition model, which is used for, before inputting the time series image data into the pre-trained image recognition model, for each first time series sample data, inputting the first time series sample data into a convolutional neural network to be trained, and modifying a first network parameter in the convolutional neural network to be trained based on a first output result to generate a target convolutional neural network; determining second time series sample data based on the first time series sample data and the target convolutional neural network; inputting the second time series sample data into a recurrent neural network to be trained, and modifying a second network parameter in the recurrent neural network to be trained based on a second output result output by the recurrent neural network to be trained to generate a target recurrent neural network; and generating the image recognition model based on the target convolutional neural network and the target recurrent neural network.

[0071] Based on any optional technical solution in the embodiments of the present invention, optionally, generating an image recognition model includes:

[0072] Determine a second time series sample data unit, which is used to input the first time series sample data into the target convolutional neural network, extract the output data obtained after the first time series sample data passes through the first fully connected layer of the target convolutional neural network; and determine the output data as the second time series sample data.

[0073] Based on any optional technical solution in the embodiments of the present invention, optionally, generating an image recognition model further includes:

[0074] The output test image information unit is used to obtain each time series test data in a pre-established test set, input each time series test data into the trained image recognition model, and output the test image information; based on the test image information and the image type identification of the pre-stored time series test data, determine the recognition accuracy of the image recognition model, and feed back the recognition accuracy to the user terminal.

[0075] Based on any optional technical solution in the embodiments of the present invention, optionally, generating an image recognition model includes:

[0076] A target convolutional neural sub-network unit is formed to extract the convolutional layer, five residual network units, the flattening layer and the first fully connected layer in the target convolutional neural network to form a target convolutional neural sub-network; the target convolutional neural sub-network is connected to the target recurrent neural network to form the image recognition model.

[0077] Based on any optional technical solution in the embodiments of the present invention, optionally, generating an image recognition model further includes:

[0078] A preprocessing unit is used to perform data preprocessing on each first time series sample data according to a preset length requirement before inputting the first time series sample data into the convolutional neural network to be trained.

[0079] Based on any optional technical solution in the embodiments of the present invention, optionally, the preprocessing unit includes:

[0080] A preprocessing subunit is used to determine, when the data length of the first time-series image data is greater than the required length in the preset length requirement, the first length quantity by which the first time-series image data exceeds the required length, select first data points of the same length quantity from the first time-series image data in an evenly spaced manner, and delete the first data points; or, when the data length of the first time-series image data is less than the required length in the preset length requirement, determine the second length quantity by which the first time-series image data is less than the required length; select second data points of the second length quantity from the time-series image data, and insert the second data points into the first time-series image data.

[0081] The apparatus for determining image information provided by the embodiment of the present invention can execute the following method: obtaining a medical image to be processed, extracting the corresponding time series image data in the medical image; inputting the time series image data into a pre-trained image recognition model, and determining the time series image information of the time series image data; wherein the image recognition model is composed based on a combination of a convolutional neural network and a recurrent neural network. The convolutional neural network can determine the graphic features of the time series image data, and the recurrent neural network can determine the time series features of the time series image data, thereby comprehensively determining the time series image information by combining the graphic features and the time series features, thereby improving the accuracy and effectiveness of the determined time series image information.

[0082] It is worth noting that in the embodiment of the above-mentioned image information determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0083] Embodiment 4

[0084] Figure 4 A structural diagram of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 20 suitable for implementing an embodiment of the present invention is shown. The electronic device 20 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0085] like Figure 4 As shown, the electronic device 20 is in the form of a general computing device. The components of the electronic device 20 may include but are not limited to: one or more processors or processing units 201, a system memory 202, and a bus 203 connecting different system components (including the system memory 202 and the processing unit 201).

[0086] Bus 203 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0087] The electronic device 20 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 20, including volatile and non-volatile media, removable and non-removable media.

[0088] The system memory 202 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 204 and / or cache memory 205. The electronic device 20 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 206 may be used to read and write non-removable, non-volatile magnetic media. A disk drive for reading and writing removable non-volatile disks (such as "floppy disks"), and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROM, DVD-ROM or other optical media) may be provided. In these cases, each drive may be connected to the bus 203 via one or more data medium interfaces. The memory 202 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0089] A program / utility 208 having a set (at least one) of program modules 207 may be stored, for example, in the memory 202, such program modules 207 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 207 generally perform the functions and / or methods of the embodiments described herein.

[0090] The electronic device 20 may also communicate with one or more external devices 209 (e.g., keyboards, pointing devices, displays 210, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 20, and / or may communicate with any device that enables the electronic device 20 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 211. Furthermore, the electronic device 20 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 212. As shown, the network adapter 212 communicates with other modules of the electronic device 20 via a bus 203. It should be understood that other hardware and / or software modules may be used in conjunction with the electronic device 20, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0091] The processing unit 201 executes various functional applications and data processing by running programs stored in the system memory 202 .

[0092] An electronic device provided by the present invention can implement the following method: obtaining a medical image to be processed, extracting the corresponding time-series image data in the medical image; inputting the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data; wherein the image recognition model is composed of a combination of a convolutional neural network and a recurrent neural network. The convolutional neural network can determine the graphic features of the time-series image data, and the recurrent neural network can determine the time-series features of the time-series image data, thereby comprehensively determining the time-series image information by combining the graphic features and the time-series features, thereby improving the accuracy and effectiveness of the determined time-series image information.

[0093] Embodiment 5

[0094] An embodiment of the present invention provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to perform a method for determining image information when executed by a computer processor, the method comprising:

[0095] Obtain a medical image to be processed, extract the corresponding time series image data in the medical image; input the time series image data into a pre-trained image recognition model to determine the time series image information of the time series image data; wherein the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network. The convolutional neural network can determine the graphic features of the time series image data, and the recurrent neural network can determine the time series features of the time series image data, thereby comprehensively determining the time series image information by combining the graphic features and the time series features, thereby improving the accuracy and effectiveness of the determined time series image information.

[0096] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, but can also execute related operations in the method for determining image information provided by any embodiment of the present invention.

[0097] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0098] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0099] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0100] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for determining image information, characterized in that: include: Acquire a medical image to be processed, and extract corresponding time-series image data from the medical image; Inputting the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data; Wherein, the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network; Before inputting the time series image data into the pre-trained image recognition model, the method further includes: For each first time series sample data, input the first time series sample data into a convolutional neural network to be trained, and modify a first network parameter in the convolutional neural network to be trained based on a first output result to generate a target convolutional neural network; Determining second time series sample data based on the first time series sample data and the target convolutional neural network; Inputting the second time series sample data into the recurrent neural network to be trained, and modifying the second network parameter in the recurrent neural network to be trained based on the second output result output by the recurrent neural network to be trained, so as to generate a target recurrent neural network; Extracting the convolutional layer, five residual network units, the flattening layer and the first fully connected layer in the target convolutional neural network to form a target convolutional neural sub-network; Connecting the target convolutional neural sub-network with the target recurrent neural network to form the image recognition model; wherein the fully connected layer of the target convolutional neural sub-network is connected to the target recurrent neural network; Acquire each time series test data in a pre-established test set, input each time series test data into a trained image recognition model, and output test image information; Based on the test image information and the image type identifier of the pre-stored time series test data, the recognition accuracy of the image recognition model is determined, and the recognition accuracy is fed back to the user terminal.

2. The method according to claim 1, characterized in that The determining the second time series sample data based on the first time series sample data and the target convolutional neural network includes: Inputting the first time series sample data into the target convolutional neural network, and extracting output data obtained after the first time series sample data passes through the first fully connected layer of the target convolutional neural network; The output data is determined as the second time series sample data.

3. The method according to claim 1, characterized in that Before inputting each first time series sample data into the convolutional neural network to be trained, the method further includes: Data preprocessing is performed on each first time series sample data according to a preset length requirement.

4. The method according to claim 3, characterized in that The data preprocessing of each first time series sample data according to a preset length requirement includes: When the data length of the first time-series image data is greater than the required length in the preset length requirement, determining the first length amount by which the first time-series image data exceeds the required length, selecting first data points having the same number as the first length from the first time-series image data in an evenly spaced manner, and deleting the first data points; or When the data length of the first time-series image data is less than the required length in the preset length requirement, determining that the first time-series image data is less than a second length amount of the required length; Select the second data points of the second length from the time-series image data, and insert the second data points into the first time-series image data.

5. A device for determining image information, characterized in that: include: A medical image acquisition module is used to acquire the medical image to be processed and extract the corresponding time series image data in the medical image; A time-series image information determination module, used for inputting the time-series image data into a pre-trained image recognition model to determine the time-series image information of the time-series image data; Wherein, the image recognition model is based on a combination of a convolutional neural network and a recurrent neural network; Generate an image recognition model, which is used for, before inputting the time series image data into the pre-trained image recognition model, for each first time series sample data, inputting the first time series sample data into the convolutional neural network to be trained, and modifying the first network parameter in the convolutional neural network to be trained based on the first output result to generate a target convolutional neural network; inputting the first time series sample data into the target convolutional neural network, extracting the output data obtained after the first time series sample data passes through the first fully connected layer of the target convolutional neural network; determining the output data as the second time series sample data; inputting the second time series sample data into the recurrent neural network to be trained, and based on the The second output result of the training recurrent neural network output is used to correct the second network parameter in the recurrent neural network to be trained to generate a target recurrent neural network; based on the target convolutional neural network and the target recurrent neural network, the image recognition model is generated; wherein, the generated image recognition model includes: forming a target convolutional neural sub-network unit, which is used to extract the convolutional layer, five residual network units, flattening layer and the first fully connected layer in the target convolutional neural network to form a target convolutional neural sub-network; connecting the target convolutional neural sub-network with the target recurrent neural network to form the image recognition model; wherein, the fully connected layer of the target convolutional neural sub-network is connected to the target recurrent neural network; The generating of the image recognition model further comprises: The output test image information unit is used to obtain each time series test data in a pre-established test set, input each time series test data into the trained image recognition model, and output the test image information; based on the test image information and the image type identification of the pre-stored time series test data, determine the recognition accuracy of the image recognition model, and feed back the recognition accuracy to the user terminal.

6. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining image information as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining image information as claimed in any one of claims 1 to 4 is implemented.

Images