A method and apparatus for object category recognition
By using a Long Short-Term Memory (LSTM) network to combine global and local features in chest X-ray image recognition, the problem that CNNs cannot effectively process global and local information is solved, achieving more accurate lesion and lesion identification, enhancing robustness to changes in image quality, and improving the early detection capability of lung disease diagnosis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 喀什地区第一人民医院
- Filing Date
- 2024-03-06
- Publication Date
- 2026-05-26
AI Technical Summary
In existing technologies, convolutional neural networks (CNNs) cannot effectively process global and local information in chest X-ray image recognition, resulting in low accuracy in identifying lesions and abnormalities in chest X-rays, and they are not robust enough to changes in image quality.
Long Short-Term Memory (LSTM) network is used as the image feature extraction network. By iteratively optimizing and combining global and local features, and utilizing the temporal unfolding characteristics and gating mechanism of LSTM, a fused image is generated for iterative optimization, thereby improving the accuracy and robustness of the recognition results.
It improves the accuracy of identifying lesions and diseased areas in chest X-ray images, enhances robustness to noise and image quality variations, and improves the early detection capability of lung disease diagnosis.
Smart Images
Figure CN122090103A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for object category recognition. Background Technology
[0002] Chest X-rays play an important role in the prevention and treatment of lung diseases. They can show abnormal shadows in the lungs, such as nodules, spots, and shadowy masses. These abnormal shadows may be manifestations of lesions. At the same time, chest X-rays can also observe whether the mediastinum (the area in the center of the lungs) and the pleura (the thin membrane covering the lungs) are affected by lesions.
[0003] With the development of artificial intelligence (AI) technology, especially its rapid advancements in natural image processing, the potential applications of related technologies in the medical field have become apparent. In the prevention and treatment of lung diseases, AI can provide beneficial support and assistance in chest X-ray analysis. For example, AI technology can analyze the structures and features in chest X-ray images and extract relevant information helpful for lung disease diagnosis, such as locating lesions, assessing the size and extent of lesions, and providing more objective image evaluations. Simultaneously, AI can process and analyze large amounts of chest X-ray data, learning and recognizing patterns and features related to lung diseases. Through deep learning and pattern recognition on large-scale data, the early detection and prediction capabilities of lung diseases can be improved, contributing to the development of more effective prevention and treatment strategies. Therefore, there is an urgent need for a method capable of identifying the presence of lesions and diseased areas in chest X-rays. Summary of the Invention
[0004] This application provides an object category recognition method and apparatus, which can achieve more accurate category recognition results corresponding to the predicted target part image, and can selectively ignore or store some input information, thereby improving robustness to noise and changes.
[0005] In a first aspect, this application provides an object category recognition method, which is applied to a category prediction model, the category prediction model comprising K image feature extraction networks, where K is a positive integer; the method includes:
[0006] Acquire images of the target area;
[0007] Using the K image feature extraction networks, K category recognition results corresponding to the target area image are obtained; wherein, for the first image feature extraction network, the target area image is input into the first image feature extraction network to obtain the first category recognition result and the first radiographic feature map corresponding to the target area image; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target area image, and the fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target area image, where K≥i≥2;
[0008] Based on the K category recognition results corresponding to the target area image, the category corresponding to the target area image is determined.
[0009] Secondly, this application provides an object category recognition device, which is applied to a category prediction model, the category prediction model including K image feature extraction networks, where K is a positive integer; the device includes:
[0010] The first unit is used to acquire images of the target area;
[0011] The second unit is used to obtain K category recognition results corresponding to the target part image using the K image feature extraction networks; wherein, for the first image feature extraction network, the target part image is input into the first image feature extraction network to obtain the first category recognition result and the first radiographic feature map corresponding to the target part image; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target part image, and the fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target part image, where K≥i≥2;
[0012] The third unit is used to determine the category corresponding to the target part image based on the K category recognition results corresponding to the target part image.
[0013] Thirdly, this application provides a readable medium including executable instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform any of the methods described in the first aspect.
[0014] Fourthly, this application provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.
[0015] As can be seen from the above technical solution, this application provides an object category recognition method. The method is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer. First, an image of the target body part is acquired. Then, the K image feature extraction networks are used to obtain K category recognition results corresponding to the target body part image. Specifically, for the first image feature extraction network, the target body part image is input into the first image feature extraction network to obtain a first category recognition result and a first radiographic feature map corresponding to the target body part image. For the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target body part image. The fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target body part image, where K ≥ i ≥ 2. Next, the category corresponding to the target body part image can be determined based on the K category recognition results corresponding to the target body part image. In this embodiment, the image feature maps extracted by the K image feature extraction networks in the category prediction model can reflect the global features of the target area image. Therefore, the fused image input to the 2nd to Kth image feature extraction networks can reflect both the global and local features of the target area image. Thus, by iteratively optimizing the fused image generated based on the image feature maps and the target area image as the input to the image feature extraction network at the next time step, the global and local features of the target area image can be effectively combined during the prediction of the category recognition result by the image feature extraction network. In this way, the category recognition result corresponding to the target area image output by the 2nd to Kth image features is more accurate, and some input information can be selectively ignored or stored, thereby improving robustness to noise and changes.
[0016] The further effects of the aforementioned non-conventional preferred method will be explained below in conjunction with specific embodiments. Attached Figure Description
[0017] To more clearly illustrate the embodiments of this application or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating an object category identification method in an embodiment of this application;
[0019] Figure 2 This is a schematic diagram of a network structure for a category prediction model in an embodiment of this application;
[0020] Figure 3 This is a schematic diagram of the structure of an object category recognition device according to an embodiment of this application;
[0021] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0023] In existing technologies, artificial intelligence recognition techniques for chest X-rays mainly rely on Convolutional Neural Networks (CNNs). CNNs excel at extracting local features but suffer from poor generalization to changes in image quality. In other words, current CNN technologies generally cannot effectively process both global and local information and are not robust enough to image quality variations. Therefore, the accuracy of using CNNs to detect abnormal regions in chest X-rays is relatively low. Thus, there is an urgent need for a method capable of identifying the presence of lesions or abnormalities in chest X-ray images.
[0024] To address the problem that existing technologies cannot optimize the interaction between features extracted by deep learning and clinical features, resulting in poor performance of the established category prediction models.
[0025] This application provides an object category recognition method. This method is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer. First, an image of the target body part is acquired. Then, the K image feature extraction networks are used to obtain K category recognition results corresponding to the target body part image. Specifically, for the first image feature extraction network, the target body part image is input into the first image feature extraction network to obtain a first category recognition result and a first radiographic feature map corresponding to the target body part image. For the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target body part image. The fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target body part image, where K ≥ i ≥ 2. Next, the category corresponding to the target body part image can be determined based on the K category recognition results corresponding to the target body part image. In this embodiment, the image feature maps extracted by the K image feature extraction networks in the category prediction model can reflect the global features of the target area image. Therefore, the fused image input to the 2nd to Kth image feature extraction networks can reflect both the global and local features of the target area image. Thus, by iteratively optimizing the fused image generated based on the image feature maps and the target area image as the input to the image feature extraction network at the next time step, the global and local features of the target area image can be effectively combined during the prediction of the category recognition result by the image feature extraction network. In this way, the category recognition result corresponding to the target area image output by the 2nd to Kth image features is more accurate, and some input information can be selectively ignored or stored, thereby improving robustness to noise and changes.
[0026] The various non-limiting embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0027] See Figure 1 This paper illustrates an object category recognition method according to an embodiment of this application. In this embodiment, the method is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer. The method may include, for example, the following steps:
[0028] S101: Acquire image of the target area.
[0029] In this embodiment, the image of a part that needs to be categorized can be referred to as the target part image. It should be noted that the target part image can be obtained by scanning the user with a scanning device, or it can be received by sending it through other devices. In this embodiment, the method of acquiring the target part image is not limited. In one implementation, the target part image can be a chest X-ray image.
[0030] S102: Using the K image feature extraction networks, obtain the K category recognition results corresponding to the target part image.
[0031] In this embodiment, each image feature extraction network in the category prediction model is a feature extraction network whose function is to extract image features, which can be understood as vectors or matrices. In one implementation, each image feature extraction network in the category prediction model is a Long Short-Term Memory (LSTM) network; that is, the first image feature extraction network, the second image feature extraction network, ..., the Kth image feature extraction network are all LSTM networks. Compared to common Convolutional Neural Networks (CNNs), LSTMs, through a combination of gating units and activation functions, can perform non-linear modeling of input data. This allows LSTMs to learn more complex patterns and relationships, making them suitable for tasks requiring higher-level feature extraction and abstraction. Simultaneously, LSTMs, through their internal memory units and gating mechanisms, can effectively store and access long-term memory, enabling them to handle global dependencies. This is very useful for tasks requiring global information, while CNNs, lacking explicit memory units, are more suitable for handling local dependencies. Finally, because LSTM has a gating mechanism, it can selectively ignore or store partial information from the input, thereby improving its robustness to noise and variations. This makes LSTM more advantageous when processing noisy or incomplete data, which is more practical than CNN in scenarios with inconsistent image quality, such as chest X-rays.
[0032] It should be noted that the network structures of the first image feature extraction network, the second image feature extraction network, ..., the Kth image feature extraction network are completely identical. The only difference is that the values of the network parameters change with each iteration. Therefore, the first image feature extraction network, the second image feature extraction network, ..., the Kth image feature extraction network represent different parameter states of the same network. For example, ... Figure 2 As shown, the category prediction model includes the first image feature extraction network N1, the second image feature extraction network N2, ..., the Kth image feature extraction network N... k .
[0033] For the first image feature extraction network, the target region image can be input into the first image feature extraction network to obtain a first category recognition result and a first radiographic feature map corresponding to the target region image. The size of the first radiographic feature map is the same as the size of the target region image.
[0034] For the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target region image. This fused image is then input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target region image; where K ≥ i ≥ 2. For example, as... Figure 2 As shown, a fused image x2 is generated based on the first image feature map h1 and the target area image x1. The fused image x2 is input into the second image feature extraction network N2 to obtain the second category recognition result O2 and the second image feature map h2 corresponding to the target area image x1. A fused image x3 is generated based on the second image feature map h2 and the target area image x1. The fused image x3 is input into the third image feature extraction network N3 to obtain the third category recognition result O3 and the third image feature map h3 corresponding to the target area image. ...; based on the (K-1)th image feature map h... K-1 Generate a fused image x with the target region image x1. K The fused image x K Inputting the Kth image feature extraction network yields the Kth category recognition result O corresponding to the target region image. K and the Kth imaging feature map h K It should be noted that the size of the (i-1)th radiographic feature map is the same as the size of the target region image.
[0035] It should be noted that the image feature maps (i.e., the first image feature map, the second image feature map, ..., the Kth image feature map) output by each image feature extraction network (i.e., the first image feature map, the second image feature map, ..., the Kth image feature map) are all output by the hidden layers in the image feature extraction network, and each image feature map (i.e., the first image feature map, the second image feature map, ..., the Kth image feature map) is a global attention map. In other words, the first image feature map, the second image feature map, ..., the Kth image feature map can all reflect the global features of the target area image, that is, the attention focus area and the key focus area in the target area image.
[0036] In one implementation, each radiographic feature map (i.e., the first radiographic feature map, the second radiographic feature map, ..., the Kth radiographic feature map) can mainly reflect the following aspects of the target area image:
[0037] 1. Morphological characteristics: These include the shape of the object (e.g., round, elliptical, irregular, etc.), boundary features (e.g., smooth, serrated, lobed, etc.), and size or volume, which can be assessed through measurement.
[0038] 2. Density characteristics: This relates to the density of an object or tissue, which can be described as low density, high density, uniform density, or anomalous density. Low density typically corresponds to darker grayscale values, while high density corresponds to brighter grayscale values.
[0039] 3. Enhancement features: This category of features involves the enhancement of an object or tissue under specific imaging techniques, such as absorption enhancement (used to assess blood perfusion) and enhancement manifestations (such as contrast agent enhancement, contrast agent enhancement, etc.).
[0040] 4. Signal characteristics: Signal characteristics mainly describe the intensity and distribution of signals of objects or tissues in the image, such as signal intensity and signal distribution, which can be represented by grayscale values.
[0041] 5. The phenomenon of rapidly flowing blood in blood vessels during MR imaging: This phenomenon is called "blood flow void effect". When blood in blood vessels is excited and MR signals are recorded, due to some reason (such as the termination of radiofrequency pulse), the blood has left the imaging plane, resulting in no signal display in that area, forming the so-called "no signal shadow".
[0042] 6. MRI contrast enhancement effect: This is an important phenomenon in magnetic resonance imaging, which involves the signal enhancement produced by certain paramagnetic and superparamagnetic materials after the use of contrast agents in MRI scans, thereby improving image quality and discernibility.
[0043] 7. False-color functional imaging: This is a special image processing technique that can utilize the signal differences produced by different types of contrast agents in different tissues or disease states to achieve more intuitive visualization.
[0044] 8. Other features: These also include ultrasound features such as frequency range, sensitivity, and application scenarios; and CT image features such as density resolution and CT values used to interpret density.
[0045] It should be noted that, in one implementation, generating the fused image based on the (i-1)th radiographic feature map and the target region image can be achieved by: determining the attention focus region in the target region image based on the (i-1)th radiographic feature map; and performing image enhancement processing on the attention focus region in the target region image to obtain the fused image. For example, specifically, the fused image can be obtained by performing matrix multiplication on the (i-1)th radiographic feature map and the target region image. It should be noted that enhancing the attention focus region (i.e., image enhancement processing) makes the attention focus region in the fused image appear more prominent, while the remaining regions (i.e., non-attention focus regions) are weakened and appear more blurry.
[0046] It should be noted that the category recognition results (i.e., first category recognition result, second category recognition result, ..., Kth category recognition result) output by each image feature extraction network (i.e., the first image feature extraction network, the second image feature extraction network, ..., the Kth image feature extraction network) all include: probability values corresponding to each of the multiple preset categories. In one implementation, the category recognition result can be a binary classification probability, such as a 1x2 array (p1, p2) representing the binary classification probability; for example, the multiple preset categories can include: normal category and abnormal category (e.g., lesion area category, non-lesion area category). That is, the first category recognition result includes the probability value of belonging to the normal category and the probability value of belonging to the abnormal category; the second category recognition result includes the probability value of belonging to the normal category and the probability value of belonging to the abnormal category; ...; the Kth category recognition result includes the probability value of belonging to the normal category and the probability value of belonging to the abnormal category.
[0047] S103: Determine the category corresponding to the target part image based on the K category recognition results corresponding to the target part image.
[0048] In this embodiment, the probability values corresponding to each preset category in the K category recognition results corresponding to the target part image can be weighted and averaged to obtain the final probability value corresponding to each preset category. For example, for the normal category, the probability values corresponding to each normal category in the K category recognition results are weighted and summed, and the sum is divided by K to obtain an average value. This average value is used as the final probability value corresponding to the normal category. In one implementation, since the prediction results output by the image feature extraction network that is ranked later are more accurate, the probability value corresponding to the normal category output by the image feature extraction network that is ranked later has a larger weight, and the probability value corresponding to the normal category output by the image feature extraction network that is ranked earlier has a smaller weight. That is, the probability value corresponding to the normal category output by the first image feature extraction network has the smallest weight, and the probability value corresponding to the normal category output by the Kth image feature extraction network has a larger weight. Then, the preset category with the largest final probability value can be used as the category O corresponding to the target part image. final .
[0049] As can be seen from the above technical solution, this application provides an object category recognition method. The method is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer. First, an image of the target body part is acquired. Then, the K image feature extraction networks are used to obtain K category recognition results corresponding to the target body part image. Specifically, for the first image feature extraction network, the target body part image is input into the first image feature extraction network to obtain a first category recognition result and a first radiographic feature map corresponding to the target body part image. For the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target body part image. The fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target body part image, where K ≥ i ≥ 2. Next, the category corresponding to the target body part image can be determined based on the K category recognition results corresponding to the target body part image. In this embodiment, the image feature maps extracted by the K image feature extraction networks in the category prediction model can reflect the global features of the target area image. Therefore, the fused image input to the 2nd to Kth image feature extraction networks can reflect both the global and local features of the target area image. Thus, by iteratively optimizing the fused image generated based on the image feature maps and the target area image as the input to the image feature extraction network at the next time step, the global and local features of the target area image can be effectively combined during the prediction of the category recognition result by the image feature extraction network. In this way, the category recognition result corresponding to the target area image output by the 2nd to Kth image features is more accurate, and some input information can be selectively ignored or stored, thereby improving robustness to noise and changes.
[0050] In other words, the method provided in this application solves the problems of CNN networks being unable to effectively process global and local information and being insufficiently robust to image quality in the prior art. The method provided in this application utilizes the temporal unfolding characteristics of LSTM to supplement global image features in target area images (e.g., chest X-ray images), and at the same time uses a gating mechanism to improve the diagnostic performance of target area images (e.g., chest X-ray images) with uneven image quality levels.
[0051] Existing technologies primarily utilize Convolutional Neural Networks (CNNs). While CNNs excel at extracting local features, they exhibit biased generalization to changes in image quality. Therefore, the method presented in this application employs Long Short-Term Memory (LSTM) networks to overcome these shortcomings. Furthermore, leveraging the temporal unfolding characteristics of LTM networks allows for more refined model design. For instance, the output of hidden layers (i.e., image feature maps) can be trained into a global attention map, serving as one of the inputs for the next time step. Through this iterative optimization, global and local features can be effectively combined. Thus, the method presented in this application employs iterative optimization to find the visual learning focus, fusing the results of multiple iterations for comprehensive decision-making. Unlike traditional CNN networks that focus more on learning local features, this method utilizes LTM networks to learn both global and local image features and fuses them through a memory mechanism. This results in more accurate category recognition of the target area image output by the category prediction model. Additionally, it allows for the selective ignoring or storage of partial input information, thereby improving robustness to noise and variations.
[0052] Based on the above object category identification method, see [link to relevant documentation]. Figure 3 This application also provides an object category recognition device, which is applied to a category prediction model, the category prediction model including K image feature extraction networks, where K is a positive integer; the device includes:
[0053] Unit 301 is used to acquire images of the target area;
[0054] The second unit 302 is used to obtain K category recognition results corresponding to the target part image using the K image feature extraction networks; wherein, for the first image feature extraction network, the target part image is input into the first image feature extraction network to obtain the first category recognition result and the first radiographic feature map corresponding to the target part image; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target part image, and the fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target part image, where K≥i≥2;
[0055] The third unit 303 is used to determine the category corresponding to the target part image based on the K category recognition results corresponding to the target part image.
[0056] Optionally, each image feature extraction network is a long short-term memory network.
[0057] Optionally, the image feature maps output by each image feature extraction network are output by the hidden layers in the image feature extraction network, and each image feature map is a global attention map.
[0058] Optionally, the second unit 302 is used for:
[0059] Based on the i-1th radiographic feature map, the attention focus region in the target area image is determined, and the attention focus region in the target area image is subjected to image enhancement processing to obtain a fused image.
[0060] Optionally, the second unit 302 is used for:
[0061] The (i-1)th radiographic feature map and the target area image are multiplied by a matrix to obtain a fused image.
[0062] Optionally, each category recognition result includes: the probability values corresponding to each of the multiple preset categories;
[0063] Accordingly, the third unit 303 is used for:
[0064] The probability values of each preset category in the K category recognition results corresponding to the target part image are weighted and averaged to obtain the final probability value of each preset category.
[0065] The category with the highest final probability value is selected as the category corresponding to the target part image.
[0066] Optionally, the target area image is a chest X-ray image; the multiple preset categories include: normal category and abnormal category.
[0067] As can be seen, the device includes: a first unit for acquiring an image of a target body part; a second unit for using the K image feature extraction networks to obtain K category recognition results corresponding to the image of the target body part; wherein, for the first image feature extraction network, the image of the target body part is input into the first image feature extraction network to obtain a first category recognition result and a first radiographic feature map corresponding to the image of the target body part; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the image of the target body part, and the fused image is input into the i-th image feature extraction network to obtain an i-th category recognition result and an i-th radiographic feature map corresponding to the image of the target body part, where K ≥ i ≥ 2; and a third unit for determining the category corresponding to the image of the target body part based on the K category recognition results corresponding to the image of the target body part. In this embodiment, the image feature maps extracted by the K image feature extraction networks in the category prediction model can reflect the global features of the target area image. Therefore, the fused image input to the 2nd to Kth image feature extraction networks can reflect both the global and local features of the target area image. Thus, by iteratively optimizing the fused image generated based on the image feature maps and the target area image as the input to the image feature extraction network at the next time step, the global and local features of the target area image can be effectively combined during the prediction of the category recognition result by the image feature extraction network. In this way, the category recognition result corresponding to the target area image output by the 2nd to Kth image features is more accurate, and some input information can be selectively ignored or stored, thereby improving robustness to noise and changes.
[0068] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.
[0069] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.
[0070] Memory is used to store instructions for execution. Specifically, instructions for execution are computer programs that can be executed. Memory can include main memory and non-volatile memory, and it provides the processor with execution instructions and data.
[0071] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into memory and then executes them. Alternatively, it may obtain the corresponding execution instructions from other devices to form an object category recognition device at the logical level. The processor executes the execution instructions stored in memory to implement the object category recognition method provided in any embodiment of this application through the executed instructions.
[0072] The above is as stated in this application. Figure 1 The object category identification method provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.
[0073] The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0074] This application also proposes a readable medium that stores execution instructions. When the stored execution instructions are executed by the processor of an electronic device, the electronic device can perform the object category identification method provided in any embodiment of this application.
[0075] The electronic devices described in the foregoing embodiments may be computers.
[0076] Those skilled in the art will understand that the embodiments of this application can be provided as methods or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.
[0077] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0078] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0079] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for object category recognition, characterized in that, The method is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer; the method includes: Acquire images of the target area; Using the K image feature extraction networks, K category recognition results corresponding to the target area image are obtained; wherein, for the first image feature extraction network, the target area image is input into the first image feature extraction network to obtain the first category recognition result and the first radiographic feature map corresponding to the target area image; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target area image, and the fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target area image, where K≥i≥2; Based on the K category recognition results corresponding to the target area image, the category corresponding to the target area image is determined.
2. The method according to claim 1, characterized in that, Each image feature extraction network is a long short-term memory network.
3. The method according to claim 2, characterized in that, The image feature maps output by each image feature extraction network are output by the hidden layers in the image feature extraction network, and each image feature map is a global attention map.
4. The method according to claim 1, characterized in that, The size of the (i-1)th radiographic feature map is the same as the size of the target region image; the generation of the fused image based on the (i-1)th radiographic feature map and the target region image includes: Based on the i-1th radiographic feature map, the attention focus region in the target area image is determined, and image enhancement processing is performed on the attention focus region in the target area image to obtain a fused image.
5. The method according to claim 4, characterized in that, The step of determining the attention focus region in the target area image based on the (i-1)th radiographic feature map, and performing image enhancement processing on the attention focus region in the target area image to obtain a fused image, includes: The (i-1)th radiographic feature map and the target area image are multiplied by a matrix to obtain a fused image.
6. The method according to claim 1, characterized in that, Each category recognition result includes: the probability value corresponding to each of the multiple preset categories; Accordingly, determining the category corresponding to the target part image based on the K category recognition results corresponding to the target part image includes: The probability values of each preset category in the K category recognition results corresponding to the target part image are weighted and averaged to obtain the final probability value of each preset category. The category with the highest final probability value is selected as the category corresponding to the target part image.
7. The method according to claim 6, characterized in that, The target area image is a chest X-ray image; the multiple preset categories include: normal category and abnormal category.
8. An object category recognition device, characterized in that, The device is applied to a category prediction model, which includes K image feature extraction networks, where K is a positive integer; the device includes: The first unit is used to acquire images of the target area; The second unit is used to obtain K category recognition results corresponding to the target part image using the K image feature extraction networks; wherein, for the first image feature extraction network, the target part image is input into the first image feature extraction network to obtain the first category recognition result and the first radiographic feature map corresponding to the target part image; for the i-th image feature extraction network, a fused image is generated based on the (i-1)-th radiographic feature map and the target part image, and the fused image is input into the i-th image feature extraction network to obtain the i-th category recognition result and the i-th radiographic feature map corresponding to the target part image, where K≥i≥2; The third unit is used to determine the category corresponding to the target part image based on the K category recognition results corresponding to the target part image.
9. A readable medium, characterized in that, include: An execution instruction is provided, wherein when the processor of the electronic device executes the execution instruction, the processor performs the method described in any one of claims 1-7.
10. An electronic device, characterized in that, include: A processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method according to any one of claims 1-7.