Methods, devices, equipment, and media for lesion location detection based on edge devices

Abstract

This application relates to the field of digital medical technology, and particularly to a method, apparatus, device, and medium for lesion location detection based on edge devices. The method acquires N real-time images acquired by an image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in a replay storage area. Based on the upper limit of the input quantity of the detection model in one round of execution, the number of images in the N real-time images, and the historical number, a first quantity and a second quantity are determined, and a target activation value and a target image are determined. An encoding layer is used to encode the target image to obtain an encoding result. A replay layer is used to connect the encoding result and the target activation value to obtain a connection result. A fully connected layer is used to detect the lesion location in the connection result, obtaining the detection result of the lesion location in the target image. By employing a lightweight model in the edge device to achieve lesion location detection and setting a replay storage area to store activation values, the storage space occupied can be greatly reduced.

Description

Technical Field

[0001] This application relates to the field of digital medical technology, and in particular to a method, device, equipment and medium for lesion location detection based on edge devices. Background Technology

[0002] With the development of digital medical technology, the use of advanced instruments to acquire corresponding human images to assist doctors in diagnosis has become an integral part of the medical process. Among these, the classification of lesions in medical images is a crucial component of the digital processing of medical images. Currently, deep neural network technology is commonly used in image processing to identify lesions in images. Medical images refer to images of internal tissues obtained non-invasively for medical treatment or research, such as images of the stomach, abdomen, heart, knees, and brain. These images are generated by medical instruments, including computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), X-ray imaging, electroencephalography (EEG), and optical imaging. In medical procedures, doctors generally need to use their professional knowledge to determine whether lesions or foci are present in images. However, this process is inefficient, and doctors with different professional skills may make different judgments. For example, when determining the location of lesions such as low-grade astrocytomas, ganglia, and arteriovenous malformations in the brain, slight shape changes may not be visible to the naked eye, leading to inaccurate determination of the lesion location.

[0003] Current methods for detecting lesions in images require large, pre-trained models on servers, relying on vast amounts of historical images. This necessitates significant storage space and involves processing large volumes of data, resulting in low efficiency. Therefore, achieving lightweight lesion detection on edge devices to improve efficiency and practicality is a pressing issue. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, device, and medium for lesion location detection based on edge devices, in order to solve the problem of how to achieve lightweight lesion location detection on edge devices, thereby improving detection efficiency and practicality.

[0005] In a first aspect, embodiments of this application provide a lesion location detection method based on an edge device. The lesion location detection method is executed in an edge device running a lightweight detection model. The edge device is provided with a replay storage area supporting the calculation of the detection model. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The lesion location detection method includes:

[0006] Obtain N real-time images acquired by the image acquisition instrument connected to the edge device and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero;

[0007] Based on the upper limit of the input in one round of the detection model, the number of images in the N real-time images, and the number of historical images, a first quantity and a second quantity are determined, wherein the sum of the first quantity and the second quantity is the upper limit of the input, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the number of historical images;

[0008] The second number of activation values ​​are selected from the replay storage area as target activation values, and the first number of real-time images are selected from the N real-time images as target images. The target images are then encoded using the coding layer to obtain the encoding result.

[0009] The replay layer is used to connect the encoding result and the target activation value to obtain a connection result. The fully connected layer is used to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image.

[0010] Optionally, determining the first and second quantities based on the upper limit of the input quantity of the detection model in one round of execution, the number of images in the N real-time images, and the number of historical images includes:

[0011] Based on the number of images in the N real-time images and the number of historical images, calculate the first proportion of the number of images in the sum of the number of images and the number of historical images, and the second proportion of the number of historical images in the sum of the number of images and the number of historical images;

[0012] The first quantity is obtained by multiplying the first proportion by the upper limit of the number of inputs in one round of execution of the detection model. The second quantity is obtained by multiplying the second proportion by the upper limit of the number of inputs.

[0013] Optionally, the replay layer is used to concatenate the encoding result and the target activation value to obtain a concatenation result including:

[0014] Obtain the activation function of the replay layer;

[0015] Based on the target activation value and the activation function, an activation operation is performed on the encoding result to obtain an activation result, which is then determined to be a connection result.

[0016] Optionally, the fully connected layer is used to detect the lesion location in the connection result, and the detection result of the lesion location in the target image is obtained by detecting the lesion location in the connection result.

[0017] The fully connected layer is used to perform a fully connected calculation on the connection result to obtain the calculation results for all preset categories corresponding to the fully connected layer;

[0018] For any preset category, the classification result of whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image belongs to the preset category;

[0019] By traversing all preset categories, the detection result of the lesion location in the target image is determined as the probability of each pixel in the target image in each preset category.

[0020] Optionally, after encoding the target image using the encoding layer to obtain the encoding result, the method further includes:

[0021] Return to the previous step of selecting the first number of real-time images from the N real-time images as the target images, until all real-time images have been selected, and obtain the encoding result corresponding to each real-time image;

[0022] For any real-time image, the encoding result of the real-time image is used as the activation value of the real-time image to obtain the activation values ​​of the N real-time images;

[0023] A preset number of activation values ​​are selected from the activation values ​​of the N real-time images as updated activation values, and the activation values ​​of the replay storage area are updated using the updated activation values.

[0024] Optionally, updating the activation value of the replay storage area using the updated activation value includes:

[0025] Select the preset number of activation values ​​from the replay storage area, and delete the selected preset number of activation values;

[0026] The updated activation value is stored in the replay storage area.

[0027] Optionally, before selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as the updated activation values, the method further includes:

[0028] Multiply the historical data by a preset ratio to obtain the result.

[0029] The result of the multiplication is taken as the preset number.

[0030] Secondly, embodiments of this application provide a lesion location detection device based on an edge device. The lesion location detection device is executed in an edge device running a lightweight detection model. The edge device is provided with a replay storage area supporting the calculation of the detection model. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The lesion location detection device includes:

[0031] The information acquisition module is used to acquire N real-time images acquired by the image acquisition instrument connected to the edge device and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero;

[0032] The quantity determination module is used to determine a first quantity and a second quantity based on the upper limit of the input quantity of the detection model in one round of execution, the number of images of the N real-time images, and the historical quantity, wherein the sum of the first quantity and the second quantity is the upper limit of the input quantity, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the historical quantity;

[0033] The image encoding module is used to select the second number of activation values ​​as target activation values ​​from the replay storage area, select the first number of real-time images as target images from the N real-time images, and use the encoding layer to encode the target images to obtain the encoding result;

[0034] The detection and classification module is used to connect the encoding result and the target activation value using the replay layer to obtain a connection result, and to use the fully connected layer to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image.

[0035] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lesion location detection method as described in the first aspect.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lesion location detection method as described in the first aspect.

[0037] The beneficial effects of this application's embodiments compared to the prior art are as follows: The method of this application is executed in an edge device running a lightweight detection model. The edge device is provided with a replay storage area to support the calculation of the detection model. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. This method acquires N real-time images acquired by an image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in the replay storage area. Based on the upper limit of the input of the detection model in one round of execution, the number of images in the N real-time images, and the historical number, a first quantity and a second quantity are determined, and the activation values ​​are selected from the replay storage area. The second number of activation values ​​is taken as the target activation value. The first number of real-time images are selected from N real-time images as the target images. The target images are encoded using an encoding layer to obtain the encoding result. The encoding result and the target activation value are connected using a replay layer to obtain the connection result. The connection result is then used to detect the lesion location using a fully connected layer to obtain the detection result of the lesion location in the target image. A lightweight model is used in the edge device to realize the lesion location detection of the image. The replay storage area is set to store the activation values ​​instead of storing the historical images themselves, which can greatly reduce the storage space occupation and accurately combine the real-time images to obtain the detection result. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of an application environment for a lesion location detection method based on an edge device provided in Embodiment 1 of this application;

[0040] Figure 2 This is a schematic flowchart of a lesion location detection method based on an edge device provided in Embodiment 2 of this application;

[0041] Figure 3 This is a schematic flowchart of a lesion location detection method based on an edge device provided in Embodiment 3 of this application;

[0042] Figure 4 This is a schematic diagram of the structure of a lesion location detection device based on an edge device provided in Embodiment 4 of this application;

[0043] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Detailed Implementation

[0044] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0045] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0046] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0047] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0048] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0049] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0050] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0051] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0052] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0053] To illustrate the technical solution of this application, specific embodiments are described below.

[0054] The lesion location detection method based on edge devices provided in Embodiment 1 of this application can be applied to, for example... Figure 1 In this application environment, the image acquisition instrument communicates with edge devices. Edge devices include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The image acquisition instrument can refer to the medical instrument used to capture images such as CT scans, MRIs, and ultrasound images taken during a patient's medical examination.

[0055] See Figure 2 This is a flowchart illustrating a lesion location detection method based on an edge device, provided in Embodiment 2 of this application. The aforementioned lesion location detection method based on an edge device is applied to... Figure 1 The edge device in the system contains a lightweight detection model within its corresponding computer device. The edge device also includes a replay storage area to support the detection model's calculations. This replay storage area stores activation values ​​based on historical images. The detection model comprises a coding layer, a replay layer, and a fully connected layer. This computer device can also be connected to a corresponding image acquisition instrument to acquire medical images of patients captured by that instrument. Figure 2As shown, the lesion location detection method based on edge devices may include the following steps:

[0056] Step S201: Obtain N real-time images acquired by the image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in the replay storage area.

[0057] In this application, when the patient reaches the designated position of the image acquisition instrument according to the instructions, the image acquisition instrument is activated by controlling it to acquire human images of the patient's affected area. The acquisition quantity is set to N, where N is an integer greater than zero. This can reduce the result error caused by the error of the image acquisition instrument in subsequent use.

[0058] The replay storage area can refer to a subspace allocated within the storage space of an edge device for storing activation values, and this replay storage area can be used when the edge device runs the method of this application. The stored activation values ​​are the activation values ​​of historical images obtained after processing historical images during previous runs of the method of this application, and are stored in the replay storage area.

[0059] The number of activation values ​​stored in the replay storage area is counted to obtain the historical number. The number of activation values ​​is the same as the number of historical images, that is, one historical image corresponds to one activation value.

[0060] Step S202: Determine the first quantity and the second quantity based on the upper limit of the number of inputs in one round of the detection model, the number of real-time images (N), and the number of historical images.

[0061] In this application, the detection model includes an encoding layer, a replay layer, and a fully connected layer. The encoding layer is used to encode the image, the replay layer is used to receive the encoding result and activation value of the image and fuse the two, and the fully connected layer is used to perform fully connected prediction on the fused result to obtain the prediction result (that is, the detection result).

[0062] In one execution process (i.e., one round of execution), the encoding layer of the detection model has a fixed upper limit on the number of input images. That is, the upper limit of input images cannot be processed if the number of images exceeds the upper limit of input images. Therefore, in order to determine the first number of real-time input images and the second number of historical images, it is necessary to calculate based on this upper limit of input images.

[0063] The sum of the first and second quantities represents the upper limit of the input. The first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the number of historical images. In other words, a higher first quantity indicates a higher number of images, and a higher second quantity indicates a higher number of historical images. This makes the detection model more closely suited to real-time requirements, satisfying both the real-time image bias and historical image bias when there are many real-time images, thus improving accuracy. Conversely, when there are fewer real-time images, the detection biases towards historical images, satisfying computational requirements and maintaining a certain level of accuracy.

[0064] Optionally, the first and second quantities are determined based on the upper limit of the number of inputs to the detection model in one round, the number of N real-time images, and the number of historical images.

[0065] Based on the number of real-time images and the number of historical images, calculate the first proportion of the number of real-time images in the sum of the number of real-time images and the number of historical images, and the second proportion of the number of historical images in the sum of the number of real-time images and the number of historical images.

[0066] The first quantity is obtained by multiplying the first proportion by the upper limit of the number of inputs in one round of the detection model. The second quantity is obtained by multiplying the second proportion by the upper limit of the number of inputs.

[0067] The quantity can be calculated using a formula, as follows:

[0068]

[0069]

[0070] In the formula, Indicates the first quantity. Indicates the second quantity. Indicates the maximum number of inputs. Indicates the number of images. Indicates historical quantity.

[0071] Step S203: Select a second number of activation values ​​from the replay storage area as target activation values, select a first number of real-time images from N real-time images as target images, and use the coding layer to encode the target images to obtain the coding result.

[0072] In this application, the activation values ​​in the replay storage area are called, and the number of calls is the second number. A first number of real-time images are selected from N real-time images as target images. The first number of real-time images and the second number of activation values ​​are input into the detection model, that is, input into the encoding layer for encoding.

[0073] The encoding layer is used to encode the image into a data format that the detection model can process. Specifically, it can be used to perform vector encoding on the image, that is, to express the image in the form of vectors, and the expression result is the encoding result.

[0074] In addition, the coding layer can be composed of multiple convolutional layers, with the size of the convolutional kernels of each layer decreasing sequentially to focus on the detailed features of the image.

[0075] The target activation value is equivalent to the encoding result of the already encoded historical image, which can be directly used in subsequent steps without going through the encoding layer.

[0076] Step S204: The replay layer is used to connect the encoding result and the target activation value to obtain the connection result. The fully connected layer is used to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image.

[0077] In this application, the input of the replay layer is connected to the output of the coding layer to obtain the coding result of the coding layer output. The input of the replay layer is also connected to the replay storage area to obtain the activation value of the replay storage area.

[0078] The replay layer is a connection layer used to connect the encoded result and the activation value to obtain the connection result. The connection is specifically to activate the encoded result, and the activation methods can include weighted activation, merge activation, multiplicative activation, etc.

[0079] Regarding the connection results, the classification result of each pixel in each image can be obtained through the calculation of the fully connected layer. After training, the fully connected layer has the ability to determine whether a pixel is a lesion. The classification result is the probability of determining whether it is a lesion. The higher the probability, the more likely it is a lesion, and the lower the probability, the less likely it is a lesion.

[0080] Optionally, a replay layer can be used to concatenate the encoded result and the target activation value, resulting in the following concatenation result:

[0081] Obtain the activation function of the replay layer;

[0082] Based on the target activation value and activation function, an activation operation is performed on the encoding result to obtain the activation result, which is then determined to be the connection result.

[0083] Activation functions are functions used to enable the detection model to learn and understand highly complex and nonlinear capabilities, introducing nonlinear characteristics into the network. In neurons, the inputs are weighted, summed, and then subjected to a function—the activation function. Using the target activation value as the weight and the activation function as the calculation formula, the encoded result is calculated; that is, the activation operation is performed, yielding the activation result, which is the connection result.

[0084] Optionally, a fully connected layer can be used to detect the lesion location in the connection result, and the detection results of the lesion location in the target image include:

[0085] The fully connected layer is used to perform fully connected calculations on the connection results, and the calculation results of all preset categories of the corresponding fully connected layer are obtained.

[0086] For any given preset category, the classification result of whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image belongs to the preset category;

[0087] Iterate through all preset categories and determine the detection result of the lesion location in the target image as the probability of each pixel in the target image in each preset category.

[0088] The fully connected computation of the connection results is used to map the learned "distributed feature representation" to the sample label space. Each node is connected to all nodes of the previous layer, which is used to combine the features extracted from the previous layers to obtain the calculation result of each preset category, that is, the probability of whether each pixel is a lesion.

[0089] Furthermore, after obtaining the detection results, the results are analyzed, and pixels with a probability greater than the probability threshold are integrated to obtain the lesion area.

[0090] The method in this embodiment is executed on an edge device running a lightweight detection model. The edge device has a replay storage area supporting the detection model's calculations. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The method acquires N real-time images acquired by an image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in the replay storage area. Based on the upper limit of the input count for the detection model in one round of execution, the number of N real-time images, and the historical number of activation values, a first quantity and a second quantity are determined. The second quantity of activation values ​​is then selected from the replay storage area as... Using the target activation value as the target, a first number of real-time images are selected from N real-time images as the target image. The target image is encoded using an encoding layer to obtain the encoding result. The encoding result and the target activation value are concatenated using a replay layer to obtain the concatenation result. The concatenation result is then used to detect the lesion location in the target image using a fully connected layer. A lightweight model is used in the edge device to realize the lesion location detection in the image. Furthermore, a replay storage area is set to store the activation value instead of storing the historical image itself, which can greatly reduce the storage space occupied and accurately combine the real-time images to obtain the detection result.

[0091] See Figure 3 This is a flowchart illustrating a lesion location detection method based on an edge device, provided in Embodiment 3 of this application. The aforementioned lesion location detection method based on an edge device is applied to... Figure 1 The edge device in the system contains a lightweight detection model within its corresponding computer device. The edge device also includes a replay storage area to support the detection model's calculations. This replay storage area stores activation values ​​based on historical images. The detection model comprises a coding layer, a replay layer, and a fully connected layer. This computer device can also be connected to a corresponding image acquisition instrument to acquire medical images of patients captured by that instrument. Figure 3 As shown, the lesion location detection method based on edge devices may include the following steps:

[0092] Step S301: Obtain N real-time images acquired by the image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in the replay storage area.

[0093] Step S302: Determine the first quantity and the second quantity based on the upper limit of the number of inputs in one round of the detection model, the number of N real-time images, and the number of historical images.

[0094] Step S303: Select a second number of activation values ​​from the replay storage area as target activation values, select a first number of real-time images from N real-time images as target images, and use the coding layer to encode the target images to obtain the coding result.

[0095] Step S304: The replay layer is used to connect the encoding result and the target activation value to obtain the connection result. The fully connected layer is used to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image.

[0096] The contents of steps S301 to S304 are partially the same as those of steps S201 to S204 above. Please refer to the description of steps S201 to S204 for details. They will not be repeated here.

[0097] Step S305: Return to the previous step of selecting a first number of real-time images from N real-time images as target images, until all real-time images have been selected, and obtain the encoding result corresponding to each real-time image.

[0098] In this application, after each round of execution of the method of the embodiment, each real-time image is also encoded to obtain the encoding result for use in subsequent steps.

[0099] Specifically, it involves traversing all real-time images, treating each real-time image as a target image, and inputting the target image into the coding layer to obtain the corresponding coding result.

[0100] Step S306: For any real-time image, use the encoding result of the real-time image as the activation value of the real-time image to obtain the activation values ​​of N real-time images.

[0101] In this application, the encoding result is also the activation value. After obtaining each real-time image, the activation values ​​for N real-time images are obtained. These activation values ​​can be used for subsequent operations to update the playback storage area.

[0102] Step S307: Select a preset number of activation values ​​from the activation values ​​of N real-time images as updated activation values, and use the updated activation values ​​to update the activation values ​​of the replay storage area.

[0103] In this application, for use in the next round, the corresponding activation value is selected from the N activation values ​​corresponding to the N real-time images in the current round as the updated activation value. The number of selected activation values ​​is determined by the total number of activation values ​​in the replay storage area.

[0104] An update can be to add a new activation value to the replay store, overwrite the activation value in the replay store, or replace some of the activation values ​​in the replay store.

[0105] Optionally, updating the activation value of the replay storage area using the updated activation value includes:

[0106] Select a preset number of activation values ​​from the replay storage area, and delete the preset number of activation values.

[0107] The updated activation value will be stored in the replay storage area.

[0108] In this process, a certain number of activation values ​​are selected from the replay storage area, deleted, and then the updated activation values ​​are stored in the replay storage area. This ensures that the number of activation values ​​in the replay storage area is constant, thus ensuring good consistency in each round of processing.

[0109] Optionally, before selecting a preset number of activation values ​​from the activation values ​​of N real-time images as the updated activation values, the following steps are also included:

[0110] Multiply the historical data by the preset ratio to obtain the result;

[0111] The result of the multiplication is used as the preset number.

[0112] The preset number is set proportionally. If the number of historical activation values ​​stored in the replay storage area is large, the corresponding preset number is large. If the number of historical activation values ​​stored in the replay storage area is small, the corresponding preset number is low, which improves the universality of the replay storage area.

[0113] The method in this embodiment is executed on an edge device running a lightweight detection model. The edge device has a replay storage area supporting the detection model's calculations. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The method acquires N real-time images acquired by an image acquisition instrument connected to the edge device, as well as the historical number of activation values ​​stored in the replay storage area. Based on the upper limit of the input count for the detection model in one round of execution, the number of images in the N real-time images, and the historical number, a first quantity and a second quantity are determined. The second quantity of activation values ​​is selected from the replay storage area as target activation values, and the first quantity of real-time images is selected from the N real-time images as target images. The encoding layer is used to process the target images. The process involves encoding the data to obtain the encoding result, then using a replay layer to connect the encoding result and the target activation value to obtain the connection result. A fully connected layer is then used to detect the lesion location in the connection result, yielding the detection result of the lesion location in the target image. The process then returns to the previous step of selecting a first number of real-time images from N real-time images as the target image, continuing until all real-time images have been selected, obtaining the encoding result for each real-time image. For any given real-time image, the encoding result is used as the activation value of the real-time image, resulting in the activation values ​​of N real-time images. A preset number of activation values ​​are selected from the activation values ​​of the N real-time images as update activation values. These update activation values ​​are then used to update the activation values ​​of the replay storage area, ensuring that the replay storage area can be updated in real time.

[0114] Corresponding to the lesion location detection method based on edge devices in the above embodiments, Figure 4This diagram illustrates the structural block diagram of a lesion location detection device based on an edge device according to Embodiment 4 of this application. The lesion location detection device is applied to... Figure 1 The edge device in the diagram has a corresponding computer device equipped with a lightweight detection model. The edge device also includes a replay storage area to support the calculations of the detection model, storing activation values ​​based on historical images. The detection model comprises an encoding layer, a replay layer, and a fully connected layer. The computer device can also be connected to a corresponding image acquisition instrument to acquire medical images of patients captured by that instrument. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0115] See Figure 4 The lesion location detection device includes:

[0116] The information acquisition module 41 is used to acquire N real-time images acquired by the image acquisition instrument connected to the edge device and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero;

[0117] The quantity determination module 42 is used to determine a first quantity and a second quantity based on the upper limit of the input quantity of the detection model in one round of execution, the number of images of N real-time images, and the historical quantity. The sum of the first quantity and the second quantity is the upper limit of the input quantity. The first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the historical quantity.

[0118] Image encoding module 43 is used to select a second number of activation values ​​from the replay storage area as target activation values, select a first number of real-time images from N real-time images as target images, and use an encoding layer to encode the target images to obtain the encoding result;

[0119] The detection and classification module 44 is used to connect the encoding result and the target activation value using the replay layer to obtain the connection result, and then use the fully connected layer to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image.

[0120] Optionally, the quantity determination module 42 includes:

[0121] The proportion determination unit is used to calculate, based on the number of images and the number of historical images in N real-time images, a first proportion of the number of images in the sum of the number of images and the number of historical images, and a second proportion of the number of historical images in the sum of the number of images and the number of historical images.

[0122] The quantity calculation unit is used to calculate the product of the first proportion and the upper limit of the input quantity of the detection model in one round of execution to obtain the first quantity, and to calculate the product of the second proportion and the upper limit of the input quantity to obtain the second quantity.

[0123] Optionally, the detection and classification module 44 includes:

[0124] The activation function acquisition unit is used to acquire the activation function of the replay layer;

[0125] The activation operation unit is used to perform an activation operation on the encoding result based on the target activation value and the activation function, obtain the activation result, and determine the activation result as the connection result.

[0126] Optionally, the detection and classification module 44 includes:

[0127] The fully connected computation unit is used to perform fully connected computation on the connection results using the fully connected layer, and obtain the computation results of all preset categories of the corresponding fully connected layer;

[0128] The preset category determination unit is used to determine whether each pixel in the target image is a lesion for any preset category. The classification result is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image belongs to the preset category.

[0129] The category probability determination unit iterates through all preset categories and determines the detection result of the lesion location in the target image as the probability of each pixel in the target image in each preset category.

[0130] Optionally, the lesion location detection device also includes:

[0131] The encoding traversal module is used to encode the target image using the encoding layer. After obtaining the encoding result, it returns to the step of selecting the first number of real-time images from N real-time images as the target image, until all real-time images have been selected, and obtains the encoding result corresponding to each real-time image.

[0132] The real-time activation value determination module is used to determine the activation value of N real-time images by taking the encoding result of the real-time image as the activation value of the real-time image for any given real-time image.

[0133] The storage area update module is used to select a preset number of activation values ​​from the activation values ​​of N real-time images as update activation values, and use the update activation values ​​to update the activation values ​​of the replay storage area.

[0134] Optionally, the storage area update module includes:

[0135] The deletion unit is used to select a preset number of activation values ​​from the replay storage area and delete the selected preset number of activation values.

[0136] Storage units are used to store updated activation values, such as in the replay storage area.

[0137] Optionally, the lesion location detection device also includes:

[0138] The proportional multiplication module is used to multiply the historical number by a preset proportional value before selecting a preset number of activation values ​​from the activation values ​​of N real-time images as the updated activation values, and obtain the multiplication result.

[0139] The preset number determination module is used to use the multiplication result as the preset number.

[0140] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0141] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, wherein the processor executes the computer program to implement the steps in any of the above embodiments of the edge device-based lesion location detection method.

[0142] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0143] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0144] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of the computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0146] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A lesion location detection method based on edge devices, characterized in that, The lesion location detection method is executed in an edge device running a lightweight detection model. The edge device is equipped with a replay storage area to support the calculations of the detection model. The replay storage area stores activation values ​​formed based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The lesion location detection method includes: Obtain N real-time images acquired by the image acquisition instrument connected to the edge device and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; Based on the upper limit of the input in one round of the detection model, the number of images in the N real-time images, and the number of historical images, a first quantity and a second quantity are determined, wherein the sum of the first quantity and the second quantity is the upper limit of the input, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the number of historical images; The second number of activation values ​​are selected from the replay storage area as target activation values, and the first number of real-time images are selected from the N real-time images as target images. The target images are then encoded using the coding layer to obtain the encoding result. The replay layer is used to connect the encoding result and the target activation value to obtain a connection result. The fully connected layer is used to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image. The first and second quantities are determined based on the upper limit of the input quantity of the detection model in one round of execution, the number of images in the N real-time images, and the number of historical images, including: Based on the number of images in the N real-time images and the number of historical images, calculate the first proportion of the number of images in the sum of the number of images and the number of historical images, and the second proportion of the number of historical images in the sum of the number of images and the number of historical images; The first quantity is obtained by multiplying the first proportion by the upper limit of the number of inputs in one round of execution of the detection model. The second quantity is obtained by multiplying the second proportion by the upper limit of the number of inputs.

2. The lesion location detection method according to claim 1, characterized in that, The replay layer is used to concatenate the encoded result and the target activation value to obtain the following concatenation result: Obtain the activation function of the replay layer; Based on the target activation value and the activation function, an activation operation is performed on the encoding result to obtain an activation result, which is then determined to be a connection result.

3. The lesion location detection method according to claim 1, characterized in that, Using the fully connected layer to detect the lesion location in the connection result, the detection result of the lesion location in the target image includes: The fully connected layer is used to perform a fully connected calculation on the connection result to obtain the calculation results for all preset categories corresponding to the fully connected layer; For any preset category, the classification result of whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image belongs to the preset category; By iterating through all preset categories, the detection result of the lesion location in the target image is determined as the probability of each pixel in the target image in each preset category.

4. The lesion location detection method according to any one of claims 1 to 3, characterized in that, After encoding the target image using the coding layer to obtain the encoding result, the process further includes: Return to the previous step of selecting the first number of real-time images from the N real-time images as the target images, until all real-time images have been selected, and obtain the encoding result corresponding to each real-time image; For any real-time image, the encoding result of the real-time image is used as the activation value of the real-time image to obtain the activation values ​​of the N real-time images; A preset number of activation values ​​are selected from the activation values ​​of the N real-time images as updated activation values, and the activation values ​​of the replay storage area are updated using the updated activation values.

5. The lesion location detection method according to claim 4, characterized in that, Updating the activation value of the replay storage area using the updated activation value includes: Select the preset number of activation values ​​from the replay storage area, and delete the selected preset number of activation values; The updated activation value is stored in the replay storage area.

6. The lesion location detection method according to claim 4, characterized in that, Before selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as the updated activation values, the process also includes: Multiply the historical data by a preset ratio to obtain the result. The result of the multiplication is taken as the preset number.

7. A lesion location detection device based on edge devices, characterized in that, The lesion location detection device is executed in an edge device running a lightweight detection model. The edge device includes a replay storage area to support the calculations of the detection model. The replay storage area stores activation values ​​based on historical images. The detection model includes an encoding layer, a replay layer, and a fully connected layer. The lesion location detection device includes: The information acquisition module is used to acquire N real-time images acquired by the image acquisition instrument connected to the edge device and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; The quantity determination module is used to determine a first quantity and a second quantity based on the upper limit of the input quantity of the detection model in one round of execution, the number of images of the N real-time images, and the historical quantity, wherein the sum of the first quantity and the second quantity is the upper limit of the input quantity, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the historical quantity; The image encoding module is used to select the second number of activation values ​​as target activation values ​​from the replay storage area, select the first number of real-time images as target images from the N real-time images, and use the encoding layer to encode the target images to obtain the encoding result; The detection and classification module is used to connect the encoding result and the target activation value using the replay layer to obtain a connection result, and to use the fully connected layer to detect the lesion location in the connection result to obtain the detection result of the lesion location in the target image. The quantity determination module includes: The proportion determination unit is used to calculate, based on the number of images in the N real-time images and the number of historical images, a first proportion of the number of images in the sum of the number of images and the number of historical images, and a second proportion of the number of historical images in the sum of the number of images and the number of historical images; The quantity calculation unit is used to calculate the product of the first proportion and the upper limit of the input quantity of the detection model in one round of execution to obtain the first quantity, and to calculate the product of the second proportion and the upper limit of the input quantity to obtain the second quantity.

8. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lesion location detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the lesion location detection method as described in any one of claims 1 to 6.

Images