Focus position detection method and device based on edge device, equipment and medium

Abstract

The invention is suitable for the technical field of digital medical treatment, and particularly relates to a focus position detection method and device based on edge equipment, equipment and a medium. The method comprises the following steps: acquiring N real-time images acquired by connecting an edge device with an image acquisition instrument and a historical number of activation values stored in a playback storage area, and determining a first number and a second number according to an input upper limit number of a detection model in one-round execution, an image number of the N real-time images and the historical number, determining a target activation value and a target image, encoding the target image by using an encoding layer to obtain an encoding result, connecting the encoding result and the target activation value by using a playback layer to obtain a connection result, and performing focus position detection on the connection result by using a full connection layer to obtain a detection result of the focus position in the target image. The lesion position detection of the image is realized by adopting a lightweight model in the edge equipment, and the storage activation value of the replay storage area is set, so that the occupation of the storage space can be greatly reduced.

Description

Technical Field

[0001] The present application is applicable to the field of digital medical technology, and in particular, relates to a lesion location detection method, apparatus, device and medium based on edge devices. Background Art

[0002] With the development of digital medical technology, the use of advanced instruments to collect corresponding human body images to assist doctors in diagnosis has become an integral part of the medical process. Among them, the classification of lesions in medical images is an important part of the digital processing of medical images. The current deep neural network technology is often used in image processing to achieve the identification of lesions in images. Medical images refer to images of internal tissues obtained in a non-invasive manner for medical treatment or medical research, such as images of the stomach, abdomen, heart, knees, and brain, such as computed tomography (CT) imaging, magnetic resonance imaging (MRI), ultrasound (US) imaging, X-ray imaging, electroencephalogram, and optical photography lights generated by medical instruments. In the medical process, doctors are generally required to judge whether there are lesions or lesions in the image based on their professional knowledge. However, this process is inefficient and the results of judgments by doctors with different professional abilities may be different. For example, when judging the location of lesions such as low-grade astrocytomas, ganglia, and arteriovenous malformations in the brain, slight changes in shape may not be observed by the naked eye, resulting in inaccurate judgment of the lesion location.

[0003] The existing detection of lesion locations in images requires the use of a large model trained in the server, which relies on a large number of historical images. This requires a large amount of storage space, and the detection process requires a large amount of data to be processed, resulting in low efficiency. Therefore, how to implement lightweight lesion location detection on edge devices to improve detection efficiency and practicality has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a lesion location detection method, apparatus, device and medium based on an edge device to solve the problem of how to achieve lightweight lesion location detection on an edge device to improve detection efficiency and practicality.

[0005] In a first aspect, an embodiment of the present application provides a lesion location detection method based on an edge device, wherein the lesion location detection method is executed in an edge device running a lightweight detection model, wherein a replay storage area supporting calculation of the detection model is provided in the edge device, wherein the replay storage area stores activation values ​​formed based on historical images, wherein the detection model includes a coding layer, a replay layer, and a fully connected layer, and the lesion location detection method includes: Obtaining N real-time images acquired by connecting the edge device to an image acquisition instrument and a historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; Determine a first number and a second number according to the input upper limit number of the detection model in one round of execution, the number of the N real-time images, and the history number, wherein the sum of the first number and the second number is the input upper limit number, the first number is positively correlated with the number of images, and the second number is positively correlated with the history number; Selecting the second number of activation values ​​from the replay storage area as target activation values, selecting the first number of real-time images from the N real-time images as target images, and encoding the target images using the encoding layer to obtain encoding results; The encoding result and the target activation value are connected using the replay layer to obtain a connection result, and the connection result is detected by the fully connected layer to obtain a detection result of the lesion position in the target image.

[0006] Optionally, determining the first quantity and the second quantity according to the input upper limit quantity of the detection model in one round of execution, the number of the N real-time images, and the number of history images includes: According to the number of images of the N real-time images and the number of history images, calculating a first proportion of the number of images in the sum of the number of images and the number of history images, and a second proportion of the number of history images in the sum of the number of images and the number of history images; The first proportion is multiplied by the upper limit number of inputs of the detection model in one round of execution to obtain a first number, and the second proportion is multiplied by the upper limit number of inputs to obtain a second number.

[0007] Optionally, the encoding result and the target activation value are connected using the replay layer, and the connection result obtained includes: Obtaining an activation function of the replay layer; Based on the target activation value and the activation function, an activation operation is performed on the encoding result to obtain an activation result, and the activation result is determined to be a connection result.

[0008] Optionally, using the fully connected layer to perform lesion position detection on the connection result, and obtaining the detection result of the lesion position in the target image includes: Using the fully connected layer to perform a fully connected calculation on the connection result, to obtain calculation results corresponding to all preset categories of the fully connected layer; For any preset category, the classification result of determining whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image is the preset category; All preset categories are traversed to determine that the detection result of the lesion position in the target image is the probability of each pixel point in the target image in each preset category.

[0009] Optionally, after encoding the target image using the encoding layer to obtain an encoding result, the method further includes: Return to the step of selecting the first number of real-time images from the N real-time images as target images until all real-time images are selected, and obtain the encoding result corresponding to each real-time image; For any real-time image, taking the encoding result of the real-time image as the activation value of the real-time image, to obtain the activation values ​​of the N real-time images; A preset number of activation values ​​are selected from the activation values ​​of the N real-time images as update activation values, and the update activation values ​​are used to update the activation values ​​of the replay storage area.

[0010] Optionally, using the update activation value to update the activation value of the replay storage area includes: Selecting the preset number of activation values ​​from the replay storage area, and deleting the selected preset number of activation values; The updated activation value is stored in the replay storage area.

[0011] Optionally, before selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as update activation values, the method further includes: Multiplying the historical quantity by a preset ratio value to obtain a multiplication result; The multiplication result is used as the preset number.

[0012] In a second aspect, an embodiment of the present application provides a lesion position detection device based on an edge device, wherein the lesion position detection device is executed in an edge device running a lightweight detection model, wherein a replay storage area supporting calculation of the detection model is provided in the edge device, wherein the replay storage area stores activation values ​​formed based on historical images, wherein the detection model includes a coding layer, a replay layer, and a fully connected layer, and wherein the lesion position detection device includes: An information acquisition module, used to acquire N real-time images acquired by the edge device connected to the image acquisition instrument and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; a quantity determination module, configured to determine a first quantity and a second quantity according to an upper limit quantity of inputs of the detection model in one round of execution, the number of images of the N real-time images, and the number of history, wherein the sum of the first quantity and the second quantity is the upper limit quantity of inputs, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the number of history; an image encoding module, configured to select the second number of activation values ​​from the replay storage area as target activation values, select the first number of real-time images from the N real-time images as target images, and use the encoding layer to encode the target images to obtain encoding results; A detection and classification module is used to use the replay layer to connect the encoding result and the target activation value to obtain a connection result, and use the fully connected layer to detect the lesion position on the connection result to obtain a detection result of the lesion position in the target image.

[0013] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the lesion location detection method as described in the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting the location of a lesion as described in the first aspect is implemented.

[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the method of the present application is executed in an edge device running a lightweight detection model, a replay storage area supporting the calculation of the detection model is provided in the edge device, the replay storage area stores activation values ​​formed based on historical images, the detection model includes a coding layer, a replay layer and a fully connected layer, the method obtains N real-time images acquired by the edge device connected to an image acquisition instrument and the historical number of activation values ​​stored in the replay storage area, determines the first number and the second number according to the input upper limit number of the detection model in one round of execution, the number of images of the N real-time images and the historical number, and selects from the replay storage area The second number of activation values ​​is taken as the target activation values, the first number of real-time images are selected from N real-time images as the target images, the target images are encoded using the encoding layer to obtain the encoding result, the encoding result and the target activation value are connected using the replay layer to obtain the connection result, the connection result is performed on the connection result for lesion position detection, and the detection result of the lesion position in the target image is obtained, a lightweight model is used in the edge device to realize the lesion position detection of the image, and a replay storage area is set to store the activation value instead of the historical image itself, which can greatly reduce the storage space occupied and can accurately obtain the detection result in combination with the real-time image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 This is a schematic diagram of an application environment of a lesion location detection method based on an edge device provided in Example 1 of the present application; Figure 2 It is a flow chart of a lesion location detection method based on an edge device provided in Embodiment 2 of the present application; Figure 3 It is a flowchart of a lesion location detection method based on an edge device provided in Example 3 of the present application; Figure 4 It is a structural schematic diagram of a lesion location detection device based on an edge device provided in Embodiment 4 of the present application; Figure 5 It is a structural diagram of a computer device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0018] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0019] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0020] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0021] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0022] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0023] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0024] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0025] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0026] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0027] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0028] The first embodiment of the present application provides a method for detecting the location of a lesion based on an edge device, which can be applied in the following aspects: Figure 1 In the application environment, the image acquisition instrument communicates with the edge device. The edge device includes but is not limited to a PDA, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computer device, a personal digital assistant (PDA) and other computer devices. The image acquisition instrument may refer to a medical instrument corresponding to the CT, MRI, US and other images taken by the patient during the medical examination.

[0029] See also Figure 2 , is a flow chart of a lesion location detection method based on an edge device provided in Example 2 of the present application. The lesion location detection method based on an edge device is applied to Figure 1 The edge device in the edge device, the computer device corresponding to the edge device is provided with a lightweight detection model, the edge device is provided with a replay storage area supporting the calculation of the detection model, the replay storage area stores activation values ​​formed based on historical images, and the detection model includes a coding layer, a replay layer and a fully connected layer. The above-mentioned computer device can also be connected to a corresponding image acquisition instrument to obtain medical images of the patient collected by the image acquisition instrument. Figure 2As shown, the edge device-based lesion location detection method may include the following steps: Step S201, obtaining N real-time images acquired by connecting the edge device to an image acquisition instrument and the historical number of activation values ​​stored in a replay storage area.

[0030] In the present application, when the patient reaches the designated position of the image acquisition instrument according to the instructions, the image acquisition instrument is controlled to start to acquire a human body image of the patient's affected part. The acquisition number is set to N, where N is an integer greater than zero. This can reduce the result error caused by the error of the image acquisition instrument in subsequent use.

[0031] The replay storage area may refer to a subspace allocated in the storage space in the edge device for storing activation values, and the replay storage area can be used when the edge device runs the method of the present application. The stored activation value is the activation value of the historical image obtained after the historical image is processed when the method of the present application is run historically, and is stored in the replay storage area.

[0032] The number of activation values ​​stored in the replay storage area is counted to obtain a historical number, wherein the number of activation values ​​is the same as the number of historical images, that is, one historical image corresponds to one activation value.

[0033] Step S202, determining a first quantity and a second quantity according to an upper limit number of inputs of the detection model in one round of execution, the number of N real-time images, and the number of historical images.

[0034] In this application, the detection model includes a coding layer, a playback layer and a fully connected layer. The coding layer is used to encode the image, the playback layer is used to receive the encoding result and activation value of the image and fuse the two, and the fully connected layer is used to perform a fully connected prediction on the fused result to obtain the prediction result (that is, the detection result).

[0035] During one execution process (also known as one round of execution), the coding layer of the detection model has a fixed upper limit on the number of input images, i.e., the upper limit of input number. Images exceeding this upper limit cannot be processed. Therefore, in order to clearly determine the first number of input real-time images and the second number of historical images, calculations need to be based on this upper limit of input number.

[0036] The sum of the first number and the second number is the input upper limit number, the first number is positively correlated with the number of images, and the second number is positively correlated with the number of history, that is, the higher the first number, the higher the number of images, and the higher the second number, the higher the number of history. This makes the detection model more suitable for the requirements of use, and at the same time, when there are more real-time images, the detection basis is biased towards real-time, making the result more accurate. When there are fewer real-time images, the detection basis is biased towards history, meeting the calculation conditions and a certain degree of accuracy.

[0037] Optionally, determining the first number and the second number according to an upper limit number of inputs of the detection model in one round of execution, the number of N real-time images, and the number of history images includes: According to the number of images and the number of history of the N real-time images, a first proportion of the number of images in the sum of the number of images and the number of history, and a second proportion of the number of history in the sum of the number of images and the number of history are calculated; The product of the first proportion and the upper limit number of inputs of the detection model in one round of execution is calculated to obtain a first number, and the product of the second proportion and the upper limit number of inputs is calculated to obtain a second number.

[0038] Among them, the calculation formula can be used for quantity calculation, the formula is as follows: In the formula, represents the first quantity, represents the second quantity, Indicates the upper limit of input quantity. Indicates the number of images, Indicates the historical quantity.

[0039] Step S203, selecting a second number of activation values ​​from the replay storage area as target activation values, selecting a first number of real-time images from the N real-time images as target images, and encoding the target images using the encoding layer to obtain encoding results.

[0040] In the present application, the activation values ​​in the replay storage area are called, the number of calls is the second number, and a first number of real-time images are selected from N real-time images as target images, and the first number of real-time images and the second number of activation values ​​are input into the detection model, that is, input into the coding layer for encoding.

[0041] The encoding layer is used to encode the image into a data format that can be processed by the detection model. Specifically, the image can be vector encoded, that is, the image is expressed in the form of a vector, and the expression result is the encoding result.

[0042] In addition, the encoding layer may be a layer composed of multiple convolutional layers, and the sizes of the convolution kernels of each convolutional layer decrease successively to form a focus on the detailed features of the image.

[0043] The target activation value is equivalent to the encoding result of the encoded historical image and can be directly used in subsequent steps without going through the encoding processing of the encoding layer.

[0044] Step S204, using the replay layer to connect the encoding result and the target activation value to obtain a connection result, using the fully connected layer to detect the lesion position on the connection result, and obtaining the detection result of the lesion position in the target image.

[0045] In the present application, the input of the playback layer is connected to the output of the encoding layer to obtain the encoding result output by the encoding layer, and the input of the playback layer is also connected to the playback storage area to obtain the activation value of the playback storage area.

[0046] The replay layer is a connection layer, which is used to connect the encoding result and the activation value to obtain a connection result. The connection is specifically to activate the encoding result. The activation method may include weighted activation, merged activation, multiplication activation, etc.

[0047] As for the connection results, the classification results of each pixel in each image can be obtained through calculation of the fully connected layer. Among them, after training, the fully connected layer has the ability to determine whether a pixel is a lesion. The classification result is the probability of determining whether it is a lesion. The larger the probability, the more likely it is a lesion, and the smaller the probability, the less likely it is a lesion.

[0048] Optionally, the encoding result and the target activation value are connected using a replay layer, and the connection result includes: Get the activation function of the replay layer; Based on the target activation value and the activation function, an activation operation is performed on the encoding result to obtain an activation result, and the activation result is determined to be a connection result.

[0049] Among them, activation functions are used to learn and understand very complex and nonlinear functions for the detection model, and can introduce nonlinear characteristics into the network. In neurons, the inputs are weighted, summed, and then a function is applied, which is the activation function. The target activation value is used as the weight value, and the activation function is used as the calculation formula to calculate the encoding result, that is, to perform an activation operation to obtain the activation result, which is the connection result.

[0050] Optionally, a fully connected layer is used to detect the lesion position on the connection result, and the detection result of the lesion position in the target image includes: Use the fully connected layer to perform full connection calculation on the connection result to obtain the calculation results of all preset categories of the corresponding fully connected layer; For any preset category, the classification result of determining whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image is the preset category; All preset categories are traversed to determine that the detection result of the lesion position in the target image is the probability of each pixel point in the target image in each preset category.

[0051] Among them, the full connection calculation of the connection results is used to map the learned "distributed feature representation" to the sample label space. Each node is connected to all the nodes in the previous layer to integrate the features extracted from the previous layers to obtain the calculation results of each preset category, that is, the probability of whether each pixel is a lesion.

[0052] Furthermore, after the detection results are obtained, the detection results are analyzed, and the pixel points with probabilities greater than the probability threshold are integrated to obtain the lesion area.

[0053] The method of the embodiment of the present application is executed in an edge device running a lightweight detection model. A replay storage area supporting the calculation of the detection model is set in the edge device. The replay storage area stores activation values ​​formed based on historical images. The detection model includes a coding layer, a replay layer, and a fully connected layer. The method obtains N real-time images acquired by the edge device connected to an image acquisition instrument and the historical number of activation values ​​stored in the replay storage area. According to the input upper limit number of the detection model in one round of execution, the number of images of the N real-time images, and the historical number, the first number and the second number are determined, and the second number of activation values ​​are selected from the replay storage area as A target activation value is selected, a first number of real-time images are selected from N real-time images as target images, the target images are encoded using a coding layer to obtain a coding result, the coding result and the target activation value are connected using a replay layer to obtain a connection result, a fully connected layer is used to detect the connection result for lesion positions, and a detection result of the lesion position in the target image is obtained, a lightweight model is used in the edge device to implement image lesion position detection, and a replay storage area is set to store activation values ​​instead of storing historical images themselves, which can greatly reduce the storage space occupied and can accurately obtain detection results in combination with real-time images.

[0054] See also Figure 3 , is a flow chart of a lesion location detection method based on an edge device provided in Example 3 of the present application. The lesion location detection method based on an edge device is applied to Figure 1The edge device in the edge device, the computer device corresponding to the edge device is provided with a lightweight detection model, the edge device is provided with a replay storage area supporting the calculation of the detection model, the replay storage area stores activation values ​​formed based on historical images, and the detection model includes a coding layer, a replay layer and a fully connected layer. The above-mentioned computer device can also be connected to a corresponding image acquisition instrument to obtain medical images of the patient collected by the image acquisition instrument. Figure 3 As shown, the edge device-based lesion location detection method may include the following steps: Step S301, obtaining N real-time images acquired by the edge device connected to the image acquisition instrument and the historical number of activation values ​​stored in the replay storage area.

[0055] Step S302, determining a first quantity and a second quantity according to an upper limit number of inputs of the detection model in one round of execution, the number of N real-time images, and the number of historical images.

[0056] Step S303, select a second number of activation values ​​from the replay storage area as target activation values, select a first number of real-time images from the N real-time images as target images, use the coding layer to encode the target images, and obtain a coding result.

[0057] Step S304, using the replay layer to connect the encoding result and the target activation value to obtain a connection result, using the fully connected layer to detect the lesion position on the connection result, and obtaining the detection result of the lesion position in the target image.

[0058] Among them, the contents of step S301 to step S304 are partially the same as the contents of the above-mentioned steps S201 to step S204, and reference may be made to the description of steps S201 to step S204, which will not be repeated here.

[0059] Step S305, returning to the step of selecting a first number of real-time images from the N real-time images as target images, until all the real-time images are selected, and obtaining the encoding result corresponding to each real-time image.

[0060] In the present application, after each round of executing the method of the embodiment, each real-time image is encoded to obtain an encoding result for use in subsequent steps.

[0061] Specifically, all real-time images are traversed so that each real-time image is used as a target image, and the target image is input into the encoding layer, thereby obtaining a corresponding encoding result.

[0062] Step S306: for any real-time image, the encoding result of the real-time image is used as the activation value of the real-time image to obtain the activation values ​​of N real-time images.

[0063] In this application, the encoding result is also the activation value. After each real-time image is obtained, the activation value of N real-time images is obtained. This activation value can be used for subsequent operations to update the replay storage area. Step S307 , selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as update activation values, and using the update activation values ​​to update the activation values ​​in the replay storage area.

[0064] In this application, for use in the next round, the corresponding activation value is selected from the N activation values ​​corresponding to the N real-time images of this round as the updated activation value, wherein the number of selected activation values ​​is determined by the total number of activation values ​​in the replay storage area.

[0065] The update may be to add a new activation value to the replay storage area, or to overwrite the activation value in the replay storage area, or to replace part of the activation value in the replay storage area. Optionally, updating the activation value of the replay storage area using the updated activation value includes: Selecting a preset number of activation values ​​from the replay storage area, and deleting the selected preset number of activation values; The updated activation value is stored in the replay storage area.

[0066] Among them, a certain number of activation values ​​are selected from the replay storage area, a deletion operation is adopted, and then the updated activation values ​​are stored in the replay storage area to ensure that the number of activation values ​​in the replay storage area is inconvenient, so that each round of processing has good consistency.

[0067] Optionally, before selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as the updated activation values, the method further includes: Multiply the historical quantity by the preset ratio value to get the multiplication result; The multiplication result is used as the preset number.

[0068] Among them, the preset number is set in a proportional manner. If the historical number of activation values ​​stored in the replay storage area is large, the corresponding preset number is large. If the historical number of activation values ​​stored in the replay storage area is small, the corresponding preset number is low, thereby improving the universality of the replay storage area.

[0069] The method of the embodiment of the present application is executed in an edge device running a lightweight detection model. A replay storage area supporting the calculation of the detection model is set in the edge device. The replay storage area stores activation values ​​formed based on historical images. The detection model includes a coding layer, a replay layer and a fully connected layer. The method obtains N real-time images acquired by the edge device connected to an image acquisition instrument and the historical number of activation values ​​stored in the replay storage area. According to the input upper limit number of the detection model in one round of execution, the number of images of the N real-time images and the historical number, a first number and a second number are determined, the second number of activation values ​​are selected from the replay storage area as the target activation value, the first number of real-time images are selected from the N real-time images as the target image, and the coding layer is used to generate a target image. The image is encoded to obtain an encoding result, the encoding result and the target activation value are connected using a replay layer to obtain a connection result, the connection result is detected by a fully connected layer to obtain a detection result of the lesion position in the target image, and the step of selecting a first number of real-time images from N real-time images as target images is returned to execute until all real-time images are selected to obtain an encoding result corresponding to each real-time image, for any real-time image, the encoding result of the real-time image is used as the activation value of the real-time image to obtain the activation values ​​of the N real-time images, a preset number of activation values ​​are selected from the activation values ​​of the N real-time images as update activation values, and the activation value of the replay storage area is updated using the updated activation value, so that the replay storage area can be updated in real time.

[0070] Corresponding to the lesion location detection method based on edge device in the above embodiment, Figure 4 The structure block diagram of the lesion position detection device based on the edge device provided in the fourth embodiment of the present application is shown. The lesion position detection device is applied to Figure 1 The edge device in the edge device, the computer device corresponding to the edge device is provided with a lightweight detection model, the edge device is provided with a replay storage area supporting the calculation of the detection model, the replay storage area stores activation values ​​formed based on historical images, and the detection model includes a coding layer, a replay layer and a fully connected layer. The above-mentioned computer device can also be connected to a corresponding image acquisition instrument to obtain medical images of the patient acquired by the image acquisition instrument. For ease of explanation, only the parts related to the embodiments of the present application are shown.

[0071] See also Figure 4 , the lesion position detection device comprises: The information acquisition module 41 is used to obtain N real-time images acquired by the edge device connected to the image acquisition instrument and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; A quantity determination module 42 is used to determine a first quantity and a second quantity according to an input upper limit quantity, a number of images of N real-time images, and a history quantity of the detection model in one round of execution, wherein the sum of the first quantity and the second quantity is the input upper limit quantity, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the history quantity; The image encoding module 43 is used to select a second number of activation values ​​from the replay storage area as target activation values, select a first number of real-time images from the N real-time images as target images, and use the encoding layer to encode the target images to obtain encoding results; The detection and classification module 44 is used to use the replay layer to connect the encoding result and the target activation value to obtain a connection result, and use the fully connected layer to detect the lesion position on the connection result to obtain a detection result of the lesion position in the target image.

[0072] Optionally, the quantity determination module 42 includes: A proportion determination unit, used to calculate a first proportion of the number of images in the sum of the number of images and the number of history, and a second proportion of the number of history in the sum of the number of images and the number of history, according to the number of images and the number of history of the N real-time images; The quantity calculation unit is used to calculate the product of the first proportion and the upper limit number of inputs of the detection model in one round of execution to obtain the first quantity, and calculate the product of the second proportion and the upper limit number of inputs to obtain the second quantity.

[0073] Optionally, the detection and classification module 44 includes: An activation function acquisition unit, used to acquire the activation function of the replay layer; The activation operation unit is used to perform an activation operation on the encoding result based on the target activation value and the activation function to obtain an activation result, and determine that the activation result is a connection result.

[0074] Optionally, the detection and classification module 44 includes: A fully connected computing unit, used to perform a fully connected computing on the connection result using the fully connected layer, and obtain computing results of all preset categories corresponding to the fully connected layer; A preset category determination unit, for determining, for any preset category, whether each pixel in the target image is a lesion, as a classification result of the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image is the preset category; The category probability determination unit traverses all preset categories and determines that the detection result of the lesion position in the target image is the probability of each pixel point in the target image in each preset category.

[0075] Optionally, the lesion location detection device further includes: The coding traversal module is used to encode the target image using the coding layer, and after obtaining the coding result, return to execute the step of selecting a first number of real-time images from the N real-time images as the target image, until all the real-time images are selected, and obtain the coding result corresponding to each real-time image; A real-time activation value determination module is used to obtain activation values ​​of N real-time images by using the encoding result of the real-time image as the activation value of the real-time image for any real-time image; The storage area updating module is used to select a preset number of activation values ​​from the activation values ​​of N real-time images as update activation values, and use the update activation values ​​to update the activation values ​​of the replay storage area.

[0076] Optionally, the storage area update module includes: A deleting unit, used for selecting a preset number of activation values ​​from the replay storage area and deleting the selected preset number of activation values; The storage unit is used to store the updated activation value such as a replay storage area.

[0077] Optionally, the lesion location detection device further includes: A proportional multiplication module, used for multiplying the historical quantity by a preset proportional value to obtain a multiplication result before selecting a preset number of activation values ​​from the activation values ​​of N real-time images as update activation values; The preset number determination module is used to use the multiplication result as the preset number.

[0078] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0079] Figure 5 This is a schematic diagram of the structure of a computer device provided in Example 5 of this application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, and when the processor executes the computer program, the steps in any of the above-mentioned embodiments of the lesion location detection method based on the edge device are implemented.

[0080] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 5 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0081] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0082] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0083] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0084] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0087] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A lesion location detection method based on edge devices, characterized in that: The lesion location detection method is executed in an edge device running a lightweight detection model, the edge device is provided with a replay storage area supporting the calculation of the detection model, the replay storage area stores activation values ​​formed based on historical images, the detection model includes a coding layer, a replay layer and a fully connected layer, and the lesion location detection method includes: Obtaining N real-time images acquired by connecting the edge device to an image acquisition instrument and a historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; Determine a first number and a second number according to the input upper limit number of the detection model in one round of execution, the number of the N real-time images, and the history number, wherein the sum of the first number and the second number is the input upper limit number, the first number is positively correlated with the number of images, and the second number is positively correlated with the history number; Selecting the second number of activation values ​​from the replay storage area as target activation values, selecting the first number of real-time images from the N real-time images as target images, and encoding the target images using the encoding layer to obtain encoding results; The encoding result and the target activation value are connected using the replay layer to obtain a connection result, and the connection result is detected by the fully connected layer to obtain a detection result of the lesion position in the target image.

2. The method for detecting the position of a lesion according to claim 1, characterized in that: Determining the first quantity and the second quantity according to the input upper limit quantity of the detection model in one round of execution, the number of the N real-time images, and the history quantity includes: According to the number of images of the N real-time images and the number of history images, calculating a first proportion of the number of images in the sum of the number of images and the number of history images, and a second proportion of the number of history images in the sum of the number of images and the number of history images; The first proportion is multiplied by the upper limit number of inputs of the detection model in one round of execution to obtain a first number, and the second proportion is multiplied by the upper limit number of inputs to obtain a second number.

3. The method for detecting the lesion position according to claim 1, characterized in that: The encoding result and the target activation value are connected using the replay layer, and the connection result obtained includes: Obtaining an activation function of the playback layer; Based on the target activation value and the activation function, an activation operation is performed on the encoding result to obtain an activation result, and the activation result is determined to be a connection result.

4. The method for detecting the lesion position according to claim 1, characterized in that: Using the fully connected layer to perform lesion position detection on the connection result, and obtaining the detection result of the lesion position in the target image includes: Using the fully connected layer to perform a fully connected calculation on the connection result, to obtain calculation results corresponding to all preset categories of the fully connected layer; For any preset category, the classification result of determining whether each pixel in the target image is a lesion is the preset category, and the calculation result corresponding to the preset category is the probability that each pixel in the target image is the preset category; All preset categories are traversed to determine that the detection result of the lesion position in the target image is the probability of each pixel point in the target image in each preset category.

5. The method for detecting the location of a lesion according to any one of claims 1 to 4, characterized in that: After encoding the target image using the encoding layer to obtain an encoding result, the method further includes: Return to the step of selecting the first number of real-time images from the N real-time images as target images until all real-time images are selected, and obtain the encoding result corresponding to each real-time image; For any real-time image, taking the encoding result of the real-time image as the activation value of the real-time image, to obtain the activation values ​​of the N real-time images; A preset number of activation values ​​are selected from the activation values ​​of the N real-time images as update activation values, and the update activation values ​​are used to update the activation values ​​of the replay storage area.

6. The method for detecting the lesion position according to claim 5, characterized in that: Updating the activation value of the replay storage area using the update activation value includes: Selecting the preset number of activation values ​​from the replay storage area, and deleting the selected preset number of activation values; The updated activation value is stored in the replay storage area.

7. The method for detecting the lesion position according to claim 5, characterized in that: Before selecting a preset number of activation values ​​from the activation values ​​of the N real-time images as update activation values, the method further includes: Multiplying the historical quantity by a preset ratio value to obtain a multiplication result; The multiplication result is used as the preset number.

8. A lesion location detection device based on edge device, characterized in that: The lesion position detection device is executed in an edge device running a lightweight detection model, the edge device is provided with a replay storage area supporting the calculation of the detection model, the replay storage area stores activation values ​​formed based on historical images, the detection model includes a coding layer, a replay layer and a fully connected layer, and the lesion position detection device includes: An information acquisition module, used to acquire N real-time images acquired by the edge device connected to the image acquisition instrument and the historical number of activation values ​​stored in the replay storage area, where N is an integer greater than zero; a quantity determination module, configured to determine a first quantity and a second quantity according to an upper limit quantity of inputs of the detection model in one round of execution, the number of images of the N real-time images, and the number of history, wherein the sum of the first quantity and the second quantity is the upper limit quantity of inputs, the first quantity is positively correlated with the number of images, and the second quantity is positively correlated with the number of history; an image encoding module, configured to select the second number of activation values ​​from the replay storage area as target activation values, select the first number of real-time images from the N real-time images as target images, and use the encoding layer to encode the target images to obtain encoding results; A detection and classification module is used to use the replay layer to connect the encoding result and the target activation value to obtain a connection result, and use the fully connected layer to detect the lesion position on the connection result to obtain a detection result of the lesion position in the target image.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting the location of a lesion according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting the location of a lesion as claimed in any one of claims 1 to 7 is implemented.

Images