Image detection method and device and storage medium

Abstract

The invention relates to an image detection method and device and a storage medium, and the method comprises the steps: obtaining a to-be-detected image; determining an indicator area of the to-be-detected image; based on the obtained expansion coefficient, performing expansion and amplification processing on the indicator area to obtain an expansion image; calculating position information of an indicator according to the expanded image; and generating target motion information corresponding to the to-be-detected image based on the indicator position information. According to the invention, the problem of low accuracy of image detection is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image detection method, device and storage medium. Background Art

[0002] During a head Magnetic Resonance (MR) scan, the patient has uncontrollable movement relative to the head coil and magnetic field, resulting in blurred images, which is not conducive to subsequent analysis and diagnosis. To improve the imaging quality in MRI or other scenarios, an algorithm is needed to capture the head's motion information in real time and feed it back to the system, thereby achieving motion caption (Mocap) of the magnetic field. The mainstream MR Mocap methods currently on the market are limited by the sensitivity of the hardware, image resolution, and the robustness of the algorithm, and often cannot achieve high-precision motion information capture, resulting in unsatisfactory motion correction effects, which in turn affects the accuracy of image detection.

[0003] Currently, no effective solution has been proposed for the problem of low accuracy of image detection in related technologies. Summary of the invention

[0004] The embodiments of the present application provide an image detection method, device, and storage medium to at least solve the problem of low accuracy of image detection in the related art.

[0005] In a first aspect, an embodiment of the present application provides an image detection method, the method comprising:

[0006] Acquire the image to be detected;

[0007] Determine the indicator region of the image to be detected; and based on the obtained expansion coefficient, perform expansion and magnification processing on the indicator region to obtain an expanded image;

[0008] Calculating the indicator position information according to the dilated image;

[0009] Based on the indicator position information, target motion information corresponding to the image to be detected is generated.

[0010] In some embodiments, determining the indicator area of ​​the image to be detected includes:

[0011] Get the preset indicator template;

[0012] Performing template matching between the indicator template and the image to be detected to obtain an initial matching area;

[0013] The edge information is determined based on the initial matching area, and the indicator area is obtained according to the initial matching area and the edge information.

[0014] In some embodiments, the method further comprises:

[0015] Acquire first scale data and second scale data; the first scale data is scale data of the image to be detected, and the second scale data is scale data of the indicator area; determine the expansion coefficient according to the correlation relationship between the first scale data and the second scale data; or,

[0016] Acquire third scale data; the third scale data is scale data of a preset target expansion area; and acquire the expansion coefficient according to the association relationship between the third scale data and the second scale data of the indicator area.

[0017] In some embodiments, calculating the indicator position information according to the dilated image includes:

[0018] The dilated image is input into a trained position detection model for detection processing, and the indicator position information is output.

[0019] In some embodiments, the step of inputting the dilated image into a trained position detection model for detection processing and outputting the indicator position information includes:

[0020] Inputting the expansion image into the position detection model to obtain a thermal map corresponding to the expansion image;

[0021] Determine a target feature point in the thermal map whose response value reaches a preset response value threshold; and obtain the indicator position information according to the feature point coordinates of the target feature point.

[0022] In some embodiments, the method further comprises:

[0023] Acquire an original training image; determine a training area of ​​the original training image, and perform dilation and magnification processing on the training area to obtain a target training image; the target training image carries a coordinate label;

[0024] Inputting the target training image into the initial neural network model for training, and outputting a position prediction result;

[0025] Based on the coordinate labels and the position prediction results, a loss function result is calculated; the gradient of the loss function result is back-transmitted to the initial neural network model for iterative training to obtain the position detection model.

[0026] In some embodiments, generating target motion information corresponding to the image to be detected based on the indicator position information includes:

[0027] Obtaining original coordinates of the indicator area in the image to be detected;

[0028] According to the original coordinates, the expansion coefficient and the indicator position information, the target position information of the feature point in the expansion image on the image to be detected is calculated, and based on the target position information, the target motion information is determined.

[0029] In some embodiments, the method further comprises:

[0030] Acquire a video stream, and determine a current reference frame from the video stream;

[0031] If determining the indicator area based on the reference frame fails, obtaining the current cumulative number of failed frames;

[0032] When it is detected that the accumulated number of failed frames is greater than a preset frame number threshold, determining a new reference frame from the video stream, and acquiring the indicator position information based on the new reference frame;

[0033] When it is detected that the accumulated number of failed frames is less than or equal to the frame number threshold, the indicator position information is acquired based on a previous reference frame.

[0034] In a second aspect, an embodiment of the present application provides an image detection device, the device comprising:

[0035] An acquisition module, used for acquiring an image to be detected;

[0036] An expansion module is used to determine the indicator area of ​​the image to be detected; and to perform expansion and amplification processing on the indicator area to obtain an expanded image;

[0037] A position detection module, used for inputting the dilated image into the trained position detection module and outputting indicator position information;

[0038] A generating module is used to generate target motion information corresponding to the image to be detected based on the indicator position information.

[0039] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the image detection method as described in the first aspect above is implemented.

[0040] Compared with the related art, the image detection method, device and storage medium provided in the embodiment of the present application obtain the image to be detected; determine the indicator area of ​​the image to be detected; based on the preset expansion coefficient, the indicator area is expanded and enlarged to obtain an expanded image; the expanded image is input into the trained position detection model to output the indicator position information; based on the indicator position information, the target motion information corresponding to the image to be detected is generated. Based on this, the significance of the indicator in the image is effectively enhanced, so that the subsequent position detection step can capture the position of the indicator more easily and accurately; through the expansion and enlargement process, even if the indicator is relatively small or blurred in the original image, it can be effectively enlarged and clarified in the expanded image, thereby realizing the accurate recognition of the position of the indicator with high-definition characteristics, which can meet the requirements of super-resolution, high real-time and high-precision image feature point detection in the camera imaging plane, better stimulate the development potential of downstream tasks such as motion capture, and effectively solve the problem of low accuracy of image detection.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 is a schematic structural diagram of a magnetic resonance system according to an embodiment of the present application;

[0044] Figure 2 is a flow chart of an image detection method according to an embodiment of the present application;

[0045] Figure 3 is a schematic diagram of an indicator template according to an embodiment of the present application;

[0046] Figure 4 is a schematic diagram of an image to be detected according to an embodiment of the present application;

[0047] Figure 5 is a flow chart of another image detection method according to an embodiment of the present application;

[0048] Figure 6 It is a structural block diagram of an image detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0050] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0051] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0052] The methods, devices, equipment or computer-readable storage media involved in this application can be used not only for non-invasive imaging, such as diagnosis and research of diseases, but also for industrial fields, etc.; the systems involved may include magnetic resonance systems (MR systems), positron emission computed tomography-magnetic resonance multimodal hybrid (Positron Emission Computed Tomography-Magnetic Resonance, PET-MR) systems, etc. The methods, devices, equipment or computer-readable storage media involved in this application can be integrated with the above-mentioned systems or can be relatively independent. In some embodiments, the magnetic resonance images can be obtained from a magnetic resonance image database instead of directly from the magnetic resonance imaging system to implement the methods, devices, equipment or computer-readable storage media involved in this application.

[0053] The embodiments of the present application will be described below using a magnetic resonance system as an example.

[0054] This embodiment provides a magnetic resonance system. Figure 1 is a schematic diagram of the structure of a magnetic resonance system according to an embodiment of the present application, such as Figure 1 As shown, the magnetic resonance system includes: a scanner and a computer, wherein the computer includes a memory 125 , a processor 122 , and a computer program stored in the memory 125 and executable on the processor 122 .

[0055] The scanner has a bore with an imaging field of view, which usually includes a magnetic resonance gantry, in which there is a main magnet 101, which can be composed of superconducting coils to generate a main magnetic field, and in some cases a permanent magnet can also be used. The main magnet 101 can be used to generate a main magnetic field strength of 0.2 Tesla, 0.5 Tesla, 1.0 Tesla, 1.5 Tesla, 3.0 Tesla or higher. During magnetic resonance imaging, the imaging object 150 will be carried by the patient bed 106, and as the bed moves, the imaging object 150 is moved into the area 105 where the magnetic field distribution of the main magnetic field is relatively uniform. Usually for a magnetic resonance system, such as Figure 1 As shown, the z direction of the spatial coordinate system (i.e., the coordinate system of the device) is set to be the same as the axial direction of the magnetic resonance system frame. The patient's body length direction is usually kept consistent with the z direction for imaging. The horizontal plane of the magnetic resonance system is set to the xz plane, the x direction is perpendicular to the z direction, and the y direction is perpendicular to both the x and z directions.

[0056] In magnetic resonance imaging, the pulse control unit 111 controls the RF pulse generating unit 116 to generate RF pulses. After being amplified by the amplifier, the RF pulses are sent out by the body coil 103 or the local coil 104 through the switch control unit 117 to perform RF excitation on the imaging object 150. The imaging object 150 will generate corresponding RF signals by resonance according to the RF excitation. When receiving the RF signal generated by the imaging object 150 according to the excitation, it can be received by the body coil 103 or the local coil 104. There can be many RF receiving links. After the RF signal is sent to the RF receiving unit 118, it is further sent to the image reconstruction unit 121 for image reconstruction to form a magnetic resonance image.

[0057] The magnetic resonance scanner also includes a gradient coil 102, which can be used to spatially encode radio frequency signals during magnetic resonance imaging. A pulse control unit 111 controls a gradient signal generating unit 112 to generate a gradient signal, which is usually divided into three mutually orthogonal direction signals: x direction, y direction and z direction. The gradient signals in different directions are amplified by gradient amplifiers (113, 114, 115) and then emitted by the gradient coil 102 to generate a gradient magnetic field in the region 105.

[0058] The pulse control unit 111, the image reconstruction unit 121 and the processor 122, the display unit 123, the input / output device 124, the memory 125, and the communication port 126 can perform data transmission via the communication bus 127, thereby realizing the control of the magnetic resonance imaging process.

[0059] The processor 122 may be composed of one or more processors, which may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0060] The display unit 123 may be a display provided to the user for displaying images.

[0061] The input / output device 124 may be a keyboard, a mouse, a control box or other related devices, which supports input / output of corresponding data streams.

[0062] Among them, the memory 125 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 125 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 125 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 125 may be inside or outside the data processing device. In a specific embodiment, the memory 125 is a non-volatile memory. In a specific embodiment, the memory 125 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory (FLASH), or a combination of two or more of these. The memory 125 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible program instructions executed by the processor 122.Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0063] The communication port 126 can realize data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0064] The communication bus 127 includes hardware, software or both, and couples the components of the magnetic resonance system to each other. The communication bus 127 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, the communication bus 127 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of the above. Where appropriate, the communication bus 127 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0065] This embodiment provides an image detection method. Figure 2 is a flow chart of an image detection method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0066] Step S210, obtaining an image to be detected.

[0067] The above-mentioned image to be detected refers to an image of the target object that is obtained and contains real-time motion information to be analyzed. For example, in the application scenario of head MRI scanning, the target object may be the head part of the human body to be scanned, and the image to be detected is an MRI scan image containing the human head; for another example, in application scenarios such as gesture recognition, the target object may be the hand part, and the image to be detected is a color image containing the human hand. The image to be detected may be obtained by collecting a series of image frames and taking the key frame or any frame of the image as the image to be detected.

[0068] Step S220, determining the indicator region of the image to be detected; and performing expansion and magnification processing on the indicator region based on the obtained expansion coefficient to obtain an expanded image.

[0069] The above-mentioned indicator refers to a pre-set object with specific characteristics that is rigidly connected to the target object; when the target object moves slightly during the detection process, the indicator will also move accordingly. The indicator can be used to achieve subsequent motion capture. Since the computing power required to process the entire image is large and it is easily interfered by noise and useless information; therefore, the indicator can be first detected from the original image to be detected, so that the effective feature screening range can be framed while reducing the amount of calculation. Specifically, the indicator area can be determined by attaching at least one indicator on the surface of the target object such as the human body to be scanned, and in the actual application process, the image containing the target object is identified by using methods such as the target detection algorithm to obtain the above-mentioned indicator area.

[0070] It should be noted that, limited by the working distance of the camera, the number of effective pixels occupied by the indicator on the original image to be detected is relatively small. If only the part occupied by the indicator in the original image is directly processed, not only can the pixel coordinates not be directly and effectively extracted, but the image size input to the network is also not uniform. In order to solve the above problem, the indicator area obtained above can be expanded according to a specific expansion coefficient, and the image can be enlarged to a uniform size. The expansion coefficient can be set in advance by the staff. For example, the target detection algorithm detects that the coordinates of the upper left corner of the edge of the indicator area are (100,100) and the coordinates of the lower right corner are (200,200). The length and width are interpolated and enlarged by 1.4 times each, and finally the expanded image with the coordinates of the upper left corner (80,80) and the coordinates of the lower right corner (220,220) is obtained.

[0071] Step S230: Calculate the indicator position information according to the dilated image.

[0072] The target detection process is performed on the above-mentioned expansion image, so that the accurate position information of the indicator in the image can be calculated.

[0073] In addition, in an optional embodiment, the process of calculating the position information of the indicator can also be: inputting the dilated image into a trained position detection model for detection processing, and outputting the position information of the indicator. The position detection model refers to a neural network model that detects the current coordinate position of the dilated image; that is, the position detection model is used to provide a mapping relationship between the dilated image and the image coordinates. Specifically, the position detection model is used to detect feature points of the dilated image, and the two-dimensional image coordinates of each feature point are output in a predetermined order. In actual application, the position detection model outputs the indicator position information as the coordinates of each feature point in the dilated image.

[0074] Step S240: generating target motion information corresponding to the image to be detected based on the indicator position information.

[0075] In this step, the coordinates of the indicator position information adaptively detected and output by the neural network model are restored to the original image to be detected, and finally the coordinates of each feature point in the expanded image on the image to be detected are calculated. In addition, in another embodiment, the position information of at least two frames of images can be counted, so as to obtain the motion trajectory of the target object in consecutive frames.

[0076] In the above-mentioned image detection method, the detected indicator area is expanded and enlarged, and the expanded image obtained after the expansion is processed and input into the position detection model to obtain the indicator position information, thereby meeting the needs of super-resolution, high real-time and high-precision image feature point detection in the camera imaging plane, better stimulating the development potential of downstream tasks such as motion capture, and effectively solving the problem of low accuracy of image detection.

[0077] In some embodiments, the above-mentioned determination of the indicator area of ​​the image to be detected further includes the following steps:

[0078] Step S221, obtaining a preset indicator template; performing template matching between the indicator template and the image to be detected to obtain an initial matching area.

[0079] The indicator template refers to a template image containing a preset indicator. Figure 3, a pre-designed indicator template image with special marks is shown in the figure. Then the indicator template is matched with the collected image to be detected. The matching method can be: obtaining at least one image including a preset indicator, and using the image as the indicator template image, using traditional processing methods such as scale-invariant feature transform (SIFT) to extract the feature results of the area where the indicator is located, and then using a sliding window to search and match the feature results on the image to be detected to obtain the initial matching area where the indicator is located on the image to be detected; or, the pyramid template matching algorithm fused with fast non-maximum suppression (NMSFast) can be used to achieve template matching; or, a neural network model can be used, for example, using target detection technology such as target detection algorithm (You Only Look Once version 4, YoloV4), after marking the template image and training, the trained model is directly used to detect the indicator target on the image to be detected, and then the initial matching area is obtained.

[0080] Step S222: determining edge information based on the initial matching area, and obtaining the indicator area according to the initial matching area and the edge information.

[0081] It should be noted that the initial matching area obtained in the above steps is an area image that only contains the indicator. Figure 4 , the initial matching area is the part marked by the solid line frame in the figure. If the initial matching area is used as the image to be input into the position detection model later, there may be factors such as the close distance between the target object and the camera, which may cause some edges of the indicator to be unable to be detected in the initial matching area, resulting in partial information loss.

[0082] In order to solve the above problem, in this step, it is necessary to retain some edge information. The edge information refers to the area adjacent to the initial matching area in the image to be detected. For example, Figure 4 The portion between the dotted frame and the solid frame is the edge information of the initial matching area. Specifically, the center point position, width and height of the initial matching area can be determined first; then, based on the scale data of the initial matching area, the N pixel values ​​required to expand the initial matching area are determined, that is, the edge information is determined; N is a positive integer. Based on the determined edge information, the boundary of the initial matching area is expanded by N pixels in the four directions of up, down, left and right, respectively, to finally obtain the above-mentioned indicator area.

[0083] Through the above steps S221 to S222, the indicator area is obtained according to the edge information of the initial matching area obtained by template matching, thereby retaining the edge information, avoiding partial information loss, and effectively improving the accuracy of image detection.

[0084] In some embodiments, the above image detection method further includes the following steps:

[0085] Acquire first scale data and second scale data; the first scale data is the scale data of the image to be detected, and the second scale data is the scale data of the indicator area; determine the expansion coefficient according to the association relationship between the first scale data and the second scale data. The first scale data refers to data with image size characteristics such as the width, height or diagonal length of the image to be detected; similarly, the second scale data refers to data with image size characteristics such as the width, height or diagonal length of the image of the indicator area.

[0086] Taking the example that both the first scale data and the second scale data adopt the width of the image or the image area, in an optional embodiment, the first ratio between the width of the image to be detected and the width of the image of the indicator area can be calculated first, and the expansion coefficient can be determined according to the first ratio. Alternatively, a preset target ratio can be obtained, and then the expansion coefficient can be determined according to the product of the first ratio and the target ratio. It should also be noted that in the process of calculating the expansion coefficient, the target ratio involved refers to the ratio value pre-set by the staff in combination with the actual situation; considering that when the target frame is expanded and enlarged, in order to ensure a certain amount of error capacity, the indicator area can not be fully expanded to the size of the entire image to be detected, but the preset target ratio is used to expand the indicator area to a certain ratio relative to the entire image. For example, if the indicator area needs to be expanded to 80% of the size of the entire image, the target ratio can be set to 80%, and the expansion coefficient is determined by using the first ratio between the width of the image to be detected and the width of the image of the indicator area, and the product of the target ratio, and finally the adaptive expansion processing of the indicator area is completed.

[0087] Alternatively, third scale data is obtained; the third scale data is the scale data of the preset target expansion area; the expansion coefficient is determined according to the association between the third scale data and the second scale data of the indicator area. Wherein, the target expansion area refers to the area to which the indicator area needs to be expanded in advance in combination with actual application requirements; for example, if the indicator area needs to be expanded to an area that occupies 80% of the entire image in the current application scenario, the target expansion area can be determined to be the area that occupies 80% of the entire image, and then the scale data such as width and height of the target expansion area can be determined. More specifically, in order to be able to enlarge the image to a uniform size, the staff can pre-set the target size to be enlarged in the expansion area, that is, the third scale data of the target expansion area. Next, in the actual application process, the processor calculates the second ratio between the third scale data and the second scale data, and then the second ratio can be used as the expansion coefficient.

[0088] Through the above embodiments, the corresponding expansion coefficient is determined based on the size of the identified indicator area, thereby providing a variety of ways to adaptively determine the expansion coefficient, which can flexibly set the expansion coefficient and effectively improve the flexibility and intelligence of image detection.

[0089] In some embodiments, the step of inputting the dilated image into a trained position detection model and outputting the indicator position information further includes the following steps:

[0090] The expansion image is input into the position detection model to obtain a thermal map corresponding to the expansion image; a target feature point in the thermal map whose response value exceeds a preset response value threshold is determined; and the position information of the indicator is obtained according to the feature point coordinates of the target feature point.

[0091] Among them, the above-mentioned position detection model is a lightweight segmentation network. The position detection model can be a neural network model such as a UNet network, a UNet++ network, a deep residual network (Deep residual network, referred to as ResNet) model, a VGG-19 model, a DenseNet model or a high-resolution network (High-Resolution network, referred to as HRNet) model. Taking the UNet network as an example, the UNet network is a classic convolutional neural network image segmentation model, a fully convolutional network, and the entire network presents a U-shaped structure. Specifically, the above-mentioned expanded image is input into the UNet network for feature extraction, and the UNet network is used to perform a 1×1 convolution on the last layer of the feature map to obtain a thermal map corresponding to each extracted feature point.

[0092] Next, the response values ​​of each feature point on the heat map are counted, and the feature points whose response values ​​on the heat map reach the preset response value threshold are used as the above-mentioned target feature points. The response value threshold can be set in advance in combination with the actual situation. For example, historical data of similar scenes can be collected, and the response value distribution of feature points in these data can be analyzed to set a reasonable threshold; the response value threshold can be set to a certain percentile (such as 90% or 95% percentile) in the historical data to ensure that only the most significant feature points are selected for the current heat map. Of course, if the embodiment allows, the response value threshold can also be dynamically adjusted according to the changes in the specific scene. For example, in the scene of continuous scanning or dynamic detection, the threshold can be dynamically adjusted according to the positioning result of the previous time point and the change trend of the current heat map. Alternatively, the response value threshold can also be determined by the response value statistics of the feature points in the current heat map, for example, it is determined as the maximum response value of the feature points in the current heat map, that is, the feature point with the largest response value in the current heat map can be directly determined as the target feature point. Finally, the coordinates of the target feature point are output as the above-mentioned indicator position information.

[0093] Through the above embodiments, a deep learning algorithm based on heat maps and neural networks is applied to motion capture tasks to ensure that the algorithm can output a set of ordered feature point coordinates in a predetermined order, thereby improving the robustness of the algorithm while ensuring real-time performance, and at the same time improving the generalization and compatibility of downstream tasks.

[0094] In some embodiments, the training method of the position detection model comprises the following steps:

[0095] Step S201, obtaining an original training image; determining a training area of ​​the original training image, and performing dilation and magnification processing on the training area to obtain a target training image; the target training image carries a coordinate label.

[0096] In this step, a private data set is first obtained. Specifically, a large number of infrared images of volunteers can be collected in corresponding actual scenes such as nuclear magnetic resonance, or historical images can be collected before for algorithm training and research; and each video stream is framed, the indicator target frame is marked, and the indicator key point is marked. Then, the target frame of the indicator in the original image is expanded, cropped, and interpolated and enlarged according to a specific coefficient to obtain a target training image containing only the indicator and a small amount of background, which can be used as a subsequent training and test image. Obtain a target training image for training. It can be understood that based on the original training image, determining the training area where the indicator is located and the expansion and enlargement processing method of the training area can be similar to the processing method of the image to be detected in any of the above method embodiments, and will not be repeated here. In particular, the staff can also manually annotate both the original training image and the enlarged target training image so that the two types of images carry corresponding coordinate labels to achieve algorithm performance evaluation in two dimensions.

[0097] It should be noted that during training, each of the above target training images can be enhanced by using affine transformation, adding noise, adjusting contrast, adjusting sharpness, etc. to obtain new target training images and corresponding coordinate labels, so as to improve the accuracy of the neural network model generated by training.

[0098] Step S202, input the target training image into the initial neural network model for training, and output the position prediction result; calculate the loss function result based on the coordinate label and the position prediction result; reversely transmit the gradient of the loss function result to the initial neural network model for iterative training to obtain the position detection model.

[0099] Taking the lightweight Unet network as an example, after the target training image is input into the initial Unet network, the initial network is used to perform 1x1 convolution on the last layer of feature maps to obtain the heat map of each feature point in the target training image, and then the point with the largest response value on the heat map is used as the predicted coordinate and output, that is, the above position prediction result is output. Then, the minimum mean square error loss function or cross entropy loss function is used to calculate the loss function result between the predicted heat map and the heat map generated by the real coordinates. Finally, the training model is optimized using the loss function result to obtain the above position detection model.

[0100] On the other hand, during training and testing, the network input uses the enlarged indicator image and corresponding coordinates instead of the original image, which effectively improves the generalization performance of the model. In particular, during testing, the error evaluation of the predicted coordinates and the mapped original image coordinates was performed separately, and the results showed that the above algorithm achieved sub-pixel feature point prediction accuracy.

[0101] Through the above steps S201 to S202, the loss function result is calculated by the position prediction result output by training the target training image through the initial neural network model, and the initial neural network model is iteratively trained based on the loss function result to obtain an optimized position detection model, which is beneficial to improve the accuracy of the output data of the position detection model and further improve the accuracy of image detection.

[0102] In some embodiments, the step of generating target motion information corresponding to the image to be detected based on the indicator position information further includes the following steps:

[0103] Step S241, obtaining the original coordinates of the indicator area in the image to be detected.

[0104] Among them, after determining the indicator area through the above steps, the original coordinates of the indicator area in the above-mentioned image to be detected can be statistically obtained; more specifically, the original coordinates can be the pixel point coordinates of the four vertices of the indicator area in the image to be detected, or the coordinates of other key points in the image to be detected, which are not limited here.

[0105] Step S242, calculating the target position information of the feature point in the expanded image on the image to be detected according to the original coordinates, the expansion coefficient and the indicator position information, and determining the target motion information based on the target position information.

[0106] Exemplarily, the calculation process of the above target position information can be shown as the following formula: (Pred_x', Pred_y') = (bbox_x, bbox_y) + (Pred_x, Pred_y) / scale. In the above formula, (Pred_x', Pred_y') is used to represent the X coordinate and Y coordinate in the target position information obtained after coordinate restoration; (bbox_x, bbox_y) is used to represent the X coordinate and Y coordinate in the original coordinate; (Pred_x, Pred_y) is used to represent the X coordinate and Y coordinate in the indicator position information; scale is used to represent the expansion coefficient. Among them, the target position information calculated by the above formula is the target motion information finally output by the algorithm.

[0107] In another embodiment, considering the need to retain edge information, the above indicator position information can also be expressed as (Pred_x, Pred_y) = (Pred_x0, Pred_y0) + N; wherein (Pred_x0, Pred_y0) is used to represent the X coordinate and Y coordinate of the above initial matching area; N is used to represent the pixel value required to be expanded to retain edge information. Therefore, the calculation process of the above target position information can also be expressed as the following formula: Pred_x', Pred_y') = (bbox_x, bbox_y) + ((Pred_x0, Pred_y0) + N) / scale.

[0108] Through the above embodiment, the coordinates of the indicator position information are mapped to the original image to be detected based on the original coordinates, thereby avoiding the problem of position detection errors caused by inconsistent coordinate systems and further effectively improving the accuracy of image detection.

[0109] In some embodiments, the method further comprises the following steps:

[0110] Obtain a video stream, and determine the current reference frame from the video stream; if the indicator area determination based on the reference frame fails, obtain the current cumulative number of failed frames. Specifically, the reference frame can be determined by detecting and extracting key frames in the video stream, and using the key frames as the reference frames to improve the accuracy of image detection; or, if permitted by the embodiment, the first frame image or a randomly extracted image in the video stream can also be used as the reference frame. Next, use the indicator area detection method of any of the above method embodiments to perform target detection, and calculate the confidence of the indicator target frame; when it is detected that the current confidence is lower than the preset confidence threshold, determine that the indicator area detection fails at this time, and accumulate the cumulative number of failed frames. The accumulation process can be: first determine a preset initial cumulative value; for example, the initial cumulative value can be 0; each time the indicator area detection fails, the cumulative value is added by 1.

[0111] When it is detected that the cumulative number of failed frames is greater than the preset frame number threshold, a new reference frame is determined from the video stream, and the indicator position information is obtained based on the new reference frame. If the indicator area is still not detected in multiple consecutive frames, it means that the algorithm is wrong at this time. In order to correct it in time, it is necessary to re-determine a new reference frame, and perform template matching and other processing again based on the new reference frame to determine a new indicator area, and obtain the corresponding indicator position information based on the new indicator area using the above steps. In the case where it is detected that the cumulative number of failed frames is less than or equal to the frame number threshold, it means that it may be that only a single frame image has unclear factors, which is an accidental situation; in order to save computing power and calculation time, the indicator area of ​​the previous reference frame can be directly used as the indicator area of ​​the current frame, and the corresponding indicator position information can be obtained, so there is no need to re-detect the indicator area. It can be understood that the above frame number threshold can be set in advance by the staff, for example, it can be set to 5 frames.

[0112] Through the above embodiment, the corresponding coordinate determination method is determined by judging the accumulated number of frames in which the indicator area detection fails, thereby ensuring both the efficiency and accuracy of image detection.

[0113] The embodiments of the present application are described in detail below in conjunction with actual application scenarios. Figure 5 is a flow chart of another image detection method according to an embodiment of the present application. Figure 5 As shown, the process includes the following steps:

[0114] Step S501, obtaining a data stream; wherein the data stream refers to a data stream during a head magnetic resonance imaging scan.

[0115] Step S502: based on the above data stream, determine a reference frame, perform template matching on the reference frame, and determine the indicator area.

[0116] Step S503, determine whether the indicator area target frame is detected. If the determination result of this step is yes, then execute step S505.

[0117] Step S504: If the result of the judgment in the above step S503 is no, it is determined whether the target frame of the indicator area has not been detected for M consecutive frames; M is a positive integer. If the result of the judgment in this step is yes, it returns to the above step S502; if the result of the judgment in this step is no, the target frame of the previous frame is used and step S505 is executed.

[0118] Step S505 : Based on the expansion coefficient, the indicator area is expanded and enlarged to obtain an expanded image.

[0119] Step S506: input the expansion image into the position detection model, and output the coordinates of each feature point in the expansion image, that is, output the indicator position information.

[0120] Step S507 , mapping the coordinates of the indicator position information to the original image, that is, mapping to the reference frame image, and finally outputting the target motion information.

[0121] It should be noted that, in this embodiment, the infrared camera used to collect images is tested under the conditions of a resolution of 1280 pixels × 960 pixels and a working distance of 12 to 15 cm, and the feature point accuracy can reach 0.00 ± 0.25 px, which is equivalent to the actual size of the Mocap scene of 0 ± 25 (μm), 0.00 ± 0.07°. Through the above embodiment, an image feature point detection algorithm based on indicators and deep learning is provided, which realizes sub-pixel level feature point coordinate detection, breaking the limitations of camera resolution and effective working distance. The results of the algorithm output can serve any subsequent pose estimation algorithm based on feature point positioning, and can effectively improve the accuracy of motion correction in, for example, nuclear magnetic resonance scenes.

[0122] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0123] The present embodiment also provides an image detection device, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module", "unit", "subunit" and the like can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0124] Figure 6 is a structural block diagram of an image detection device according to an embodiment of the present application, such as Figure 6 As shown, the device includes: an acquisition module 62, used for acquiring an image to be detected; an expansion module 64, used for determining an indicator region of the image to be detected; based on the acquired expansion coefficient, the indicator region is expanded and enlarged to obtain an expanded image; a position detection module 66, used for inputting the expanded image into a trained position detection module and outputting indicator position information; and a generation module 68, used for generating target motion information corresponding to the image to be detected based on the indicator position information.

[0125] In some embodiments, the expansion module 64 is also used to obtain a preset indicator template; the expansion module 64 performs template matching on the indicator template and the image to be detected to obtain an initial matching area; the expansion module 64 determines edge information based on the initial matching area, and obtains the indicator area based on the initial matching area and the edge information.

[0126] In some embodiments, the above-mentioned image detection device also includes a coefficient determination module; the coefficient determination module is used to obtain first scale data and second scale data; the first scale data is the scale data of the image to be detected, and the second scale data is the scale data of the indicator area; the expansion coefficient is determined according to the association relationship between the first scale data and the second scale data; or, the coefficient determination module obtains third scale data; the third scale data is the scale data of the target expansion area; the expansion coefficient is determined according to the association relationship between the third scale data and the second scale data of the indicator area.

[0127] In some embodiments, the position detection module 66 is further configured to input the expansion image into a trained position detection model for detection processing, and output the indicator position information.

[0128] In some embodiments, the position detection module 66 is further used to input the expansion image into the position detection model to obtain a thermal map corresponding to the expansion image; the position detection module 66 determines a target feature point in the thermal map whose response value reaches a preset response value threshold; and obtains the indicator position information based on the feature point coordinates of the target feature point.

[0129] In some embodiments, the image detection device further includes a training module; the training module is used to obtain an original training image; determine a training area of ​​the original training image, and perform expansion and enlargement processing on the training area to obtain a target training image; the target training image carries a coordinate label; the training module inputs the target training image into an initial neural network model for training, and outputs a position prediction result; the training module calculates a loss function result based on the coordinate label and the position prediction result; and reversely transmits the gradient of the loss function result to the initial neural network model for iterative training to obtain the position detection model.

[0130] In some embodiments, the image detection device further includes a frame accumulation module, which is used to acquire a video stream and determine a current reference frame from the video stream. If the frame accumulation module fails to determine the indicator area based on the reference frame, the frame accumulation module acquires the current failed accumulated frame number. When the frame accumulation module detects that the failed accumulated frame number is greater than a preset frame number threshold, the frame accumulation module determines a new reference frame from the video stream and acquires the indicator position information based on the new reference frame. When the frame accumulation module detects that the failed accumulated frame number is less than or equal to the frame number threshold, the frame accumulation module acquires the indicator position information based on the previous reference frame.

[0131] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0132] This embodiment further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0133] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0134] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0135] S1, obtaining the image to be detected.

[0136] S2, determining the indicator region of the image to be detected; and performing expansion and magnification processing on the indicator region based on the obtained expansion coefficient to obtain an expanded image.

[0137] S3, calculating the pointer position information based on the expanded image.

[0138] S4, generating target motion information corresponding to the image to be detected based on the indicator position information.

[0139] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0140] In addition, in combination with the image detection method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the image detection methods in the above embodiment is implemented.

[0141] Those skilled in the art should understand that the various technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image detection method, characterized in that: The method comprises: Acquire the image to be detected; Determine the indicator region of the image to be detected; and based on the obtained expansion coefficient, perform expansion and magnification processing on the indicator region to obtain an expanded image; Calculating the indicator position information according to the dilated image; Based on the indicator position information, target motion information corresponding to the image to be detected is generated.

2. The image detection method according to claim 1, characterized in that: The determining of the indicator area of ​​the image to be detected includes: Get the preset indicator template; Performing template matching between the indicator template and the image to be detected to obtain an initial matching area; The edge information is determined based on the initial matching area, and the indicator area is obtained according to the initial matching area and the edge information.

3. The image detection method according to claim 1, characterized in that: The method further comprises: Acquire first scale data and second scale data; the first scale data is scale data of the image to be detected, and the second scale data is scale data of the indicator area; determine the expansion coefficient according to the correlation relationship between the first scale data and the second scale data; or, Acquire third scale data; the third scale data is scale data of a preset target expansion area; determine the expansion coefficient according to the association relationship between the third scale data and the second scale data of the indicator area.

4. The image detection method according to claim 1, characterized in that: The step of calculating the indicator position information according to the dilated image includes: The dilated image is input into a trained position detection model for detection processing, and the indicator position information is output.

5. The image detection method according to claim 4, characterized in that: The step of inputting the dilated image into a trained position detection model for detection processing and outputting the indicator position information includes: Inputting the expansion image into the position detection model to obtain a thermal map corresponding to the expansion image; Determine a target feature point in the thermal map whose response value reaches a preset response value threshold; and obtain the indicator position information according to the feature point coordinates of the target feature point.

6. The image detection method according to claim 4, characterized in that: The method further comprises: Acquire an original training image; determine a training area of ​​the original training image, and perform dilation and magnification processing on the training area to obtain a target training image; the target training image carries a coordinate label; Inputting the target training image into the initial neural network model for training, and outputting a position prediction result; Based on the coordinate labels and the position prediction results, a loss function result is calculated; the gradient of the loss function result is back-transmitted to the initial neural network model for iterative training to obtain the position detection model.

7. The image detection method according to claim 1, characterized in that: The step of generating target motion information corresponding to the image to be detected based on the indicator position information includes: Obtaining original coordinates of the indicator area in the image to be detected; According to the original coordinates, the expansion coefficient and the indicator position information, the target position information of the feature point in the expansion image on the image to be detected is calculated, and based on the target position information, the target motion information is determined.

8. The image detection method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquire a video stream, and determine a current reference frame from the video stream; If determining the indicator area based on the reference frame fails, obtaining the current cumulative number of failed frames; When it is detected that the accumulated number of failed frames is greater than a preset frame number threshold, determining a new reference frame from the video stream, and acquiring the indicator position information based on the new reference frame; When it is detected that the accumulated number of failed frames is less than or equal to the frame number threshold, the indicator position information is acquired based on a previous reference frame.

9. An image detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be detected; An expansion module is used to determine the indicator region of the image to be detected; based on the obtained expansion coefficient, the indicator region is expanded and enlarged to obtain an expanded image; A position detection module, used to calculate the position information of the indicator according to the expansion image; A generating module is used to generate target motion information corresponding to the image to be detected based on the indicator position information.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the image detection method according to any one of claims 1 to 8 when executed.

Images