Image detection model training method, image detection method, device and equipment

Abstract

The present application discloses a training method, an image detection method, an apparatus and a device for an image detection model, and belongs to the field of artificial intelligence technology. The method includes: obtaining a sample image and annotated geometric figure parameters of the sample image, wherein the annotated geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained by annotation; obtaining predicted geometric figure parameters of the sample image according to a first network model, wherein the predicted geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained by prediction; adjusting the first network model based on the annotated geometric figure parameters and the predicted geometric figure parameters of the sample image to obtain an image detection model. The embodiment of the present application characterizes the geometric figure through the parameter information of the geometric figure, thereby realizing rapid determination of the geometric figure in the image, simplifying the image detection steps, and improving the image detection efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a training method for an image detection model, an image detection method, a device, and equipment. Background Art

[0002] With the development of computers, more and more data exists in the form of images in computers, and computers are used to process images. In image processing technology, image detection technology is an important technology. Image detection technology can detect geometric figures in images, including straight lines, ellipses, planes, etc.

[0003] Taking the detection of a target plane in an image as an example, related technologies obtain the plane of a sample image and annotation information indicating whether the plane belongs to the target plane. A network model is then used to obtain probability information that the plane belongs to the target plane. Based on this annotation information and probability information, an image detection model is obtained. When detecting the target plane of a target image, the image detection model obtained in this way needs to first obtain a certain plane of the target image, input this plane into the image detection model, and let the image detection model determine whether the plane is the target plane. If the plane is not the target plane, another plane of the target image is obtained, and the image detection model again determines whether the plane is the target plane.

[0004] When detecting the target plane of an image, the image detection model acquired in the above manner needs to continuously acquire different planes of the image and input the planes into the image detection model until the image detection model determines the target plane. This method is relatively cumbersome and results in low image detection efficiency. Summary of the Invention

[0005] The embodiments of the present application provide a training method, image detection method, device and equipment for an image detection model, which can be used to solve the problems that the image detection model trained by related technologies is cumbersome in determining planes and has low image detection efficiency. The technical solution includes the following contents.

[0006] In one aspect, an embodiment of the present application provides a method for training an image detection model, the method comprising:

[0007] Acquire a sample image and annotated geometric figure parameters of the sample image, where the annotated geometric figure parameters of the sample image are parameter information of a geometric figure in the sample image obtained through annotation;

[0008] Acquire predicted geometric figure parameters of the sample image according to the first network model, where the predicted geometric figure parameters of the sample image are parameter information of geometric figures in the sample image obtained through prediction;

[0009] Based on the annotated geometric parameters of the sample image and the predicted geometric parameters, the first network model is adjusted to obtain an image detection model.

[0010] On the other hand, an embodiment of the present application provides an image detection method, the method comprising:

[0011] Acquire the target image;

[0012] Obtaining predicted geometric figure parameters of the target image according to an image detection model, wherein the image detection model is obtained according to any of the above-mentioned image detection model training methods, and the predicted geometric figure parameters of the target image are parameter information of geometric figures in the target image obtained through prediction;

[0013] Based on the predicted geometry parameters of the target image, a target geometry in the target image is determined.

[0014] On the other hand, an embodiment of the present application provides a training device for an image detection model, the device comprising:

[0015] A first acquisition module is configured to acquire a sample image and annotated geometric figure parameters of the sample image, where the annotated geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained through annotation;

[0016] A second acquisition module is configured to acquire predicted geometric figure parameters of the sample image according to the first network model, where the predicted geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained through prediction;

[0017] An adjustment module is used to adjust the first network model based on the annotated geometric figure parameters of the sample image and the predicted geometric figure parameters to obtain an image detection model.

[0018] In one possible implementation, the second acquisition module is used to obtain a predicted key point heat map of the sample image based on the first network model, where the predicted key point heat map of the sample image is a heat map of at least one key point of the sample image obtained through prediction; and to obtain predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

[0019] In one possible implementation, the second acquisition module is used to obtain a downsampled feature map of the sample image according to the first network model, where the downsampled feature map of the sample image includes semantic information of the sample image; determine a spatial feature map of the sample image based on the downsampled feature map of the sample image, where the spatial feature map of the sample image includes spatial position information of the sample image; and determine a predicted key point heat map of the sample image based on the spatial feature map of the sample image.

[0020] In one possible implementation, the second acquisition module is used to determine at least one key point of the sample image based on the predicted key point heat map of the sample image; based on the position information of each key point, filter out the target key point from the at least one key point; and determine the predicted geometric parameters of the sample image based on the target key point.

[0021] In one possible implementation, the second acquisition module is used to determine the size information of the target area in the sample image and the position information of the center point of the target area; based on the position information of the center point and the position information of each key point, determine the distance information corresponding to each key point, and the distance information corresponding to any key point is the distance information between the center point and any key point; based on the size information of the target area and the distance information corresponding to each key point, filter out the target key point from the at least one key point.

[0022] In one possible implementation, the second acquisition module is used to determine, for any key point, a ratio of the size information of the target area and the distance information corresponding to any key point, wherein the ratio represents the distance relationship between the target area and any key point; and to filter out, from the at least one key point, a target key point whose ratio is not greater than a ratio threshold.

[0023] In one possible implementation, the adjustment module is used to obtain a heat map of annotated key points of the sample image, where the heat map of annotated key points of the sample image is a heat map of at least one key point of the sample image obtained through annotation; based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map and the predicted key point heat map, the first network model is adjusted to obtain an image detection model.

[0024] In a possible implementation, the first acquisition module is further configured to acquire a labeled and rectified image of the sample image, where the labeled and rectified image of the sample image is an image obtained by geometrically correcting the sample image through labeling;

[0025] The second acquisition module is further configured to acquire a predicted and rectified image of the sample image according to the first network model, where the predicted and rectified image of the sample image is an image obtained by geometrically correcting the sample image through prediction;

[0026] The adjustment module is used to adjust the first network model based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated corrected image and the predicted corrected image to obtain an image detection model.

[0027] In one possible implementation, the second acquisition module is used to obtain the spatial feature map of the sample image based on the first network model; perform linear transformation processing on the spatial feature map of the sample image to obtain an affine matrix, and the affine matrix contains geometric correction information of the sample image; perform affine transformation processing on the affine matrix to obtain a predicted corrected image of the sample image.

[0028] In one possible implementation, the adjustment module is used to obtain the annotated key point heat map, predicted key point heat map, predicted corrected image and annotated corrected image of the sample image; based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map, the predicted key point heat map, the annotated corrected image and the predicted corrected image, the first network model is adjusted to obtain an image detection model.

[0029] On the other hand, an embodiment of the present application provides an image detection device, comprising:

[0030] A first acquisition module is used to acquire a target image;

[0031] a second acquisition module, configured to acquire predicted geometric figure parameters of the target image based on an image detection model, wherein the image detection model is obtained according to any of the above-mentioned image detection model training methods, and the predicted geometric figure parameters of the target image are parameter information of geometric figures in the target image obtained through prediction;

[0032] A determination module is used to determine a target geometry in the target image based on the predicted geometry parameters of the target image.

[0033] In one possible implementation, the second acquisition module is used to obtain a predicted key point heat map of the target image according to the image detection model, where the predicted key point heat map of the target image is a heat map of at least one key point of the target image obtained through prediction; and according to the image detection model, obtain predicted geometric parameters of the target image based on the predicted key point heat map of the target image.

[0034] In a possible implementation, the determination module is further used to determine the designated key points in the target image based on the predicted key point heat map of the target image; and determine the target frame image containing the designated key points from the multiple frames of images.

[0035] In a possible implementation, the second acquisition module is further configured to acquire a predicted corrected image of the target image based on the image detection model, where the predicted corrected image of the target image is an image obtained by geometrically correcting the target image through prediction.

[0036] In a possible implementation, the first acquisition module is further configured to acquire actual geometric figures in the target image;

[0037] The determination module is further configured to determine an offset between the target geometric figure in the target image and the actual geometric figure in the target image based on the target geometric figure in the target image and the actual geometric figure in the target image.

[0038] On the other hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor so that the electronic device implements any of the above-mentioned image detection model training methods or any of the above-mentioned image detection methods.

[0039] On the other hand, a computer-readable storage medium is also provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor so that the computer implements any of the above-mentioned image detection model training methods or any of the above-mentioned image detection methods.

[0040] On the other hand, a computer program or computer program product is also provided, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor so that the computer implements any of the above-mentioned image detection model training methods or any of the above-mentioned image detection methods.

[0041] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

[0042] The technical solution provided by the embodiment of the present application is to obtain an image detection model using the annotated geometric figure parameters and predicted geometric figure parameters of the sample image, so that the image detection model learns the parameter information of the geometric figures in the image, represents the geometric figures through the parameter information of the geometric figures, and realizes the rapid determination of the geometric figures in the image, thereby simplifying the image detection steps and improving the image detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 Schematic diagram of an image detection model training method and an implementation environment of the image detection method provided in an embodiment of the present application;

[0045] Figure 2 This is a flowchart of a method for training an image detection model provided in an embodiment of the present application;

[0046] Figure 3 This is a schematic diagram of a brain bony key point provided in an embodiment of the present application;

[0047] Figure 4 This is a schematic diagram of key points inside brain tissue provided by an embodiment of the present application;

[0048] Figure 5 This is a schematic diagram of determining predicted geometric figure parameters based on key points provided by an embodiment of the present application;

[0049] Figure 6 This is a flow chart of an image detection method provided by an embodiment of the present application;

[0050] Figure 7 is a schematic diagram of a straight line in a two-dimensional image provided by an embodiment of the present application;

[0051] Figure 8 is a schematic diagram of a plane in a three-dimensional image provided by an embodiment of the present application;

[0052] Figure 9 is a schematic diagram of the maximum offset between a target geometric figure and an actual geometric figure provided in an embodiment of the present application;

[0053] Figure 10 This is a schematic diagram of the structure of a first network model provided in an embodiment of the present application;

[0054] Figure 11 Schematic diagram of a training device for an image detection model provided in an embodiment of the present application;

[0055] Figure 12 1 is a schematic structural diagram of an image detection device provided in an embodiment of the present application;

[0056] Figure 13 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0057] Figure 14 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0059] Figure 1 This is a schematic diagram of an image detection model training method and an implementation environment of the image detection method provided in an embodiment of the present application, such as Figure 1 The implementation environment shown includes an electronic device 11, and the image detection model training method and the image detection method in the embodiment of the present application can be executed by the electronic device 11. Exemplarily, the electronic device 11 can include at least one of a terminal device or a server.

[0060] The terminal device can be at least one of a smart phone, a game console, a desktop computer, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player and a laptop computer.

[0061] The server can be a single server, a server cluster consisting of multiple servers, or any one of a cloud computing platform and a virtualization center, which are not limited in the embodiments of the present application. The server can communicate with the terminal device via a wired network or a wireless network. The server can have functions such as data processing, data storage, and data transmission and reception, which are not limited in the embodiments of the present application.

[0062] The various optional embodiments of the present application can be implemented based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0063] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0064] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying, tracking, and measuring objects, and then further processing the images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional (3D) object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0065] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0066] Based on the above implementation environment, the present application embodiment provides a training method for an image detection model. Figure 2 Taking the flowchart of a training method of an image detection model provided in the embodiment of the present application as an example, the method can be performed by Figure 1 The electronic device 11 in the embodiment is executed. Figure 2 As shown, the method includes steps 201 to 203.

[0067] Step 201 : Acquire a sample image and annotated geometric figure parameters of the sample image. The annotated geometric figure parameters of the sample image are parameter information of geometric figures in the sample image obtained through annotation.

[0068] The embodiment of the present application does not limit the method for obtaining the sample image. For example, a computed tomography (CT) image or a magnetic resonance imaging (MRI) image can be obtained, and the obtained CT image or MRI image can be used as the sample image.

[0069] The present embodiment does not limit the image type of the sample image. For example, the sample image is a brain image, a fetal image, a cell image, a photographic image, etc. The sample image can be a two-dimensional image such as a cell image or a photographic image, or a three-dimensional image such as a brain image or a fetal image.

[0070] In the embodiment of the present application, the parameter information of the geometric figures in the sample image is annotated to obtain the annotated geometric figure parameters of the sample image. The embodiment of the present application does not limit the geometric figures, and illustratively, the geometric figures include but are not limited to points, lines, ellipses, planes, etc.

[0071] Taking a plane as an example, the plane equation in three-dimensional space can be defined as Ax+By+Cz+D=0, where A, B, C, and D are the four plane equation coefficients, and x, y, and z are the three independent variables. In this case, the plane's parameter information can be the four plane equation coefficients A, B, C, and D. In the least squares method, the plane equation can be expressed as Y=AX, where Y is the zero vector, A is the parameter vector (there are four parameter vectors), and X is a matrix consisting of multiple points in the plane. In this case, the plane's parameter information can be the parameter vector A.

[0072] Step 202: Obtain predicted geometric figure parameters of the sample image according to the first network model. The predicted geometric figure parameters of the sample image are parameter information of the geometric figures in the sample image obtained through prediction.

[0073] In an embodiment of the present application, a sample image is input into a first network model, and the first network model outputs predicted geometric parameters of the sample image, wherein the predicted geometric parameters of the sample image are parameter information of the geometric figures in the sample image obtained through prediction. The description of the parameter information of the geometric figures is as described above and will not be repeated here.

[0074] In one possible implementation, obtaining predicted geometric parameters of a sample image according to a first network model includes: obtaining a predicted key point heat map of the sample image according to the first network model, where the predicted key point heat map of the sample image is a heat map of at least one key point of the sample image obtained through prediction; and obtaining predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

[0075] In an embodiment of the present application, a sample image includes at least one key point. For example, when the sample image is a brain image, the key points of the brain image include, but are not limited to, eight key points: the center point of the brainstem near the spine, the starting point of the nasal drum at the midline between the two eyeballs, the center of the short bone near the midbrain, the cockscomb structure on the front and back sides of the skull, the triangular point at the top of the cerebellum, the center of the third ventricle, and the junction of the left and right hemispheres of the skull. Any key point of the brain image is a bony key point of the brain or a key point inside the brain tissue.

[0076] See Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of a brain bony key point provided in an embodiment of the present application. Figure 4 This is a schematic diagram of key points inside brain tissue provided by an embodiment of the present application. Figure 3 Nine images are shown, each of which has a crosshair intersection representing a bony key point in the brain. Figure 4 Six images are shown, and the intersection of the crosshairs in each image represents a key point inside the brain tissue.

[0077] In an embodiment of the present application, a predicted key point heat map of a sample image can be obtained based on a first network model, thereby obtaining a heat map of at least one key point of the sample image, and predicted geometric parameters of the sample image can be obtained based on the predicted key point heat map of the sample image. The first network model includes a key point detection network and a geometric regression network. A sample image is input into the first network model, and the key point detection network obtains a predicted key point heat map of the sample image. The geometric regression network obtains predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

[0078] Optionally, obtaining a predicted key point heat map of the sample image according to the first network model includes: obtaining a downsampled feature map of the sample image according to the first network model, the downsampled feature map of the sample image including semantic information of the sample image; determining a spatial feature map of the sample image based on the downsampled feature map of the sample image, the spatial feature map of the sample image including spatial position information of the sample image; and determining a predicted key point heat map of the sample image based on the spatial feature map of the sample image.

[0079] In the embodiment of the present application, the keypoint detection network includes a stem block network and a stacked hourglass network. When the sample image is a three-dimensional image, the keypoint detection network is a three-dimensional keypoint detection network; when the sample image is a two-dimensional image, the keypoint detection network is a two-dimensional keypoint detection network.

[0080] The backbone network consists of, in order, a convolutional module with a kernel size of 7×7×7 and a stride of 2×2×2, a residual block, a maximum pooling module with a window width of 3×3×3 and a stride of 2×2×2, and a residual block. The residual block is composed of a bottleneck layer. The sample image is input into the backbone network, which first extracts the initial feature map of the sample image and downsamples it to obtain a downsampled feature map of the sample image. The size of the downsampled feature map is 1 / 4 of the size of the sample image. Since the size of the downsampled feature map is smaller than that of the sample image, it can increase the training and inference speed of the network, thereby improving the efficiency of image detection.

[0081] The stacked hourglass network includes at least one hourglass network with a top-down to bottom-up structure. The hourglass network has an hourglass-shaped structure and can capture the spatial position information of objects in the sample image based on the multi-scale features in the downsampled feature map, thereby determining the spatial feature map of the sample image, that is, the stacked hourglass network determines and outputs the spatial feature map of the sample image based on the downsampled feature map of the sample image. Subsequently, the predicted key point heat map of the sample image is determined and output based on the spatial feature map of the sample image. Subsequently, the geometric regression network obtains the predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

[0082] Optionally, obtaining predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image includes: determining at least one key point in the sample image based on the predicted key point heat map of the sample image; and determining the predicted geometric parameters of the sample image based on the at least one key point.

[0083] The predicted key point heat map of the sample image is a heat map of at least one key point in the sample image. In the embodiment of the present application, the geometric regression network can perform segmentation processing on the predicted key point heat map of the sample image to obtain a heat map of any key point in the sample image, and determine the coordinates corresponding to the highest response in the heat map of any key point to obtain the coordinates of the any key point.

[0084] In one possible implementation, the Arguments Of The Maxima (Argmax) function can be used to determine the key point based on the heat map of any key point in the sample image. The Argmax function is a function that calculates the parameters (or parameter sets) of a function and has the characteristic of being non-differentiable. Since the Argmax function has the characteristic of being non-differentiable, it is impossible to perform end-to-end training in the geometric regression network. In an embodiment of the present application, the geometric regression network is based on the Global Soft Argmax (GSA) algorithm. Regardless of whether the sample image is a two-dimensional image or a three-dimensional image, the key point can be determined based on the heat map of any key point in the sample image.

[0085] Taking the sample image as a three-dimensional image as an example, the geometric figure regression network is based on the GSA algorithm shown in the following formula (1), and determines the key point based on the heat map of any key point in the sample image.

[0086]

[0087] Wherein, SoftArgmax(I) is the coordinate corresponding to the highest response in the heat map of any key point in the sample image, that is, the coordinate of any key point in the sample image, i and j are the coordinates of any point in the heat map of any key point in the sample image, i, j∈[0,D×H×W-1], β is the adjustment coefficient, and the embodiment of the present application does not limit the value of the adjustment coefficient, x i is the response value of i in the heat map of any key point in the sample image, x j is the response value of j in the heat map of any key point in the sample image, Grid is the coordinate grid in three-dimensional space, and each element in the coordinate grid adopts (I x ,I y ,I z ) indicates that the size of the coordinate grid is D×H×W×3, where D, H, and W are the depth, height, and width of the heat map of any key point in the sample image, respectively, and e is a natural constant.

[0088] When the sample image is a two-dimensional image, the geometric figure regression network determines the key point based on the heat map of any key point in the sample image based on a principle similar to formula (1), which will not be repeated here.

[0089] According to the above method, each key point in the sample image can be determined based on the predicted key point heat map of the sample image, and then the predicted geometric figure parameters of the sample image can be determined based on each key point.

[0090] Optionally, determining the predicted geometric parameters of the sample image based on at least one key point includes: screening out a target key point from at least one key point based on position information of each key point; and determining the predicted geometric parameters of the sample image based on the target key point.

[0091] The accuracy of the predicted geometric parameters of the sample image is closely related to two factors: the first factor is the number of key points, and the second factor is the accuracy of the key points. The more key points there are, the higher the accuracy of the predicted geometric parameters of the sample image is, and the higher the accuracy of the key points, the higher the accuracy of the predicted geometric parameters of the sample image is.

[0092] The sample image contains at least one keypoint. During application, the accuracy of some keypoints may be low, resulting in low accuracy of the predicted geometric parameters of the sample image. Therefore, it is necessary to screen out at least one target keypoint from the sample image, with high accuracy. The predicted geometric parameters of the sample image are then determined based on the target keypoints, and the accuracy of the predicted geometric parameters of the sample image is also high.

[0093] For example, when the sample image is a brain image, there are a large number of key points in the brain image. These key points can be divided into brain bony key points and brain tissue internal key points. Brain bony key points are not easily affected by hematomas, tumors, etc., and therefore have higher accuracy. However, brain tissue internal key points are easily affected by hematomas, tumors, etc. and deviate from their normal positions, resulting in inaccurate positions of some brain tissue internal key points. In other words, the accuracy of some brain tissue internal key points is not high. Therefore, it is necessary to filter out the brain tissue internal key points with lower accuracy and select the brain tissue internal key points with higher accuracy from the brain tissue internal key points. In this way, the predicted geometric parameters of the brain image can be determined using the brain tissue internal key points and the brain bony key points with higher accuracy.

[0094] In an embodiment of the present application, based on the position information of each key point, target key points whose position information meets the position condition are screened from at least one key point. Optionally, based on the position information of each key point, screening the target key points from at least one key point includes: determining a target area in the sample image; and screening the target key points from at least one key point based on the target area and the position information of each key point.

[0095] In the embodiment of the present application, image segmentation processing is performed on the sample image to obtain the target area in the sample image. The embodiment of the present application does not limit the image segmentation processing method. For example, the image segmentation processing can be implemented based on the color, grayscale, texture, gradient, etc. of the image, or the image segmentation processing can be implemented using a neural network model. The embodiment of the present application also does not limit the content, size, etc. of the target area. For example, when the sample image is a brain image, the target area is the area of ​​a hematoma or tumor.

[0096] After determining the target area in the sample image, based on the target area and the position information of each key point, a target key point whose position information meets the position condition is screened out from at least one key point.

[0097] In one possible implementation, based on the target area and the position information of each key point, the target key point is filtered out from at least one key point, including: determining the size information of the target area and the position information of the center point of the target area; based on the position information of the center point and the position information of each key point, determining the distance information corresponding to each key point, the distance information corresponding to any key point is the distance information between the center point and any key point; based on the size information of the target area and the distance information corresponding to each key point, the target key point is filtered out from at least one key point.

[0098] After determining the target region in the sample image, each point in the target region is determined. Based on the position information of each point in the target region, the size information of the target region and the position information of the center point of the target region can be determined. The position information includes, but is not limited to, coordinate information, and the size information includes, but is not limited to, area, volume, and other information.

[0099] For any keypoint in the sample image, the distance between the target region's center and the keypoint is determined based on the positional information of the target region's center and the keypoint's location. This allows the distance between the target region's center and each keypoint to be determined. Subsequently, the target keypoint is selected from the at least one keypoint based on the target region's size and the distances between the target region's center and each keypoint.

[0100] Optionally, based on the size information of the target area and the distance information corresponding to each key point, the target key point is screened out from at least one key point, including: for any key point, determining the ratio of the size information of the target area and the distance information corresponding to any key point, the ratio representing the distance relationship between the target area and any key point; and screening out the target key point whose ratio is not greater than the ratio threshold from at least one key point.

[0101] For any key point, determine the ratio of the target area's size information to the distance information between the center point and any key point. If the ratio is not greater than a ratio threshold, the key point is a target key point. If the ratio is greater than the ratio threshold, the key point is not a target key point. In this way, the target key point is screened out from at least one key point. The above process of determining whether any key point is a target key point can be expressed as the following formula (2).

[0102]

[0103] Among them, D is the distance information between the center point of the target area and any key point, V is the size information of the target area, D0 is the distance warning information, V0 is the size warning information, and Threshold is the discrimination threshold.

[0104] Based on formula (2), it can be determined that: is the ratio of the size information of the target area to the distance information between the center point and any key point, is the ratio threshold.

[0105] The embodiment of the present application does not limit the distance warning information, size warning information and the discrimination threshold. For example, the distance warning information and size warning information are determined based on the scene or manual experience, and the discrimination threshold is 1.

[0106] When the distance between the center point of the target area and any key point is greater than or equal to the distance warning information, When the distance between the center point of the target area and any key point is less than the distance warning information, When the size information of the target area is greater than or equal to the size warning information, When the size information of the target area is smaller than the size warning information, Therefore, the embodiment of the present application can filter out key points that are relatively far away from the target area.

[0107] After the target keypoints are screened, the predicted geometric parameters of the sample image are determined based on the target keypoints. Optionally, the annotated geometric parameters of the sample image are determined according to the formula Y = AX. Where Y is the zero vector, A is the parameter vector, i.e., the annotated geometric parameters of the sample image, and X is the matrix composed of multiple points in the geometric figure, i.e., the matrix composed of the target keypoints.

[0108] Next, see Figure 5 , Figure 5The following is a schematic diagram of a method for determining predicted geometric parameters based on key points, provided by an embodiment of the present application. First, based on the key points and the target area in the sample image, distance information D between the center point of the target area and the key points is determined. Based on the target area in the sample image, size information V of the target area is determined. Next, a determination is made as to whether the ratio of V to D is less than a ratio threshold. If the ratio of V to D is less than the ratio threshold, the key point is determined to be a target key point. The target key points are then used to determine the predicted geometric parameters of the sample image.

[0109] Step 203: Based on the annotated geometric parameters and the predicted geometric parameters of the sample image, the first network model is adjusted to obtain an image detection model.

[0110] In the embodiment of the present application, based on the annotated geometric parameters of the sample image and the predicted geometric parameters of the sample image, a loss value of the geometric parameters is determined, and based on the loss value of the geometric parameters, the first network model is adjusted to obtain an image detection model. The loss value of the geometric parameters is determined according to the mean square error (MSE) function, and the MSE function is shown in the following formula (3).

[0111]

[0112] in, is the loss value of the geometric parameters, is the predicted geometric parameters of the sample image, w i is the labeled geometric parameter of the sample image, i is the serial number, and C is the number of geometric parameters of the sample image.

[0113] In one possible implementation, the first network model is adjusted based on the labeled geometric parameters and predicted geometric parameters of the sample image to obtain an image detection model, including: obtaining a labeled key point heat map of the sample image, where the labeled key point heat map of the sample image is a heat map of at least one key point in the sample image obtained through labeling; based on the labeled geometric parameters, predicted geometric parameters, labeled key point heat map and predicted key point heat map of the sample image, the first network model is adjusted to obtain an image detection model.

[0114] In the embodiment of the present application, a heat map of annotated key points of the sample image can also be obtained. When the sample image is a medical image, the heat map of annotated key points of the sample image is a gold standard image, which is a standard image recognized by the medical community. In order to make the training process smoother, a Gaussian kernel is used to convert each key point in the medical image into a Gaussian sphere, thereby obtaining a heat map of annotated key points of the sample image. Among them, the embodiment of the present application does not limit the Gaussian kernel. For example, a Gaussian kernel with a size of 15×15×15 and a sigma (i.e., σ) of 1 is used to convert the key points into Gaussian spheres at the positions of the key points, thereby obtaining a heat map of annotated key points of the sample image.

[0115] In the embodiment of the present application, a loss value of the geometric parameters is determined based on the annotated geometric parameters of the sample image and the predicted geometric parameters of the sample image, and a loss value of the key point heat map is determined based on the annotated key point heat map of the sample image and the predicted key point heat map of the sample image. Based on the loss values ​​of the geometric parameters and the loss values ​​of the key point heat map, the first network model is adjusted to obtain an image detection model.

[0116] The loss value of the key point heat map is determined according to the following formula (4).

[0117]

[0118] in, is the loss value of the key point heat map, K is the number of key points in the sample image, N is the number of voxels (i.e. points) in the sample image, k and i are serial numbers, is the predicted key point heat map of the sample image, is the heat map of the labeled key points of the sample image. Optionally, N = D × H × W, where D is the depth of the sample image, H is the height of the sample image, and W is the width of the sample image.

[0119] In an embodiment of the present application, the first network model includes two end-to-end networks, which are a key point detection network and a geometric figure regression network. The key point detection network is used to determine the predicted key point heat map of the image, which is semantic information related to the geometric figure. The geometric figure regression network is used to determine the predicted geometric figure parameters of the image based on the predicted key point heat map of the image, so that the accuracy of the predicted geometric figure parameters depends on the accuracy of the predicted key point heat map. The first network model is adjusted by the loss value of the key point heat map and the loss value of the plane parameter, so that the model can accurately determine the key point heat map, accurately extract the key points located on the geometric figure based on the key point heat map, and determine the geometric figure based on the extracted key points, and the method of fitting the geometric figure based on the key point heat map is more robust.

[0120] In one possible implementation, after obtaining the sample image and the annotated geometric parameters of the sample image, it also includes: obtaining the annotated corrected image of the sample image, which is the image obtained by annotating the sample image after geometric correction; obtaining the predicted corrected image of the sample image according to the first network model, which is the image obtained by predicting the sample image after geometric correction; adjusting the first network model based on the annotated geometric parameters and the predicted geometric parameters of the sample image to obtain an image detection model, including: adjusting the first network model based on the annotated geometric parameters, the predicted geometric parameters, the annotated corrected image and the predicted corrected image of the sample image to obtain an image detection model.

[0121] In an embodiment of the present application, the first network model also includes an image correction learning network. The sample image is input into the first network model, and the image correction learning network obtains a predicted corrected image of the sample image.

[0122] Optionally, obtaining a predicted corrected image of the sample image according to the first network model includes: obtaining a spatial feature map of the sample image according to the first network model; performing linear transformation processing on the spatial feature map of the sample image to obtain an affine matrix, wherein the affine matrix contains geometric correction information of the sample image; and performing affine transformation processing on the affine matrix to obtain a predicted corrected image of the sample image.

[0123] The first network model includes a key point detection network and an image correction learning network. The sample image is input into the first network model. The key point detection network first obtains the spatial feature map of the sample image, and then the key point detection network determines and outputs the predicted key point heat map of the sample image based on the spatial feature map of the sample image. The image correction learning network determines and outputs the predicted corrected image of the sample image based on the spatial feature map of the sample image.

[0124] In one possible implementation, the image correction learning network includes a global average pooling (GAP) network, a linear transformation network, and an affine transformation network. The GAP network is used to perform global pooling on the spatial feature map of the sample image to obtain the spatial feature map after global pooling. The linear transformation network is used to determine the affine matrix based on the spatial feature map after global pooling. The affine matrix contains the geometric correction information of the sample image. The affine transformation network is used to determine the predicted corrected image based on the affine matrix. Among them, the affine matrix can be expressed as T∈R 3×4 , T is an affine matrix, and R is a real number.

[0125] It should be noted that the key point detection network includes a backbone network and a stacked hourglass network. The backbone network can determine the downsampled feature map of the sample image. The stacked hourglass network can first determine the spatial feature map of the sample image based on the downsampled feature map of the sample image, and then determine the predicted key point heat map of the sample image based on the spatial feature map of the sample image.

[0126] In an embodiment of the present application, a labeled corrected image of a sample image can be obtained first. The labeled corrected image of the sample image is an image obtained by geometrically correcting the sample image through labeling. The geometric correction includes but is not limited to translation, flipping, rotation, scaling, etc. At the same time, a predicted corrected image of the sample image can be obtained by an image correction learning network. Afterwards, based on the labeled geometric parameters of the sample image and the predicted geometric parameters of the sample image, the loss value of the geometric parameters is determined. Based on the labeled corrected image of the sample image and the predicted corrected image of the sample image, the loss value of the corrected image is determined. Based on the loss value of the geometric parameters and the loss value of the corrected image, the first network model is adjusted to obtain an image detection model. Among them, the loss value of the corrected image is determined according to the normalized cross correlation loss (NCC) function. The NCC function is shown in the following formula (5).

[0127]

[0128] in, is the loss value of the rectified image, V t is the predicted rectified image of the sample image, V s is the labeled rectified image of the sample image.

[0129] Optionally, based on the labeled geometric parameters, predicted geometric parameters, labeled corrected image and predicted corrected image of the sample image, the first network model is adjusted to obtain an image detection model, including: obtaining the labeled key point heat map, predicted key point heat map, predicted corrected image and labeled corrected image of the sample image; based on the labeled geometric parameters, predicted geometric parameters, labeled key point heat map, predicted key point heat map, labeled corrected image and predicted corrected image of the sample image, the first network model is adjusted to obtain an image detection model.

[0130] In an embodiment of the present application, based on the annotated geometric parameters of the sample image and the predicted geometric parameters of the sample image, the loss value of the geometric parameters is determined; based on the annotated key point heat map of the sample image and the predicted key point heat map of the sample image, the loss value of the key point heat map is determined; based on the annotated corrected image of the sample image and the predicted corrected image of the sample image, the loss value of the corrected image is determined. Based on the loss value of the geometric parameters, the loss value of the key point heat map and the loss value of the corrected image, the loss value of the first network model is determined, and based on the loss value of the first network model, the first network model is adjusted to obtain an image detection model. The loss value of the first network model is determined according to the following formula (6).

[0131]

[0132] Among them, l is the loss value of the first network model, is the loss value of the key point heat map, is the loss value of the rectified image, l plane is the loss value of the geometric parameters, λ1 and λ2 are two weight parameters. The embodiment of the present application does not limit the weight parameters. For example, λ1 = 0.1, λ2 = 2.0.

[0133] The embodiment of the present application adjusts the first network model through the loss value of the key point heat map, the loss value of the corrected image and the loss value of the geometric figure parameters, so that the network has the ability to determine the key point heat map, the ability to correct the image and the ability to determine the geometric figure, and the three promote each other, which can achieve better training results and improve the accuracy of the network.

[0134] It can be understood that any one of the backbone network, stacked hourglass network, GAP network, linear transformation network and affine transformation network in the embodiments of the present application can be replaced by other networks. For example, the stacked hourglass network can be replaced by a U network (U-net).

[0135] The above method uses the annotated geometric figure parameters and predicted geometric figure parameters of the sample image to obtain the image detection model, so that the image detection model learns the parameter information of the geometric figures in the image, represents the geometric figures through the parameter information of the geometric figures, and realizes the rapid determination of the geometric figures in the image, simplifies the image detection steps, and improves the image detection efficiency.

[0136] Based on the above implementation environment, the present application embodiment provides an image detection method. Figure 6 As an example, the flowchart of an image detection method provided in the embodiment of the present application is shown. The method can be performed by Figure 1 The electronic device 11 in the embodiment is executed. Figure 6 As shown, the method includes steps 601 to 603.

[0137] Step 601: Acquire a target image.

[0138] The embodiments of this application do not limit the target image. Exemplary target images include brain images, fetal images, cell images, photographic images, etc. Specifically, the target image can be a two-dimensional image such as a cell image or a photographic image, or a three-dimensional image such as a brain image or a fetal image. The target image can be a CT image or an MRI image.

[0139] In a possible implementation, after acquiring the target image, the method further includes: acquiring a predicted corrected image of the target image according to an image detection model, where the predicted corrected image of the target image is an image obtained by geometrically correcting the target image through prediction.

[0140] In an embodiment of the present application, the image detection model includes an image correction learning network. The target image is input into the image detection model, and the image correction learning network obtains a predicted corrected image of the target image.

[0141] Optionally, the image detection model includes a key point detection network and an image correction learning network. The target image is input into the image detection model, and the key point detection network first obtains the spatial feature map of the target image, and then the key point detection network determines and outputs the predicted key point heat map of the target image based on the spatial feature map of the target image, and the image correction learning network determines and outputs the predicted corrected image of the target image based on the spatial feature map of the target image.

[0142] Step 602: Obtain predicted geometric parameters of the target image according to the image detection model.

[0143] The image detection model is obtained according to the training method of the image detection model provided in any of the above optional embodiments, and the predicted geometric figure parameters of the target image are parameter information of the geometric figures of the target image obtained through prediction.

[0144] In an embodiment of the present application, the target image is input into an image detection model, and the image detection model outputs the predicted geometric parameters of the target image, that is, the image detection model outputs the parameter information of geometric figures such as points, lines, ellipses, and planes in the target image.

[0145] Optionally, obtaining predicted geometric parameters of the target image according to the image detection model includes: obtaining a predicted key point heat map of the target image according to the image detection model, the predicted key point heat map of the target image being a heat map of at least one key point in the target image obtained through prediction; obtaining predicted geometric parameters of the target image based on the predicted key point heat map of the target image according to the image detection model.

[0146] The image detection model consists of a keypoint detection network and a geometry regression network. When a target image is input into the image detection model, the keypoint detection network obtains a predicted keypoint heatmap of the target image. The geometry regression network then uses this predicted keypoint heatmap to obtain the predicted geometry parameters of the target image.

[0147] Optionally, after step 602, the method further includes: obtaining actual geometric parameters of the target image, where the actual geometric parameters of the target image are parameter information of the actual geometric figures in the target image. Based on the actual geometric parameters of the target image and the predicted geometric parameters of the target image, correction information for the target image is determined. The correction information includes, but is not limited to, translation, flipping, rotation, scaling, etc. The target image is corrected based on the correction information of the target image to obtain a corrected image. For example, when the target image is a three-dimensional brain image, the three-dimensional brain image can be corrected using the correction information of the three-dimensional brain image to obtain a corrected brain image, that is, a straightened brain image.

[0148] In one possible implementation, the target image includes multiple frame images. After obtaining the predicted key point heat map of the target image according to the image detection model, it also includes: determining the specified key points in the target image based on the predicted key point heat map of the target image; and determining the target frame image containing the specified key points from the multiple frame images.

[0149] In an embodiment of the present application, a heat map of predicted key points of a target image can be segmented to obtain a heat map of any key point in the target image, and the coordinates corresponding to the highest response in the heat map of any key point can be determined to obtain the coordinates of the any key point. An Argmax function can be used to determine the key point based on the heat map of any key point in the target image. By detecting the key points in the target image, it is easier to understand the target image and to efficiently analyze and process the target image.

[0150] After determining each key point in the target image, since the target image includes multiple frames and any key point in the target image is located on a frame included in the target image, a designated operation for any key point in the target image can be obtained. In this case, the key point corresponding to the designated operation is the designated key point. The target frame image containing the designated key point is determined from the multiple frames included in the target image, achieving rapid and accurate positioning of the target frame image. By specifying different key points, different target frames can be quickly positioned, achieving rapid frame switching and saving time.

[0151] It should be noted that after determining the key points in the target image, the key points can be grouped to obtain at least two groups, each of which includes at least one key point. For each group, the key points in that group are used to determine the geometric figures in the target image. In other words, one group corresponds to one geometric figure, thereby enabling the simultaneous detection of at least two geometric figures by determining the key points in the target image.

[0152] Step 603: Determine the target geometry in the target image based on the predicted geometry parameters of the target image.

[0153] The embodiment of the present application can determine the target geometric figure in the target image based on the predicted geometric figure parameters of the target image. The embodiment of the present application does not limit the target geometric figure. For example, the target geometric figure includes but is not limited to a point, a straight line, an ellipse, a plane, etc.

[0154] Taking a plane as an example, the predicted geometric figure parameters of the target image can be substituted into the plane equation in three-dimensional space or the plane equation in the least squares method to obtain the plane equation of the target geometric figure in the target image, thereby determining the plane in the target image.

[0155] Next, see Figure 7 , Figure 7 This is a schematic diagram of a straight line in a two-dimensional image provided by an embodiment of the present application. In this case, the target image is a two-dimensional image, and the target geometric figure in the target image is a straight line in the two-dimensional image. It can be understood that the two-dimensional image can be regarded as a section of a three-dimensional image. In this case, Figure 7 The straight lines shown in represent the plane of the three-dimensional image.

[0156] Next, see Figure 8 , Figure 8 is a schematic diagram of a plane in a three-dimensional image provided by an embodiment of the present application. In this case, the target image is a three-dimensional image (the three-dimensional image is a brain image), the target geometric figure in the target image is a plane in the three-dimensional image, and the three-dimensional image includes three key points (i.e. Figure 8 These three key points are located on a plane in the 3D image.

[0157] In one possible implementation, after determining the target geometry in the target image based on the predicted geometry parameters of the target image, it also includes: obtaining the actual geometry in the target image; and determining the offset between the target geometry in the target image and the actual geometry in the target image based on the target geometry in the target image and the actual geometry in the target image.

[0158] The embodiments of the present application do not limit the method for obtaining the actual geometric figure in the target image. After obtaining the target geometric figure in the target image and the actual geometric figure in the target image, the offset between the target geometric figure in the target image and the actual geometric figure in the target image can be quickly calculated. This offset can be a maximum offset. Since the embodiments of the present application determine the target geometric figure in the target image, by comparing it with the actual geometric figure in the target image, the offset and offset direction can be intuitively and accurately indicated.

[0159] See Figure 9 , Figure 9 The figure is a schematic diagram of the maximum offset between a target geometric figure and an actual geometric figure provided in an embodiment of the present application. The target image is a cross-section of a brain image. The black straight line represents the target geometric figure in the target image, which can be understood as the target brain midline. The white curved line represents the actual geometric figure in the target image, which can be understood as the actual brain midline. The distance indicated by the double-headed arrow is the maximum offset between the target geometric figure in the target image and the actual geometric figure in the target image.

[0160] The actual brain midline is the anatomical structure that separates the left and right hemispheres of the brain. It appears as a curve in a single-frame image and a curved surface in a three-dimensional image. The target brain midline, also known as the standard brain midline, is an ideal plane when the brain is not deformed.

[0161] When the target image is a fetal image, geometric figures such as planes and straight lines in the fetal image can represent the standard plane of the fetal abdomen, the standard plane of the fetal human brain, etc. That is to say, the image detection model in the embodiment of the present application can be used to determine the standard brain midline in the brain image, various standard planes in the fetal image, etc.

[0162] It is understandable that the image detection model is obtained by adjusting the first network model. Therefore, the image detection model and the first network model have the same network structure, size, processing method, etc., and the only difference between the two is the model parameters. Therefore, the image detection model processes the target image in the same way as the first network model processes the sample image. The above optional embodiment has described the processing method of the first network model for the sample image. The processing method of the image detection model for the target image can be found in the description of the above optional embodiment and will not be repeated here.

[0163] The image detection model in the above method is derived from the annotated and predicted geometric parameters of the sample image, enabling the model to learn the parameter information of the geometric figures in the image. Using this image detection model to obtain the predicted geometric parameters of the target image, the geometric figure is characterized by its parameter information, enabling rapid identification of the target geometric figure in the image, simplifying the image detection process and improving image detection efficiency.

[0164] The above describes in detail the training method and image detection method of the image detection model of the embodiment of the present application from the perspective of method steps. The following describes the embodiment of the present application from the perspective of medical scenarios. The embodiment of the present application uses brain images in medical scenarios as sample images, and uses the sample images to train the first network model to obtain an image detection model.

[0165] See Figure 10 , Figure 10 1 is a schematic diagram of the structure of a first network model provided in an embodiment of the present application. The first network model includes a key point detection network, an image correction learning network, and a geometric figure regression network. The geometric figure in the embodiment of the present application includes a plane, and the plane of the brain image includes the brain midline of the brain image.

[0166] In an embodiment of the present application, a brain image is input into a first network model, and the key point detection network first obtains an initial feature map of the brain image, and obtains a downsampled feature map of the brain image based on the initial feature map. The key point detection network includes two hourglass networks, which obtain the spatial feature map of the brain image based on the downsampled feature map of the brain image, and output a predicted key point heat map of the brain image based on the spatial feature map of the brain image. The embodiment of the present application can also obtain an annotated key point heat map of the brain image, and determine the loss value of the key point heat map based on the predicted key point heat map of the brain image and the annotated key point heat map of the brain image. For an explanation of this part, please refer to the relevant description of "obtaining a predicted key point heat map of the sample image according to the first network model" and "the loss value of the key point heat map" in the above optional embodiment, which will not be repeated here.

[0167] Among them, the spatial feature map of the brain image will also be input into the image correction learning network, which includes a global average pooling network and two linear transformation networks. After the spatial feature map of the brain image is processed by the global average pooling network and the two linear transformation networks in turn, an affine matrix is ​​obtained, and the affine transformation is performed on the affine matrix to obtain a predicted corrected image of the brain image. The embodiment of the present application can also obtain an annotated corrected image of the brain image, and determine the loss value of the corrected image based on the predicted corrected image of the brain image and the annotated corrected image of the brain image. The explanation of this part can be found in the above optional embodiment regarding the relevant description of "obtaining a predicted corrected image of the sample image according to the first network model" and "the loss value of the corrected image", which will not be repeated here.

[0168] In addition, after the key point detection network determines and outputs the predicted key point heat map of the brain image, the geometric regression network first uses the GSA algorithm to determine each key point (of the brain image) based on the predicted key point heat map of the brain image, and then uses the least squares method to determine the predicted plane parameters of the brain image based on each key point of the brain image. The embodiment of the present application can also obtain the annotated plane parameters of the brain image, and determine the loss value of the plane parameters based on the predicted plane parameters of the brain image and the annotated plane parameters of the brain image. For an explanation of this part, please refer to the relevant description of "obtaining the predicted geometric parameters of the sample image according to the first network model" and "the loss value of the geometric parameters" in the above optional embodiment, which will not be repeated here.

[0169] Afterwards, based on the loss values ​​of the key point heat map, the loss values ​​of the corrected image, and the loss values ​​of the plane parameters, the loss value of the first network model is determined, and the first network model is adjusted based on the loss value of the first network model to obtain an image detection model. For an explanation of this part, please refer to the description of "adjusting the first network model based on the labeled geometric parameters of the sample image, the predicted geometric parameters, the labeled key point heat map, the predicted key point heat map, the labeled corrected image, and the predicted corrected image to obtain the image detection model" in the above optional embodiment, which will not be repeated here.

[0170] After the image detection model is obtained through training, the image detection model can be used to obtain the predicted key point heat map, predicted correction image, predicted plane parameters, etc. of the target image. Among them, in order to speed up the network operation speed, after the image detection model is obtained through training, the embodiment of the present application can crop the image correction learning network in the image detection model to obtain the cropped image detection model, and use the cropped image detection model to obtain the predicted key point heat map, predicted plane parameters, etc. of the target image. For the explanation of this part, please refer to the relevant description of "obtaining the predicted geometric parameters of the target image according to the image detection model", "obtaining the predicted key point heat map of the target image according to the image detection model", "obtaining the predicted correction image of the target image according to the image detection model" and so on in the above optional embodiments, which will not be repeated here.

[0171] Figure 11 FIG. 1 is a structural diagram of a training device for an image detection model provided in an embodiment of the present application. Figure 11 As shown, the device includes:

[0172] A first acquisition module 1101 is configured to acquire a sample image and annotated geometric figure parameters of the sample image, where the annotated geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained through annotation;

[0173] A second acquisition module 1102 is configured to acquire predicted geometric figure parameters of the sample image according to the first network model, where the predicted geometric figure parameters of the sample image are parameter information of geometric figures in the sample image obtained through prediction;

[0174] The adjustment module 1103 is used to adjust the first network model based on the annotated geometric figure parameters and the predicted geometric figure parameters of the sample image to obtain an image detection model.

[0175] In one possible implementation, the second acquisition module 1102 is used to obtain a predicted key point heat map of the sample image based on the first network model, where the predicted key point heat map of the sample image is a heat map of at least one key point of the sample image obtained through prediction; and to obtain predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

[0176] In one possible implementation, the second acquisition module 1102 is used to obtain a downsampled feature map of the sample image according to the first network model, where the downsampled feature map of the sample image includes semantic information of the sample image; determine a spatial feature map of the sample image based on the downsampled feature map of the sample image, where the spatial feature map of the sample image includes spatial position information of the sample image; and determine a predicted key point heat map of the sample image based on the spatial feature map of the sample image.

[0177] In one possible implementation, the second acquisition module 1102 is used to determine at least one key point of the sample image based on the predicted key point heat map of the sample image; based on the position information of each key point, filter out the target key point from at least one key point; and determine the predicted geometric parameters of the sample image based on the target key point.

[0178] In one possible implementation, the second acquisition module 1102 is used to determine the size information of the target area in the sample image and the position information of the center point of the target area; based on the position information of the center point and the position information of each key point, determine the distance information corresponding to each key point, and the distance information corresponding to any key point is the distance information between the center point and any key point; based on the size information of the target area and the distance information corresponding to each key point, filter out the target key point from at least one key point.

[0179] In one possible implementation, the second acquisition module 1102 is used to determine, for any key point, the ratio of the size information of the target area and the distance information corresponding to any key point, where the ratio represents the distance relationship between the target area and any key point; and to filter out the target key point whose ratio is not greater than the ratio threshold from at least one key point.

[0180] In one possible implementation, the adjustment module 1103 is used to obtain a heat map of annotated key points of a sample image, where the heat map of annotated key points of a sample image is a heat map of at least one key point of the sample image obtained through annotation; based on the annotated geometric parameters, predicted geometric parameters, annotated key point heat map and predicted key point heat map of the sample image, the first network model is adjusted to obtain an image detection model.

[0181] In a possible implementation, the first acquisition module 1101 is further configured to acquire a labeled and rectified image of the sample image, where the labeled and rectified image of the sample image is an image obtained by geometrically correcting the sample image through labeling.

[0182] The second acquisition module 1102 is further configured to acquire a predicted and corrected image of the sample image according to the first network model, where the predicted and corrected image of the sample image is an image obtained by geometrically correcting the sample image through prediction;

[0183] The adjustment module 1103 is used to adjust the first network model based on the annotated geometric parameters, the predicted geometric parameters, the annotated corrected image and the predicted corrected image of the sample image to obtain an image detection model.

[0184] In one possible implementation, the second acquisition module 1102 is used to obtain a spatial feature map of the sample image based on the first network model; perform linear transformation processing on the spatial feature map of the sample image to obtain an affine matrix, and the affine matrix contains geometric correction information of the sample image; perform affine transformation processing on the affine matrix to obtain a predicted corrected image of the sample image.

[0185] In one possible implementation, the adjustment module 1103 is used to obtain the annotated key point heat map, predicted key point heat map, predicted corrected image and annotated corrected image of the sample image; based on the annotated geometric parameters, predicted geometric parameters, annotated key point heat map, predicted key point heat map, annotated corrected image and predicted corrected image of the sample image, the first network model is adjusted to obtain an image detection model.

[0186] The above-mentioned device uses the annotated geometric figure parameters and predicted geometric figure parameters of the sample image to obtain the image detection model, so that the image detection model learns the parameter information of the geometric figures in the image, represents the geometric figures through the parameter information of the geometric figures, and realizes the rapid determination of the geometric figures in the image, simplifies the image detection steps, and improves the image detection efficiency.

[0187] Figure 12 FIG. 1 is a schematic diagram of the structure of an image detection device provided in an embodiment of the present application. Figure 12 As shown, the device includes:

[0188] A first acquisition module 1201 is used to acquire a target image;

[0189] A second acquisition module 1202 is configured to acquire predicted geometric figure parameters of a target image based on an image detection model, wherein the image detection model is obtained according to any of the above-mentioned image detection model training methods, and the predicted geometric figure parameters of the target image are parameter information of geometric figures in the target image obtained through prediction;

[0190] The determination module 1203 is configured to determine the target geometry in the target image based on the predicted geometry parameters of the target image.

[0191] In one possible implementation, the second acquisition module 1202 is used to obtain a predicted key point heat map of the target image according to the image detection model, where the predicted key point heat map of the target image is a heat map of at least one key point of the target image obtained through prediction; and according to the image detection model, obtain predicted geometric parameters of the target image based on the predicted key point heat map of the target image.

[0192] In a possible implementation, the determination module 1203 is further configured to determine designated key points in the target image based on the predicted key point heat map of the target image; and determine a target frame image containing the designated key points from multiple frames of images.

[0193] In a possible implementation, the second acquisition module 1202 is further configured to acquire a predicted corrected image of the target image according to the image detection model. The predicted corrected image of the target image is an image obtained by geometrically correcting the target image through prediction.

[0194] In a possible implementation, the first acquisition module 1201 is further configured to acquire actual geometric figures in the target image;

[0195] The determination module 1203 is further configured to determine an offset between the target geometric figure in the target image and the actual geometric figure in the target image based on the target geometric figure in the target image and the actual geometric figure in the target image.

[0196] The image detection model in the above-mentioned device is derived from the annotated and predicted geometric parameters of the sample image, enabling the model to learn the parameter information of the geometric figures in the image. Using this image detection model to obtain the predicted geometric parameters of the target image, the geometric figure is characterized by the geometric parameter information, enabling rapid identification of the target geometric figure in the image, simplifying the image detection process and improving image detection efficiency.

[0197] It should be understood that the above Figure 11 、 Figure 12 The provided device is illustrated only by the division of the above-mentioned functional modules when implementing its functions. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0198] Figure 13The following is a block diagram of a terminal device 1300 according to an exemplary embodiment of the present application. Terminal device 1300 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop computer, or desktop computer. Terminal device 1300 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0199] Typically, the terminal device 1300 includes a processor 1301 and a memory 1302 .

[0200] The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0201] The memory 1302 may include one or more computer-readable storage media, which may be non-transitory. The memory 1302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, which is used to be executed by the processor 1301 to implement the image detection model training method or image detection method provided in the method embodiment of the present application.

[0202] In some embodiments, terminal device 1300 may optionally include a peripheral device interface 1303 and at least one peripheral device. The processor 1301, memory 1302, and peripheral device interface 1303 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1303 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 1304, a display screen 1305, a camera assembly 1306, an audio circuit 1307, and a power supply 1309.

[0203] The peripheral device interface 1303 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1301 and the memory 1302. In some embodiments, the processor 1301, the memory 1302, and the peripheral device interface 1303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1301, the memory 1302, and the peripheral device interface 1303 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0204] The RF circuit 1304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1304 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1304 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 1304 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 1304 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1304 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0205] The display screen 1305 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1305 is a touch screen display, the display screen 1305 also has the ability to collect touch signals on the surface or above the surface of the display screen 1305. The touch signal can be input as a control signal to the processor 1301 for processing. In this case, the display screen 1305 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1305, which is set on the front panel of the terminal device 1300; in other embodiments, there can be at least two display screens 1305, which are respectively set on different surfaces of the terminal device 1300 or in a folding design; in other embodiments, the display screen 1305 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal device 1300. Even more, the display screen 1305 can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1305 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0206] The camera assembly 1306 is used to capture images or videos. Optionally, the camera assembly 1306 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1306 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0207] The audio circuit 1307 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 1301 for processing, or input into the RF circuit 1304 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there can be multiple microphones, each located in different parts of the terminal device 1300. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 1301 or the RF circuit 1304 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1307 may also include a headphone jack.

[0208] Power supply 1309 is used to power the various components in terminal device 1300. Power supply 1309 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1309 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0209] In some embodiments, the terminal device 1300 further includes one or more sensors 1310 , including but not limited to: an acceleration sensor 1311 , a gyroscope sensor 1312 , a pressure sensor 1313 , an optical sensor 1315 , and a proximity sensor 1316 .

[0210] The accelerometer 1311 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal device 1300. For example, the accelerometer 1311 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1301 can control the display screen 1305 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1311. The accelerometer 1311 can also be used to collect game or user motion data.

[0211] The gyroscope sensor 1312 can detect the body orientation and rotation angle of the terminal device 1300. The gyroscope sensor 1312 can work with the acceleration sensor 1311 to collect the user's 3D movements of the terminal device 1300. Based on the data collected by the gyroscope sensor 1312, the processor 1301 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0212] The pressure sensor 1313 can be set on the side frame of the terminal device 1300 and / or the lower layer of the display screen 1305. When the pressure sensor 1313 is set on the side frame of the terminal device 1300, it can detect the user's grip signal of the terminal device 1300, and the processor 1301 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 1313. When the pressure sensor 1313 is set on the lower layer of the display screen 1305, the processor 1301 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 1305. The operable controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0213] Optical sensor 1315 is used to detect ambient light intensity. In one embodiment, processor 1301 can control the display brightness of display screen 1305 based on the ambient light intensity detected by optical sensor 1315. Specifically, when the ambient light intensity is high, the display brightness of display screen 1305 is increased; when the ambient light intensity is low, the display brightness of display screen 1305 is decreased. In another embodiment, processor 1301 can also dynamically adjust the shooting parameters of camera assembly 1306 based on the ambient light intensity detected by optical sensor 1315.

[0214] Proximity sensor 1316, also known as a distance sensor, is typically located on the front panel of terminal device 1300. Proximity sensor 1316 is used to detect the distance between the user and the front of terminal device 1300. In one embodiment, when proximity sensor 1316 detects that the distance between the user and the front of terminal device 1300 is gradually decreasing, processor 1301 controls display screen 1305 to switch from the screen-on state to the screen-off state. When proximity sensor 1316 detects that the distance between the user and the front of terminal device 1300 is gradually increasing, processor 1301 controls display screen 1305 to switch from the screen-off state to the screen-on state.

[0215] Those skilled in the art will understand that Figure 13 The structure shown in the figure does not constitute a limitation on the terminal device 1300, and the terminal device 1300 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0216] Figure 14 This is a schematic diagram of the structure of the server provided in the embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more processors 1401 and one or more memories 1402, wherein the one or more memories 1402 store at least one program code, and the at least one program code is loaded and executed by the one or more processors 1401 to implement the image detection model training method or image detection method provided in the above-mentioned various method embodiments. Exemplarily, the processor 1401 is a CPU. Of course, the server 1400 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 1400 may also include other components for implementing device functions, which will not be described in detail here.

[0217] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one program code, and the at least one program code is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned image detection model training methods or image detection methods.

[0218] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0219] In an exemplary embodiment, a computer program or computer program product is also provided, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to enable the computer to implement any of the above-mentioned image detection model training methods or image detection methods.

[0220] It should be understood that the term "plurality" used herein refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0221] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0222] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A training method for an image detection model, characterized in that: The method comprises: Acquire a sample image and annotated geometric figure parameters of the sample image, where the annotated geometric figure parameters of the sample image are parameter information of a geometric figure in the sample image obtained through annotation, and the parameter information of the geometric figure includes coefficients of an equation of the geometric figure in three-dimensional space or a parameter vector of the equation in a least squares method; Acquire predicted geometric figure parameters of the sample image according to the first network model, where the predicted geometric figure parameters of the sample image are parameter information of geometric figures in the sample image obtained through prediction; Based on the annotated geometric parameters of the sample image and the predicted geometric parameters, the first network model is adjusted to obtain an image detection model.

2. The method according to claim 1, characterized in that The obtaining of predicted geometric parameters of the sample image according to the first network model includes: Obtaining a predicted key point heat map of the sample image according to the first network model, where the predicted key point heat map of the sample image is a heat map of at least one key point of the sample image obtained through prediction; The predicted geometric parameters of the sample image are obtained based on the predicted key point heat map of the sample image.

3. The method according to claim 2, characterized in that The obtaining of a predicted key point heat map of the sample image according to the first network model includes: Acquire a downsampled feature map of the sample image according to the first network model, where the downsampled feature map of the sample image includes semantic information of the sample image; determining a spatial feature map of the sample image based on the downsampled feature map of the sample image, where the spatial feature map of the sample image includes spatial position information of the sample image; A predicted key point heat map of the sample image is determined based on the spatial feature map of the sample image.

4. The method according to claim 2, characterized in that The obtaining of predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image includes: determining at least one key point of the sample image based on the predicted key point heat map of the sample image; Filtering a target key point from the at least one key point based on the position information of each key point; Predicted geometric parameters of the sample image are determined based on the target key points.

5. The method according to claim 4, characterized in that The step of selecting a target key point from the at least one key point based on the position information of each key point includes: Determining size information of a target area in the sample image and position information of a center point of the target area; Based on the position information of the center point and the position information of each key point, determining the distance information corresponding to each key point, where the distance information corresponding to any key point is the distance information between the center point and any key point; Filtering target key points from the at least one key point based on the size information of the target area and the distance information corresponding to each key point.

6. The method according to claim 5, characterized in that The filtering out a target key point from the at least one key point based on the size information of the target area and the distance information corresponding to each key point includes: For any key point, determining a ratio between the size information of the target area and the distance information corresponding to the key point, wherein the ratio represents a distance relationship between the target area and the key point; A target key point whose ratio is not greater than a ratio threshold is selected from the at least one key point.

7. The method according to any one of claims 2 to 6, characterized in that: The adjusting the first network model based on the annotated geometric figure parameters and the predicted geometric figure parameters of the sample image to obtain an image detection model includes: Obtaining a heat map of annotated key points of the sample image, where the heat map of annotated key points of the sample image is a heat map of at least one key point of the sample image obtained through annotation; Based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map, and the predicted key point heat map, the first network model is adjusted to obtain an image detection model.

8. The method according to any one of claims 1 to 6, characterized in that: After obtaining the sample image and the annotated geometric figure parameters of the sample image, the method further includes: Acquire a marked and corrected image of the sample image, where the marked and corrected image of the sample image is an image obtained by geometrically correcting the sample image through marking; Acquire a predicted and rectified image of the sample image according to the first network model, where the predicted and rectified image of the sample image is an image obtained by geometrically correcting the sample image through prediction; The adjusting the first network model based on the annotated geometric figure parameters and the predicted geometric figure parameters of the sample image to obtain an image detection model includes: Based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated corrected image, and the predicted corrected image, the first network model is adjusted to obtain an image detection model.

9. The method according to claim 8, characterized in that The obtaining of a predicted and rectified image of the sample image according to the first network model includes: Acquire a spatial feature map of the sample image according to the first network model; Performing linear transformation on the spatial feature map of the sample image to obtain an affine matrix, where the affine matrix contains geometric correction information of the sample image; Affine transformation is performed on the affine matrix to obtain a predicted corrected image of the sample image.

10. The method according to claim 1, characterized in that The adjusting the first network model based on the annotated geometric figure parameters and the predicted geometric figure parameters of the sample image to obtain an image detection model includes: Obtaining a labeled key point heat map, a predicted key point heat map, a predicted corrected image, and a labeled corrected image of the sample image; Based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map, the predicted key point heat map, the annotated corrected image and the predicted corrected image, the first network model is adjusted to obtain an image detection model.

11. An image detection method, characterized in that: The method comprises: Acquire the target image; Acquiring predicted geometric parameters of the target image according to an image detection model, wherein the image detection model is obtained according to the method of any one of claims 1 to 10, and the predicted geometric parameters of the target image are parameter information of geometric figures in the target image obtained through prediction; Based on the predicted geometry parameters of the target image, a target geometry in the target image is determined.

12. The method according to claim 11, characterized in that The step of obtaining the predicted geometric parameters of the target image according to the image detection model includes: Obtaining a predicted key point heat map of the target image according to the image detection model, where the predicted key point heat map of the target image is a heat map of at least one key point of the target image obtained through prediction; According to the image detection model, predicted geometric parameters of the target image are obtained based on the predicted key point heat map of the target image.

13. The method according to claim 12, characterized in that The target image includes multiple frames of images. After obtaining the predicted key point heat map of the target image according to the image detection model, the method further includes: determining a designated key point in the target image based on a predicted key point heat map of the target image; A target frame image containing the designated key point is determined from the multiple frame images.

14. The method according to any one of claims 11 to 13, characterized in that: After acquiring the target image, the method further includes: A predicted corrected image of the target image is acquired according to the image detection model, where the predicted corrected image of the target image is an image obtained by geometrically correcting the target image through prediction.

15. The method according to any one of claims 11 to 13, characterized in that: After determining the target geometry in the target image based on the predicted geometry parameters of the target image, the method further includes: Acquire actual geometric figures in the target image; An offset between the target geometry in the target image and the actual geometry in the target image is determined based on the target geometry in the target image and the actual geometry in the target image.

16. A training device for an image detection model, characterized in that: The device comprises: A first acquisition module is configured to acquire a sample image and annotated geometric figure parameters of the sample image, wherein the annotated geometric figure parameters of the sample image are parameter information of a geometric figure in the sample image obtained through annotation, and the parameter information of the geometric figure includes coefficients of an equation of the geometric figure in three-dimensional space or a parameter vector of the equation in a least squares method; A second acquisition module is configured to acquire predicted geometric figure parameters of the sample image according to the first network model, where the predicted geometric figure parameters of the sample image are parameter information of the geometric figure in the sample image obtained through prediction; An adjustment module is used to adjust the first network model based on the annotated geometric figure parameters of the sample image and the predicted geometric figure parameters to obtain an image detection model.

17. The device according to claim 16, characterized in that The second acquisition module is used to obtain the predicted key point heat map of the sample image according to the first network model, where the predicted key point heat map of the sample image is a heat map of at least one key point of the sample image obtained through prediction; and obtain the predicted geometric parameters of the sample image based on the predicted key point heat map of the sample image.

18. The device according to claim 17, characterized in that The second acquisition module is configured to acquire a downsampled feature map of the sample image according to the first network model, wherein the downsampled feature map of the sample image includes semantic information of the sample image; and determine a spatial feature map of the sample image based on the downsampled feature map of the sample image, wherein the spatial feature map of the sample image includes spatial position information of the sample image; A predicted key point heat map of the sample image is determined based on the spatial feature map of the sample image.

19. The device according to claim 17, characterized in that The second acquisition module is configured to determine at least one key point of the sample image based on the predicted key point heat map of the sample image; and filter out a target key point from the at least one key point based on position information of each key point; Predicted geometric parameters of the sample image are determined based on the target key points.

20. The device according to claim 19, characterized in that The second acquisition module is used to determine the size information of the target area in the sample image and the position information of the center point of the target area; Based on the position information of the center point and the position information of each key point, determining the distance information corresponding to each key point, where the distance information corresponding to any key point is the distance information between the center point and any key point; Filtering target key points from the at least one key point based on the size information of the target area and the distance information corresponding to each key point.

21. The device according to claim 20, characterized in that The second acquisition module is configured to determine, for any key point, a ratio of the size information of the target area to the distance information corresponding to the key point, wherein the ratio represents a distance relationship between the target area and the key point; A target key point whose ratio is not greater than a ratio threshold is selected from the at least one key point.

22. The device according to any one of claims 17 to 21, characterized in that The adjustment module is configured to obtain a heat map of annotated key points of the sample image, where the heat map of annotated key points of the sample image is a heat map of at least one key point of the sample image obtained through annotation; Based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map, and the predicted key point heat map, the first network model is adjusted to obtain an image detection model.

23. The device according to any one of claims 16 to 21, characterized in that The first acquisition module is further configured to acquire a labeled and corrected image of the sample image, where the labeled and corrected image of the sample image is an image obtained by geometrically correcting the sample image through labeling; The second acquisition module is further configured to acquire a predicted and rectified image of the sample image according to the first network model, where the predicted and rectified image of the sample image is an image obtained by geometrically correcting the sample image through prediction; The adjustment module is used to adjust the first network model based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated corrected image and the predicted corrected image to obtain an image detection model.

24. The device according to claim 23, characterized in that The second acquisition module is used to obtain the spatial feature map of the sample image based on the first network model; perform linear transformation processing on the spatial feature map of the sample image to obtain an affine matrix, and the affine matrix contains geometric correction information of the sample image; perform affine transformation processing on the affine matrix to obtain a predicted corrected image of the sample image.

25. The device according to claim 16, characterized in that The adjustment module is used to obtain the annotated key point heat map, the predicted key point heat map, the predicted corrected image and the annotated corrected image of the sample image; Based on the annotated geometric parameters of the sample image, the predicted geometric parameters, the annotated key point heat map, the predicted key point heat map, the annotated corrected image and the predicted corrected image, the first network model is adjusted to obtain an image detection model.

26. An image detection device, characterized in that: The device comprises: A first acquisition module is used to acquire a target image; a second acquisition module, configured to acquire predicted geometric figure parameters of the target image according to an image detection model, wherein the image detection model is obtained according to the method of any one of claims 1 to 10, and the predicted geometric figure parameters of the target image are parameter information of geometric figures in the target image obtained through prediction; A determination module is used to determine a target geometry in the target image based on the predicted geometry parameters of the target image.

27. The device according to claim 26, characterized in that The second acquisition module is used to obtain a predicted key point heat map of the target image according to the image detection model, where the predicted key point heat map of the target image is a heat map of at least one key point of the target image obtained through prediction; and obtain predicted geometric parameters of the target image based on the predicted key point heat map of the target image according to the image detection model.

28. The device according to claim 27, characterized in that The target image includes multiple frame images, and the determination module is further used to determine the specified key points in the target image based on the predicted key point heat map of the target image; and determine the target frame image containing the specified key points from the multiple frame images.

29. The device according to any one of claims 26 to 28, characterized in that The second acquisition module is further configured to acquire a predicted corrected image of the target image according to the image detection model, wherein the predicted corrected image of the target image is an image obtained by geometrically correcting the target image through prediction.

30. The device according to any one of claims 26 to 28, characterized in that The first acquisition module is further used to acquire the actual geometric figure in the target image; The determination module is further configured to determine an offset between the target geometric figure in the target image and the actual geometric figure in the target image based on the target geometric figure in the target image and the actual geometric figure in the target image.

31. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor so that the electronic device implements the training method of the image detection model described in any one of claims 1 to 10 or the image detection method described in any one of claims 11 to 15.

32. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one program code, which is loaded and executed by the processor so that the computer implements the training method of the image detection model as described in any one of claims 1 to 10 or the image detection method as described in any one of claims 11 to 15.

33. A computer program product, characterized in that The computer program product stores at least one computer instruction, which is loaded and executed by a processor so that the computer implements the image detection model training method as described in any one of claims 1 to 10 or the image detection method as described in any one of claims 11 to 15.

Images