A corner point coordinate determination method, system, device and storage medium

Abstract

The application discloses a corner point coordinate determination method, system, device and storage medium. The method comprises the following steps: acquiring a target image; inputting the target image into a preset model to obtain a first feature map, a second feature map and a third feature map, wherein the preset model is used for segmenting the target image; processing the first feature map, the second feature map and the third feature map based on a heat map head and a detection head of the preset model to obtain corresponding detection boxes and corner point positioning heat maps, wherein the detection head is used for outputting the positions and categories of the detection boxes, and the heat map head is used for generating four corner point heat maps corresponding to each detection box; and determining the position coordinates of the points in each corner point positioning heat map that meet a preset condition as corner point coordinates for each detection box, that is, segmenting the target image, and then determining the corner point coordinates corresponding to each detection box based on the detection boxes and the corner point heat maps corresponding to the feature maps obtained through segmentation, so that the accuracy of the corner point coordinate determination is improved.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to a method, system, device and storage medium for determining corner coordinates. Background Technology

[0002] Corner detection is a crucial part of image detection. Current image detection methods require identifying the positions of the four corners of each document or invoice in both mixed and non-mixed document images. Mixed placement refers to a single image containing multiple invoices or documents, such as in a financial reimbursement scenario where multiple invoices are pasted onto a single sheet of paper.

[0003] Current strategies for corner localization in document images include corner regression schemes. This approach follows a top-down framework, treating the corner localization problem in mixed images as the detection and localization of a single target region. Based on the feature map extracted from the target image, it uses fully connected or convolutional layers to regress the corner position and bounding box for each feature point, thereby obtaining the corner position. However, this direct corner regression scheme only analyzes each feature point and lacks consideration of the overall structure, resulting in low localization accuracy.

[0004] Therefore, how to accurately determine the corner locations of the mixed-attachment target area is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method, system, device, and storage medium for determining corner coordinates, thereby optimizing the low positioning accuracy problem of existing corner regression schemes.

[0006] To address the above problems, the technical solutions provided in this application are as follows:

[0007] The first aspect of this application provides a method for determining corner coordinates, including:

[0008] Acquire the target image;

[0009] The target image is input into a preset model to obtain a first feature map, a second feature map, and a third feature map. The preset model is used to segment the target image.

[0010] The heatmap head and detection head based on the preset model process the first feature map, the second feature map and the third feature map to obtain the corresponding detection box and corner location heatmap. The detection head is used to output the position and category of the detection box, and the heatmap head is used to generate the four corner heatmaps corresponding to each detection box.

[0011] For each detection frame, the coordinates of the points that meet the preset conditions in the corner point positioning heatmap are determined as the corner point coordinates.

[0012] Optionally, before acquiring the target image, the process further includes:

[0013] Obtain the preliminary image;

[0014] The long and short sides of the prepared image are adjusted to obtain a target image that conforms to the preset image size.

[0015] Optionally, the step of inputting the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map includes:

[0016] The target image is processed based on a feature extraction structure and a feature fusion structure of a preset model to obtain a first feature map, a second feature map, and a third feature map. The feature extraction structure is used to extract features from the target image, and the feature fusion structure is used to fuse the low-level texture features and high-level semantic features of the target image.

[0017] Optionally, for each detection frame, determining the coordinates of points in the corner location heatmap that meet preset conditions as corner coordinates includes:

[0018] For each detection box, the maximum value of the independent variable is taken for each heatmap corresponding to the detection box, and the coordinates of the maximum point in the heatmap of each corner point are obtained, which are used as the corner point coordinates.

[0019] Optionally, the preset model further includes a preset segmentation head, which is used to generate a segmentation map for each detection box to improve the accuracy of corner heatmap localization.

[0020] Optionally, the preset model further includes a classification head, which is used to output the position and category of the detection box.

[0021] Optionally, the method for determining the corner location heatmap includes:

[0022] A fourth feature map is predicted based on a preset convolutional structure, a first feature map, and a heatmap header; the weights of the heatmaps for each corner point of the detection box are determined based on the first feature map, the second feature map, the third feature map, and the heatmap header.

[0023] Based on the fourth feature map and the determined weights, several preliminary feature maps are obtained;

[0024] Based on the determined preliminary feature map and S-shaped growth curve, the corner location heatmap corresponding to the detection box is obtained.

[0025] A second aspect of this application provides a corner point coordinate determination system, comprising:

[0026] The first acquisition unit is used to acquire the target image;

[0027] The input unit is used to input the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map; the preset model is used to segment the target image.

[0028] The processing unit is used to process the first feature map, the second feature map and the third feature map based on the heatmap head and the detection head of the preset model to obtain the corresponding detection box and corner point localization heatmap. The detection head is used to output the position and category of the detection box, and the heatmap head is used to generate the four corner point heatmaps corresponding to each detection box.

[0029] The corner coordinate determination unit is used to determine the position coordinates of points that meet preset conditions in the corner positioning heatmap for each detection frame.

[0030] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the corner coordinate determination method described in any one of the first aspects above.

[0031] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the corner coordinate determination method as described in any of the preceding first aspects.

[0032] Compared with the prior art, this application has the following beneficial effects:

[0033] This application acquires a target image; inputs the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map, the preset model being used to segment the target image; processes the first, second, and third feature maps based on a heatmap head and a detection head of the preset model to obtain corresponding detection boxes and corner location heatmaps, the detection head being used to output the position and category of the detection boxes, and the heatmap head being used to generate four corner heatmaps corresponding to each detection box; for each detection box, the coordinates of points in each corner location heatmap that meet preset conditions are determined as corner coordinates, i.e., the target image is segmented, and then the corner coordinates corresponding to each detection box are determined based on the detection boxes and corner heatmaps corresponding to the segmented feature maps. Compared to the prior art that processes the complete target image, this application splits the target image and then processes multiple parts, reducing the processing target, and simultaneously processes multiple split smaller targets, improving the efficiency and accuracy of corner coordinate determination. Thus, efficient prediction of corner location within a mixed target area is achieved. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of a corner point coordinate determination method provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of the backbone and neck structure in the preset model provided in the embodiments of this application;

[0037] Figure 3 This is a schematic diagram of the head structure for the feature map provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the head structure for the feature map provided in an embodiment of this application;

[0039] Figure 5 This is a structural diagram of a corner coordinate determination system provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0042] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0043] As mentioned earlier, existing technologies for corner localization in document images typically employ two approaches: First, the corner regression approach, which follows a top-down framework. It treats corner localization in mixed images as a problem of detecting and locating corners within a single target region. Based on the extracted feature maps, it regresses the corner position and bounding box for each feature point using fully connected or convolutional layers. This direct corner regression approach has the advantage of being time-efficient, but its localization accuracy is relatively low. Second, the heatmap-based corner localization approach, which follows a bottom-up approach. It detects all corner positions in a single image and then determines the relationships between the corners, grouping corners belonging to the same target region together. In other words, it treats corner localization in mixed images as a problem of corner detection and association. The model generates a heatmap to obtain corner positions and calculates the grouping relationships between corners by generating grouped feature vectors. This approach offers relatively high corner positioning accuracy in non-mixed-lay scenarios, but suffers from unstable corner correlation in mixed-lay scenarios.

[0044] In summary, the two existing processing schemes—corner regression and heatmap-based schemes—each have their advantages and disadvantages. Corner regression can reliably detect mixed-area target regions, but its corner location accuracy within these regions is not high. Heatmap-based schemes can identify corner locations, but in mixed-area scenarios, they are prone to misjudging the relationships between corners in different target regions, leading to chaotic corner location in mixed-area scenarios. In other words, neither scheme can accurately determine corner coordinates in mixed-area scenarios.

[0045] To address the aforementioned technical problems, this application provides a corner coordinate determination method. The method involves acquiring a target image; inputting the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map, whereby the preset model is used to segment the target image; processing the first, second, and third feature maps based on a heatmap head and a detection head of the preset model to obtain corresponding detection boxes and corner location heatmaps. The detection head outputs the position and category of the detection boxes, and the heatmap head generates four corner heatmaps corresponding to each detection box; for each detection box, the coordinates of points in each corner location heatmap that meet preset conditions are determined as corner coordinates. In other words, the target image is segmented, and the corner coordinates corresponding to each detection box are determined based on the detection boxes and corner heatmaps obtained from the segmentation. Compared to the prior art that processes the complete target image, this application splits the target image and then processes multiple parts, reducing the processing target and simultaneously processing multiple smaller targets, thus improving the efficiency and accuracy of corner coordinate determination. This achieves efficient prediction of corner location within a mixed target area.

[0046] Furthermore, the implementation of the corner coordinate determination method in this application is not limited to a specific entity. For example, the corner coordinate determination method in this application can be applied to data processing devices such as terminal devices or servers. The terminal device can be an electronic device such as a smartphone, computer, personal digital assistant (PDA), or tablet computer. The server can be a standalone server, a cloud server, or a cluster server composed of multiple servers.

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] The following embodiment illustrates a method for determining corner coordinates provided in this application. See also... Figure 1 The figure is a flowchart of a corner coordinate determination method provided in an embodiment of this application. The execution subject of this method is a server, and the method includes:

[0049] S101. Obtain the target image.

[0050] The target image is an image whose image format, size, and content all conform to the preset model standard. After being input into the preset model, it can be processed directly without any other processing.

[0051] In one possible implementation, before acquiring the target image, steps A1-A2 are further included:

[0052] Step A1: Obtaining a preliminary image can be based on selection within a mixed area. For example, after selecting area a for mixed area A, the content in area a is converted into a preliminary image.

[0053] Step A2: Adjust the long and short sides of the prepared image to obtain a target image that conforms to the preset image size.

[0054] Based on the preliminary image obtained in the above steps, the image size of the preliminary image is adjusted. For example, for a target image of size H*W, the longer side of the image can be modified to 640, the shorter side scaled proportionally, and then the image is padded to a target image of size 640*640. It should be noted that this preset image size can be adaptively adjusted according to actual needs. The size of the input feature map of the convolutional layer is H(input)×W(input)×C(input), which represents the height, width, and number of channels of the input feature map, respectively.

[0055] S102. Input the target image into the preset model to obtain the first feature map, the second feature map and the third feature map.

[0056] The preset model can be used to segment and split the target image, that is, to split the target image into three feature maps.

[0057] In one possible implementation, the step of inputting the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map includes:

[0058] The target image is processed based on a feature extraction structure and a feature fusion structure of a preset model to obtain a first feature map, a second feature map, and a third feature map. The feature extraction structure is used to extract features from the target image, and the feature fusion structure is used to fuse the low-level texture features and high-level semantic features of the target image.

[0059] The target image is input into a preset model. Through the feature extraction backbone and classification head within the model, three feature maps are obtained. The feature map sizes are 80*80*, 40*40*, and 20*20*, respectively, representing the first, second, and third feature maps. In practical applications, the feature map sizes can be adjusted according to actual needs. The three sizes mentioned above are merely illustrative and do not affect the protection scope.

[0060] The preset model is used to transform and process the target image. This instance segmentation model consists of CSPDarknet (backbone, the basic model for feature extraction), PANNet (neck, the feature fusion structure), and Dethead (detection head), cls head (classification head), Yolact head (segmentation head), and heatmap head (heatmap head), which do not require pre-anchor boxes. A schematic diagram of the preset model structure is attached. Figure 2 Appendix Figure 3 Appendix Figure 4 , Figure 2This is a schematic diagram of the backbone and neck structure in the preset model provided in the embodiments of this application. Here, conv represents a convolutional layer, unsample represents deconvolution, and Conv block represents a convolutional block, which consists of multiple layers. P1 is the preliminary image, P2 is the target image, P3 is the first feature map, P4 is the second feature map, and P5 is the third feature map. Figure 3 This is a schematic diagram of the head structure for the feature map provided in an embodiment of this application, namely... Figure 3 The diagram illustrates the processing flow and results for the second and third feature maps. Figure 4 This is a schematic diagram of the head structure for the feature map provided in an embodiment of this application, namely... Figure 4 The diagram illustrates the processing flow and results for the first feature map. The backbone extracts image features, while the neck fuses low-level texture features and high-level semantic features to enhance the model's feature representation capabilities. The Det head and cls head output the location and category of the target detection boxes. The Yolact head is an auxiliary branch that generates an instance segmentation mask for each detection box, improving the accuracy of corner heatmap localization. The heatmap head generates heatmaps for the four corners of each detection box.

[0061] Figure 3 The `dethead` uses a 1x1 convolution to predict the coordinates of the four edges (top, bottom, left, and right), resulting in an h*w*4 feature map. The `clshead` uses a 1x1 convolution to predict the category of the target region, also resulting in an h*w*1 feature map. The `yolacthead`, corresponding to the prototype map weight head `yolactprotoweight`, uses a 1x1 convolution to predict the weights of the segmentation prototype features of the target region, resulting in an h*w*32 feature map. The `headmaphead`, corresponding to the `heatmapprotoweight`, uses a 1x1 convolution to predict the prototype feature weights of the four heatmaps corresponding to the target region, resulting in an h*w*64 feature map.

[0062] Figure 4The `dethead` predicts the coordinates of the four edges (top, bottom, left, and right) using a 1x1 convolution, resulting in an h*w*4 feature map. The `clshead` predicts the category of the target region using a 1x1 convolution, also resulting in an h*w*1 feature map. The `yolact` head, corresponding to the prototype map feature extraction network `yolactproto net`, predicts the prototype features of image segmentation components using a convolutional structure, resulting in a 2h*2w*32 feature map. The `yolactproto weight head` predicts the weights of the prototype features of the target region using a 1x1 convolution, resulting in an h*w*32 feature map. The `heatmap proto net` predicts the prototype features of the image heatmap using a convolutional structure, resulting in a 2h*2w*16 feature map. The `heatmap protoweight` predicts the prototype feature weights of the four heatmaps corresponding to the target region using a 1x1 convolution, resulting in an h*w*64 feature map.

[0063] The loss function for the object detection head consists of two parts: classification loss and bounding box localization loss. The classification loss uses the cross-entropy loss function, with two classes representing the target region and the background region respectively. The bounding box localization loss function comprises the DFL loss (Distributed Focusing Loss) and the DIOU loss (Intersection over Union based on diagonal distance). The loss function for the heatmap head combines FocalLoss... (聚焦损失函数) The point-wise logistic regression loss is used. The loss function of the Yolact head is a binary cross-entropy loss function based on each point.

[0064] The training set distribution for the pre-defined model training process can include: medical scenarios (medical invoices, medical insurance settlement statements, expense lists, CT reports, etc.) accounting for 45%, financial scenarios (fixed-amount invoices, machine-printed invoices, VAT invoices, train tickets, taxi receipts, etc.) accounting for 40%, and insurance scenarios (insurance policies, endorsements, etc.) accounting for 15%, with a total training set of over 50,000 images.

[0065] S103. Based on the preset model, the heat map head and the detection head process the first feature map, the second feature map and the third feature map to obtain the corresponding detection box and corner point positioning heat map.

[0066] The detection head is used to output the position and category of the detection box, and the heat map head is used to generate a heat map of the four corner points corresponding to each detection box.

[0067] The three feature maps obtained from the above steps are input into the heatmap head and the detection head to obtain the corner localization heatmap and the predicted output of the target detection box.

[0068] S104. For each detection frame, determine the coordinates of the points that meet the preset conditions in the corner point positioning heatmap as the corner point coordinates.

[0069] Each detection box corresponds to a heatmap of four corner points. For each detection box, the coordinates of points in the four corner heatmaps that meet preset conditions are determined and used as the corner coordinates of the current detection box. This process is repeated for each detection box in the current target image, thereby obtaining the corner coordinates of each detection box.

[0070] In practical applications, threshold filtering and non-maximum suppression algorithms can be applied to the target detection box to obtain the final target box region. This allows the target detection box to be adjusted according to actual needs.

[0071] In one possible implementation, determining the coordinates of points in the corner location heatmap that meet preset conditions as corner coordinates for each detection frame includes:

[0072] For each detection box, the maximum value of the independent variable is taken for each heatmap corresponding to the detection box, and the coordinates of the maximum point in the heatmap of each corner point are obtained, which are used as the corner point coordinates.

[0073] The preset condition is the location of the maximum point in the heatmap. For each target detection box, the maximum value of the variable argmax is taken for each heatmap, and the coordinates of the maximum point are obtained as the corner coordinates. In this way, the position of each corner point of each document image is obtained.

[0074] In one possible implementation, the preset model further includes a preset segmentation head, which is used to generate a mask (segmentation map) for each detection box segmentation to improve the accuracy of corner heatmap localization.

[0075] In one possible implementation, the preset model further includes a classification head, which is used to output the position and category of the detection box.

[0076] In one possible implementation, the method for determining the corner location heatmap includes:

[0077] A fourth feature map is predicted based on a preset convolutional structure, a first feature map, and a heatmap header; the weights of the heatmaps for each corner point of the detection box are determined based on the first feature map, the second feature map, the third feature map, and the heatmap header.

[0078] Based on the fourth feature map and the determined weights, several preliminary feature maps are obtained;

[0079] Based on the determined preliminary feature map and S-shaped growth curve, the corner location heatmap corresponding to the detection box is obtained.

[0080] The calculation method for the corner heatmap is as follows: obtain the heatmap proto based on the first feature map, and obtain the weights of each corner heatmap corresponding to the detection box by the heatmap proto weight head corresponding to the first, second and third feature maps. Multiply and sum them to obtain four feature maps of size 160*160. Process these four feature maps based on the S-shaped growth curve sigmoid to obtain the predicted heatmap result.

[0081] For document image corner location tasks in mixed-image scenarios, the solution provided in this application has the advantages of high corner location accuracy, strong generalization ability, and low time consumption. This solution combines the technical features of target detection, instance segmentation, and heatmap corner location, optimizes the low location accuracy problem of general corner regression solutions, solves the problem of unstable heatmap corner correlation, and achieves high-precision determination of corner coordinates.

[0082] The above are some specific implementations of the corner coordinate determination method provided in the embodiments of this application. Based on this, this application also provides a corresponding system for corner coordinate determination. The system provided in the embodiments of this application will be described below from the perspective of functional modularity. Figure 5 This is a structural diagram of a corner point coordinate determination system provided in an embodiment of this application.

[0083] The system includes:

[0084] The first acquisition unit 110 is used to acquire the target image;

[0085] Input unit 111 is used to input the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map; the preset model is used to segment the target image;

[0086] Processing unit 112 is used to process the first feature map, the second feature map and the third feature map based on the heat map head and the detection head of the preset model to obtain the corresponding detection box and corner point positioning heat map. The detection head is used to output the position and category of the detection box, and the heat map head is used to generate the four corner point heat map corresponding to each detection box.

[0087] The corner coordinate determination unit 113 is used to determine the position coordinates of points that meet preset conditions in the corner positioning heatmap for each detection frame as corner coordinates.

[0088] Optionally, the system further includes:

[0089] The second acquisition unit is used to acquire the preliminary image;

[0090] The adjustment unit is used to adjust the long and short sides of the prepared image to obtain a target image that conforms to the preset image size.

[0091] Optionally, the input unit is specifically used for:

[0092] The target image is processed based on a feature extraction structure and a feature fusion structure of a preset model to obtain a first feature map, a second feature map, and a third feature map. The feature extraction structure is used to extract features from the target image, and the feature fusion structure is used to fuse the low-level texture features and high-level semantic features of the target image.

[0093] Optionally, for each detection frame, determining the coordinates of points in the corner location heatmap that meet preset conditions as corner coordinates includes:

[0094] For each detection box, the maximum value of the independent variable is taken for each heatmap corresponding to the detection box, and the coordinates of the maximum point in the heatmap of each corner point are obtained, which are used as the corner point coordinates.

[0095] Optionally, the preset model further includes a preset segmentation head, which is used to generate a mask for each detection box segmentation to improve the accuracy of corner heatmap localization.

[0096] Optionally, the preset model further includes a classification head, which is used to output the position and category of the detection box.

[0097] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0098] like Figure 6 As shown, the computer device 01 is represented in the form of a general-purpose computing device. The components of the computer device 01 may include, but are not limited to: one or more processors or processing units 03, system memory 08, and bus 04 connecting different system components (including system memory 08 and processing unit 03).

[0099] Bus 04 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0100] Computer device 01 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 01, including volatile and non-volatile media, removable and non-removable media.

[0101] System memory 08 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 09 and / or cache memory 10. Computer device 01 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 04 via one or more data media interfaces. Memory 08 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0102] A program / utility 12 having a set (at least one) of program modules 13 may be stored in, for example, memory 08. Such program modules 13 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of this application.

[0103] Computer device 01 can also communicate with one or more external devices 02 (e.g., keyboard, pointing device, display 07, etc.), and with one or more devices that enable a user to interact with the computer device 01, and / or with any device that enables the computer device 01 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 06. Furthermore, computer device 01 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 05. Figure 6 As shown, network adapter 05 communicates with other modules of computer device 01 via bus 04. It should be understood that, although... Figure 6 As not shown in the diagram, it can be used in conjunction with computer device 01 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0104] The processor unit 03 executes various functional applications and data processing by running programs stored in the system memory 08, such as implementing a front-end button permission control method provided in the embodiments of this application.

[0105] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0106] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0107] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0108] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining corner point coordinates, characterized in that, include: Acquire the target image; The target image is input into a preset model to obtain a first feature map, a second feature map, and a third feature map. The preset model is used to segment the target image. The heatmap head and detection head based on the preset model process the first feature map, the second feature map and the third feature map to obtain the corresponding detection box and corner location heatmap. The detection head is used to output the position and category of the detection box, and the heatmap head is used to generate the four corner heatmaps corresponding to each detection box. For each detection frame, the coordinates of the points that meet the preset conditions in the corner point positioning heatmap are determined as the corner point coordinates.

2. The method according to claim 1, characterized in that, Before acquiring the target image, the process also includes: Obtain the preliminary image; The long and short sides of the prepared image are adjusted to obtain a target image that conforms to the preset image size.

3. The method according to claim 1, characterized in that, The step of inputting the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map includes: The target image is processed based on a feature extraction structure and a feature fusion structure of a preset model to obtain a first feature map, a second feature map, and a third feature map. The feature extraction structure is used to extract features from the target image, and the feature fusion structure is used to fuse the low-level texture features and high-level semantic features of the target image.

4. The method according to claim 1, characterized in that, For each detection frame, determining the coordinates of points in the corner location heatmap that meet preset conditions as corner coordinates includes: For each detection box, the maximum value of the independent variable is taken for each heatmap corresponding to the detection box, and the coordinates of the maximum point in the heatmap of each corner point are obtained, which are used as the corner point coordinates.

5. The method according to claim 1, characterized in that, The preset model also includes a preset segmentation head, which is used to generate a segmentation map for each detection box to improve the accuracy of corner heatmap positioning.

6. The method according to claim 1, characterized in that, The preset model also includes a classification head, which is used to output the position and category of the detection box.

7. The method according to claim 1, characterized in that, The method for determining the corner location heatmap includes: A fourth feature map is predicted based on a preset convolutional structure, a first feature map, and a heatmap header; the weights of the heatmaps for each corner point of the detection box are determined based on the first feature map, the second feature map, the third feature map, and the heatmap header. Based on the fourth feature map and the determined weights, several preliminary feature maps are obtained; Based on the determined preliminary feature map and S-shaped growth curve, the corner location heatmap corresponding to the detection box is obtained.

8. A corner point coordinate determination system, characterized in that, The system includes: The first acquisition unit is used to acquire the target image; The input unit is used to input the target image into a preset model to obtain a first feature map, a second feature map, and a third feature map; the preset model is used to segment the target image. The processing unit is used to process the first feature map, the second feature map and the third feature map based on the heatmap head and the detection head of the preset model to obtain the corresponding detection box and corner point localization heatmap. The detection head is used to output the position and category of the detection box, and the heatmap head is used to generate the four corner point heatmaps corresponding to each detection box. The corner coordinate determination unit is used to determine the position coordinates of points that meet preset conditions in the corner positioning heatmap for each detection frame.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the corner coordinate determination method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the corner coordinate determination method as described in any one of claims 1-7.

Images