Graphic code detection method and device, mobile terminal, storage medium and product

Abstract

The application provides a kind of graphic code detection method, device, mobile terminal, storage medium and product, belong to artificial intelligence technical field.The method includes: the feature map set of target image is acquired, at least one first feature map is included in feature map set;Determine the pooling window in the first feature map, the pooling window includes multiple first feature points, and the first feature point in the pooling window is used to carry out pooling processing;For any first feature point in the pooling window, the first feature point is offset to obtain second feature point;Second feature point is carried out bilinear interpolation, and the feature value of multiple third feature points around second feature point is obtained;Determine the pooling feature value of first feature point based on the feature value of multiple third feature points;Based on the pooling feature value of multiple first feature points in the pooling window, determine the graphic code detection result of target image.The application can improve the detection accuracy of the deformed graphic code.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a graphic code detection method and device, a mobile terminal, a storage medium and a product. BACKGROUND

[0002] Due to the characteristics of storing more information and reading fast, a two-dimensional code is widely used in various scenes in life. For example, the two-dimensional code is applied to common scenes such as payment codes and shared bicycle unlocking. As the most common mobile terminal, a mobile phone needs to realize detection and recognition of the two-dimensional code. The detection refers to detecting whether the two-dimensional code is contained in an image and, in the case that the two-dimensional code is contained in the image, determining the position of the two-dimensional code in the image. The recognition refers to recognizing information stored by the two-dimensional code based on the position of the two-dimensional code in the image.

[0003] With the development of artificial intelligence, a target monitoring network based on a convolutional neural network has become a mainstream target detection method. Therefore, a mobile terminal usually uses a graphic code detection network based on a convolutional neural network to detect a two-dimensional code. However, in actual application scenarios, the two-dimensional code in an image is often deformed due to an incorrect shooting angle, and the graphic code detection network in the related art has low detection accuracy for the deformed two-dimensional code. SUMMARY

[0004] Embodiments of the present application provide a graphic code detection method and device, a mobile terminal, a storage medium and a product, which can improve the detection accuracy of a deformed graphic code. The technical solution is as follows:

[0005] In one aspect, a graphic code detection method is provided, and the method comprises:

[0006] obtaining a feature map set of a target image, wherein the feature map set comprises at least one first feature map;

[0007] determining a pooling window in the first feature map, wherein the pooling window comprises a plurality of first feature points, and the first feature points in the pooling window are used for pooling processing;

[0008] for any first feature point in the pooling window, offsetting the first feature point to obtain a second feature point;

[0009] performing bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point;

[0010] determining a pooling feature value of the first feature point based on the feature values of the plurality of third feature points;

[0011] determine the graphical code detection result of the target image based on the pooling feature values of the plurality of first feature points within the pooling window.

[0012] In another aspect, a graphical code detection apparatus is provided, the apparatus comprising:

[0013] an obtaining module configured to obtain a feature map set of a target image, the feature map set comprising at least one first feature map;

[0014] a first determining module configured to determine a pooling window in the first feature map, the pooling window comprising a plurality of first feature points, and the first feature points within the pooling window being used for pooling processing;

[0015] an offsetting module configured to offset any first feature point within the pooling window to obtain a second feature point;

[0016] an interpolating module configured to perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point;

[0017] a second determining module configured to determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points;

[0018] a third determining module configured to determine the graphical code detection result of the target image based on the pooling feature values of the plurality of first feature points within the pooling window.

[0019] In another aspect, a mobile terminal is provided, the mobile terminal comprising one or more processors and one or more memories, the one or more memories having stored therein at least one program code, the at least one program code being loaded and executed by the one or more processors to implement the graphical code detection method according to any of the above implementation manners.

[0020] In another aspect, a computer-readable storage medium is provided, the computer-readable storage medium having stored therein at least one program code, the at least one program code being loaded and executed by a processor to implement the graphical code detection method according to any of the above implementation manners.

[0021] In another aspect, a computer program product is provided, the computer program product having stored therein at least one program code, the at least one program code being used to be executed by a processor to implement the graphical code detection method according to any of the above implementation manners.

[0022] In the embodiment of the present application, since the offset is applied when the feature points are taken in the first feature map during the pooling processing of the feature map, that is, the first feature points are offset, the features of the deformed object can be better extracted. Therefore, the problem of inaccurate detection of the graphical code with deformation by the ordinary pooling can be solved by applying the variable pooling, that is, the detection accuracy of the deformed graphical code can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0024] Figure 2 A schematic diagram of selecting feature points according to an example embodiment of the present application is shown;

[0025] Figure 3 A schematic diagram of determining the feature value of the second feature point according to an example embodiment of the present application is shown;

[0026] Figure 4 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0027] Figure 5 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0028] Figure 6 A schematic diagram of a graphical code detection network according to an example embodiment of the present application is shown;

[0029] Figure 7 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0030] Figure 8 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0031] Figure 9 A schematic diagram of a first neural network and a second neural network according to an example embodiment of the present application is shown;

[0032] Figure 10 A flow chart of a graphical code detection method according to an example embodiment of the present application is shown;

[0033] Figure 11 A block diagram of a graphical code detection apparatus according to an example embodiment of the present application is shown;

[0034] Figure 12 A block diagram of a mobile terminal according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0036] In the present application, "a plurality of" refers to two or more. The "and / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0037] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the target image and sample image involved in the present application are obtained under full authorization.

[0038] In some embodiments, the graphical code detection method provided by the embodiments of the present application is executed by a mobile terminal, that is, the mobile terminal performs graphical code detection according to the method provided by the embodiments of the present application, wherein the graphical code detection refers to detecting whether the target image contains a graphical code and, in the case that the target image contains a graphical code, determining the position of the graphical code in the target image.

[0039] For example, a graphical code detection network is deployed in the mobile terminal, and the mobile terminal performs graphical code detection on the target image through the graphical code detection network. The graphical code detection network can be trained by the mobile terminal itself or by a server and then deployed on the mobile terminal. In the embodiments of the present application, the subject of training the graphical code detection network is not specifically limited.

[0040] The embodiments of the present application provide an application scenario:

[0041] In some embodiments, the graphical code detection method provided by the embodiments of the present application is applied in the scenario of a payment code. When a user pays for an account corresponding to the payment code, the user uses a mobile terminal to scan the payment code, that is, the mobile terminal captures a target image containing the payment code, detects the position of the graphical code in the target image through the graphical code detection method provided by the embodiments of the present application, and then identifies the account identifier stored in the graphical code based on the determined position, and pays for the account corresponding to the payment code based on the account identifier.

[0042] In some embodiments, the graphic code detection method provided by the embodiments of the present application is applied in a scene of unlocking a shared bicycle. When a user uses the shared bicycle, the user scans a graphic code of the shared bicycle using a mobile terminal, i.e., the mobile terminal captures a target image containing the graphic code, and the graphic code detection method provided by the embodiments of the present application is used to detect the position of the graphic code in the target image. Then, based on the determined position, identification information of the shared bicycle stored by the graphic code is recognized, and based on the identification information, an unlocking request is sent to a background server, the unlocking request carrying the identification information of the shared bicycle and the identification information of the mobile terminal, to request the background server to unlock.

[0043] In some embodiments, the graphic code detection method provided by the embodiments of the present application is applied in a scene of developing an invoice. When a user wants to develop an invoice, the user scans a graphic code on a checkout bill using a mobile terminal, i.e., the mobile terminal captures a target image containing the graphic code, and the graphic code detection method provided by the embodiments of the present application is used to detect the position of the graphic code in the target image. Then, based on the determined position, interface information stored by the graphic code is recognized, and based on the interface information, an invoice filling interface is displayed, invoice information is filled in the invoice filling interface, and an invoicing request is sent to a background server, the invoicing request carrying the invoice information, to request the background server to develop the invoice.

[0044] It should be noted that the above three application scenarios are only examples and do not limit the application scenarios of the embodiments of the present application. The present application can be applied in any scene for detecting a graphic code. In addition, the graphic code can be a two-dimensional code, or a three-dimensional code or a four-dimensional code developed in the future, etc. The mobile terminal can be a mobile phone, a tablet computer, or a smart wearable device, etc.

[0045] Reference is made to Figure 1 which shows a flowchart of a graphic code detection method according to an example embodiment of the present application. Referring to Figure 1 The method comprises the following steps:

[0046] Step 101: Obtain a feature map set of a target image, the feature map set including at least one first feature map.

[0047] The target image is a currently captured image, or the target image is an image stored in an image library of the mobile terminal, or the target image is an image in a chat record of a social application. In the embodiments of the present application, the source of the target image is not limited.

[0048] The feature map set includes at least one first feature map; when the feature map set includes multiple first feature maps, the sizes of the multiple first feature maps can be different. Accordingly, this step can be: the mobile terminal acquires the target image, determines multiple first feature maps of the target image, the different first feature maps have different sizes, and combines the multiple first feature maps into a feature map set.

[0049] In some embodiments, the mobile terminal may also apply the region of interest to determine the feature map set; correspondingly, this step may be: the mobile terminal acquires the target image, determines the region of interest in the target image, determines multiple first feature maps of the region of interest, and combines the multiple first feature maps into a feature map set.

[0050] In the embodiments of this application, the accuracy of detection can be improved by determining multiple feature maps of different scales for subsequent detection.

[0051] Step 102: Determine a pooling window in the first feature map. The pooling window includes multiple first feature points, and the first feature points within the pooling window are used for pooling processing.

[0052] A pooling window is a regular window; for example, if a mobile terminal determines a pooling window R in the first feature map, and the size of the pooling window R is 3x3, then the pooling window R can be represented by the following formula:

[0053] Formula 1: R = {(-1,-1),(-1,0),…,(0,1),(1,1)}.

[0054] In some embodiments, when the feature map set includes multiple first feature maps, the mobile terminal determines a pooling window in each of the multiple first feature maps. The size of the pooling window in different feature maps may be the same or different, and no specific limitation is made in this embodiment.

[0055] It should be noted that the pooling window sizes in different first feature maps can be the same or different; and the shapes of the pooling windows in different first feature maps can also be the same or different; in this embodiment, a square shape for the pooling window is used as an example for illustration.

[0056] Step 103: For any first feature point within the pooling window, offset the first feature point to obtain the second feature point.

[0057] In related technologies, pooling processes determine the maximum or average value of the first feature point within the pooling window as the output. For example, if the pooling process determines the maximum value of the first feature point within the pooling window, then the pooling process can be implemented using the following formula:

[0058] Formula 2:

[0059] y1(p0) is the pooling eigenvalue of the first feature point p0 determined by the pooling processing method of related techniques; p0 is the first feature point within the pooling window, n = 1, ..., N, N = |R|. x(p0+p n ) is p0+p n eigenvalues.

[0060] In this embodiment of the application, it is necessary to improve the detection accuracy of the deformed graphic code. Therefore, an offset is added to the coordinates of the first feature point within the pooling window of the rule. The offset can be an integer or a decimal, and the offsets of different first feature points within the pooling window can be different. The offset can be set and changed as needed. In this embodiment of the application, the offset is not specifically limited.

[0061] Accordingly, the pooling process in this application embodiment can be implemented using the following formula three:

[0062] Formula 3:

[0063] y2(p0) is the pooling feature value of the first feature point p0 determined by the pooling method provided in this application embodiment; p0 is the first feature point within the pooling window, n = 1, ..., N, N = |R|. x(p0+p n +Δp n ) is p0+p n +Δp n eigenvalues.

[0064] For example, see Figure 2 The first feature point within the pooling window R is Figure 2 The nine points on the left; and an offset Δp is added to the coordinates of the first feature point within the pooling window R. n The first feature point within the increased pooling window R is Figure 2 The nine dots on the right.

[0065] Step 104: Perform bilinear interpolation on the second feature point to obtain the feature values ​​of multiple third feature points surrounding the second feature point.

[0066] The number of third feature points can be 4 or 8, etc.; in this embodiment, the example of 4 third feature points is used for illustration. Additionally, since Δp nIt is possible that the coordinate of the second feature point is a decimal number, while the coordinate of the second feature point after the offset is an integer; therefore, the second feature point is interpolated by a bilinear interpolation method in the embodiment of the present application. The bilinear interpolation is linear interpolation in two directions (horizontal axis and vertical axis). For example, the coordinate of the second feature point is (i+u, j+v), and the four third feature points determined by the mobile terminal have the coordinates of (i, j), (i+1, j), (i, j+1) and (i+1, j+1) respectively; wherein u and v are decimals less than 1.

[0067] For any third feature point, the mobile terminal determines the feature value of the third feature point by the following process: the mobile terminal determines the feature values of a plurality of fourth feature points around the third feature point, determines the average of the feature values of the fourth feature points, and obtains the feature value of the third feature point.

[0068] The process of determining the feature value of the third feature point is only an example; the mobile terminal can determine the feature value of the third feature point by any interpolation method, which is not limited in the embodiment of the present application. For example, the mobile terminal determines the feature value of the nearest feature point of the third feature point, and takes the feature value of the nearest feature point of the third feature point as the feature value of the third feature point.

[0069] Step 105: determining the pooling feature value of the first feature point based on the feature values of the plurality of third feature points.

[0070] For example, referring to Figure 3 , the feature value of the second feature point p is determined by the feature values of the four third feature points around the second feature point p, the coordinate of the second feature point p is (i+u, j+v), the feature value of the second feature point p is represented as x(p), the four third feature points around the second feature point p are q1, q2, q3 and q4 respectively, the coordinates of the third feature points q1, q2, q3 and q4 are (i, j), (i+1, j), (i, j+1) and (i+1, j+1) respectively, and the feature values of the third feature points q1, q2, q3 and q4 are x(q1), x(q2), x(q3) and x(q4) respectively.

[0071] The mobile terminal determines the pooling feature value of the first feature point by any preset pooling algorithm based on the feature values of the plurality of third feature points; for example, the mobile terminal determines the average of the feature values of the plurality of third feature points, and obtains the pooling feature value of the first feature point; for another example, the mobile terminal determines the maximum value among the feature values of the plurality of third feature points, and obtains the pooling feature value of the first feature point; for another example, the mobile terminal determines the pooling feature value of the first feature point based on the feature values of the plurality of third feature points by the following steps 405-406.

[0072] Step 106: determining the graphic code detection result of the target image based on the pooling feature values of the plurality of first feature points in the pooling window.

[0073] The mobile terminal inputs the pooled feature values of the plurality of first feature points into a classifier to obtain a classification result, the classification result being whether the target image contains a graphic code, and inputs the pooled features of the plurality of first feature points into a regressor to obtain a regression result, the regression result being a position of the graphic code in the target image in the case where the target image contains the graphic code, and the classification result and the regression result constitute a graphic code detection result, that is, the graphic code detection result being whether the target image contains a graphic code and, in the case where the target image contains the graphic code, a position of the graphic code in the target image.

[0074] It should be noted that after the mobile terminal determines the graphic code detection result of the target image, the graphic code is identified based on the graphic code detection result; for example, the graphic code detection result is a position of the graphic code in the target image; then the mobile terminal identifies information stored by the graphic code based on the position of the graphic code in the target image, and then performs a subsequent operation; wherein the subsequent operation can be set to different operations according to different scenes; for example, in the payment transfer scene, the subsequent operation is a transfer operation; in the shared bicycle unlocking scene, the subsequent operation is a request for unlocking operation; in the invoice development scene, the subsequent operation is an application for invoice operation.

[0075] In the embodiments of the present application, since the offset is applied when the feature points are taken in the first feature map during the pooling processing of the feature map, that is, the first feature points are offset, the features of the deformed object can be better extracted; therefore, the application of the variable pooling can solve the problem of inaccurate detection of the graphic code with deformation by the ordinary pooling, that is, the present application can improve the detection accuracy of the deformed graphic code.

[0076] Reference is made to Figure 4 which shows a flowchart of a graphic code detection method according to an example embodiment of the present application. Referring to Figure 4 The method comprises the following steps:

[0077] Step 401: The mobile terminal acquires a feature map set of a target image, the feature map set comprising at least one first feature map.

[0078] It should be noted that this step is the same as step 101 and will not be repeated here.

[0079] Step 402: The mobile terminal determines a pooling window in the first feature map, the pooling window comprising a plurality of first feature points, and the first feature points in the pooling window being used for pooling processing.

[0080] It should be noted that this step is the same as step 102 and will not be repeated here.

[0081] Step 403: For any first feature point within the pooling window, the mobile terminal offsets the first feature point to obtain a second feature point.

[0082] It should be noted that this step is the same as step 103, and will not be repeated here.

[0083] Step 404: The mobile terminal performs bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point.

[0084] It should be noted that this step is the same as step 104, and will not be repeated here.

[0085] Step 405: For any third feature point, the mobile terminal determines a pooling feature value of the third feature point based on the feature value of the third feature point.

[0086] This step can be implemented through the following steps (1) and (2), comprising:

[0087] (1) The mobile terminal determines the pooling rate of the third feature point based on the coordinate information of the second feature point and the coordinate information of the third feature point.

[0088] This step can be implemented through the following steps (1-1) to (1-3), comprising:

[0089] (1-1) The mobile terminal determines a first offset corresponding to the horizontal axis coordinate and a second offset corresponding to the vertical axis coordinate based on the coordinate information of the second feature point and the coordinate information of the third feature point.

[0090] For example, the second feature point is p, the coordinate information of the second feature point p is (p x ,p y ); the third feature point is q, the coordinate information of the third feature point q is (q x ,q y ), then the first offset corresponding to the horizontal axis coordinate is |p x -q x |; the second offset corresponding to the vertical axis coordinate is |p y -q y |.

[0091] (1-2) The mobile terminal determines a first pooling rate corresponding to the horizontal axis coordinate and a second pooling rate corresponding to the vertical axis coordinate based on the first offset and the second offset.

[0092] The mobile terminal determines the first pooling rate based on the first offset through the following formula four, and determines the second pooling rate based on the second offset through the following formula five:

[0093] Formula four: g(q x ,q x) = max(0, 1 - |q x -p x |)

[0094] Formula Five: g(q y ,q y ) = max(0, 1 - |q y -p y |)

[0095] wherein g(q x ,q x ) is the first pooling rate, |q x -p x | is the first offset; g(q y ,q y ) is the second pooling rate; |q y -p y | is the second offset.

[0096] Correspondingly, the actual process of step (1-2) is that the mobile terminal determines whether the first offset is greater than 1, and in the case that the first offset is greater than 1, determines the first pooling rate as 0; and in the case that the first offset is not greater than 1, determines the first pooling rate as the difference between 1 and the first offset. The mobile terminal determines whether the second offset is greater than 1, and in the case that the second offset is greater than 1, determines the second pooling rate as 0; and in the case that the second offset is not greater than 1, determines the second pooling rate as the difference between 1 and the second offset.

[0097] (1-3) The mobile terminal determines the product of the first pooling rate and the second pooling rate to obtain the pooling rate of the third feature point.

[0098] The mobile terminal determines the pooling rate of the third feature point based on the first pooling rate and the second pooling rate through the following Formula Six:

[0099] Formula Six: G(p, q) = g(q x ,p x )g(g y ,p y )

[0100] wherein G(p, q) is the pooling rate of the third feature point, g(q x ,p x ) is the first pooling rate, and g(g y ,p y ) is the second pooling rate.

[0101] It should be noted that steps (1-1) to (1-3) are examples of determining the pooling rate of the third feature point based on the coordinate information of the second feature point and the coordinate information of the third feature point, and do not limit the determination of the pooling rate of the third feature point.

[0102] (2) The mobile terminal determines the product of the pooling rate of the third feature point and the feature value, and obtains the pooling feature value of the third feature point.

[0103] It should be noted that the mobile terminal can also determine the pooling feature value of the third feature point through other algorithms; for example, the mobile terminal determines the product of the pooling rate of the third feature point and the feature value, and obtains the pooling feature value of the third feature point by weighting the product; or the mobile terminal determines the product of the pooling rate of the third feature point and the feature value, and obtains the pooling feature value of the third feature point by adding an increment to the product.

[0104] Step 406: The mobile terminal determines the sum of the pooling feature values of the plurality of third feature points, and obtains the pooling feature value of the first feature point.

[0105] The mobile terminal determines the pooling feature value of the first feature point based on the pooling feature values of the plurality of third feature points, through the following Formula Seven:

[0106] Formula Seven: x(p) = ∑ q G(p,q)·x(q)

[0107] Wherein, x(p) is the pooling feature value of the first feature point, and G(p,q)·x(q) is the pooling feature value of the third feature point.

[0108] It should be noted that the mobile terminal can also determine the pooling rate of the first feature point through other algorithms; for example, the mobile terminal obtains the pooling feature value of the first feature point by weighting and summing the pooling feature values of the plurality of third feature points; for another example, the mobile terminal obtains the pooling feature value of the first feature point by adding an increment to the sum of the pooling feature values of the plurality of feature points.

[0109] Step 407: The mobile terminal determines the graphic code detection result of the target image based on the pooling feature values of the plurality of first feature points in the pooling window.

[0110] It should be noted that this step is the same as step 106, and will not be repeated here.

[0111] In the embodiments of the present application, since the offset is applied when the feature points are taken in the first feature map during the pooling processing of the feature map, that is, the first feature points are offset, the features of the deformed object can be better extracted; therefore, the application of the variable pooling can solve the problem of inaccurate detection of the graphic code with deformation by the ordinary pooling, that is, the present application can improve the detection accuracy of the deformed graphic code.

[0112] Please refer to Figure 5Fig. 5 shows a flowchart of a method for detecting a graphic code according to an example embodiment of the present application. As shown in Fig. 5, the method comprises the following steps.

[0113] Step 501: training a graphic code detection network by a mobile terminal.

[0114] The graphic code detection network comprises a backbone network, a variable pooling layer, a convolution layer, a classifier and a regressor.

[0115] It should be noted that step 501 needs to be performed only once. After the graphic code detection network is trained, the graphic code detection network can be directly used to detect a graphic code by steps 502 and 503.

[0116] It should be noted that the graphic code detection network can also be trained by a server, and the trained graphic code detection network can be deployed to the mobile terminal.

[0117] Step 502: obtaining a target image by the mobile terminal.

[0118] The target image can be an image currently captured by the mobile terminal, or an image stored in an image library of the mobile terminal, or an image in a chat record of a social application. In the example embodiment of the present application, the source of the target image is not limited. In addition, after the target image is obtained by the mobile terminal, a region of interest can be cropped from the target image, and the graphic code detection is performed based on the region of interest.

[0119] Step 503: inputting the target image into the graphic code detection network, and outputting a graphic code detection result of the target image.

[0120] The graphic code detection result is whether the target image contains a graphic code, and the position of the graphic code in the target image in the case that the target image contains a graphic code. For example, as shown in Fig. 6, the graphic code detection result is that the target image contains a graphic code, and the position of the graphic code in the target image is indicated by a bounding box. Figure 6 The graphic code detection network comprises a backbone network, a variable pooling layer, a convolution layer, a classifier and a regressor. The backbone network is a lightweight MobilenetV2 network; the variable pooling layer is a maximum pooling pyramid (PPN); for example, the variable pooling layer is represented as Maxpool, and the convolution layer is represented as Shared conv; the classifier is a Box classifier, and the regressor is a Box regressor.

[0121] The backbone network is used to obtain a feature map set of the target image.

[0122] The variable pooling layer is configured to determine a pooling window in the feature map, for any first feature point within the pooling window, offset the first feature point to obtain a second feature point, perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point, and determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points.

[0123] The convolution layer is configured to perform convolution processing on the pooling feature values of the plurality of first feature points within the pooling window to obtain a convolution processing result, and the convolution layer is a shared weight convolution layer.

[0124] The classifier is configured to detect whether the target image contains a graphic code based on the convolution processing result.

[0125] The regressor is configured to detect a position of the graphic code in the target image based on the convolution processing result in a case where the target image contains the graphic code.

[0126] Step 504: The mobile terminal identifies information stored in the graphic code based on the position of the graphic code in the target image.

[0127] For example, the graphic code is a payment code, and the mobile terminal identifies account information in the payment code. For another example, the graphic code is an unlocking code of a shared bicycle, and the mobile terminal identifies identification information of the shared bicycle in the unlocking code.

[0128] In the embodiments of the present application, a graphic code detection model is trained in advance, and the graphic code detection model is capable of performing deformable pooling on a feature map. Therefore, the detection of the graphic code based on the graphic code detection model can solve the problem of inaccurate detection of a deformed graphic code by using ordinary pooling, that is, the present application can improve the detection accuracy of the deformed graphic code.

[0129] Please refer to Figure 7 which shows a flowchart of training a graphic code detection network according to an example embodiment of the present application. As shown in Figure 7 The method comprises the following steps:

[0130] Step 701: The mobile terminal acquires a set of images, and the set of images comprises sample images used for training a graphic code detection network.

[0131] The sample images comprise positive sample images and negative sample images. The positive sample images are sample images containing graphic codes, and the positions of the graphic codes in the positive sample images are labeled. The negative sample images are sample images not containing graphic codes. For example, as shown in Figure 8 The mobile terminal acquires the set of images.

[0132] Step 702: The mobile terminal constructs a first neural network and a second neural network. The magnitude of the first neural network is smaller than that of the second neural network, and the second neural network includes a variable pooling layer.

[0133] Since the first neural network is a lightweight neural network, the image code detection network trained based on the first neural network is also lightweight, making it easy to deploy on mobile terminals. Variable pooling layers can better extract features of deformed image codes; therefore, networks using variable pooling layers can solve the problem of inaccurate target localization by ordinary detection networks for deformed or occluded targets. The second neural network can be a lightweight neural network, or it can be a regular-weighted neural network. In this embodiment, the second neural network is not specifically limited; in this embodiment, a lightweight neural network is used as an example for explanation. Accordingly, see [link to relevant documentation]. Figure 8 The mobile terminal constructs a lightweight image code detection network, which includes a student network (first neural network) and a teacher network (second neural network) that applies deformable pooling.

[0134] For example, see Figure 9 The mobile terminal constructs the first neural network for Figure 9 The neural network on the left side of the middle, the second neural network is Figure 9 In the neural networks on the right, the first neural network uses max-pooling layers, while the second neural network uses deformable max-pooling layers. Both the backbone networks of the first and second neural networks are MobileNet V2. The convolutional layers of both neural networks are shared convolutional layers. Both the classifiers of the first and second neural networks are box classifiers. Both the regressors of the first and second neural networks are box regressors.

[0135] Step 703: The mobile terminal trains a second neural network based on the image set until the second neural network converges.

[0136] The mobile terminal uses a supervised training method to train a second neural network based on an image set until the second neural network converges; for example, see [link to previous section]. Figure 8 .

[0137] Step 704: The mobile terminal performs knowledge distillation training on the first neural network based on the second neural network to obtain the image code detection network.

[0138] The mobile terminal takes the second neural network as a teacher network and takes the first neural network as a student network, so in this step, the mobile terminal performs knowledge distillation training on the student network based on the teacher network using the deformable pooling; determines whether the accuracy of the student network meets the standard or the number of training steps reaches the maximum number of steps; in the case that the accuracy of the student network meets the standard or the number of training steps reaches the maximum number of steps, the training is ended, and the student network obtained by the training is the graphic code detection network; in the case that the accuracy of the student network does not meet the standard and the number of training steps does not reach the maximum number of steps, the mobile terminal continues to perform knowledge distillation training on the student network based on the teacher network until the accuracy of the student network meets the standard or the number of training steps reaches the maximum number of steps, for example, continue to refer to Figure 8 .

[0139] In the embodiments of the present application, since the mobile terminal has strict requirements on the size and inference speed of the graphic code detection network, the deformable pooling layer can better extract deformation features to accurately locate the position of the graphic code; but at the same time, the storage of the offset coordinates and the irregular point picking mode will increase the size of the graphic code detection network and the memory transfer time. In the embodiments of the present application, the second neural network containing deformable pooling is taken as a teacher network to assist in training the first neural network deployed by the mobile terminal, so that the graphic code detection network based on the first neural network can not only learn the representation ability of the teacher network, but also ensure the model size and accuracy.

[0140] Please refer to Figure 10 , which shows a flowchart of training a graphic code detection network according to an example embodiment of the present application. Referring to Figure 10 , the method comprises:

[0141] Step 1001: The mobile terminal obtains an image set, and the image set comprises sample images for training a graphic code detection network.

[0142] It should be noted that this step is the same as step 701 and will not be repeated here.

[0143] Step 1002: The mobile terminal constructs a first neural network and a second neural network, the magnitude of the first neural network is smaller than the magnitude of the second neural network, and the second neural network comprises a deformable pooling layer.

[0144] It should be noted that this step is the same as step 702 and will not be repeated here.

[0145] Step 1003: The mobile terminal trains the second neural network based on the image set until the second neural network converges.

[0146] It should be noted that this step is the same as step 703 and will not be repeated here.

[0147] Step 1004: The mobile terminal determines a second feature map and a third feature map of the sample image based on the first neural network and the second neural network respectively, the second feature map including first feature values corresponding to the plurality of calculation layers, and the third feature map including second feature values corresponding to the plurality of calculation layers.

[0148] The mobile terminal determines the second feature map of the sample image through the first neural network including the backbone network, and determines the third feature map of the sample image through the second neural network including the backbone network. The calculation layer is a network layer that needs to be calculated by attention. For example, the calculation layer is a layer in the first neural network except for a classification output layer corresponding to a classifier and a regression output layer corresponding to a regressor.

[0149] For example, the size of the second feature map is h x w x c, where h and w are the height and width of the second feature map, and c is the number of channels of the second feature map. The second feature map can be represented as A ∈ R h*w*c .

[0150] Step 1005: The mobile terminal performs attention calculation on the second feature map and the third feature map respectively to obtain a fourth feature map and a fifth feature map, the fourth feature map including third feature values corresponding to the plurality of calculation layers, and the fifth feature map including sixth feature values corresponding to the plurality of calculation layers.

[0151] The mobile terminal performs attention calculation on the second feature map to obtain the fourth feature map, and performs attention calculation on the third feature map to obtain the fifth feature map. The algorithm for attention calculation can be any algorithm, for example, the sum of absolute values, the average of absolute values w times, or the maximum of absolute values w times.

[0152] In some embodiments, the algorithm for attention calculation is the sum of absolute values. The mobile terminal performs attention calculation on the second feature map to obtain the fourth feature map by the following formula eight:

[0153] Formula eight:

[0154] wherein A attention is the third feature value in the fourth feature map, and A i is the feature value of the i-th channel.

[0155] In some embodiments, the algorithm for attention calculation is the average of absolute values w times. The mobile terminal performs attention calculation on the second feature map to obtain the fourth feature map by the following formula nine:

[0156] Formula nine:

[0157] wherein A attention is the third feature value in the fourth feature map, and A iis the feature value of the i-th channel. w is the power, and w is an integer greater than or equal to 1.

[0158] In some embodiments, the algorithm of attention calculation is the maximum value of the absolute value w power; then the mobile terminal performs attention calculation on the second feature map by the following formula ten to obtain the fourth feature map:

[0159] Formula ten: A attention = max i＝1,c |A i | w

[0160] wherein A attention is the third feature value in the fourth feature map, A i is the feature value of the i-th channel. w is the power, and w is an integer greater than or equal to 1.

[0161] It should be noted that the process of the mobile terminal performing attention calculation on the third feature map to obtain the fifth feature map is similar to the process of the mobile terminal performing attention calculation on the second feature map to obtain the fourth feature map, which will not be repeated here.

[0162] It should be noted that the mobile terminal can also determine the fourth feature map and the fifth feature value by other attention algorithms, which are not limited in the embodiments of the present application.

[0163] Step 1006: The mobile terminal determines the loss function value corresponding to the first neural network based on the second feature map, the third feature map, the fourth feature map and the fifth feature map.

[0164] This step can be implemented by the following steps (1) to (3), including:

[0165] (1) For any calculation layer, the mobile terminal determines the first loss value corresponding to the calculation layer based on the third feature value and the fourth feature value corresponding to the calculation layer, and determines the first average loss value of the first loss values corresponding to the plurality of calculation layers.

[0166] The mobile terminal determines the first difference value of the third feature value and the fourth feature value, determines the w power of the absolute value of the first difference value, and obtains the first loss value corresponding to the calculation layer. Wherein, w can be set and changed according to the needs, which is not limited in the embodiments of the present application.

[0167] (2) The mobile terminal respectively determines the second loss value corresponding to the classifier and the third loss value corresponding to the regressor based on the first feature value and the second feature value corresponding to the classifier and the regressor in the second feature map and the point feature map, and determines the second average loss value of the second loss value and the third loss value.

[0168] The mobile terminal determines a second difference value of the first feature value corresponding to the classifier in the second feature map and the second feature value corresponding to the classifier in the third feature map, determines w-th power of the absolute value of the second difference value, and obtains a second loss value corresponding to the classifier; determines a third difference value of the first feature value corresponding to the regressor in the third feature map and the second feature group corresponding to the regressor in the third feature map, determines w-th power of the absolute value of the third difference value, and obtains a third loss value corresponding to the regressor; and determines an average value of the second loss value and the third loss value, and obtains a second average loss value.

[0169] (3) The mobile terminal determines a sum of the first average loss value and the second average loss value, and obtains a loss function value corresponding to the first neural network.

[0170] The steps (1) to (3) can be actually represented by the following formula eleven.

[0171] Formula eleven:

[0172] wherein, L KL_res is the loss function value corresponding to the first neural network, A att_teacher is the fourth feature value in the fifth feature map, A att_student is the third feature value in the fourth feature map (i.e. A attention determined by the formula eight, the formula nine, and the formula ten), A teacher is the second feature value in the third feature map, and A student is the first feature value in the second feature map.

[0173] Step 1007: The mobile terminal performs iterative training on the first neural network based on the loss function value, and obtains a graphic code detection network.

[0174] The mobile terminal performs iterative training on the first neural network based on the loss function value until the accuracy of the first neural network meets the standard or the training step number of the first neural network reaches the maximum step number, at which time the trained first neural network is the graphic code detection network.

[0175] In the embodiments of the present application, not only the final classification result (whether containing a graphic code) and the position prediction result (the position of the graphic code in the image) are distilled, but also the feature maps in the neural network are distilled with attention mechanism, so that the student network can learn the teacher network not only in the final prediction result but also in the network feature extraction process, thereby improving the inference accuracy of the trained graphic code detection model.

[0176] Please refer to Figure 11 which shows a block diagram of a graphic code detection apparatus according to an example embodiment of the present application, and Figure 11 the apparatus includes:

[0177] The acquisition module 1101 is configured to acquire a feature map set of a target image, the feature map set including at least one first feature map;

[0178] The first determination module 1102 is configured to determine a pooling window in the first feature map, the pooling window including a plurality of first feature points, and the first feature points in the pooling window being used for pooling processing;

[0179] The offset module 1103 is configured to, for any first feature point in the pooling window, offset the first feature point to obtain a second feature point;

[0180] The interpolation module 1104 is configured to perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point;

[0181] The second determination module 1105 is configured to determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points;

[0182] The third determination module 1106 is configured to determine a graphical code detection result of the target image based on the pooling feature values of the plurality of first feature points in the pooling window.

[0183] In some embodiments, the second determination module 1105 includes:

[0184] The first determination unit is configured to, for any third feature point, determine a pooling feature value of the third feature point based on the feature value of the third feature point;

[0185] The second determination unit is configured to determine a sum of the pooling feature values of the plurality of third feature points to obtain the pooling feature value of the first feature point.

[0186] In some embodiments, the first determination unit is configured to determine a pooling rate of the third feature point based on the coordinate information of the second feature point and the coordinate information of the third feature point; and determine a product of the pooling rate and the feature value of the third feature point to obtain the pooling feature value of the third feature point.

[0187] In some embodiments, the first determination unit is configured to determine a first offset corresponding to a horizontal axis coordinate and a second offset corresponding to a vertical axis coordinate based on the coordinate information of the second feature point and the coordinate information of the third feature point; determine a first pooling rate corresponding to the horizontal axis coordinate and a second pooling rate corresponding to the vertical axis coordinate based on the first offset and the second offset; and determine a product of the first pooling rate and the second pooling rate to obtain the pooling rate of the third feature point.

[0188] In some embodiments, the acquisition module 1101 is configured to determine a plurality of first feature maps of the target image, and group the plurality of first feature maps into the feature map set.

[0189] Different first feature maps have different sizes.

[0190] In some embodiments, the apparatus further includes:

[0191] a training module configured to train a graphic code detection network, the graphic code detection network including a backbone network, a variable pooling layer, a convolution layer, a classifier, and a regressor;

[0192] The backbone network is configured to obtain a feature map set of a target image.

[0193] The variable pooling layer is configured to determine a pooling window in the feature map, for any first feature point within the pooling window, offset the first feature point to obtain a second feature point, perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point, and determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points.

[0194] The convolution layer is configured to perform convolution processing on the pooling feature values of the plurality of first feature points within the pooling window to obtain a convolution processing result.

[0195] The classifier is configured to detect whether the target image contains a graphic code based on the convolution processing result.

[0196] The regressor is configured to detect a position of the graphic code in the target image based on the convolution processing result in a case where the target image contains the graphic code.

[0197] In some embodiments, the training module includes:

[0198] an obtaining unit configured to obtain an image set, the image set including sample images for training the graphic code detection network;

[0199] a constructing unit configured to construct a first neural network and a second neural network, the first neural network having a smaller magnitude than the second neural network, and the second neural network including the variable pooling layer;

[0200] a training unit configured to train the second neural network based on the image set until the second neural network converges;

[0201] the training unit is configured to perform knowledge distillation training on the first neural network based on the second neural network to obtain the graphic code detection network.

[0202] In some embodiments, the training unit is configured to determine, based on the first neural network and the second neural network, a second feature map and a third feature map of the sample image respectively, the second feature map comprising first feature values corresponding to the plurality of calculation layers, and the third feature map comprising second feature values corresponding to the plurality of calculation layers; perform attention calculation on the second feature map and the third feature map respectively to obtain a fourth feature map and a fifth feature map, the fourth feature map comprising third feature values corresponding to the plurality of calculation layers, and the fifth feature map comprising fourth feature values corresponding to the plurality of calculation layers; determine a loss function value corresponding to the first neural network based on the second feature map, the third feature map, the fourth feature map and the fifth feature map; and perform iterative training on the first neural network based on the loss function value to obtain the graph code detection network.

[0203] In some embodiments, the training unit is configured to, for any calculation layer, determine a first loss value corresponding to the calculation layer based on the third feature value and the fourth feature value corresponding to the calculation layer, and determine a first average loss value of the first loss values corresponding to the plurality of calculation layers; determine a second loss value corresponding to the classifier and a third loss value corresponding to the regressor based on the first feature value and the second feature value corresponding to the classifier and the regressor in the second feature map and the third feature map respectively, and determine a second average loss value of the second loss value and the third loss value; and determine a sum of the first average loss value and the second average loss value to obtain the loss function value corresponding to the first neural network.

[0204] In the embodiments of the present application, since the offset is applied when the feature points are taken in the first feature map during the pooling processing of the feature map, that is, the first feature points are offset, the features of the deformed object can be better extracted. Therefore, the application of the variable pooling can solve the problem of inaccurate detection of the graph code with deformation by the ordinary pooling, that is, the present application can improve the detection accuracy of the deformed graph code.

[0205] It should be noted that the above-mentioned embodiments of the graph code detection apparatus are only used for example to illustrate the division of the above-mentioned functional modules, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the mobile terminal is divided into different functional modules to complete all or part of the above-described functions. In addition, the graph code detection apparatus and the graph code detection method provided in the above-mentioned embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0206] For reference Figure 12Fig. 12 shows a block diagram of a mobile terminal 1200 according to an example embodiment of the present application. The mobile terminal 1200 can be a smartphone, a phablet, or a smart wearable device. The mobile terminal 1200 runs a first operating system and a second operating system, and the power consumption of the mobile terminal 1200 running the first operating system is lower than the power consumption of the mobile terminal 1200 running the second operating system. The mobile terminal 1200 can include one or more of the following components: a processor 1210, a memory 1220, and a display screen 1230.

[0207] The processor 1210 can include one or more processing cores. The one or more processing cores can be a first core controller and a second core controller. The processor 1210 connects various parts of the mobile terminal 1200 through various interfaces and lines, and performs various functions of the mobile terminal 1200 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1220, and calling data stored in the memory 1220. Alternatively, the processor 1210 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1210 can be integrated with one or more of a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), and a modem. Among them, the CPU is mainly used to process an operating system, a user interface, and an application program; the GPU is used to render and draw the content to be displayed on the display screen 1230; the NPU is used to implement an artificial intelligence (AI) function; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1210, but can be implemented by a separate chip.

[0208] The memory 1220 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 1220 includes a non-transitory computer-readable storage medium. The memory 1220 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1220 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the methods described below, etc., and the data storage area can store data created according to the use of the mobile terminal 1200 (such as audio data, a phone book, etc.).

[0209] The display screen 1230 is a display component for displaying a user interface. Optionally, the display screen 1230 is a display screen with a touch function, through which a user can perform a touch operation on the display screen 1230 using a finger, a stylus, or any suitable object.

[0210] The display screen 1230 is usually arranged on the front panel of the mobile terminal 1200. The display screen 1230 can be designed to be a full screen, a curved screen, a special-shaped screen, a double-sided screen, or a folding screen. The display screen 1230 can also be designed to be a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., and the present embodiment does not limit this.

[0211] In addition, those skilled in the art can understand that the structure of the mobile terminal 1200 shown in the above-described figures does not constitute a limitation on the mobile terminal 1200, and the mobile terminal 1200 can include more or fewer components than those shown in the figures, or combine certain components, or different component arrangements. For example, the mobile terminal 1200 also includes audio acquisition devices, speakers, radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (Wi-Fi) modules, power supplies, Bluetooth modules, etc., which are not described here.

[0212] The embodiments of the present application also provide a computer readable medium, which stores at least one program code, the at least one program code being loaded and executed by the processor to implement the graphical code detection method shown in each of the above embodiments.

[0213] The embodiments of the present application also provide a computer program product, which stores at least one program code, the at least one program code being loaded and executed by the processor to implement the graphical code detection method shown in each of the above embodiments.

[0214] In some embodiments, the computer program product related to the embodiments of the present application can be deployed on a mobile terminal for execution, or on a plurality of mobile terminals located at a site, or on a plurality of mobile terminals distributed at a plurality of sites and interconnected through a communication network, which can constitute a blockchain system.

[0215] Those skilled in the art should be aware that, in one or more examples described above, the functions described in the embodiments of the present application can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer storage medium and communication medium, wherein the communication medium includes any medium that facilitates the transfer of computer program from one place to another. The storage medium can be any available medium accessible by a general or special purpose computer.

[0216] The above description is only optional embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting graphic codes, characterized in that, The method comprises: obtaining a feature map set of a target image, the feature map set comprising at least one first feature map; determining a pooling window in the first feature map, the pooling window comprising a plurality of first feature points, and the first feature points within the pooling window being used for pooling processing; for any first feature point within the pooling window, offsetting the first feature point to obtain a second feature point; performing bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point; determining a pooling feature value of the first feature point based on the feature values of the plurality of third feature points; determining a graphical code detection result of the target image based on the pooling feature values of the plurality of first feature points within the pooling window.

2. The method of claim 1, wherein, The determination of the pooling feature value of the first feature point based on the feature values of the plurality of third feature points comprises: for any third feature point, determining a pooling feature value of the third feature point based on the feature value of the third feature point; determining a sum of the pooling feature values of the plurality of third feature points to obtain the pooling feature value of the first feature point.

3. The method of claim 2, wherein, The determination of the pooling feature value of the third feature point based on the feature value of the third feature point comprises: determining a pooling rate of the third feature point based on coordinate information of the second feature point and coordinate information of the third feature point; determining a product of the pooling rate and the feature value of the third feature point to obtain the pooling feature value of the third feature point.

4. The method of claim 3, wherein, The determination of the pooling rate of the third feature point based on the coordinate information of the second feature point and the coordinate information of the third feature point comprises: determining a first offset corresponding to a horizontal axis coordinate and a second offset corresponding to a vertical axis coordinate based on the coordinate information of the second feature point and the coordinate information of the third feature point, respectively; determining a first pooling rate corresponding to the horizontal axis coordinate and a second pooling rate corresponding to the vertical axis coordinate based on the first offset and the second offset, respectively; determining a product of the first pooling rate and the second pooling rate to obtain the pooling rate of the third feature point.

5. The method of claim 1, wherein, The obtaining of the feature map set of the target image comprises: determining a plurality of first feature maps of the target image, and grouping the plurality of first feature maps to form the feature map set; wherein the sizes of different first feature maps are different.

6. The method of claim 1, wherein, The method further comprises: training a graphical code detection network, the graphical code detection network comprising a backbone network, a variable pooling layer, a convolution layer, a classifier, and a regressor; wherein the backbone network is used to obtain a feature map set of the target image; the variable pooling layer is used to determine a pooling window in the first feature map, for any first feature point within the pooling window, offset the first feature point to obtain a second feature point, perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point, and determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points; the convolution layer is used to perform convolution processing on the pooling feature values of the plurality of first feature points within the pooling window to obtain a convolution processing result; The classifier is configured to detect whether the target image contains a graphic code based on the convolution processing result. The regressor is configured to detect a position of the graphic code in the target image based on the convolution processing result in a case where the target image contains the graphic code.

7. The method of claim 6, wherein, The training graphic code detection network comprises: an image set is acquired, the image set comprising sample images for training the graphic code detection network; a first neural network and a second neural network are constructed, the first neural network having a smaller scale than the second neural network, and the second neural network comprising a variable pooling layer; the second neural network is trained based on the image set until the second neural network converges; the first neural network is trained based on the second neural network by knowledge distillation to obtain the graphic code detection network.

8. The method of claim 7, wherein, The training of the first neural network based on the second neural network by knowledge distillation to obtain the graphic code detection network comprises: second feature maps and third feature maps of the sample images are respectively determined based on the first neural network and the second neural network, the second feature maps comprising first feature values corresponding to a plurality of calculation layers, and the third feature maps comprising second feature values corresponding to the plurality of calculation layers; fourth feature maps and fifth feature maps are obtained by performing attention calculation on the second feature maps and the third feature maps respectively, the fourth feature maps comprising third feature values corresponding to the plurality of calculation layers, and the fifth feature maps comprising fourth feature values corresponding to the plurality of calculation layers; a loss function value corresponding to the first neural network is determined based on the second feature maps, the third feature maps, the fourth feature maps, and the fifth feature maps; the first neural network is iteratively trained based on the loss function value to obtain the graphic code detection network.

9. The method of claim 8, wherein, The determination of the loss function value corresponding to the first neural network based on the second feature maps, the third feature maps, the fourth feature maps, and the fifth feature maps comprises: for any calculation layer, a first loss value corresponding to the calculation layer is determined based on the third feature value and the fourth feature value corresponding to the calculation layer, and a first average loss value of the first loss values corresponding to the plurality of calculation layers is determined; second loss values corresponding to the classifier and third loss values corresponding to the regressor are respectively determined based on first feature values and second feature values corresponding to the classifier and the regressor in the second feature maps and the third feature maps, and a second average loss value of the second loss values and the third loss values is determined; a sum of the first average loss value and the second average loss value is determined to obtain the loss function value corresponding to the first neural network.

10. A graphic code detection apparatus, characterized by, The apparatus comprises: an acquisition module configured to acquire a feature map set of a target image, the feature map set comprising at least one first feature map; a first determination module configured to determine a pooling window in the first feature map, the pooling window comprising a plurality of first feature points, and the first feature points in the pooling window being used for pooling processing; The offset module is configured to offset any first feature point in the pooling window to obtain a second feature point; The interpolation module is configured to perform bilinear interpolation on the second feature point to obtain feature values of a plurality of third feature points around the second feature point; The second determination module is configured to determine a pooling feature value of the first feature point based on the feature values of the plurality of third feature points; The third determination module is configured to determine a graphical code detection result of the target image based on the pooling feature values of the plurality of first feature points in the pooling window.

11. A mobile terminal, characterized by The mobile terminal comprises one or more processors and one or more memories, and the one or more memories store at least one program code, which is loaded and executed by the one or more processors to implement the graphical code detection method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one program code, which is loaded and executed by the processor to implement the graphical code detection method according to any one of claims 1 to 9.

13. A computer program product, characterised in that, The computer program product stores at least one program code, which is used to be executed by the processor to implement the graphical code detection method according to any one of claims 1 to 9.

Images