Improved calculation method and system for adaptive spatial relationship correlation

By introducing an adaptive spatial relationship correlation algorithm in computer vision and image processing, the potential correlation of bounding boxes is evaluated using penalty coefficients and coordinate interchange, the problem of inability to evaluate the intersection-free bounding boxes in the prior art is solved, and the more Accurate correlation assessment.

CN119027485APending Publication Date: 2024-11-26BEIJING FUYOUDUODUO INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411193057.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art cannot effectively evaluate bounding boxes that have no intersection in space but have potential associations, especially in the fields of computer vision and image processing. Intersection ratio (IoU) can only evaluate the overlap ratio of two bounding boxes, for no actual overlap. But semantically or logically relevant bounding boxes cannot provide effective evaluation.

Method used

Adaptive spatial relationship correlation improved calculation method, by calculating the overlap and penalty coefficients of bounding boxes in the image two-dimensional coordinate system, recalculating the overlap after coordinate interchange, and combining the logical Stie function to evaluate the space of bounding boxes Relevance, solves the problem of evaluating potential correlations between bounding boxes.

Benefits of technology

This method can improve evaluation accuracy through multi-dimensional IoU calculation and punishment function without actual intersection of bounding boxes, enhance the judgment of potential correlation, and is suitable for processing text detection and association tasks, providing more accurate and Effective assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027485A_ABST
    Figure CN119027485A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an improved calculation method and system for adaptive spatial relationship correlation, and the method comprises the steps: calculating the overlap ratio between a key bounding box and a value bounding box based on an image two-dimensional coordinate system, judging whether the overlap ratio is smaller than a preset threshold value or not, and if yes, judging that a key and a value are not key values; penalty coefficients of the key boundary box and the value boundary box are calculated; performing coordinate interchange on the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the image two-dimensional coordinate system, and recalculating the overlap ratio between the key bounding box and the value bounding box based on the penalty coefficient and the key bounding box and the value bounding box after coordinate interchange; and based on the penalty coefficient and the recalculated coincidence degree, obtaining the spatial correlation of the key and the value. The problem that in the prior art, bounding boxes which do not have intersections in space but have potential correlation cannot be evaluated is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an improved computing system, method, electronic device and storage medium for adaptive spatial relationship correlation. Background Art

[0002] In the field of computer vision and image processing, Intersection over Union (IoU) is a widely used evaluation metric to measure the degree of overlap between two bounding boxes. The calculation formula of IoU is the ratio of the intersection area of ​​two bounding boxes (key bounding box and value bounding box) to the union area. Its mathematical expression is:

[0003]

[0004] In the formula, A intersection is the intersection area of ​​the key bounding box and the value bounding box, A union Represents the union area of ​​the key bounding box and the value bounding box. The I o U value ranges from 0 to 1, and a larger value indicates a higher degree of overlap between the two bounding boxes.

[0005] IoU is widely used in tasks such as object detection and image segmentation, mainly to evaluate the degree of overlap between the bounding box predicted by the model and the true bounding box. However, IoU has some limitations, especially when dealing with bounding boxes that do not actually overlap but have potential associations. Specifically, IoU can only evaluate the overlap ratio of two bounding boxes. For those bounding boxes that have no spatial intersection but are semantically or logically related, IoU cannot provide an effective evaluation.

[0006] Therefore, there is an urgent need for a method that can effectively evaluate bounding boxes that have no spatial intersection but are potentially associated. Summary of the invention

[0007] The purpose of the embodiments of the present invention is to provide an improved computing system, method, electronic device and storage medium for adaptive spatial relationship correlation, so as to solve the problem in the prior art that bounding boxes that have no spatial intersection but have potential association cannot be evaluated.

[0008] To achieve the above object, an embodiment of the present invention provides an improved calculation method for adaptive spatial relationship correlation, and the improved calculation method for adaptive spatial relationship correlation specifically includes:

[0009] Calculate the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, and determine whether the overlap is less than a preset threshold, and if so, determine that the key and the value are unrelated key values;

[0010] Calculate the penalty coefficients for the key bounding box and the value bounding box;

[0011] Respectively swapping the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional coordinate system of the image, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the key bounding box and the value bounding box after the coordinate swap;

[0012] The spatial correlation between keys and values ​​is obtained based on the penalty coefficient and the recalculated coincidence degree.

[0013] On the basis of the above technical solution, the present invention can also be improved as follows:

[0014] Further, the calculating the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, determining whether the overlap is less than a preset threshold, and if so, determining that the key and the value are unrelated key values, includes:

[0015] Calculate the overlap between the key bounding box and the value bounding box using formula (1);

[0016]

[0017] Where IoU is the overlap between the key bounding box and the value bounding box, A intersection is the intersection area of ​​the key bounding box and the value bounding box, A box1 is the area of ​​the key bounding box, A box2 is the area of ​​the value bounding box.

[0018] Further, the calculating of the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, determining whether the overlap is less than a preset threshold, and if so, determining that the key and the value are unrelated key values, further includes:

[0019] If the overlap degree is greater than a preset threshold, the key and value are determined to be related key values, and subsequent calculations are terminated.

[0020] Further, the calculating of the penalty coefficients of the key bounding box and the value bounding box includes:

[0021] The coefficient is calculated by formula (2);

[0022]

[0023] Where coef is the coefficient, Δy1 is the difference between the y coordinate of the upper boundary of the second frame and the y coordinate of the upper boundary of the first frame, Δy2 is the difference between the y coordinate of the lower boundary of the second frame and the y coordinate of the lower boundary of the first frame, and H box1 The difference between the x coordinate of the left border of the second border and the x coordinate of the left border of the first border, H box2The difference is the x-coordinate of the right border of the second border minus the x-coordinate of the right border of the first border.

[0024] Furthermore, the calculation of the penalty coefficients of the key bounding box and the value bounding box further includes:

[0025] The penalty coefficients of the key bounding box and the value bounding box are calculated by formula (3);

[0026]

[0027] Where D gap is the shortest distance between the key bounding box and the value bounding box, D center is the distance between the center point of the key and value border, and coef is the coefficient.

[0028] Furthermore, the step of swapping the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional coordinate system of the image, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the key bounding box and the value bounding box after the coordinate swapping includes:

[0029] The overlap between key values ​​is calculated by formula (4);

[0030]

[0031] Wherein, IoU1 is the overlap between the first frame and the second frame after the abscissa of the first frame is replaced by the abscissa of the second frame, IoU2 is the overlap between the first frame and the second frame after the abscissa of the second frame is replaced by the abscissa of the first frame, IoU3 is the overlap between the first frame and the second frame after the ordinate of the first frame is replaced by the ordinate of the second frame, and IoU4 is the overlap between the first frame and the second frame after the ordinate of the second frame is replaced by the ordinate of the first frame.

[0032] Furthermore, the obtaining of the spatial correlation between the key and the value based on the penalty coefficient and the recalculated overlap degree further includes:

[0033] The spatial correlation is calculated by formula (5);

[0034]

[0035] Wherein, Match is the spatial correlation, σ is the logistic function, when Match>=0, the key bounding box and the value bounding box match, and when Match<=0, the key bounding box and the value bounding box do not match.

[0036] An improved calculation system for adaptive spatial relationship correlation, comprising:

[0037] A first coincidence calculation module, used for calculating the coincidence between the key boundary box and the value boundary box based on the two-dimensional coordinate system of the image, and judging whether the coincidence is less than a preset threshold value, and if so, judging that the key and the value are unrelated key values;

[0038] A penalty coefficient calculation module, used to calculate the penalty coefficients of the key bounding box and the value bounding box;

[0039] A second coincidence calculation module is used to exchange the horizontal coordinates and the vertical coordinates of the key boundary box and the value boundary box in the two-dimensional coordinate system of the image, and recalculate the coincidence between the key boundary box and the value boundary box based on the penalty coefficient and the key boundary box and the value boundary box after the coordinate exchange;

[0040] The spatial correlation calculation module is used to obtain the spatial correlation of the key and the value based on the penalty coefficient and the recalculated coincidence degree.

[0041] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0042] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0043] The embodiments of the present invention have the following advantages:

[0044] The improved calculation method of adaptive spatial relationship correlation in the present invention adds multi-dimensional I o U calculation, uses innovative key value frames to interchange x and y coordinates, can measure the correlation of unrelated key values ​​at any angle, adds a penalty function, and gains the correlation results to further increase the accuracy. This solves the problem in the prior art that bounding boxes that have no intersection in space but have potential correlation cannot be evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0046] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0047] Figure 1 A flowchart of an improved calculation method for adaptive spatial relationship correlation of the present invention;

[0048] Figure 2 The architecture diagram of the improved computing system for adaptive spatial relationship correlation of the present invention;

[0049] Figure 3 A schematic diagram of the distance between a key bounding box and a value bounding box of the present invention;

[0050] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided by the present invention.

[0051] The accompanying drawings are denoted as follows:

[0052] A first coincidence calculation module 10 , a penalty coefficient calculation module 20 , a second coincidence calculation module 30 , a spatial correlation calculation module 40 , an electronic device 50 , a processor 501 , a memory 502 , and a bus 503 . DETAILED DESCRIPTION

[0053] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Figure 1 FIG. 1 is a flow chart of an improved calculation method for adaptive spatial relationship correlation according to an embodiment of the present invention. Figure 1 As shown, an improved calculation method for adaptive spatial relationship correlation provided by an embodiment of the present invention includes the following steps:

[0055] S101, calculating the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, and determining whether the overlap is less than a preset threshold, and if so, determining that the key and the value are unrelated key values;

[0056] Specifically: the overlap between the key bounding box and the value bounding box is calculated by formula (1);

[0057]

[0058] Where IoU is the overlap between the key bounding box and the value bounding box, A intersection is the intersection area of ​​the key bounding box and the value bounding box, A box1 is the area of ​​the key bounding box, A box2 is the area of ​​the value bounding box.

[0059] If the overlap is greater than a preset threshold, the key and value are determined to be related key values, and subsequent calculations are terminated. For example, if the overlap is 0.99, it means that one entity is completely contained in the other entity. This situation can be considered as complete identification and is related, and subsequent calculations are terminated.

[0060] S102, calculating the penalty coefficients of the key bounding box and the value bounding box;

[0061] Specifically, the coefficient is calculated by formula (2);

[0062]

[0063] Where coef is the coefficient, Δy1 is the difference between the y coordinate of the upper boundary of the second frame and the y coordinate of the upper boundary of the first frame, Δy2 is the difference between the y coordinate of the lower boundary of the second frame and the y coordinate of the lower boundary of the first frame, and H box1 The difference between the x coordinate of the left border of the second border and the x coordinate of the left border of the first border, H box2 The difference is the x-coordinate of the right border of the second border minus the x-coordinate of the right border of the first border.

[0064] Δy1=max(0,min(999,y box2,top -y box1,top ));

[0065] Δy2=max(0,min(999,y box2,bottom -y box1,bottom ));

[0066] D gap =distance between edges of box1 and box2;

[0067] D center =distance between centroids of box1 and box2;

[0068] The penalty coefficients of the key bounding box and the value bounding box are calculated by formula (3);

[0069]

[0070] Where D gap is the shortest distance between the key bounding box and the value bounding box, such as Figure 3 As shown, the short arrow spacing between the key border and the two value borders, D center is the distance between the center points of the key and value borders, the distance between the long arrows of the key border and the two value borders, and coef is the coefficient.

[0071] The spacing between the matching key value boxes is very small, and the spacing between the mismatched key value boxes is very large. However, relative to the distance between the center points of the two, the increase in the distance between the mismatched and matched ones is small. A variable D with a larger increase gap Divide by the variable D which has a smaller growth rate center , the penalty coefficient for the key-value box that is truly matched will be very small, otherwise the penalty coefficient will increase.

[0072] Box1 is the bounding box of the key, and box2 is the bounding box of the value. The calculation of coef is the height difference between box1 and box2 divided by the maximum height of box1 or box2. If the heights of the two boxes are exactly the same, the height difference is 0, so the coefficient is also 0, and the final penalty coefficient is also 0;

[0073] If the key value box is a normal box, and the value content wraps to make the box larger, then the height difference between the two boxes is not 0, but is not greater than the height of the value box, so the coefficient will not amplify the final result, but will only reduce it or remain unchanged; if the key box and the value box do not actually match, and there is a misalignment in position, that is, the height difference is greater than the height of box1 or box2, the coefficient will become larger, thereby amplifying the final result.

[0074] S103, swapping the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional coordinate system of the image, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the swapped key bounding box and the value bounding box;

[0075] Specifically, the overlap between key values ​​is calculated by formula (4);

[0076]

[0077] Wherein, IoU1 is the overlap between the first frame and the second frame after the abscissa of the first frame is replaced by the abscissa of the second frame, IoU2 is the overlap between the first frame and the second frame after the abscissa of the second frame is replaced by the abscissa of the first frame, IoU3 is the overlap between the first frame and the second frame after the ordinate of the first frame is replaced by the ordinate of the second frame, and IoU4 is the overlap between the first frame and the second frame after the ordinate of the second frame is replaced by the ordinate of the first frame.

[0078] Since the key and value are both independent text frames, there is no intersection between the two, so a method is needed to associate the two. In actual situations, there are only two corresponding relationships between matching keys and values: horizontal correspondence, with the key on the left and the value on the right; vertical correspondence, with the key on the top and the value on the bottom.

[0079] Perform several variant IoU calculations, replace the x-axis coordinate of box1 with the x-axis coordinate of box2 to obtain the new bounding box coordinate, which is equivalent to translating box1 horizontally to the x-coordinate of box2, and then calculating with box2 to obtain IoU1; replace the x-axis coordinate of box2 with the x-axis coordinate of box1 to obtain the new bounding box coordinate, which is equivalent to translating box2 horizontally to the x-coordinate of box1, and then calculating with box1 to obtain IoU2; replace the y-axis coordinate of box1 with the y-axis coordinate of box2 to obtain the new bounding box coordinate, which is equivalent to translating box1 vertically to the y-coordinate of box2, and then calculating with box2 to obtain IoU3; replace the y-axis coordinate of box2 with the y-axis coordinate of box1 to obtain the new bounding box coordinate, which is equivalent to translating box2 vertically to the y-coordinate of box1, and then calculating with box1 to obtain IoU4;

[0080] IoU1 and IoU2 are key-value matching metrics for horizontal correspondence, and IoU3 and IoU4 are key-value matching metrics for vertical correspondence.

[0081]

[0082] S104, obtaining the spatial correlation between the key and the value based on the penalty coefficient and the recalculated coincidence degree;

[0083] Specifically, the spatial correlation is calculated by formula (5);

[0084]

[0085] Wherein, Match is the spatial correlation, σ is the logistic function, when Match>=0, the key bounding box and the value bounding box match, and when Match<=0, the key bounding box and the value bounding box do not match.

[0086] σ is the Logistic function: its value range is 0 to 1. When x approaches positive infinity, its value approaches 1. When x approaches negative infinity, its value approaches 0. The function is to convert continuous values ​​into discrete values.

[0087]

[0088] The logistic function converts a continuous function into a binary classification.

[0089] The improved calculation method of the adaptive spatial relationship correlation calculates the overlap between the key boundary box and the value boundary box based on the two-dimensional coordinate system of the image, determines whether the overlap is less than a preset threshold, and if so, determines that the key and value are unrelated key values; calculates the penalty coefficient of the key boundary box and the value boundary box; swaps the horizontal and vertical coordinates of the key boundary box and the value boundary box in the two-dimensional coordinate system of the image, and recalculates the overlap between the key boundary box and the value boundary box based on the penalty coefficient and the swapped key boundary box and the value boundary box; obtains the spatial correlation between the key and the value based on the penalty coefficient and the recalculated overlap. The problem that the prior art cannot evaluate bounding boxes that have no intersection in space but have potential association is solved.

[0090] Aiming at the limitations of the existing IoU evaluation index, this paper proposes an improved adaptive spatial relationship correlation algorithm (ASRCA). ASRCA can not only evaluate the overlap degree of two bounding boxes, but also evaluate the potential correlation between them through coordinate matching when the bounding boxes have no actual intersection.

[0091] ASRCA introduces distance penalty coefficients and position rationality judgments to evaluate the potential association between two bounding boxes even when they have no actual intersection. This enables ASRCA to provide more accurate and effective evaluation when dealing with text detection and association tasks.

[0092] During the calculation process, ASRCA considers the relative position relationship between bounding boxes, and can more comprehensively evaluate the correlation between bounding boxes through position rationality judgment and distance penalty coefficient. This makes ASRCA more superior in some application scenarios where position relationship needs to be considered.

[0093] The calculation method of ASRCA is flexible and can be adjusted according to specific application scenarios. For example, the calculation method of the distance penalty coefficient can be adjusted as needed, or the evaluation of other specific attributes can be introduced. This allows ASRCA to adapt to different application needs and provide a more customized evaluation solution.

[0094] In summary, the adaptive spatial relationship correlation algorithm of the present invention is superior to the existing IoU evaluation index in processing non-intersection bounding boxes, considering positional relationships and flexibility, and can provide more accurate and effective evaluation in more application scenarios.

[0095] Figure 2 FIG. 1 is a flow chart of an improved computing system embodiment of the present invention for adaptive spatial relationship correlation; Figure 2 As shown, an improved calculation system for adaptive spatial relationship correlation provided by an embodiment of the present invention includes the following steps:

[0096] A first coincidence calculation module 10 is used to calculate the coincidence between the key boundary box and the value boundary box based on the two-dimensional coordinate system of the image, and determine whether the coincidence is less than a preset threshold. If so, it is determined that the key and the value are unrelated key values;

[0097] The first overlap calculation module 10 is also used for:

[0098] Calculate the overlap between the key bounding box and the value bounding box using formula (1);

[0099]

[0100] Where IoU is the overlap between the key bounding box and the value bounding box, A intersection is the intersection area of ​​the key bounding box and the value bounding box, A box1 is the area of ​​the key bounding box, A box2 is the area of ​​the value bounding box.

[0101] If the overlap degree is greater than a preset threshold, the key and value are determined to be related key values, and subsequent calculations are terminated.

[0102] A penalty coefficient calculation module 20, used to calculate the penalty coefficients of the key bounding box and the value bounding box;

[0103] The penalty coefficient calculation module 20 is also used for:

[0104] The coefficient is calculated by formula (2);

[0105]

[0106] Where coef is the coefficient, Δy1 is the difference between the y coordinate of the upper boundary of the second frame and the y coordinate of the upper boundary of the first frame, Δy2 is the difference between the y coordinate of the lower boundary of the second frame and the y coordinate of the lower boundary of the first frame, and H box1 The difference between the x coordinate of the left border of the second border and the x coordinate of the left border of the first border, H box2 The difference is the x-coordinate of the right border of the second border minus the x-coordinate of the right border of the first border.

[0107] The penalty coefficients of the key bounding box and the value bounding box are calculated by formula (3);

[0108]

[0109] Where D gap is the shortest distance between the key bounding box and the value bounding box, D center is the distance between the center point of the key and value border, and coef is the coefficient.

[0110] A second coincidence calculation module 30 is used to exchange the horizontal coordinates and the vertical coordinates of the key boundary box and the value boundary box in the two-dimensional coordinate system of the image, and recalculate the coincidence between the key boundary box and the value boundary box based on the penalty coefficient and the key boundary box and the value boundary box after the coordinate exchange;

[0111] The second overlap calculation module 30 is also used for:

[0112] The overlap between key values ​​is calculated by formula (4);

[0113]

[0114] Wherein, IoU1 is the overlap between the first frame and the second frame after the abscissa of the first frame is replaced by the abscissa of the second frame, IoU2 is the overlap between the first frame and the second frame after the abscissa of the second frame is replaced by the abscissa of the first frame, IoU3 is the overlap between the first frame and the second frame after the ordinate of the first frame is replaced by the ordinate of the second frame, and IoU4 is the overlap between the first frame and the second frame after the ordinate of the second frame is replaced by the ordinate of the first frame.

[0115] The spatial correlation calculation module 40 is used to obtain the spatial correlation between the key and the value based on the penalty coefficient and the recalculated overlap degree.

[0116] The spatial correlation calculation module 40 is also used for:

[0117] The spatial correlation is calculated by formula (5);

[0118]

[0119] Wherein, Match is the spatial correlation, σ is the logistic function, when Match>=0, the key bounding box and the value bounding box match, and when Match<=0, the key bounding box and the value bounding box do not match.

[0120] The improved calculation system of adaptive spatial relationship correlation of the present invention calculates the overlap between the key boundary box and the value boundary box based on the two-dimensional coordinate system of the image through the first overlap calculation module 10, determines whether the overlap is less than a preset threshold, and if so, determines that the key and value are unrelated key values; calculates the penalty coefficient of the key boundary box and the value boundary box through the penalty coefficient calculation module 20; swaps the horizontal coordinates and vertical coordinates of the key boundary box and the value boundary box in the two-dimensional coordinate system of the image through the second overlap calculation module 30, and recalculates the overlap between the key boundary box and the value boundary box based on the penalty coefficient and the coordinate swapped key boundary box and the value boundary box; obtains the spatial correlation of the key and the value based on the penalty coefficient and the recalculated overlap through the spatial correlation calculation module 40. The improved calculation system of adaptive spatial relationship correlation solves the problem in the prior art that bounding boxes that have no intersection in space but have potential associations cannot be evaluated.

[0121] Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 4 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503;

[0122] The processor 501 and the memory 502 communicate with each other via the bus 503;

[0123] The processor 501 is used to call the program instructions in the memory 502 to execute the methods provided by the above-mentioned method embodiments, for example, including: calculating the overlap between the key boundary box and the value boundary box based on the image two-dimensional coordinate system, judging whether the overlap is less than a preset threshold, and if so, judging that the key and value are unrelated key values; calculating the penalty coefficient of the key boundary box and the value boundary box; respectively swapping the horizontal coordinates and the vertical coordinates of the key boundary box and the value boundary box in the image two-dimensional coordinate system, and recalculating the overlap between the key boundary box and the value boundary box based on the penalty coefficient and the key boundary box after the coordinate swap; and obtaining the spatial correlation between the key and the value based on the penalty coefficient and the recalculated overlap.

[0124] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments, for example, including: calculating the overlap between a key bounding box and a value bounding box based on a two-dimensional image coordinate system, determining whether the overlap is less than a preset threshold, and if so, determining that the key and value are unrelated key values; calculating the penalty coefficient of the key bounding box and the value bounding box; respectively swapping the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional image coordinate system, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the swapped key bounding box; and obtaining the spatial correlation between the key and the value based on the penalty coefficient and the recalculated overlap.

[0125] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various storage media that can store program codes.

[0126] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0127] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0128] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An improved calculation method for adaptive spatial relationship correlation, characterized in that: The improved calculation method of the adaptive spatial relationship correlation specifically includes: Calculate the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, and determine whether the overlap is less than a preset threshold, and if so, determine that the key and the value are unrelated key values; Calculate the penalty coefficients for the key bounding box and the value bounding box; Respectively swapping the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional coordinate system of the image, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the key bounding box and the value bounding box after the coordinate swap; The spatial correlation between keys and values ​​is obtained based on the penalty coefficient and the recalculated coincidence degree.

2. The improved calculation method of adaptive spatial relationship correlation according to claim 1, characterized in that: The step of calculating the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, and determining whether the overlap is less than a preset threshold, and if so, determining that the key and the value are unrelated key values, comprises: Calculate the overlap between the key bounding box and the value bounding box using formula (1); Where IoU is the overlap between the key bounding box and the value bounding box, A intersection is the intersection area of ​​the key bounding box and the value bounding box, A box1 is the area of ​​the key bounding box, A box2 is the area of ​​the value bounding box.

3. The improved calculation method of adaptive spatial relationship correlation according to claim 1, characterized in that: The step of calculating the overlap between the key bounding box and the value bounding box based on the two-dimensional coordinate system of the image, and determining whether the overlap is less than a preset threshold, and if so, determining that the key and the value are unrelated key values, further includes: If the overlap degree is greater than a preset threshold, the key and value are determined to be related key values, and subsequent calculations are terminated.

4. The improved calculation method of adaptive spatial relationship correlation according to claim 1, characterized in that: The calculation of the penalty coefficients of the key bounding box and the value bounding box includes: The coefficient is calculated by formula (2); Where coef is the coefficient, Δy1 is the difference between the y coordinate of the upper boundary of the second frame and the y coordinate of the upper boundary of the first frame, Δy2 is the difference between the y coordinate of the lower boundary of the second frame and the y coordinate of the lower boundary of the first frame, and H box1 The difference between the x coordinate of the left border of the second border and the x coordinate of the left border of the first border, H box2 The difference is the x-coordinate of the right border of the second border minus the x-coordinate of the right border of the first border.

5. The improved calculation method of adaptive spatial relationship correlation according to claim 4, characterized in that: The calculation of the penalty coefficients of the key bounding box and the value bounding box also includes: The penalty coefficients of the key bounding box and the value bounding box are calculated by formula (3); Where D gap is the shortest distance between the key bounding box and the value bounding box, D center is the distance between the center point of the key and value border, and coef is the coefficient.

6. The improved calculation method of adaptive spatial relationship correlation according to claim 1, characterized in that: The step of respectively exchanging the horizontal coordinates and the vertical coordinates of the key bounding box and the value bounding box in the two-dimensional coordinate system of the image, and recalculating the overlap between the key bounding box and the value bounding box based on the penalty coefficient and the key bounding box and the value bounding box after the coordinate exchange, comprises: The overlap between key values ​​is calculated by formula (4); Wherein, IoU1 is the overlap between the first frame and the second frame after the abscissa of the first frame is replaced by the abscissa of the second frame, IoU2 is the overlap between the first frame and the second frame after the abscissa of the second frame is replaced by the abscissa of the first frame, IoU3 is the overlap between the first frame and the second frame after the ordinate of the first frame is replaced by the ordinate of the second frame, and IoU4 is the overlap between the first frame and the second frame after the ordinate of the second frame is replaced by the ordinate of the first frame.

7. The improved calculation method of adaptive spatial relationship correlation according to claim 6, characterized in that: The obtaining of the spatial correlation of keys and values ​​based on the penalty coefficient and the recalculated overlap degree further includes: The spatial correlation is calculated by formula (5); Wherein, Match is the spatial correlation, σ is the logistic function, when Match>=0, the key bounding box and the value bounding box match, and when Match<=0, the key bounding box and the value bounding box do not match.

8. An improved computing system for adaptive spatial relationship correlation, characterized in that: include: A first coincidence calculation module, used for calculating the coincidence between the key boundary box and the value boundary box based on the two-dimensional coordinate system of the image, and judging whether the coincidence is less than a preset threshold value, and if so, judging that the key and the value are unrelated key values; A penalty coefficient calculation module, used to calculate the penalty coefficients of the key bounding box and the value bounding box; A second coincidence calculation module is used to exchange the horizontal coordinates and the vertical coordinates of the key boundary box and the value boundary box in the two-dimensional coordinate system of the image, and recalculate the coincidence between the key boundary box and the value boundary box based on the penalty coefficient and the key boundary box and the value boundary box after the coordinate exchange; The spatial correlation calculation module is used to obtain the spatial correlation of the key and the value based on the penalty coefficient and the recalculated coincidence degree.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.