Method and device for measuring coverage area, electronic equipment, medium and program product

Abstract

The embodiment of the invention provides a coverage area measurement method and device, electronic equipment, a medium and a program product. The method comprises the following steps: acquiring image data acquired by a camera; performing target detection on the image data by rotating a target detection model, and determining a target coverage area in the image data according to a target detection result; performing image correction on a reference coverage area in the image data; determining a size proportion relation of a reference coverage area occupied by each pixel point in the image data; wherein the reference coverage area comprises the target coverage area; and performing definite integral calculation on the target coverage area in combination with the size proportion relation to obtain the coverage area of the target coverage area. According to the embodiment of the invention, the method achieves the optimization of the measurement of the coverage area, can adapt to the oblique shooting visual angle of a camera, and improves the measurement accuracy of the coverage area.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a method, device, electronic equipment, medium and program product for measuring coverage area. Background Art

[0002] In scenarios such as warehouse location detection, the coverage area measurement of the coverage area is usually involved. However, most of the existing coverage area measurements are based on relevant calculations from the bird's-eye view of the camera. If the camera is installed diagonally above, relevant calculations are also performed from the bird's-eye view.

[0003] However, in practical applications, cameras are mostly installed obliquely above the range to be measured, which inevitably leads to certain image distortion. If the coverage area is measured on the distorted image from a bird's-eye view, the accuracy of the coverage area measurement will be affected. Summary of the invention

[0004] In view of the above problems, a method, device, electronic device, medium and program product for measuring coverage area that overcomes the above problems or at least partially solves the above problems are proposed, including:

[0005] A method for measuring coverage area, the method comprising:

[0006] Get the image data collected by the camera;

[0007] Performing target detection on the image data by rotating the target detection model, and determining a target coverage area in the image data according to the target detection result;

[0008] performing image correction on a reference coverage area in the image data;

[0009] Determine the size ratio relationship of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area;

[0010] In combination with the size ratio relationship, a definite integral calculation is performed on the target coverage area to obtain the coverage area of ​​the target coverage area.

[0011] Optionally, determining the target coverage area in the image data according to the target detection result includes:

[0012] According to the positions of all the rotation detection frames in the target detection result, a minimum circumscribed polygon is determined as the target coverage area in the image data.

[0013] Optionally, determining the size ratio relationship of the reference coverage area occupied by each pixel in the image data includes:

[0014] determining a coverage area of ​​a reference coverage region in the image data;

[0015] The size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined according to the coverage area of ​​the reference coverage area and the size information of the image data.

[0016] Optionally, determining the coverage area of ​​the reference coverage area in the image data includes:

[0017] Determine relevant attribute information of the camera and size information of the image data;

[0018] The coverage area of ​​the reference coverage area in the image data is determined according to the relevant attribute information of the camera and the size information of the image data.

[0019] Optionally, the relevant attribute information of the camera includes the height of the camera and the focal length of the camera, and the size information of the image data includes the width of the image data and the height of the image data.

[0020] Optionally, performing image correction on a reference coverage area in the image data includes:

[0021] A perspective transformation matrix is ​​obtained, and the perspective transformation matrix is ​​used to correct the positions of the pixels in the reference coverage area in the image data.

[0022] Optionally, it also includes:

[0023] Sample data is acquired, and a perspective transformation matrix is ​​determined according to coordinates of a plurality of pixel points in the sample data before image correction and after image correction.

[0024] Optionally, the image data is image data captured by the camera using an oblique viewing angle.

[0025] Optionally, the rotating target detection model performs target detection on the image data by rotating the detection frame;

[0026] The head part of the rotating target detection model has a branch for predicting the coordinate value of the rotating detection frame and a branch for predicting the angle of the rotating detection frame. The branch for predicting the coordinate value of the rotating detection frame and the branch for predicting the angle of the rotating detection frame are two decoupled branches.

[0027] Optionally, the head part of the rotating object detection model also has a branch for classification.

[0028] Optionally, the rotation target detection model reconstructs the entire feature map by setting a feature refinement module to perform feature alignment.

[0029] Optionally, the feature refinement module reconstructs the entire feature map in the following manner:

[0030] For a feature point in the input feature map, a new feature vector is obtained according to the coordinate value of the rotation detection box corresponding to the feature point, and the new feature vector is used to replace the previous feature vector of the feature point;

[0031] The feature points in the new feature map are traversed to replace the feature vectors to obtain a reconstructed feature map.

[0032] Optionally, before the step of replacing the previous feature vector of the feature point with the new feature vector, the step further includes:

[0033] The new feature vector is updated through bilinear interpolation.

[0034] Optionally, before obtaining a new feature vector based on the coordinate value of the rotation detection box corresponding to the feature point in the input feature map, the method further includes:

[0035] Through two-way convolution, the input feature map is superimposed to obtain a new feature map.

[0036] Optionally, the rotation target detection model adopts a loss function combining Inner-IoU (Intersection over Union) and SIoU during model training.

[0037] A device for measuring coverage area, the device being used for:

[0038] Get the image data collected by the camera;

[0039] Performing target detection on the image data by rotating the target detection model, and determining a target coverage area in the image data according to the target detection result;

[0040] performing image correction on a reference coverage area in the image data;

[0041] Determine the size ratio relationship of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area;

[0042] In combination with the size ratio relationship, a definite integral calculation is performed on the target coverage area to obtain the coverage area of ​​the target coverage area.

[0043] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the coverage area measurement method as described above when executed by the processor.

[0044] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for measuring coverage area as described above is implemented.

[0045] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the coverage area measurement method as described above.

[0046] The embodiments of the present invention have the following advantages:

[0047] In an embodiment of the present invention, image data collected by a camera is acquired, target detection is performed on the image data by rotating the target detection model, and the target coverage area in the image data is determined based on the target detection result, image correction is performed on the reference coverage area in the image data, and then the size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined, the reference coverage area includes the target coverage area, and the definite integral calculation is performed on the target coverage area in combination with the size ratio relationship to obtain the coverage area of ​​the target coverage area, thereby achieving optimization of the coverage area measurement, being able to adapt to the oblique shooting angle of the camera, and improving the accuracy of the coverage area measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0049] Figure 1 is a flowchart of a method for measuring a coverage area provided by some embodiments of the present invention;

[0050] Figure 2 is a flowchart of steps of another method for measuring coverage area provided by some embodiments of the present invention;

[0051] Figure 3 is a schematic diagram of rotating target detection provided by some embodiments of the present invention;

[0052] Figure 4 is a schematic diagram of another rotating target detection provided by some embodiments of the present invention;

[0053] Figure 5is a schematic diagram of a model structure provided by some embodiments of the present invention;

[0054] Figure 6a is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0055] Figure 6b is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0056] Figure 6c is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0057] Figure 6d is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0058] Figure 7 is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0059] Figure 8 is a schematic diagram of another model structure provided by some embodiments of the present invention;

[0060] Fig. 9 is a schematic diagram of a loss function structure provided by some embodiments of the present invention;

[0061] Fig.10 is a schematic diagram of image correction provided by some embodiments of the present invention;

[0062] Fig.11 is a schematic diagram of a reference coverage area relationship provided by some embodiments of the present invention;

[0063] Fig.12 It is a schematic diagram of a definite integral calculation provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] In the embodiment of the present invention, the image data is detected by rotating the target detection model, the target box obtained according to the target detection result is circumscribed with the minimum polygon, the required area is perspective transformed, and a certain image correction is performed. After the correction, the irregular measurement area is calculated using mathematical formulas such as definite integrals to achieve a more accurate value. Figure 1 , by using the rotating target detection model to detect the input image, the minimum circumscribed polygon of all detection boxes in the identifiable area of ​​the detection result is calculated, and then the reference area is perspective transformed to correct the identifiable area. After correcting the identifiable area, the final identifiable area coverage size is calculated.

[0066] Reference Figure 2 , shows a flowchart of a method for measuring a coverage area provided by some embodiments of the present invention, which may specifically include the following steps:

[0067] Step 201, obtaining image data collected by a camera.

[0068] In scenarios such as warehouse location detection, cameras are deployed to monitor the warehouse locations. Some cameras are installed diagonally above, and the image data may be image data captured by the camera using an oblique viewing angle.

[0069] Step 202: Perform target detection on the image data by rotating the target detection model, and determine the target coverage area in the image data according to the target detection result.

[0070] Since the camera is tilted, the object is distorted to varying degrees. In this case, a suitable rotating target detection model can be selected to detect the collected image data. Then, the target detection result of the rotating target detection model for the image data can be obtained. Then, the target coverage area in the image data can be determined based on the target detection result, that is, the identification area.

[0071] In some embodiments of the present invention, determining the target coverage area in the image data based on the target detection result includes: determining a minimum circumscribed polygon as the target coverage area in the image data based on the positions of all rotation detection boxes in the target detection result.

[0072] In practical applications, the rotation target detection model uses a rotation detection frame for target detection, such as Figure 3 After performing target detection on the image data, multiple rotation detection frames for selecting the target object can be determined, and then the minimum circumscribed polygon of all the rotation detection frames can be determined as the target coverage area. Figure 4 Compared with the coverage area of ​​the minimum enclosing rectangle, the minimum enclosing polygon fits the coverage formed by the detection box better than the minimum enclosing rectangle.

[0073] In some embodiments of the present invention, the rotation target detection model may be an improved PP-YOLOE-R model. In some examples, the Pytorch framework may be used to perform offline training of the PP-YOLOE-R model, and the model may be used for target detection after training.

[0074] When measuring the coverage area from a bird's-eye view, the YOLO (You Only Look Once) target detection algorithm is usually used. The approximate coverage area is simply calculated by the size of the detection frame of the detection result. However, the annotation frames of the YOLO target detection algorithm are all horizontal. Objects that are placed diagonally or deformed in the photo cannot be accurately framed with a horizontal annotation frame, and will contain more irrelevant background information, which will affect the location information of the warehouse and the subsequent calculation of the coverage area, that is, there will be certain errors and insufficient precision. Even if a rotation frame angle calculation branch is simply added to the Head part based on the YOLO overall framework, the rotation frame angle calculation branch only roughly calculates the corresponding angle change and still cannot better fit the edge of the object.

[0075] In an embodiment of the present invention, the PP-YOLOE-R rotating target detection model is adopted and the feature refinement module of R3Det is integrated. The improved PP-YOLOE-R uses the feature refinement module to re-encode the current bounding box information and reconstructs the entire feature map to achieve feature alignment, which can better solve the above problems. Compared with PP-YOLOE-R, the rotating target box can better frame the storage location, and the detection box contains less background information, so that the positioning is more accurate.

[0076] In some embodiments of the present invention, a rotating target detection model performs target detection on image data through a rotating detection frame, and a head portion of the rotating target detection model has a branch for predicting the coordinate value of the rotating detection frame, a branch for predicting the angle of the rotating detection frame, and a branch for classification.

[0077] Among them, the branch for predicting the coordinate value of the rotation detection frame and the branch for predicting the angle of the rotation detection frame are two decoupled branches.

[0078] The detection head part of PP-YOLOE-R has three branches in total, one is the classification branch, one is the branch for detecting the coordinate values ​​of the rotation detection frame (such as the coordinates of the four vertices), and one is the branch for predicting the angle of the rotation detection frame. Through the decoupling head technology, the original five values ​​of one branch of the detection frame (four coordinate values ​​+ one angle prediction value) are decoupled into two branches, so that the angle change can be calculated better separately. The specific network structure diagram of PP-YOLOE-R is as follows: Figure 5 .

[0079] RepRes-Block combines Residual Connections and Dense Connections for the trunk and neck of YOLOE. However, it simplifies the original Block (such as Figure 6a), use element-wise Add operation to replace the connection operation (such as Figure 6c ), these two operations are similar to RMNet to some extent. Therefore, in the inference phase, it can be reparameterized as RepResBlock (such as Figure 6b ). By using RepResBlock to build a network similar to ResNet (Residual Network, deep residual network), called CSPRepResNet (such as Figure 6d , ESE attention module).

[0080] In some embodiments of the present invention, although PP-YOLOE-R improves the detection performance of the rotated bounding box at the cost of adding relatively few parameters and calculations based on PP-YOLOE. However, the decoupling head may make the classification and positioning tasks independent, lacking task-specific learning, and many rotated object detectors still use the same feature map to perform multiple classifications and regressions without considering the feature misalignment caused by the change in the position of the bounding box. The rotated object detection model reconstructs the entire feature map by setting a feature refinement module (Feature Refinement Module, FRM) to achieve feature alignment, and can also reduce the number of refined bounding boxes after the first stage, thereby speeding up the model, such as Figure 7 .

[0081] In some embodiments of the present invention, the feature refinement module reconstructs the entire feature map in the following manner: for a feature point in an input feature map, a new feature vector is obtained based on the coordinate value of a rotation detection box corresponding to the feature point, and the new feature vector is used to replace the previous feature vector of the feature point; the feature points in the new feature map are traversed to replace the feature vectors to obtain a reconstructed feature map.

[0082] In some embodiments of the present invention, before obtaining a new feature vector based on the coordinate values ​​of the rotation detection box corresponding to the feature point in the input feature map, it also includes: superimposing the input feature map through two-way convolution to obtain a new feature map.

[0083] In some embodiments of the present invention, before replacing the previous feature vector of the feature point with the new feature vector, the method further includes: updating the new feature vector by bilinear interpolation.

[0084] Specifically, the feature refinement module can adopt the following operations: superimpose feature maps through two-way convolution to obtain a new feature map. In the refinement stage, only the bounding box with the highest score for each feature point is retained to improve the speed, while also ensuring that each feature point only corresponds to one refined bounding box. For each feature point in the feature map, the corresponding feature vector on the feature map is obtained according to the five coordinates of the refined bounding box (one center point and four corner points). A more accurate feature vector can be obtained through bilinear interpolation. Next, add 5 feature vectors to replace the previous feature vector. After traversing the feature points, the entire feature map will be reconstructed. Finally, the reconstructed feature map is added to the original feature map to complete the whole process.

[0085] By replacing the original Head part with FRM in the Head part of PP-YOLOE-R, the five coordinate parameters of the rotated target frame are refined, the accuracy of the rotated target frame fitting the GT frame is improved, and the fitting of the subsequent reconstructed feature map to the original feature map is enhanced. Figure 8 .

[0086] In some embodiments of the present invention, the rotation target detection model adopts a loss function combining Inner-IoU and SIoU during model training.

[0087] R3Det uses the SkewIOU loss function, which is very sensitive to angles. Even a slight offset will cause SkewIOU to drop rapidly. If a slight jitter of the predicted detection box occurs during training, SkewIOU will get inappropriate results, causing the network to continue to oscillate and fail to converge. Using the SIoU loss function calculation method instead of the SkewIOU loss function can effectively solve problems such as slow model convergence due to angle parameter offset. The SIoU loss function considers the matching direction, which greatly helps the training convergence process and effect. Inner-IoU uses the scale factor ratio to control the generation of auxiliary boxes of different scales for calculating losses. Combining it with SIoU can obtain faster and more effective regression results.

[0088] In practical applications, since there is only a scale difference between the auxiliary border and the actual border, the change trend of its IoU value during the regression process is consistent with the change trend of the IoU value of the actual border, which can reflect the quality of the actual border regression result. For high IoU samples, the absolute value of the IoU gradient of the smaller-scale auxiliary border is greater than the absolute value of the IoU gradient of the actual border. For low IoU samples, the absolute value of the IoU gradient of the larger-scale auxiliary border is greater than the absolute value of the IoU gradient of the actual border.

[0089] Based on the above analysis, using a smaller scale auxiliary bounding box to calculate the IoU loss will help the regression of high IoU samples and achieve the effect of accelerating convergence. On the contrary, using a larger scale auxiliary bounding box to calculate the IoU loss can accelerate the regression process of low IoU samples.

[0090] like Fig. 9 , Inner-IoU, GT box and anchor box are represented as B gt and B, the center point of the GT box and the interior of the GT box is Indicates that the center point of the anchor box and the internal anchor box is (x c ,y c ) represents the width and height of the GT box. gt and h gt , the width and height of the anchor box are w and h respectively. ratio corresponds to the scale factor, which is usually in the range of [0.5,1.5].

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0097]

[0098] Inner-IoU loss inherits some characteristics of IoU and has its own characteristics. Compared with IoU loss, when the ratio is less than 1, the auxiliary box size is smaller than the actual box, and its effective range of regression is smaller than IoU loss, but the gradient obtained by the absolute value of the gradient is greater than IoU loss, which can accelerate the convergence of high IoU samples. On the contrary, when the ratio is greater than 1, the larger-scale auxiliary box expands the effective range of regression, which has some benefits for the regression of low Iou.

[0099] Using the SIoU loss function calculation method instead of the SkewIOU loss function can effectively solve the problem of slow model convergence caused by angle parameter offset. The SIoU loss function considers the matching direction, which greatly helps the training convergence process and effect. In order to achieve better performance of rotating target detection, Inner-IoU is combined with SIoU to form the final loss function L Inner-SIoU .

[0100] L Inner-SIoU =L SIoU +IoU-IoU inner

[0101] In an embodiment of the present invention, the rotating target detection model can be an improved PP-YOLOE-R model, which can effectively identify targets that are not horizontally placed and have a smaller resolution, with high recognition accuracy and strong robustness, laying a solid foundation for subsequent calculation of the coverage area of ​​the identifiable area.

[0102] On the one hand, the Head part feature refinement module of R3Det is used to replace the original Head part of PP-YOLOE-R. The 5 coordinate parameters of the rotated target frame are refined through the feature refinement module to improve the accuracy of the rotated target frame fitting the GT frame. On the other hand, in order to achieve better performance of rotated target detection, Inner-IoU and SIoU are combined to form the loss function L Inner-SIoU The loss function calculation method replaces the original IoU loss function. Inner-IoU uses the scale factor ratio to control the generation of auxiliary frames of different scales for calculating the loss. The SIoU loss function considers the matching direction, which can effectively solve the problem of slow model convergence caused by angle parameter offset. Inner-SIoU Combining the two IoU loss functions greatly helps the training convergence process and results.

[0103] Step 203: performing image correction on the reference coverage area in the image data.

[0104] In some embodiments of the present invention, the image correction of the reference coverage area in the image data includes: obtaining a perspective transformation matrix, and using the perspective transformation matrix to correct the positions of pixel points in the reference coverage area in the image data.

[0105] In some embodiments of the present invention, it also includes:

[0106] Sample data is acquired, and a perspective transformation matrix is ​​determined according to coordinates of a plurality of pixel points in the sample data before image correction and after image correction.

[0107] In practical applications, since the image data is not taken by the camera from a top-down perspective, but from an oblique angle, the object in the obtained image data has a certain degree of distortion. The image can be corrected by image perspective transformation, such as Fig.10 Schematic diagram of image before and after correction.

[0108] The essence of perspective transformation is to project the image onto a new viewing plane. The perspective transformation matrix can be used to transform the positions of the pixels in the reference coverage area of ​​the original image data into new positions, as shown in the following formula:

[0109]

[0110] Among them, (x, y, z) is the homogeneous coordinates of the pixel points in the original image data, (X', Y', Z') is the homogeneous coordinates of the pixel points in the image data after perspective transformation correction, and a 11 -a 33 The perspective transformation matrix is ​​composed of The above formula can be transformed into:

[0111]

[0112] Let a 33 =1, then the above formula can be transformed to:

[0113]

[0114] For the perspective transformation matrix, sample data (which is also image data) can be used to determine the coordinates of multiple pixel points (such as selecting four vertices) in the sample data before and after image correction to form multiple pairs of coordinates, and then the perspective transformation matrix is ​​obtained by calculation.

[0115] In some examples, multiple pixel points can be 4 vertices, but in real application scenarios, the calculated point pairs may contain noise or feature point pairs may be mismatched. If only 4 pairs of coordinate points are used to calculate the perspective transformation matrix, a large error will occur. In order to make the calculation more accurate, generally far more than 4 pairs of coordinate points are used to calculate the perspective transformation matrix.

[0116] Step 204, determining the size ratio of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area.

[0117] In practical applications, a reference coverage area can be set. The reference coverage area can be pre-marked, such as setting the actual site range of 6m*10m in the image data as the reference coverage area. The reference coverage area is a larger area that includes the target coverage area, such as Figure 4, the reference area must be larger than the area of ​​the minimum circumscribed polygon, so that the size ratio of the reference coverage area occupied by each pixel in the image data can be determined for the subsequent calculation of the definite integral.

[0118] In some embodiments of the present invention, determining the size ratio relationship of the reference coverage area occupied by each pixel in the image data includes: determining the coverage area of ​​the reference coverage area in the image data; and determining the size ratio relationship of the reference coverage area occupied by each pixel in the image data based on the coverage area of ​​the reference coverage area and the size information of the image data.

[0119] In some embodiments of the present invention, determining the coverage area of ​​the reference coverage area in the image data includes: determining relevant attribute information of the camera and size information of the image data; and determining the coverage area of ​​the reference coverage area in the image data based on the relevant attribute information of the camera and the size information of the image data.

[0120] As an example, the relevant attribute information of the camera includes the height of the camera and the focal length of the camera, and the size information of the image data includes the width of the image data and the height of the image data.

[0121] In practical applications, the height H' and focal length f of the camera can be obtained, and the width w and height h of the image data can be obtained, such as Fig.11 According to the similar triangle method, the width W and length H of the reference coverage area can be determined and calculated using the following formula:

[0122]

[0123]

[0124] After obtaining the width and length H of the reference coverage area, the coverage area of ​​the reference coverage area can be determined to be W*H. Since the image data size information includes width w and height h, the entire image data includes w*h pixels. Then, the size ratio of each pixel in the image data to the reference coverage area can be calculated using the following formula: 2 / pixel):

[0125]

[0126] Step 205: performing definite integral calculation on the target coverage area in combination with the size ratio relationship to obtain the coverage area of ​​the target coverage area.

[0127] Since the target coverage area is the minimum circumscribed polygon, in order to calculate the actual coverage area of ​​the minimum circumscribed polygon, such as Fig.12, the coverage area can be calculated by definite integral. The polygon length is determined by the function F2(x)-F1(x) at a fixed value x (x is the real-time width of the minimum circumscribed polygon), and the F2(x) function can be considered to be composed of multiple piecewise functions, as is the F1(x) function. The width is determined by the upper and lower limits x2 and x1 of the definite integral (i.e., the upper and lower limits of the width of the minimum circumscribed polygon), as shown in the following formula:

[0128]

[0129] According to the previously calculated size ratio relationship and the above definite integral calculation formula, the coverage area of ​​the target coverage area (i.e., the minimum circumscribed polygon) can be calculated as S, as follows:

[0130]

[0131] In an embodiment of the present invention, image data collected by a camera is acquired, target detection is performed on the image data by rotating the target detection model, and the target coverage area in the image data is determined based on the target detection result, image correction is performed on the reference coverage area in the image data, and then the size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined, the reference coverage area includes the target coverage area, and the definite integral calculation is performed on the target coverage area in combination with the size ratio relationship to obtain the coverage area of ​​the target coverage area, thereby achieving optimization of the coverage area measurement, being able to adapt to the oblique shooting angle of the camera, and improving the accuracy of the coverage area measurement.

[0132] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0133] Some embodiments of the present invention provide a device for measuring coverage area, the device being used for:

[0134] Get the image data collected by the camera;

[0135] Performing target detection on the image data by rotating the target detection model, and determining a target coverage area in the image data according to the target detection result;

[0136] performing image correction on a reference coverage area in the image data;

[0137] Determine the size ratio relationship of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area;

[0138] In combination with the size ratio relationship, a definite integral calculation is performed on the target coverage area to obtain the coverage area of ​​the target coverage area.

[0139] In some embodiments of the present invention, determining the target coverage area in the image data according to the target detection result includes:

[0140] According to the positions of all the rotation detection frames in the target detection result, a minimum circumscribed polygon is determined as the target coverage area in the image data.

[0141] In some embodiments of the present invention, determining the size ratio relationship of the reference coverage area occupied by each pixel in the image data includes:

[0142] determining a coverage area of ​​a reference coverage region in the image data;

[0143] The size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined according to the coverage area of ​​the reference coverage area and the size information of the image data.

[0144] In some embodiments of the present invention, determining the coverage area of ​​the reference coverage area in the image data includes:

[0145] Determine relevant attribute information of the camera and size information of the image data;

[0146] The coverage area of ​​the reference coverage area in the image data is determined according to the relevant attribute information of the camera and the size information of the image data.

[0147] In some embodiments of the present invention, the relevant attribute information of the camera includes the height of the camera and the focal length of the camera, and the size information of the image data includes the width of the image data and the height of the image data.

[0148] In some embodiments of the present invention, performing image correction on the reference coverage area in the image data includes:

[0149] A perspective transformation matrix is ​​obtained, and the perspective transformation matrix is ​​used to correct the positions of the pixels in the reference coverage area in the image data.

[0150] In some embodiments of the present invention, it is also used for:

[0151] Sample data is acquired, and a perspective transformation matrix is ​​determined according to coordinates of a plurality of pixel points in the sample data before image correction and after image correction.

[0152] In some embodiments of the present invention, the image data is image data captured by the camera using an oblique viewing angle.

[0153] In some embodiments of the present invention, the rotating object detection model performs object detection on the image data by rotating the detection frame;

[0154] The head part of the rotating target detection model has a branch for predicting the coordinate value of the rotating detection frame and a branch for predicting the angle of the rotating detection frame. The branch for predicting the coordinate value of the rotating detection frame and the branch for predicting the angle of the rotating detection frame are two decoupled branches.

[0155] In some embodiments of the present invention, the head portion of the rotating object detection model further has a branch for classification.

[0156] In some embodiments of the present invention, the rotation target detection model reconstructs the entire feature map by setting a feature refinement module to perform feature alignment.

[0157] In some embodiments of the present invention, the feature refinement module reconstructs the entire feature map in the following manner:

[0158] For a feature point in the input feature map, a new feature vector is obtained according to the coordinate value of the rotation detection box corresponding to the feature point, and the new feature vector is used to replace the previous feature vector of the feature point;

[0159] The feature points in the new feature map are traversed to replace the feature vectors to obtain a reconstructed feature map.

[0160] In some embodiments of the present invention, before replacing the previous feature vector of the feature point with the new feature vector, the method further includes:

[0161] The new feature vector is updated through bilinear interpolation.

[0162] In some embodiments of the present invention, before obtaining a new feature vector based on the coordinate value of the rotation detection box corresponding to the feature point in the input feature map, the method further includes:

[0163] Through two-way convolution, the input feature map is superimposed to obtain a new feature map.

[0164] In some embodiments of the present invention, the rotation target detection model adopts a loss function combining Inner-IoU and SIoU during model training.

[0165] In an embodiment of the present invention, image data collected by a camera is acquired, target detection is performed on the image data by rotating the target detection model, and the target coverage area in the image data is determined based on the target detection result, image correction is performed on the reference coverage area in the image data, and then the size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined, the reference coverage area includes the target coverage area, and the definite integral calculation is performed on the target coverage area in combination with the size ratio relationship to obtain the coverage area of ​​the target coverage area, thereby achieving optimization of the coverage area measurement, being able to adapt to the oblique shooting angle of the camera, and improving the accuracy of the coverage area measurement.

[0166] Some embodiments of the present invention further provide an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the above coverage area measurement method is implemented.

[0167] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above coverage area measurement method is implemented.

[0168] Some embodiments of the present invention further provide a computer program product, including a computer program, wherein the computer program implements the above coverage area measurement method when executed by a processor.

[0169] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0171] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0172] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0173] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0177] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0178] The above provides a detailed introduction to the provided coverage area measurement method, device, electronic device, medium and program product. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for measuring coverage area, characterized in that: The method comprises: Get the image data collected by the camera; Performing target detection on the image data by rotating the target detection model, and determining a target coverage area in the image data according to the target detection result; performing image correction on a reference coverage area in the image data; Determine the size ratio relationship of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area; In combination with the size ratio relationship, a definite integral calculation is performed on the target coverage area to obtain the coverage area of ​​the target coverage area.

2. The method according to claim 1, characterized in that: The step of determining the target coverage area in the image data according to the target detection result includes: According to the positions of all the rotation detection frames in the target detection result, a minimum circumscribed polygon is determined as the target coverage area in the image data.

3. The method according to claim 1, characterized in that The determining the size ratio relationship of the reference coverage area occupied by each pixel in the image data includes: determining a coverage area of ​​a reference coverage region in the image data; The size ratio relationship of the reference coverage area occupied by each pixel in the image data is determined according to the coverage area of ​​the reference coverage area and the size information of the image data.

4. The method according to claim 3, characterized in that The determining the coverage area of ​​the reference coverage area in the image data comprises: Determine relevant attribute information of the camera and size information of the image data; The coverage area of ​​the reference coverage area in the image data is determined according to the relevant attribute information of the camera and the size information of the image data.

5. The method according to claim 4, characterized in that The relevant attribute information of the camera includes the height of the camera and the focal length of the camera, and the size information of the image data includes the width of the image data and the height of the image data.

6. The method according to any one of claims 1 to 5, characterized in that: The performing image correction on the reference coverage area in the image data comprises: A perspective transformation matrix is ​​obtained, and the perspective transformation matrix is ​​used to correct the positions of the pixels in the reference coverage area in the image data.

7. The method according to claim 6, characterized in that Also includes: Sample data is acquired, and a perspective transformation matrix is ​​determined according to coordinates of a plurality of pixel points in the sample data before image correction and after image correction.

8. The method according to claim 1, characterized in that The image data is image data captured by the camera using an oblique viewing angle.

9. The method according to claim 1, characterized in that: The rotating target detection model performs target detection on the image data by rotating the detection frame; The head part of the rotating target detection model has a branch for predicting the coordinate value of the rotating detection frame and a branch for predicting the angle of the rotating detection frame. The branch for predicting the coordinate value of the rotating detection frame and the branch for predicting the angle of the rotating detection frame are two decoupled branches.

10. The method according to claim 9, characterized in that The head part of the rotation object detection model also has a branch for classification.

11. The method according to claim 1, characterized in that The rotation object detection model reconstructs the entire feature map by setting a feature refinement module to perform feature alignment.

12. The method according to claim 11, characterized in that The feature refinement module reconstructs the entire feature map in the following way: For a feature point in the input feature map, a new feature vector is obtained according to the coordinate value of the rotation detection box corresponding to the feature point, and the new feature vector is used to replace the previous feature vector of the feature point; The feature points in the new feature map are traversed to replace the feature vectors to obtain a reconstructed feature map.

13. The method according to claim 12, characterized in that Before the new feature vector is used to replace the previous feature vector of the feature point, the method further includes: The new feature vector is updated through bilinear interpolation.

14. The method according to claim 12, characterized in that Before obtaining a new feature vector based on the coordinate value of the rotation detection box corresponding to the feature point in the input feature map, the method further includes: Through two-way convolution, the input feature map is superimposed to obtain a new feature map.

15. The method according to claim 1, characterized in that The rotation target detection model adopts a loss function combining Inner-IoU (Intersection over Union) and SIoU during model training.

16. A device for measuring coverage area, characterized in that: The device is used for: Get the image data collected by the camera; Performing target detection on the image data by rotating the target detection model, and determining a target coverage area in the image data according to the target detection result; performing image correction on a reference coverage area in the image data; Determine the size ratio relationship of each pixel in the image data to the reference coverage area; wherein the reference coverage area includes the target coverage area; In combination with the size ratio relationship, a definite integral calculation is performed on the target coverage area to obtain the coverage area of ​​the target coverage area.

17. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for measuring the coverage area according to any one of claims 1 to 15 is implemented.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for measuring a coverage area according to any one of claims 1 to 15 is implemented.

19. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method for measuring the coverage area according to any one of claims 1 to 15.

Images