Area detection method and related device, equipment and medium

Abstract

The present application discloses a region detection method and its related apparatus, device and medium, the method comprising: obtaining an image to be detected; performing region detection on the image to be detected using a region detection model to obtain a target prediction frame for a specific text region in the image to be detected; wherein the region detection model is obtained by optimizing the evaluation index of the region detection model, the evaluation index is obtained based on the boundary tightness between the true value frame and the prediction frame of the specific text region in the verification sample image, and the prediction frame is obtained by performing region detection on the verification sample image by the region detection model. In the above manner, the present application can improve the accuracy of the target prediction frame.

Description

Technical Field

[0001] The present application relates to the technical field of text recognition, and in particular to a region detection method and related devices, equipment and media. Background Art

[0002] Scene text recognition is to recognize text areas detected in natural scenes and translate them into reasonable computer text. It is an important topic in the field of computer vision. In recent years, scene text recognition technology has been continuously developed and innovated, and is widely used in the fields of intelligent transportation, unmanned driving and robots. The most common application is the field of non-standard license plate recognition technology. Among them, the accuracy of text area detection will directly affect the accuracy of scene text recognition. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a region detection method and related devices, equipment and media, which can improve the accuracy of the target prediction frame.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a region detection method, the method comprising: obtaining an image to be detected; performing region detection on the image to be detected using a region detection model, and obtaining a target prediction box for a specific text area in the image to be detected; wherein the region detection model is optimized based on an evaluation index of the region detection model, the evaluation index is obtained based on the boundary tightness between the true value box and the prediction box of the specific text area in the verification sample image, and the prediction box is obtained by performing region detection on the verification sample image by the region detection model.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: to provide a region detection device, which includes: an acquisition module, used to acquire an image to be detected; a detection module, used to perform region detection on the image to be detected using a region detection model, and obtain a target prediction box for a specific text area in the image to be detected; wherein the region detection model is optimized based on an evaluation index of the region detection model, and the evaluation index is obtained based on the boundary tightness between the true value box and the prediction box of the specific text area in the verification sample image, and the prediction box is obtained by performing region detection on the verification sample image by the region detection model.

[0006] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein program instructions are stored in the memory, and the processor is used to execute the program instructions to implement the above area detection method.

[0007] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium, storing program instructions that can be executed by a processor, and the program instructions are used to implement the above area detection method.

[0008] The above scheme uses the region detection model optimized based on the evaluation index of the region detection model to perform region detection on the image to be detected, and obtains the target prediction box about the specific text region in the image to be detected. Therefore, since the region detection model is optimized based on the evaluation index of the region detection model, and the evaluation index is obtained based on the boundary tightness between the true value frame and the prediction frame, the factor of the boundary tightness of the prediction frame relative to the true value frame will be taken into account during the optimization of the region detection model based on the evaluation index, and the position of each boundary of the prediction frame will be reasonably controlled, so that each boundary of the prediction frame obtained by the optimized region detection model is close to the boundary of the corresponding true value frame. Therefore, the target prediction box about the specific text region obtained by performing region detection on the image to be detected using the optimized region detection model is closer to the corresponding true value frame, that is, the target prediction box is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flow chart of an embodiment of a region detection method provided by the present application;

[0010] Figure 2 It is a flowchart of an embodiment of a training method for a region detection model provided by the present application;

[0011] Figure 3 It is a schematic diagram of an embodiment of a non-standard license plate provided by the present application;

[0012] Figure 4 is a schematic diagram of another embodiment of the non-standard license plate provided by the present application;

[0013] Figure 5 It is a flowchart of an embodiment of a training method for a text recognition model provided by the present application;

[0014] Figure 6 It is a flowchart of an embodiment of a testing method for a text recognition model provided by the present application;

[0015] Figure 7 yes Figure 2 The flowchart of step S23 is shown as an embodiment;

[0016] Figure 8 yes Figure 7 The flowchart of step S231 of an embodiment is shown;

[0017] Fig. 9It is a schematic diagram of an embodiment of a truth frame and a prediction frame provided by the present application;

[0018] Fig.10 yes Figure 8 The flowchart of step S2312 is shown as an embodiment;

[0019] Fig.11 yes Fig.10 The flowchart of step S102 of an embodiment is shown;

[0020] Fig.12 yes Figure 7 The flowchart of step S232 is shown as an embodiment;

[0021] Fig.13 yes Fig.12 The flowchart of step S2321 of an embodiment is shown;

[0022] Fig.14 It is a schematic diagram of a framework of an embodiment of a training device for a region detection model provided by the present application;

[0023] Fig.15 It is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application;

[0024] Fig.16 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0025] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0026] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0027] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0028] See also Figure 1 , Figure 1 is a flow chart of an embodiment of the area detection method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0029] Step S11: Acquire the image to be detected.

[0030] The method of this embodiment is used to detect the image to be detected and obtain the target detection frame about the specific text area in the image to be detected. The image to be detected described in this article may be, but is not limited to, multi-eye stitching images, stitched panoramic images, high-resolution images with a resolution exceeding 4K, etc. Among them, the image to be detected can be obtained from local storage or cloud storage, or collected by electronic equipment in the current environment, which is not specifically limited here.

[0031] Step S12: Perform region detection on the image to be detected using the region detection model to obtain a target prediction box for a specific text region in the image to be detected.

[0032] In this embodiment, the region detection model is used to perform region detection on the image to be detected, and the target prediction frame about the specific text region in the image to be detected is obtained. The region detection model is optimized based on the evaluation index of the region detection model, and the evaluation index is obtained based on the boundary tightness between the true value frame and the prediction frame of the specific text region in the verification sample image, and the prediction frame is obtained by the region detection model performing region detection on the verification sample image. Since the region detection model is optimized based on the evaluation index of the region detection model, and the evaluation index is obtained based on the boundary tightness between the true value frame and the prediction frame, the factor of the boundary tightness of the prediction frame relative to the true value frame is taken into account during the optimization of the region detection model based on the evaluation index, and the position of each boundary of the prediction frame is reasonably controlled, so that each boundary of the prediction frame obtained by the optimized region detection model is close to the boundary of the corresponding true value frame. Therefore, the target prediction frame about the specific text region obtained by performing region detection on the image to be detected using the optimized region detection model is closer to the corresponding true value frame, that is, the target prediction frame is more accurate.

[0033] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of a training method for a region detection model provided in the present application. It should be noted that if there are substantially the same results, this embodiment does not necessarily refer to the training method of the region detection model provided in the present application. Figure 2 The process sequence shown is limited. Figure 2 As shown in the figure, before using the region detection model to perform region detection on the image to be detected, the region detection model needs to be optimized and trained, including:

[0034] Step S21: Obtain verification sample images from the verification set.

[0035] The method of this embodiment is used to optimize the region detection model using the verification sample images in the verification set. The verification sample images described herein may be, but are not limited to, multi-eye stitched images, stitched panoramic images, high-resolution images with a resolution exceeding 4K, etc. Among them, each verification sample image constitutes a verification set, and the size, quantity, etc. of the verification sample images in the verification set are not limited, and can be specifically set according to actual use needs; the verification sample images can be obtained from local storage or cloud storage, or collected by electronic devices in the current environment, which is not specifically limited here.

[0036] Among them, the verification sample image contains a specific text area and is marked with a true value box of the specific text area, so that the parameters of the region detection model can be adjusted based on each verification sample image in the verification set, thereby optimizing the region detection model. That is, the method of training the region detection model based on the verification set in this application can be regarded as a method for optimizing the region detection model that has been initially trained using the training set. In addition, each verification sample image in the verification set is different from each training sample image in the training set for the initial training of the region detection model. Therefore, after the region detection model is subsequently optimized based on each verification sample image, the generalization ability of the region detection model can be improved, and the region detection accuracy of the region detection model can also be verified during the optimization process.

[0037] See also Figure 3 , Figure 3 is a schematic diagram of an embodiment of a non-standard license plate provided by the present application. In one implementation, the specific text area is a non-standard license plate. Figure 3 As shown, in a certain monitoring scene, the vehicle's license plate may be blocked or damaged, so the vehicle's license plate cannot be recognized all the time. Therefore, if the vehicle's identity needs to be confirmed at this time, other information is needed; compared with other information such as vehicle color and vehicle brand, the non-standard license plate has the same content as the vehicle's rear license plate, and the size of the non-standard license plate is larger than that of the rear license plate. The non-standard license plate is easier to identify, so the vehicle's identity can be confirmed more quickly through the non-standard license plate. It can be understood that in other embodiments, the specific text area can also be other areas that require text recognition, which can be set according to actual use needs and are not specifically limited here.

[0038] In one embodiment, the truth box of the specific text area marked in the verification sample image can be manually marked, that is, the user manually marks the truth box of each verification sample image in the verification set one by one. It can be understood that in other embodiments, the network algorithm can also be used to mark the truth box of each verification sample image in the verification set, wherein the network algorithm is not limited and can be specifically set according to actual use needs.

[0039] In one embodiment, before obtaining the verification sample image from the verification set, it is necessary to train the region detection model using the training set to obtain an initial region detection model trained with the training set. The training set includes a number of training sample images, the training sample images contain specific text areas, and are marked with truth boxes of the specific text areas. The size and number of the training sample images in the training set are not limited, and can be specifically set according to actual use needs. It should be noted that since the verification set is used to further optimize the initial region detection model obtained after training with the training set, the specific text area contained in the training sample images in the training set is the same as the specific text area contained in the verification sample images in the verification set. For example, the specific text area contained in the verification sample image is a non-standard license plate, and the specific text area contained in the training sample image is also a non-standard license plate.

[0040] Step S22: Perform region detection on the verification sample image using the region detection model to obtain a prediction box for a specific text region in the verification sample image.

[0041] In this embodiment, the region detection model is used to perform region detection on the verification sample image to obtain a prediction box for a specific text region in the sample image. Figure 4 As shown, Figure 4 is a schematic diagram of another embodiment of the non-standard license plate provided by the present application. Taking the verification that the specific text area contained in the sample image is a non-standard license plate as an example, Figure 4 The data is input into the regional detection model, which performs regional detection on it and obtains the prediction box for the non-standard license plate.

[0042] Step S23: Based on the boundary tightness between the true value box and the predicted box, an evaluation index of the region detection model is obtained.

[0043] like Figure 4As shown in FIG. 1 , taking the verification sample image containing a specific text area as a non-standard license plate as an example, after the verification sample image is subjected to area detection using the area detection model, the prediction box about the specific text area in the verification sample image is obtained as prediction box 1, and prediction box 1 is slightly larger than the true value box of the non-standard license plate, that is, the upper and lower boundaries of prediction box 1 are far away from the upper and lower boundaries of the true value box, and the left and right boundaries of prediction box 1 are far away from the left and right boundaries of the true value box. In other words, the upper and lower boundaries of prediction box 1 are not close enough to the upper and lower boundaries of the true value box, and the left and right boundaries of prediction box 1 are not close enough to the left and right boundaries of the true value box, so that prediction box 1 may contain other text content besides the license plate number, thereby affecting the subsequent The recognition of the text content in the non-standard license plate is continued, reducing the recognition accuracy; and when the prediction box for the specific text area in the obtained verification sample image is prediction box 2, prediction box 2 is slightly smaller than the true value box of the non-standard license plate, that is, the upper and lower boundaries of prediction box 2 are far from the upper and lower boundaries of the true value box, and the left and right boundaries of prediction box 2 are far from the left and right boundaries of the true value box, that is, the upper and lower boundaries of prediction box 2 are not close enough to the upper and lower boundaries of the true value box, and the left and right boundaries of prediction box 2 are not close enough to the left and right boundaries of the true value box, so that prediction box 2 cannot contain the complete text content in the non-standard license plate, thereby affecting the subsequent recognition of the text content in the non-standard license plate and reducing the recognition accuracy. Therefore, the region detection model is used to perform region detection on the verification sample image, and the boundary tightness between the prediction box and the true value box for the specific text area in the obtained verification sample image will affect the accuracy of text content recognition in the specific text area.

[0044] Therefore, it is necessary to evaluate the boundary tightness between the prediction box and the true value box of a specific text area obtained by the region detection model, that is, it is necessary to evaluate the region detection capability of the region detection model, so that the parameters of the region detection model can be adjusted based on the evaluation index, and the region detection model can be optimized, thereby improving the accuracy of subsequent text recognition in a specific text area. In the existing methods, most of them directly use the traditional ICDAR15 measurement method to evaluate the position of the prediction box, without considering that the boundary tightness between the prediction box and the true value box will affect the accuracy of subsequent text content recognition in a specific text area.

[0045] In the present embodiment, an evaluation index of the region detection model is obtained according to the tightness of the boundary between the true value box of the specific text region and the prediction box of the specific text region, that is, the evaluation index of the region detection model takes into account the factor of the tightness of the boundary of the prediction box of the specific text region relative to the true value box, so that the boundaries of the prediction box of the specific text region obtained by the region detection model optimized based on this factor are close to the boundaries of the corresponding true value box, thereby improving the recognition accuracy of the text content in the specific text region.

[0046] Step S24: Optimize the region detection model based on the evaluation index.

[0047] In this embodiment, the region detection model is optimized according to the evaluation index. Specifically, the evaluation index value of the region detection model is calculated to evaluate the region detection accuracy of the region detection model, and the parameters of the region detection model are adjusted according to the evaluation result to optimize the region detection model, thereby improving the region detection accuracy of the region detection model. Since the evaluation index is obtained based on the boundary tightness between the true value box and the prediction box, the factor of the boundary tightness of the prediction box relative to the true value box is taken into account in the process of optimizing the region detection model based on the evaluation index, and the position of each boundary of the prediction box is reasonably controlled, so that each boundary of the prediction box obtained by the optimized region detection model is close to the boundary of the corresponding true value box, so that the obtained prediction box is more suitable for the text recognition model, and a more accurate recognition effect can be obtained in the same text recognition model.

[0048] In the above implementation, the evaluation index of the region detection model is obtained based on the boundary tightness between the true value box and the prediction box of the specific text region, and the region detection model is optimized based on the evaluation index. Since the evaluation index of the region detection model is determined based on the boundary tightness between the true value box and the prediction box, the factor of the boundary tightness of the prediction box relative to the true value box is taken into account in the process of optimizing the region detection model based on the evaluation index, so as to reasonably control and adjust the position of the prediction box boundary relative to the true value box boundary, so that each boundary of the prediction box obtained by the optimized region detection model is close to the boundary of the corresponding true value box, thereby improving the region detection accuracy of the region detection model.

[0049] In one embodiment, if Figure 5 As shown, Figure 5 1 is a flow chart of an embodiment of a training method for a text recognition model provided in the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 5 The process sequence shown is limited. Figure 5 As shown in the figure, after optimizing the region detection model, the text recognition model needs to be trained, including:

[0050] Step S51: Obtain training sample images in a training set.

[0051] The method of this embodiment is used to train the text recognition model using the training sample images in the training set. The training sample images described herein may be, but are not limited to, multi-eye stitched images, stitched panoramic images, high-resolution images with a resolution exceeding 4K, etc. Among them, each training sample image constitutes a training set, and the size, number, etc. of the training sample images in the training set are not limited, and can be specifically set according to actual use needs; the training sample images can be obtained from local storage or cloud storage, or collected by electronic devices in the current environment, which is not specifically limited here.

[0052] Among them, the training sample image contains a specific text area and is marked with a true value box of the specific text area and the first true value text content in the specific text area, so that the parameters of the text recognition model can be adjusted based on each training sample image in the training set, thereby realizing the training of the text recognition model.

[0053] Step S52: Based on the true value frame, crop the image portion of the specific text area from the training sample image to obtain a first area image.

[0054] In this embodiment, according to the truth box of the specific text area marked on the training sample image, the image portion of the specific text area is clipped from the training sample image to obtain the first area image. In other words, only the image portion corresponding to the specific text area is used for the subsequent text recognition model training, so as to avoid the text content in other areas outside the specific text area from interfering with the training of the text recognition model, thereby improving the text recognition accuracy of the text recognition model.

[0055] In order to improve the recognition accuracy of the subsequent text recognition model and the training efficiency of the text recognition model, in one embodiment, after the first region image is obtained, the first region image may be corrected.

[0056] Step S53: training a text recognition model using the first region image and the true text content.

[0057] In this embodiment, the text recognition model is trained using the first region image and the true value text content. Specifically, taking the training of the text recognition model using a certain training sample image as an example, the text recognition model is used to perform text recognition on the first region image to obtain the predicted text content, and a loss function such as cross entropy is used to obtain a loss value based on the annotated true value text content. Further, the network parameters of the text recognition model can be adjusted based on the loss value, so that the text recognition model can recognize the text content in the first region image more and more accurately during multiple rounds of training.

[0058] In one embodiment, if Figure 6 As shown, Figure 6 1 is a flow chart of an embodiment of a test method for a text recognition model provided in the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 6 The process sequence shown is limited. Figure 6 As shown, after obtaining the trained text recognition model and the optimized region detection model, the recognition accuracy of the trained text recognition model is determined based on the optimized region detection model and the test set, specifically including:

[0059] Step S61: Acquire a test sample image of a test set.

[0060] The method of this embodiment is used to test the accuracy of the text recognition model using the test sample images in the test set, that is, to determine the accuracy. The test sample images described herein may be, but are not limited to, images stitched from multiple eyes, stitched panoramic images, high-resolution images with a resolution exceeding 4K, etc. Among them, each test sample image constitutes a test set, and the size, number, etc. of the test sample images in the test set are not limited, and can be specifically set according to actual use needs; the test sample images can be obtained from local storage or cloud storage, or collected by electronic devices in the current environment, which is not specifically limited here.

[0061] The test sample image contains a specific text area and is annotated with the second truth text content in the specific text area, so that the accuracy of the text recognition model can be determined based on each test sample image in the test set.

[0062] Step S62: Perform region detection on the test sample image using the region detection model to obtain a test prediction box of the test sample image.

[0063] In this embodiment, the region detection model is used to perform region detection on the test sample image to obtain a test prediction frame of the test sample image. Since the region detection model is optimized based on the evaluation index of the region detection model, the boundaries of the test prediction frame of the obtained test sample image are close to the true value frame corresponding to the test sample image.

[0064] Step S63: Based on the test prediction frame, crop the image portion of the specific text area from the test sample image to obtain a second area image.

[0065] In this embodiment, according to the test prediction frame of the test sample image, the image portion of the specific text area is clipped from the test sample image to obtain the second area image. In other words, only the image portion corresponding to the specific text area is used for the subsequent text recognition model test, so as to avoid the text content in other areas outside the specific text area from interfering with the text recognition model, thereby improving the text recognition accuracy of the text recognition model.

[0066] In order to improve the recognition accuracy and efficiency of the subsequent text recognition model, in one embodiment, after the second region image is obtained, the second region image may be corrected.

[0067] Step S64: Perform text recognition on the second region image using the text recognition model to obtain a text recognition result.

[0068] In this implementation, the text recognition model is used to perform text recognition on the second region image to obtain a text recognition result.

[0069] Step S65: Based on the text recognition result and the second truth text content, obtain the recognition accuracy.

[0070] In this embodiment, the recognition accuracy is obtained based on the text recognition result and the second true value text content in the specific text area marked on the test sample image. In one embodiment, the overall text content recognition accuracy can be used as the final recognition accuracy. It can be understood that in other embodiments, the recognition accuracy of individual characters can also be calculated separately to serve as the final recognition accuracy.

[0071] See also Figure 7 , Figure 7 yes Figure 2 It should be noted that if there is substantially the same result, this embodiment does not use Figure 7 The process sequence shown is limited. Figure 7 As shown, this embodiment includes:

[0072] Step S231: Determine the boundary tightness between the true value box and the predicted box.

[0073] In this implementation, first, the boundary tightness between the true value box of the specific text region and the predicted box of the specific text region is determined.

[0074] In one embodiment, if Figure 8 As shown, Figure 8 yes Figure 7 FIG. 1 is a flow chart of an embodiment of step S231. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Figure 8 The process sequence shown is limited. Figure 8 As shown, in this embodiment, based on the distance between the boundary of the true value box of the specific text area and the boundary corresponding to the prediction box, the boundary tightness is determined, specifically including:

[0075] Step S2311: Obtain the distance of the first boundary between the true value box and the predicted box, and obtain the distance of the second boundary between the true value box and the predicted box.

[0076] In this embodiment, a first boundary distance between the true value box and the predicted box of a specific text region is obtained, and a second boundary distance between the true value box and the predicted box of the specific text region is obtained. The first boundary is a boundary in a first direction, and the second boundary is a boundary in a second direction. Specifically, Fig. 9 As shown, Fig. 9 It is a schematic diagram of an embodiment of a truth box and a prediction box provided in the present application, wherein the first direction is a vertical direction, and the first boundary is an upper and lower boundary in the vertical direction; the second direction is a horizontal direction, and the second boundary is a left and right boundary in the horizontal direction.

[0077] In a specific embodiment, the first boundary includes an upper boundary and a lower boundary, and the distance of the first boundary between the true value box and the prediction box can be obtained by calculating the distance of the upper boundary between the true value box and the prediction box and the distance of the lower boundary between the true value box and the prediction box; the second boundary includes a left boundary and a right boundary, and the distance of the second boundary between the true value box and the prediction box can be obtained by calculating the distance of the left boundary between the true value box and the prediction box and the distance of the right boundary between the true value box and the prediction box.

[0078] Specifically, Fig. 9 As shown, first obtain the true value box G of a specific text area, and the true value box G is:

[0079] G={(x Glt ,y Glt ),(x Grt ,y Grt ),(x Gld ,y Gld ),(x Grd ,y Grd )}

[0080] Among them, (x Glt ,y Glt ) represents the coordinate of the upper left corner of the truth box G, (x Grt ,y Grt ) represents the coordinate of the upper right corner of the truth box G, (x Gld ,y Gld ) represents the coordinate of the lower left corner of the truth box G, (x Grd ,y Grd ) represents the lower right corner coordinate of the truth box G; the upper left corner coordinate and the upper right corner coordinate constitute the upper boundary of the truth box G, the lower left corner coordinate and the lower right corner coordinate constitute the lower boundary of the truth box G, the upper left corner coordinate and the lower left corner coordinate constitute the left boundary of the truth box G, and the upper right corner coordinate and the lower right corner coordinate constitute the right boundary of the truth box G.

[0081] Secondly, the prediction box D of a specific text area is defined as:

[0082] D={(xDlt ,y Dlt ),(x Drt ,y Drt ),(x Dld ,y Dld ),(x Drd ,y Drd )}

[0083] Among them, (x Dlt ,y Dlt ) represents the coordinate of the upper left corner of the prediction box D, (x Drt ,y Drt ) represents the coordinate of the upper right corner of the prediction box D, (x Dld ,y Dld ) represents the coordinate of the lower left corner of the prediction box D, (x Drd ,y Drd ) represents the coordinate of the lower right corner of the prediction box D; the coordinates of the upper left corner and the upper right corner constitute the upper boundary of the prediction box D, the coordinates of the lower left corner and the lower right corner constitute the lower boundary of the prediction box D, the coordinates of the upper left corner and the lower left corner constitute the left boundary of the prediction box D, and the coordinates of the upper right corner and the lower right corner constitute the right boundary of the prediction box D.

[0084] Therefore, the first distance between the upper boundary of the true value box G and the predicted box D can be obtained as the enlarged height h 扩 ; The second distance between the lower boundary of the true value box G and the predicted box D can be obtained as the shrinkage height h 缩 ; The third distance of the left boundary between the true value box G and the predicted box D can be obtained as the expansion width w 扩 ; The fourth distance of the right boundary between the true value box G and the predicted box D can be obtained as the shrinkage width w 缩 It should be noted that if Figure 8 The positional relationship between the true value box G and the predicted box D is shown, so the first distance between the upper boundary of the true value box G and the predicted box D is used as the enlarged height h 扩 , the second distance between the lower boundary of the true value box G and the predicted box D is taken as the shrinkage height h 缩 , the third distance from the left boundary between the true value box G and the predicted box D is taken as the expansion width w 扩 , the fourth distance of the right boundary between the true value box G and the predicted box D is taken as the shrinkage width w 缩 If the positions of the true value box G and the predicted box D are swapped, the first distance between the upper boundaries of the true value box G and the predicted box D is used as the shrinkage height h 缩 , the second distance between the lower boundary of the true value box G and the predicted box D is taken as the enlarged height h 扩 , the third distance from the left boundary between the true value box G and the predicted box D is taken as the shrinkage width w 缩, the fourth distance of the right boundary between the true value box G and the predicted box D is taken as the expansion width w 扩 That is to say, according to the positional relationship between the true value box G and the predicted box D, it is determined whether the height or width between the boundaries is shrunk or expanded.

[0085] Step S2312: obtaining a first boundary tightness with respect to the first boundary by using the distance of the first boundary, and obtaining a second boundary tightness with respect to the second boundary by using the distance of the second boundary.

[0086] In this embodiment, the first boundary tightness with respect to the first boundary is obtained by using the distance of the first boundary, and the second boundary tightness with respect to the second boundary is obtained by using the distance of the second boundary.

[0087] In one embodiment, the first boundary tightness about the first boundary is obtained by using the expanded height and the contracted height, and the second boundary tightness about the second boundary is obtained by using the expanded width and the contracted width. Fig.10 As shown, Fig.10 yes Figure 8 It should be noted that if there is substantially the same result, this embodiment does not use Fig.10 The process sequence shown is limited. Fig.10 As shown, in this embodiment, a reference frame is introduced to calculate the boundary tightness, which specifically includes:

[0088] Step S101: obtaining an expansion parameter using the expansion length and the first reference distance, and obtaining a contraction parameter using the contraction length and the first reference distance.

[0089] In this embodiment, the expansion parameter is obtained by using the expansion length and the first reference distance; and the contraction parameter is obtained by using the contraction length and the first reference distance. The reference frame includes a true value frame and a prediction frame. In one embodiment, the reference frame is the minimum circumscribed rectangular frame of the true value frame and the prediction frame. It can be understood that in other embodiments, the reference frame can also be a circumscribed rectangular frame whose area is larger than the sum of the areas of the true value frame and the prediction frame. Alternatively, the shape of the reference frame can also be other shapes such as a circle.

[0090] Specifically, take the reference box as the minimum bounding rectangle of the true value box and the predicted box as an example, Fig. 9 As shown, the reference frame of the true value frame G and the predicted frame D is C, and C is defined as:

[0091] C={(x Clt ,y Clt ),(x Crt ,y Crt ),(x Cld ,y Cld),(x Crd ,y Crd )}

[0092] Among them, (x Clt ,y Clt ) represents the coordinate of the upper left corner of the reference frame C, (x Crt ,y Crt ) represents the coordinate of the upper right corner of the reference frame C, (x Cld ,y Cld ) represents the coordinate of the lower left corner of the reference frame C, (x Crd ,y Crd ) represents the coordinate of the lower right corner of the reference frame C. Since the height of the reference frame C is the first reference distance, the first reference distance is h c ; The expanded height is the expanded length, so the expanded length = expanded height = h 扩 ; Take the contraction height as the contraction length, so contraction length = contraction height = h 缩 When the width of the reference frame C is taken as the first reference distance, the first reference distance is w c ; The expanded width is the expanded length, so the expanded length = expanded width = w 扩 ; The contraction width is the contraction length, so contraction length = contraction width = w 缩 .

[0093] Further, the enlargement parameter is obtained by using the enlargement length and the first reference distance. In a specific implementation, the ratio of the enlargement length and the first reference distance is obtained as the enlargement parameter, and the specific formula is as follows:

[0094] or

[0095] Among them, t 扩 Indicates the expansion parameter; h 扩 Indicates that enlarging the height means enlarging the length; w 扩 Indicates that enlarging the width means enlarging the length; h c Indicates the first reference distance.

[0096] The contraction parameter is obtained by using the contraction length and the first reference distance. In a specific implementation, the ratio of the contraction length and the first reference distance is obtained as the contraction parameter, and the specific formula is as follows:

[0097] or

[0098] Among them, t 缩 represents the shrinkage parameter; h 缩 Indicates the contraction height or contraction length; w 缩 Indicates the contraction width or contraction length; hc Indicates the first reference distance.

[0099] Step S102: Obtain the target boundary density by using the expansion parameter, the contraction parameter, the expansion area corresponding to the expansion length, and the contraction area corresponding to the contraction length.

[0100] In this embodiment, the target boundary tightness is obtained by using the expansion parameter, the contraction parameter, the expansion area corresponding to the expansion length, and the contraction area corresponding to the contraction length. When the expansion parameter and the contraction parameter are calculated using the height of the reference frame as the first reference distance, the target boundary tightness is the first boundary tightness; and when the expansion parameter and the contraction parameter are calculated using the width of the reference frame as the first reference distance, the target boundary tightness is the second boundary tightness.

[0101] In one embodiment, the enlarged area corresponding to the enlarged length is the product of the enlarged length and the second reference distance, and the contracted area corresponding to the contracted length is the product of the contracted length and the second reference distance. If the enlarged length is the enlarged height, the second reference distance is the width of the reference frame; if the enlarged length is the enlarged width, the second reference distance is the height of the reference frame. The specific formula for the enlarged area is as follows:

[0102] or

[0103] in, Indicates the enlarged area corresponding to the enlarged length; h 扩 Indicates that enlarging the height means enlarging the length; w c Indicates the width of the reference frame, i.e. the second reference distance; h c Indicates the height of the reference frame, i.e., the second reference distance; w 扩 It means that expanding the width means expanding the length.

[0104] The specific formula for the shrinkage area is as follows:

[0105] or

[0106] in, represents the contraction area corresponding to the contraction length; h 缩 Indicates the contraction height or contraction length; w c Indicates the width of the reference frame, i.e. the second reference distance; h c Indicates the height of the reference frame, i.e., the second reference distance; w 缩 Indicates the contraction width, i.e. the contraction length.

[0107] In a specific embodiment, Fig.11 As shown, Fig.11 yes Fig.10It should be noted that if there are substantially the same results, this embodiment does not necessarily use Fig.11 The process sequence shown is limited. Fig.11 As shown, this embodiment includes:

[0108] Step S1021: Obtain a first product of the square of the enlarged area, the first power operation result, and the first value.

[0109] In this embodiment, the first product of the square of the enlarged area, the first power operation result and the first numerical value is obtained. The first power operation result is the power operation result of the first fraction whose exponent is the first fraction, the numerator of the first fraction is determined based on the enlargement parameter, the denominator is determined based on the preset constant, and the first fraction is negatively correlated with the enlargement parameter, and the first numerical value is determined based on the preset constant and negatively correlated with the preset constant. The size of the preset constant is not limited, and can be specifically set according to actual use needs. For example, the preset constant is 0.4. The specific formula is as follows:

[0110] or

[0111] in, or represents the square of the enlarged area; represents a first value; Represents the result of the first power operation.

[0112] Step S1022: Obtain a second product of the square of the contraction area, the second power operation result, and the first value.

[0113] In this embodiment, the second product of the square of the contraction area, the second power operation result and the first value is obtained. The second power operation result is the power operation result of the second fraction whose exponent is a second fraction, the numerator of the second fraction is determined based on the contraction parameter, the denominator is determined based on a preset constant, and the second fraction is negatively correlated with the contraction parameter. The specific formula is as follows:

[0114] or

[0115] in, or represents the square of the contraction area; represents a first value; Represents the result of the second power operation.

[0116] Step S1023: Utilize the square of the area of ​​the reference frame, the first product, and the second product to obtain the target boundary tightness.

[0117] In this embodiment, the target boundary tightness is obtained by using the square of the area of ​​the reference frame, the first product and the second product, wherein the target boundary tightness is positively correlated with the first product and the second product, and negatively correlated with the square of the area of ​​the reference frame.

[0118] Among them, when the height of the reference frame is taken as the first reference distance, the expanded height is taken as the expanded length, and the contracted height is taken as the contracted length, the first boundary tightness is taken as the target boundary tightness, and the specific formula of the first boundary tightness is as follows:

[0119]

[0120] Among them, P(h 扩 ,h 缩 ) represents the tightness of the first boundary; represents the first product; represents the second product; Represents the square of the area of ​​the reference box.

[0121] When the width of the reference frame is taken as the first reference distance, the expanded width is taken as the expanded length, and the contracted width is taken as the contracted length, the second boundary tightness is taken as the target boundary tightness, and the specific formula of the second boundary tightness is as follows:

[0122]

[0123] Among them, P(w 扩 ,w 缩 ) indicates the tightness of the second boundary; represents the first product; represents the second product; Represents the square of the area of ​​the reference box.

[0124] Therefore, it can be seen from the first boundary tightness and the second boundary tightness that when the prediction box is the same as the true value box, the boundary function value corresponding to the first boundary tightness and the second boundary tightness is 0; when the prediction box shrinks or expands, the boundary function value decreases faster; when the prediction box shrinks or expands to a certain extent, the boundary function value is 1 and remains unchanged, and no matter how large or small the prediction box is, it will not have a worse impact on the accuracy of subsequent text recognition.

[0125] Step S232: Based on the boundary density, determine at least one boundary weighted metric to serve as an evaluation indicator of the region detection model.

[0126] In this embodiment, at least one boundary weighted metric can be determined according to the boundary tightness as an evaluation index of the region detection model. That is, the at least one boundary weighted metric determined is used as the evaluation index of the region detection model, so that the evaluation index takes into account the boundary tightness between the prediction box and the true value box, so that the region detection model optimized based on the boundary weighted metric can obtain a prediction box with a more reasonable boundary position, so that the subsequent text recognition model can more accurately recognize the text content in the specific text area.

[0127] In one embodiment, if Fig.12 As shown, Fig.12 yes Figure 7 It should be noted that if there is substantially the same result, this embodiment does not use Fig.12 The process sequence shown is limited. Fig.12 As shown, this embodiment includes:

[0128] Step S2321: Obtain a target weight based on the first boundary tightness and the second boundary tightness.

[0129] In this embodiment, the target weight is obtained according to the first boundary tightness and the second boundary tightness. In a specific embodiment, the first boundary tightness and the second boundary tightness are directly processed, for example, the average value of the first boundary tightness and the second boundary tightness is calculated as the target weight; or the first boundary tightness and the second boundary tightness are weighted and summed as the target weight, etc.

[0130] In other specific embodiments, Fig.13 As shown, Fig.13 yes Fig.12 It should be noted that if there are substantially the same results, this embodiment does not necessarily use Fig.13 The process sequence shown is limited. Fig.13 As shown, this embodiment includes:

[0131] Step S1301: Determine the sub-weight of the first boundary based on the boundary ratio of the first boundary and the first boundary density in the truth frame, and determine the sub-weight of the second boundary based on the boundary ratio of the second boundary and the second boundary density in the truth frame.

[0132] In this embodiment, in determining the evaluation index of the region detection model, in addition to considering the factor of the boundary tightness between the prediction box and the true value box, the aspect ratio of the specific text area is also considered. When the aspect ratio of the prediction box is compared with the true value box, the aspect ratio of the prediction box with the same area as the true value box can bring more accurate text recognition effect than the aspect ratio of the prediction box with a similar area to the true value box. Therefore, in this embodiment, the sub-weight of the first boundary is determined according to the boundary ratio of the first boundary in the true value frame and the first boundary tightness; and the sub-weight of the second boundary is determined according to the boundary ratio of the second boundary in the true value frame and the second boundary tightness.

[0133] In a specific implementation, the product of the boundary ratio of the first boundary in the truth frame and the first boundary tightness can be obtained as the sub-weight of the first boundary. The specific formula is as follows:

[0134]

[0135] in, represents the boundary proportion of the first boundary in the truth frame; P(h 扩 ,h 缩 ) represents the first boundary tightness. It can be understood that in other embodiments, other processing can also be performed on the boundary proportion of the first boundary and the first boundary tightness in the truth frame to obtain the sub-weight of the first boundary.

[0136] In a specific implementation, the product of the boundary ratio of the second boundary in the truth frame and the tightness of the second boundary can be obtained as the sub-weight of the second boundary. The specific formula is as follows:

[0137]

[0138] in, represents the boundary proportion of the second boundary in the truth frame; P(w 扩 ,w 缩 ) represents the second boundary tightness. It can be understood that in other embodiments, other processing can be performed on the boundary proportion of the second boundary in the truth frame and the second boundary tightness to obtain the sub-weight of the second boundary.

[0139] Step S1302: Obtain a target weight using the sub-weight of the first boundary and the sub-weight of the second boundary.

[0140] In this embodiment, the target weight is obtained by using the sub-weight of the first boundary and the sub-weight of the second boundary. In a specific embodiment, the sum of the sub-weight of the first boundary and the sub-weight of the second boundary can be obtained as the target weight. The specific formula is as follows:

[0141]

[0142] Among them, NBP(G,D) represents the target weight; represents the sub-weight of the first boundary; Represents the sub-weight of the second boundary.

[0143] It can be understood that in other embodiments, other processing may be performed on the sub-weights of the first boundary and the sub-weights of the second boundary to obtain the target weights, for example, averaging the sub-weights of the first boundary and the sub-weights of the second boundary or performing weighted summation processing, etc., which is not specifically limited here.

[0144] Step S2322: Using the target weight, obtain at least one boundary weighted measurement.

[0145] In this implementation, the target weight is used to obtain at least one boundary weighted metric, wherein the at least one boundary weighted metric includes at least one of accuracy, recall, and a comprehensive index, and the comprehensive index can be obtained based on the accuracy and recall, or based on the accuracy or recall alone, which is not specifically limited here.

[0146] In one embodiment, at least one boundary weighted metric is a recall rate, and the recall rate is obtained using the target weight. The recall rate is obtained specifically according to the target weight, the first size of the prediction box, and the second size of the union of the true value box and the prediction box. In a specific embodiment, the first difference between the second value and the target weight is obtained, and the ratio of the product of the second size and the first difference to the first size is used as the recall rate. Among them, the second value is not limited, for example, the second value is 1. The specific formula of the recall rate is as follows:

[0147]

[0148] Among them, NBIoU recall Represents the recall rate; Area(D) represents the first size of the prediction box; Area(G,D) represents the second size of the union of the true value box and the prediction box; 1 represents the second value; NBP(G,D) represents the target weight.

[0149] In one embodiment, at least one boundary weighted metric is a recall rate, and then the target weight is used to obtain the accuracy. The accuracy is obtained specifically based on the target weight, the third size of the truth box, and the second size. In a specific embodiment, the second difference between the second value and the target weight is obtained, and the ratio of the product of the second size and the second difference to the third size is used as the accuracy. Among them, the second value is not limited, for example, the second value is 1. The specific formula of the accuracy is as follows:

[0150]

[0151] Among them, NBIoU precision Represents accuracy; Area(G) represents the third size of the true value box; Area(G,D) represents the second size of the union of the true value box and the prediction box; 1 represents the second value; NBP(G,D) represents the target weight.

[0152] In one embodiment, at least one boundary weighted metric is a comprehensive indicator, and then the target weight is used to obtain the comprehensive indicator. The comprehensive indicator is obtained specifically based on the recall rate and the accuracy rate. In a specific embodiment, the third product of the recall rate and the accuracy rate, and the sum of the recall rate and the accuracy rate are obtained, and a preset multiple of the ratio of the third product to the sum is obtained as the comprehensive indicator. Among them, the preset multiple is not limited, for example, the preset multiple is 2. The specific formula of the comprehensive indicator is as follows:

[0153]

[0154] Among them, H-mean represents the comprehensive index; NBIoU precision Indicates accuracy; NBIoU reca ll represents the recall rate; 2 represents the preset multiple. H-mean is a comprehensive indicator that combines the recall rate and the precision rate into one indicator, which is conducive to the subsequent judgment of the robustness of the region detection model.

[0155] It can be understood that in other embodiments, the boundary tightness of the upper boundary, lower boundary, left boundary and right boundary can also be calculated respectively, so as to determine at least one boundary weighted measurement based on the upper boundary tightness, lower boundary tightness, left boundary tightness and boundary tightness, and use it as an evaluation indicator of the region detection model.

[0156] See also Fig.14 , Fig.14 It is a schematic diagram of the framework of an embodiment of the region detection device provided by the present application. The region detection device 140 includes: an acquisition module 1401 and a detection module 1402. The acquisition module 1401 is used to acquire the image to be detected; the detection module 1402 is used to perform region detection on the image to be detected using the region detection model, and obtain a target prediction box about a specific text region in the image to be detected; wherein the region detection model is obtained by optimizing the evaluation index of the region detection model, and the evaluation index is obtained based on the boundary tightness between the true value box and the prediction box of the specific text region in the verification sample image, and the prediction box is obtained by performing region detection on the verification sample image by the region detection model.

[0157] Among them, the region detection device 140 also includes an optimization module 1403, and the optimization module 1403 is used to perform region detection on the image to be detected using the region detection model to obtain the target prediction box about the specific text area in the image to be detected, specifically including: obtaining a verification sample image from a verification set, wherein the verification sample image contains a specific text area and is marked with a true value box of the specific text area; using the region detection model to perform region detection on the verification sample image to obtain a prediction box about the specific text area in the verification sample image; based on the boundary tightness between the true value box and the prediction box, obtaining an evaluation index of the region detection model; based on the evaluation index, optimizing the region detection model.

[0158] Among them, the optimization module 1403 is used to obtain the evaluation index of the region detection model based on the boundary tightness between the true value box and the prediction box, specifically including: determining the boundary tightness between the true value box and the prediction box; based on the boundary tightness, determining at least one boundary weighted measurement as an evaluation index of the region detection model.

[0159] Among them, the optimization module 1403 is used to determine the boundary tightness between the true value box and the prediction box, specifically including: obtaining the distance of the first boundary between the true value box and the prediction box, and obtaining the distance of the second boundary between the true value box and the prediction box, wherein the first boundary is the boundary in the first direction, and the second boundary is the boundary in the second direction; using the distance of the first boundary to obtain the first boundary tightness with respect to the first boundary, and using the distance of the second boundary to obtain the second boundary tightness with respect to the second boundary.

[0160] Among them, the above-mentioned first boundary includes an upper boundary and a lower boundary, and the second boundary includes a left boundary and a right boundary; the optimization module 1403 is used to obtain the distance of the first boundary between the true value box and the prediction box, and obtain the distance of the second boundary between the true value box and the prediction box, specifically including: obtaining the first distance of the upper boundary between the true value box and the prediction box, and the second distance of the lower boundary between the true value box and the prediction box, respectively as the expansion height and the contraction height; and obtaining the third distance of the left boundary between the true value box and the prediction box, and the fourth distance of the right boundary between the true value box and the prediction box, respectively as the expansion width and the contraction width; the optimization module 1403 is used to use the distance of the first boundary to obtain the first boundary tightness with respect to the first boundary, and use the distance of the second boundary to obtain the second boundary tightness with respect to the second boundary, specifically including: using the expansion height and the contraction height to obtain the first boundary tightness with respect to the first boundary, and using the expansion width and the contraction width to obtain the second boundary tightness with respect to the second boundary.

[0161] Among them, the optimization module 1403 is used to use the expanded height and the contracted height to obtain the first boundary tightness about the first boundary, or the optimization module 1403 is used to use the expanded width and the contracted width to obtain the second boundary tightness about the second boundary, specifically including: taking the height of the reference frame as the first reference distance, the expanded height as the expanded length, the contracted height as the contracted length, and the first boundary tightness as the target boundary tightness; or taking the width of the reference frame as the first reference distance, the expanded width as the expanded length, the contracted width as the contracted length, and the second boundary tightness as the target boundary tightness; using the expanded length and the first reference distance to obtain the expansion parameter, and using the contracted length and the first reference distance to obtain the contraction parameter, wherein the reference frame includes a true value frame and a prediction frame; using the expansion parameter, the contraction parameter, the expanded area corresponding to the expanded length, and the contracted area corresponding to the contracted length to obtain the target boundary tightness.

[0162] Among them, the above-mentioned reference frame is the minimum circumscribed rectangular frame of the true value frame and the prediction frame; and / or, the optimization module 1403 is used to use the expansion length and the first reference distance to obtain the expansion parameter, and use the contraction length and the first reference distance to obtain the contraction parameter, specifically including: obtaining the ratio of the expansion length and the first reference distance as the expansion parameter, and obtaining the ratio of the contraction length and the first reference distance as the contraction parameter.

[0163] Among them, the optimization module 1403 is used to determine at least one boundary weighted measurement based on the boundary tightness, specifically including: obtaining a target weight based on the first boundary tightness and the second boundary tightness; using the target weight, obtaining at least one boundary weighted measurement, wherein the at least one boundary weighted measurement includes at least one of accuracy, recall rate and comprehensive index, and the comprehensive index is obtained based on accuracy and / or recall rate.

[0164] Among them, the optimization module 1403 is used to obtain the target weight based on the first boundary tightness and the second boundary tightness, specifically including: determining the sub-weight of the first boundary based on the boundary ratio of the first boundary in the truth value frame and the first boundary tightness, and determining the sub-weight of the second boundary based on the boundary ratio of the second boundary in the truth value frame and the second boundary tightness; using the sub-weight of the first boundary and the sub-weight of the second boundary to obtain the target weight.

[0165] Among them, the optimization module 1403 is used to determine the sub-weight of the first boundary based on the boundary ratio of the first boundary in the truth value frame and the tightness of the first boundary, specifically including: obtaining the product of the boundary ratio of the first boundary in the truth value frame and the tightness of the first boundary as the sub-weight of the first boundary; based on the boundary ratio of the second boundary in the truth value frame and the tightness of the second boundary, determine the sub-weight of the second boundary, specifically including: obtaining the product of the boundary ratio of the second boundary in the truth value frame and the tightness of the second boundary as the sub-weight of the second boundary.

[0166] Among them, the optimization module 1403 is used to use the target weight to obtain at least one boundary weighted measurement, including any one or more of the following steps: obtaining the recall rate based on the target weight, the first size of the prediction box, and the second size of the union of the true value box and the prediction box; obtaining the accuracy rate based on the target weight, the third size of the true value box and the second size; using the recall rate and the accuracy rate to obtain a comprehensive indicator.

[0167] Among them, the above-mentioned specific text area is a non-standard license plate; and / or, the area detection device 140 also includes a training module 1404, which, before obtaining the verification sample image from the verification set, specifically also includes: using the training set to train the area detection model, wherein the training set includes a number of training sample images, the training sample images contain a specific text area, and are marked with a true value box of the specific text area.

[0168] See also Fig.15 , Fig.15 1 is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application. The electronic device 150 includes a memory 1501 and a processor 1502 coupled to each other, the memory 1501 stores program instructions, and the processor 1502 is used to execute the program instructions to implement the steps in any of the above-mentioned area detection method embodiments. Specifically, the electronic device 150 may include but is not limited to: a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, etc., which are not limited here.

[0169] Specifically, the processor 1502 is used to control itself and the memory 1501 to implement the steps in any of the above-mentioned area detection method embodiments. The processor 1502 can also be called a CPU (Central Processing Unit). The processor 1502 may be an integrated circuit chip with signal processing capabilities. The processor 1502 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 1502 can be implemented by an integrated circuit chip.

[0170] See also Fig.16 , Fig.16 The computer-readable storage medium 160 stores program instructions 1601 that can be executed by a processor, and the program instructions 1601 are used to implement the steps in any of the above-mentioned area detection method embodiments.

[0171] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0172] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0177] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A region detection method, characterized in that: include: Acquire the image to be detected; Acquire a verification sample image from a verification set, wherein the verification sample image contains a specific text area and is marked with a true value box of the specific text area; Performing region detection on the verification sample image using a region detection model to obtain a prediction box about the specific text region in the verification sample image; Acquire a first distance between the upper boundary of the true value box and the prediction box, and a second distance between the lower boundary of the true value box and the prediction box, to serve as an enlarged height and a shrunk height, respectively; and acquire a third distance between the left boundary of the true value box and the prediction box, and a fourth distance between the right boundary of the true value box and the prediction box, to serve as an enlarged width and a shrunk width, respectively; The first boundary tightness about the first boundary is obtained by using the enlarged height and the contracted height, and the second boundary tightness about the second boundary is obtained by using the enlarged width and the contracted width; wherein the first boundary includes the upper boundary and the lower boundary, and the second boundary includes the left boundary and the right boundary; Based on the boundary tightness, determining at least one boundary weighted metric as an evaluation indicator of the region detection model; Based on the evaluation index, optimizing the region detection model; The optimized region detection model is used to perform region detection on the image to be detected, and a target prediction box for a specific text region in the image to be detected is obtained.

2. The method according to claim 1, characterized in that The step of obtaining a first boundary tightness with respect to a first boundary by using the enlarged height and the contracted height, or obtaining a second boundary tightness with respect to a second boundary by using the enlarged width and the contracted width, comprises: The height of the reference frame is used as the first reference distance, the enlarged height is used as the enlarged length, the shrunk height is used as the shrunk length, and the first boundary tightness is used as the target boundary tightness; or the width of the reference frame is used as the first reference distance, the enlarged width is used as the enlarged length, the shrunk width is used as the shrunk length, and the second boundary tightness is used as the target boundary tightness; Using the expansion length and the first reference distance, an expansion parameter is obtained, and using the contraction length and the first reference distance, a contraction parameter is obtained, wherein the reference frame includes the true value frame and the prediction frame; The target boundary tightness is obtained by using the expansion parameter, the contraction parameter, the expansion area corresponding to the expansion length, and the contraction area corresponding to the contraction length.

3. The method according to claim 2, characterized in that The reference frame is the minimum circumscribed rectangular frame of the true value frame and the prediction frame; And / or, the using the expansion length and the first reference distance to obtain an expansion parameter, and using the contraction length and the first reference distance to obtain a contraction parameter, comprises: A ratio of the expansion length to the first reference distance is obtained as the expansion parameter, and a ratio of the contraction length to the first reference distance is obtained as the contraction parameter.

4. The method according to claim 1, characterized in that: The determining at least one boundary weighted metric based on the boundary tightness comprises: Obtaining a target weight based on the first boundary tightness and the second boundary tightness; Using the target weight, at least one boundary weighted metric is obtained, wherein the at least one boundary weighted metric includes at least one of accuracy, recall and a comprehensive index, and the comprehensive index is obtained based on the accuracy and / or the recall.

5. The method according to claim 4, characterized in that The obtaining of a target weight based on the first boundary tightness and the second boundary tightness includes: Determine a sub-weight of the first boundary based on a boundary ratio of the first boundary in the truth value frame and the first boundary tightness, and determine a sub-weight of the second boundary based on a boundary ratio of the second boundary in the truth value frame and the second boundary tightness; The target weight is obtained by using the sub-weight of the first boundary and the sub-weight of the second boundary.

6. The method according to claim 5, characterized in that The determining the sub-weight of the first boundary based on the boundary proportion of the first boundary in the truth frame and the first boundary tightness includes: Obtaining a product of a boundary ratio of a first boundary in the truth frame and a tightness of the first boundary as a sub-weight of the first boundary; The determining the sub-weight of the second boundary based on the boundary proportion of the second boundary in the truth frame and the tightness of the second boundary includes: The product of the boundary ratio of the second boundary in the truth frame and the tightness of the second boundary is obtained as the sub-weight of the second boundary.

7. The method according to claim 4, characterized in that The step of obtaining at least one boundary weighted metric by using the target weight comprises any one or more of the following steps: Obtaining the recall rate based on the target weight, the first size of the prediction box, and the second size of the union of the true value box and the prediction box; Obtaining the accuracy based on the target weight, the third size of the truth box, and the second size; The comprehensive index is obtained by using the recall rate and the precision rate.

8. The method according to claim 1, characterized in that The specific text area is a non-standard license plate; And / or, before obtaining the verification sample image from the verification set, the method further includes: The region detection model is trained using a training set, wherein the training set includes a plurality of training sample images, the training sample images contain specific text regions, and are annotated with true value boxes of the specific text regions.

9. A region detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected; The optimization module is used to obtain a verification sample image from a verification set, wherein the verification sample image contains a specific text area and is marked with a true value box of the specific text area; perform region detection on the verification sample image using a region detection model to obtain a prediction box about the specific text area in the verification sample image; obtain a first distance between the upper boundary of the true value box and the prediction box, and a second distance between the lower boundary of the true value box and the prediction box, to respectively serve as an enlarged height and a shrunk height, and obtain a third distance between the left boundary of the true value box and the prediction box, and a fourth distance between the right boundary of the true value box and the prediction box, to respectively serve as an enlarged width and a shrunk width; obtain a first boundary tightness about the first boundary using the enlarged height and the shrunk height, and obtain a second boundary tightness about the second boundary using the enlarged width and the shrunk width, wherein the first boundary includes the upper boundary and the lower boundary, and the second boundary includes the left boundary and the right boundary; determine at least one boundary weighted metric based on the boundary tightness as an evaluation index of the region detection model; optimize the region detection model based on the evaluation index; The detection module is used to perform region detection on the image to be detected using the optimized region detection model to obtain a target prediction box for a specific text region in the image to be detected.

10. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the area detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the area detection method according to any one of claims 1 to 8.

Images