A signboard region correction and recognition method and device in an image, equipment and medium

Abstract

The embodiment of the specification discloses a road sign region correction and recognition method, device, equipment and medium in an image, the road sign region correction method in the image comprises the following steps: after an initial image is acquired, whether the initial image can be corrected is judged according to whether the edges of the polygonal region occupied by the interested road sign in the initial image satisfy the regular condition; if it is judged that the initial image can be corrected, a road sign region correction operation is performed on the initial image; wherein the road sign region correction operation on the initial image comprises the following steps: if only one edge of the polygonal region does not satisfy the regular condition, then the to-be-adjusted point among the two vertices connected by the edge is adjusted, so that all the edges of the polygonal region satisfy the regular condition; if only two adjacent edges of the polygonal region do not satisfy the regular condition, then the common vertex of the two edges is the to-be-adjusted point, and the to-be-adjusted point is adjusted, so that all the edges of the polygonal region satisfy the regular condition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and medium for correcting and recognizing road sign areas in images. Background Technology

[0002] With the continuous development of applications such as navigation and panoramic maps, higher requirements are being placed on the accuracy of the maps used. In the process of creating high-precision maps, it is necessary to identify various objects in the images used to ensure the accuracy of the final map. Among these, accurately representing road signs, a crucial map element, on the map is an important challenge.

[0003] Therefore, a more effective road sign recognition solution is needed. Summary of the Invention

[0004] This specification provides a method, apparatus, device, and medium for correcting and recognizing road sign areas in images, in order to solve the technical problem of how to more effectively recognize road signs.

[0005] To solve the above-mentioned technical problems, the embodiments of this specification provide the following technical solutions:

[0006] This specification provides an embodiment of a method for correcting road sign regions in an image, including:

[0007] After obtaining the initial image, it is determined whether the initial image can be used for the correction of the area of ​​interest of the road sign based on whether the edges of the polygonal region occupied by the road sign in the initial image meet the regularity condition.

[0008] If it is determined that the initial image can be corrected for the road sign region of interest, then the road sign region correction operation is performed on the initial image; wherein, the road sign region correction operation on the initial image includes:

[0009] If only one edge of the polygonal region does not meet the regularity condition, then the point to be adjusted among the two vertices connected to that edge is adjusted so that all edges of the polygonal region meet the regularity condition; if only two adjacent edges of the polygonal region do not meet the regularity condition, then the common vertex of the two edges is the point to be adjusted, and the point to be adjusted is adjusted so that all edges of the polygonal region meet the regularity condition.

[0010] This specification provides an embodiment of a method for recognizing road signs in images, including:

[0011] After obtaining a set of initial images containing road signs of interest, perform the above-described road sign region correction method on any of the initial images;

[0012] The initial image that has undergone road sign region correction and the initial image that does not require road sign region correction are used as target images to form a target image sequence; the usability of the target image sequence is determined based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image.

[0013] If the target image sequence is available, the recognition result of the road sign of interest is generated based on the area occupied by the road sign of interest in each image of the target image sequence.

[0014] This specification provides an embodiment of a road sign region correction device in an image, comprising:

[0015] The region determination module is used to determine whether the initial image can be used for road sign region correction based on whether the edges of the polygonal region occupied by the road sign of interest in the initial image meet the regularity condition after acquiring the initial image.

[0016] The region correction module is used to perform a road sign region correction operation on the initial image if it is determined that the initial image can be corrected for the road sign region of interest.

[0017] The road sign region correction operation on the initial image includes: if only one edge of the polygonal region does not meet the regularization condition, then the adjustment point among the two vertices connected to that edge is adjusted so that all edges of the polygonal region meet the regularization condition; if only two adjacent edges of the polygonal region do not meet the regularization condition, then the common vertex of the two edges is the adjustment point, and the adjustment point is adjusted so that all edges of the polygonal region meet the regularization condition.

[0018] This specification provides an embodiment of a road sign recognition device in an image, comprising:

[0019] The region correction module is used to obtain a set of initial images containing road signs of interest, and then perform the above-described road sign region correction method on any of the initial images.

[0020] The image filtering module is used to select initial images that have undergone road sign region correction and initial images that do not require road sign region correction as target images to form a target image sequence; and to determine whether the target image sequence is usable based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in the target image sequence.

[0021] An image recognition module is used to generate a recognition result for the road sign of interest based on the area occupied by the road sign of interest in each image of the target image sequence if the target image sequence is available.

[0022] This specification provides an embodiment of a road sign region correction device in an image, comprising:

[0023] At least one processor;

[0024] as well as,

[0025] A memory that is communicatively connected to the at least one processor;

[0026] in,

[0027] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the above-described method for correcting road sign areas in images.

[0028] This specification provides an embodiment of a road sign recognition device in an image, comprising:

[0029] At least one processor;

[0030] as well as,

[0031] A memory that is communicatively connected to the at least one processor;

[0032] in,

[0033] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the above-described method for recognizing road signs in images.

[0034] This specification provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the above-described method for correcting road sign regions in an image or the above-described method for recognizing road signs in an image.

[0035] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0036] By correcting the region occupied by the road sign of interest in an image, the integrity, regularity, and accuracy of this region can be effectively improved. Based on this, for a set of initial images containing road signs of interest, the initial image with road sign region correction and the initial image without road sign region correction are used as target images. The accuracy of the road sign region in each target image is improved, and the road sign recognition result obtained based on the target image sequence more closely matches the actual shape of the road sign of interest in reality. Furthermore, it effectively reduces random errors between target images and random errors in road sign recognition, thereby improving the accuracy of road sign recognition. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments of this specification or the prior art will be briefly introduced below. Obviously, the drawings described below are only some of the drawings that may be involved in the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the execution subject of the road sign area correction method in the first embodiment of this specification.

[0039] Figure 2 This is a flowchart illustrating the method for correcting road sign areas in images in the first embodiment of this specification.

[0040] Figure 3 This is a schematic diagram of road sign obstruction in the first embodiment of this specification.

[0041] Figure 4 This is a schematic diagram of the area occupied by a road sign of interest in the first embodiment of this specification.

[0042] Figure 5 This is a schematic diagram of the area occupied by a road sign of interest, captured separately in the first embodiment of this specification.

[0043] Figure 6 This is a schematic diagram of the area occupied by another road sign of interest in the first embodiment of this specification.

[0044] Figure 7 This is a schematic diagram of the area occupied by another road sign of interest in the first embodiment of this specification.

[0045] Figure 8 This is a schematic diagram of the road sign area correction process in the first embodiment of this specification.

[0046] Figure 9 This is a flowchart illustrating the road sign recognition method in an image, as shown in the second embodiment of this specification.

[0047] Figure 10 This is a schematic diagram of IOU calculation in the second embodiment of this specification.

[0048] Figure 11 This is a schematic diagram of the structure of the road sign area correction device in the image in the third embodiment of this specification.

[0049] Figure 12 This is a schematic diagram of the structure of the road sign recognition device in the image in the fourth embodiment of this specification. Detailed Implementation

[0050] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions of the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described in this specification are merely some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0051] With technological advancements, maps, especially high-definition maps, are being used more widely and frequently. Among these applications, accurately representing real-world road signs on maps is a crucial issue.

[0052] The first embodiment of this specification (hereinafter referred to as "Embodiment 1") provides a method for correcting road sign regions in an image. The executing entity of Embodiment 1 can be a terminal (including but not limited to mobile phones, computers, tablets, and televisions), a server, an operating system, an application, a road sign region correction platform, or a road sign region correction system, etc. That is, the executing entity can be diverse and can be set, used, or changed as needed. Alternatively, a third-party application can assist the executing entity in performing Embodiment 1. For example... Figure 1 As shown, the image road sign region correction method in Embodiment 1 can be executed by the server, and a corresponding application can be installed on the terminal (held by the user). Data can be transmitted between the terminal or application and the server. Data can be collected, input, or output, or pages or information can be processed (to the user) through the terminal or application, thereby assisting the server in executing the image road sign region correction method in Embodiment 1.

[0053] like Figure 2 As shown, the method for correcting road sign areas in images provided in Embodiment 1 includes:

[0054] S101: (Executing entity) After obtaining the initial image, it determines whether the initial image can be used for the correction of the area of ​​interest of the road sign based on whether the edges of the polygonal region occupied by the road sign of interest in the initial image meet the regularity condition.

[0055] The executing entity in Embodiment 1 can acquire an initial image, which is an image containing a road sign of interest. Acquiring the initial image can include: capturing the initial image using an image capturing device and sending the captured initial image to the executing entity in Embodiment 1; or, the executing entity in Embodiment 1 can have an image capturing function and capture the initial image; or, the initial image can be a single frame image decomposed from a video (which can be decomposed by the executing entity in Embodiment 1 or by another entity, and the resulting image sent to the executing entity in Embodiment 1). The executing entity in Embodiment 1 can also acquire the initial image through other methods, which are not limited in Embodiment 1.

[0056] Hereinafter, any initial image obtained by the execution subject in Embodiment 1 will be denoted as image P.

[0057] In Example 1, the choice of which road sign or road sign to designate as the sign of interest can be set as needed. Generally, the sign of interest is a polygonal road sign, and the area occupied by the sign of interest in image P is also a polygonal region. Specifically, the sign of interest referred to in Example 1 can refer to the area of ​​the road sign to be processed that is outlined on the image being processed using a rectangle, circle, ellipse, irregular polygon, etc.

[0058] In reality, road signs are often partially obscured by other objects, such as tree branches. Figure 3 The image shows a road sign being obscured by tree branches.

[0059] After acquiring image P, the execution entity in Implementation Example 1 can detect (e.g., using an object detection algorithm) the area occupied by the road sign of interest in the image. This area is the portion of image P occupied by the road sign of interest that is displayed. For example... Figure 4 In the image, the road sign of interest is a rectangular sign above the road. Because the lower right part of the sign is obscured by tree branches, the area occupied by the road sign of interest resembles a trapezoid. If the area occupied by the road sign of interest is extracted separately, it looks like... Figure 5 As shown.

[0060] Generally, occluded edges are represented by straight lines so that the area occupied by the road sign of interest is a polygon. Hereinafter, the area occupied by the road sign of interest in image P will be referred to as the "road sign of interest region".

[0061] In reality, if the road sign of interest is rectangular, its sides include both horizontal and vertical types. Therefore, the angle between the edge of the road sign region in image P and the ground plane should be 0 degrees (or close to 0 degrees) or 90 degrees (or close to 90 degrees). If the road sign of interest is an equilateral triangular road sign, its sides include both horizontal and inclined types. Therefore, the angle between the edge of the road sign region in image P and the ground plane should be 0 degrees (or close to 0 degrees) or 60 degrees (or close to 60 degrees). If the road sign of interest is other polygonal shapes, the angle between its edge and the ground plane can also be calculated, and thus the angle between the edge of the road sign region in image P and the ground plane can also be calculated. Examples are not listed in Implementation 1.

[0062] The execution entity in Implementation Example 1 can set regularization conditions. By determining whether the edges of the road sign region of interest in image P satisfy the regularization conditions, it can determine whether the edges of the road sign region of interest in image P are regular or regular (regular or regular has a similar meaning here, generally referring to edges of the same type as those in reality, without occlusion). That is, if an edge of the road sign region of interest in image P satisfies the regularization conditions, then the edge is regular; and / or, if an edge of the road sign region of interest in image P does not satisfy the regularization conditions, then the edge is not regular.

[0063] The regularization condition may include the angle between the edge and the ground plane falling within a preset range. That is, for any side of the road sign region of interest in image P, if the angle between the edge and the ground plane falls within the preset range, then the edge satisfies the regularization condition; or, if the angle between the edge and the ground plane does not fall within the preset range, then the edge does not satisfy the regularization condition. Furthermore, for edges that satisfy the regularization condition, it can be determined whether the edge is a horizontal edge, a vertical edge, or another type of edge. The following is a detailed explanation:

[0064] In Embodiment 1, the preset range can be a segmented preset range, where each segment is set based on the angle between each side of the road sign of interest in reality and the ground plane. An example is given below:

[0065] Example 1: For a rectangular road sign, the preset range can include two segments. One segment is set for horizontal types, such as the range of 0 degrees to a1 degrees, where a1 is greater than 0; the other segment is set for vertical types, such as the range of b degrees to 90 degrees, where b1 is less than 90 degrees. Then, for a certain edge of the road sign region of interest in image P, if the angle between the edge and the ground plane is within the range of 0 degrees to a1 degrees, it means that the edge meets the regularity condition and that the edge is a horizontal edge of the road sign of interest; if the angle between the edge and the ground plane is within the range of b1 degrees to 90 degrees, it means that the edge meets the regularity condition and that the edge is a vertical edge of the road sign of interest.

[0066] Example 2: For an equilateral triangle road sign, the preset range can include two segments. One segment is set for horizontal types, such as the range of 0 degrees to a2 degrees, where a2 is greater than 0; the other segment is set for inclined types, such as the range of b2 degrees to 60 degrees, where b2 is less than 60 degrees. Then, for a certain side of the road sign region of interest in image P, if the angle between the side and the ground plane is within the range of 0 degrees to a2 degrees, it means that the side meets the regularity condition and that the side is a horizontal side of the road sign of interest; if the angle between the side and the ground plane is within the range of b2 degrees to 60 degrees, it means that the side meets the regularity condition and that the side is an inclined side of the road sign of interest.

[0067] If the road sign of interest is in other polygonal shapes, then each segment within a preset range can be set according to the angle between each side of the road sign of interest and the ground plane in reality. Examples will not be listed one by one in Implementation 1.

[0068] Since the segments of the preset range are set for the angles between each side of the road sign of interest in reality and the ground plane, for any side of the polygonal region, as long as the side belongs to any segment of the preset range, it can be determined that the side meets the regularity condition. Furthermore, based on the preset range segment to which the angle between the side and the ground plane belongs, the type of the side (horizontal, vertical, or inclined) can be determined.

[0069] To improve the accuracy of determining whether an edge is horizontal, vertical, or inclined, a1 or a2 can be close to 0, b1 can be close to 90, and b2 can be close to 60. In short, the range of each segment within the preset range can be as small as possible to improve the accuracy of determining whether an edge is horizontal, vertical, or inclined.

[0070] For ease of calculation, if the angle between a side and the ground plane is obtuse, then the supplementary angle of the obtuse angle is taken as the angle between the side and the ground plane.

[0071] In Implementation Example 1, the executing entity can determine whether image P can be corrected based on whether the edges of the polygonal region occupied by the road sign of interest in image P meet the regularity condition.

[0072] In Example 1, determining whether the initial image can be used for road sign region correction based on whether the edges of the polygonal region (hereinafter referred to as "polygonal region") occupied by the road sign of interest in the initial image satisfy the regularity condition can include:

[0073] (1) If the area of ​​interest in the road sign region in image P has only one edge or only two adjacent edges that do not meet the regularity condition, then it is determined that the initial image can be used for road sign region correction;

[0074] And / or,

[0075] (2) If all sides of the road sign region of interest in image P satisfy the regularity condition, then it is determined that the initial image does not need to be corrected for the road sign region of interest;

[0076] And / or,

[0077] (3) If at least two non-adjacent edges of the road sign region of interest in image P do not meet the regularity condition, then the initial image is determined to be unable to perform road sign region correction.

[0078] It should be noted that "no need for region of interest (ROI) correction," "able to perform ROI correction," and "unable to perform ROI correction" are three different concepts. If all edges of the ROI in image P satisfy the regularity condition, it means that all edges of the ROI in image P are regular, and image P does not need ROI correction. If only one edge or only two adjacent edges of the ROI in image P do not satisfy the regularity condition, it means that image P needs ROI correction, and it is possible to perform ROI correction. If at least two non-adjacent edges of the ROI in image P do not satisfy the regularity condition, it means that image P needs ROI correction, but it is not possible to perform ROI correction.

[0079] S103: (Executing entity) If it is determined that the initial image can be corrected for the road sign region of interest, then the initial image is subjected to a road sign region correction operation; wherein, the road sign region correction operation on the initial image includes: if only one edge of the polygonal region does not meet the regularization condition, then the point to be adjusted in the two vertices connected to that edge is adjusted (i.e., corrected) so that all edges of the polygonal region meet the regularization condition; if only two adjacent edges of the polygonal region do not meet the regularization condition, then the common vertex of the two edges is the point to be adjusted, and the point to be adjusted is adjusted so that all edges of the polygonal region meet the regularization condition.

[0080] If the executing entity in Embodiment 1 determines that image P can be corrected for the road sign region of interest, then the executing entity in Embodiment 1 performs road sign region correction operation on image P. The road sign region correction operation on image P may include the contents described in 1.1 and / or 1.2:

[0081] 1.1 If only one edge (hereinafter referred to as "non-regular edge") in the region of interest in image P does not meet the regularity condition, then the points to be adjusted in the two vertices connected by that edge are adjusted so that all edges in the region of interest in image P meet the regularity condition.

[0082] In Embodiment 1, before adjusting the point to be adjusted among the two vertices connected by the irregular edge, the executing entity of Embodiment 1 can determine the point to be adjusted among the two vertices connected by the irregular edge. Specifically, determining the point to be adjusted among the two vertices connected by the irregular edge may include: taking the vertex with the larger error among the two vertices connected by the irregular edge as the point to be adjusted.

[0083] In Example 1, the edges of the road sign region of interest in image P correspond to the edges of the road signs of interest in reality (hereinafter referred to as "real-world edges"). For any edge of the road sign region of interest in image P, the "angle between the edge and the ground plane" may have a certain error compared to the "angle between the corresponding real-world edge and the ground plane". As long as the error is reasonable, that is, the angle between the edge and the ground plane is within a preset range, the edge can be judged to be regular. This is also the embodiment of the meaning of the aforementioned preset range. For example, if the angle between a certain edge in image P and the ground plane is 5 degrees, and the corresponding real-world edge is a horizontal edge (with an angle of 0 degrees with the ground plane), then the "angle between the edge and the ground plane" may have an error of 5 degrees compared to the "angle between the corresponding real-world edge and the ground plane".

[0084] For any adjacent edge of the aforementioned irregular edge, the error between the adjacent edge and its corresponding real edge represents the error between the adjacent edge and the common vertex of the irregular edge. Based on this, the error between the two (common) vertices connected by the irregular edge is determined, and the vertex with the larger error among the two vertices connected by the irregular edge is selected as the point to be adjusted.

[0085] In Embodiment 1, adjusting the point to be adjusted among the two vertices connected by the irregular edge may include: adjusting the point to be adjusted using the adjacent vertices of the point to be adjusted among the two vertices connected by the irregular edge.

[0086] The method of adjusting the point to be adjusted using its adjacent vertices may include: adjusting the coordinates of the point to be adjusted using the coordinates of two adjacent vertices.

[0087] Specifically, the area of ​​interest in the road sign in image P can be placed in a three-dimensional coordinate system. Using the coordinates of two adjacent vertices of the point to be adjusted, the coordinates of the point to be adjusted can be adjusted or determined. Assume the area of ​​interest in the road sign in image P is as follows: Figure 6 As shown, the area of ​​interest for the road sign has four sides: a, b, c, and d. Side b is an irregular edge, and vertex bc (the common vertex of sides b and c) is the point to be adjusted. The adjustment of vertex bc includes:

[0088] The adjusted x-coordinate value of vertex bc is equal to vertex cd.x, which means that the x-coordinate value of vertex cd (i.e., the common vertex of edge c and edge d) is used as the adjusted x-coordinate value of vertex bc.

[0089] The adjusted y-coordinate value of vertex bc is equal to the y-coordinate value of vertex cd.y, which means that the y-coordinate value of vertex cd is used as the adjusted y-coordinate value of vertex bc.

[0090] The adjusted z-coordinate value of vertex bc is equal to vertex ab.z, which means that the z-coordinate value of vertex ab (the common vertex of edge a and edge b) is used as the adjusted z-coordinate value of vertex bc.

[0091] 1.2 If only two adjacent edges of the road sign region of interest in image P do not meet the regularity condition, then the common vertex of the two edges is the point to be adjusted. The point to be adjusted is adjusted so that all edges of the polygon region meet the regularity condition.

[0092] In section 1.2, adjusting the point to be adjusted may include: adjusting the point to be adjusted using the adjacent vertices of the point to be adjusted.

[0093] In section 1.2, adjusting the point to be adjusted using its adjacent vertices may include: adjusting the coordinates of the point to be adjusted using the coordinates of two adjacent vertices.

[0094] Specifically, the area of ​​interest in the road sign in image P can be placed in a three-dimensional coordinate system. Using the coordinates of two adjacent vertices of the point to be adjusted, the coordinates of the point to be adjusted can be adjusted or determined. Assume the area of ​​interest in the road sign in image P is as follows: Figure 7 As shown, the area of ​​interest for the road sign has four edges: e, f, g, and h. Edges f and g are irregular edges, and vertex fg (the common vertex of edges f and g) is the point to be adjusted. The adjustment of vertex fg includes:

[0095] The adjusted x-coordinate value of vertex fg is equal to vertex gh.x, which means that the x-coordinate value of vertex gh (i.e., the common vertex of edge g and edge h) is used as the adjusted x-coordinate value of vertex fg.

[0096] The adjusted y-coordinate value of vertex fg is equal to the y-coordinate value of vertex gh.y, which means that the y-coordinate value of vertex gh is used as the adjusted y-coordinate value of vertex fg.

[0097] The adjusted z-coordinate value of vertex fg is equal to vertex ef.z, which means that the z-coordinate value of vertex ef (i.e., the common vertex of edge e and edge f) is used as the adjusted z-coordinate value of vertex fg.

[0098] Whether it's 1.1 or 1.2, after the adjustments made to the point to be adjusted, the coordinates of the adjusted point match those of its two adjacent vertices. The irregular edges are also corrected as the point to be adjusted is adjusted, thus satisfying the regularity condition. In this way, all edges of the region occupied by the road sign of interest in image P (the region occupied by the road sign of interest changes with the adjustment of the point to be adjusted; here, it refers to the changed region occupied by the road sign of interest) satisfy the regularity condition, achieving the correction of the region occupied by the road sign of interest in image P.

[0099] The following examples further illustrate the effects of Example 1:

[0100] Figure 8 Examples of two road sign region correction operations are shown in Case 1 and Case 2. Both Case 1 and Case 2 use the lower right vertex as the point to be adjusted. In Case 1, from left to right, the images show the initial image, the area occupied by the road sign of interest in the initial image, the area occupied by the road sign of interest in the initial image after road sign region correction, and a comparison of the areas occupied by the road sign of interest in the initial image before and after the correction. In Case 2, from left to right, the images show the initial image, the area occupied by the road sign of interest in the initial image, the area occupied by the road sign of interest in the initial image after road sign region correction, and a comparison of the areas occupied by the road sign of interest in the initial image before and after the correction. The effect data before and after the road sign region correction operation are shown in Table 1.

[0101] Serial Number True value of road sign diameter (m) Diameter of the road sign before correction (m) Error before correction (%) Diameter of the road sign after correction (m) Corrected error (%) 1 11.7025 12.3804 5.79288 12.0275 2.7769 2 11.5988 11.1668 3.72429 11.3373 2.25479

[0102] Table 1

[0103] In Table 1, number 1 represents case 1, number 2 represents case 2, the true value of the road sign diameter represents the perimeter of the road sign of interest in reality, the road sign diameter before correction represents the perimeter of the area occupied by the road sign of interest in the initial image before the road sign region correction operation, the error before correction represents the error between "the true value of the road sign before correction representing the perimeter of the area occupied by the road sign of interest in the initial image before the road sign region correction operation" and "the perimeter of the road sign of interest in reality", and the road sign diameter after correction represents the perimeter of the area occupied by the road sign of interest in the initial image after the road sign region correction operation, and the error after correction represents the error between "the perimeter of the area occupied by the road sign of interest in the initial image after the road sign region correction operation" and "the perimeter of the road sign of interest in reality". As shown in Table 1, the road sign region correction operation can effectively reduce the error between the area occupied by the road sign of interest in the initial image and the actual road sign of interest.

[0104] In Example 1, by correcting the area occupied by the road sign of interest in the image, the integrity and regularity of the area occupied by the road sign of interest in the image can be effectively improved. It also makes the area occupied by the adjusted road sign of interest in the image more consistent with the actual shape of the road sign of interest in reality, thereby improving the accuracy of the area occupied by the road sign of interest in the image.

[0105] The second embodiment of this specification (hereinafter referred to as "Embodiment Two") provides a method for recognizing road signs in images. The executing entity of Embodiment Two can be a terminal (including but not limited to mobile phones, computers, tablets, and televisions), a server, an operating system, an application, a road sign recognition platform in images, or a road sign recognition system in images, etc. That is, the executing entity can be diverse and can be set, used, or changed as needed. Alternatively, a third-party application can assist the executing entity in performing Embodiment Two. For example... Figure 1 As shown, the image road sign recognition method in Embodiment 2 can be executed by the server, and a corresponding application can be installed on the terminal (held by the user). Data can be transmitted between the terminal or the application and the server. Data can be collected, input, or output, or pages or information can be processed (to the user) through the terminal or the application, thereby assisting the server in executing the image road sign recognition method in Embodiment 2.

[0106] like Figure 9 As shown, the image road sign recognition method provided in Embodiment 2 includes:

[0107] S202: (Executing entity) After obtaining a set of initial images containing road signs of interest, for any of the initial images, execute the road sign region correction method in the image described in Embodiment 1;

[0108] The content or meaning of "road sign of interest" in Example 2 is the same as in Example 1.

[0109] The executing entity in Embodiment 2 can acquire a set of initial images, each containing the same road sign of interest. Acquiring the initial images can include: capturing a set of initial images using an image capturing device and sending the captured initial images to the executing entity in Embodiment 2; or, the executing entity in Embodiment 2 can have an image capturing function and capture a set of initial images; or, it can decompose a video into frames, each frame containing an image of the road sign of interest as the initial images (this decomposition can be performed by the executing entity in Embodiment 2 or by another entity, and the resulting images can be sent to the executing entity in Embodiment 2). The executing entity in Embodiment 2 can also acquire the initial images through other methods, which are not limited in Embodiment 2.

[0110] Specifically, the initial images in the "set of initial images" can be arranged sequentially to form an initial image sequence, for example, according to the shooting time of the initial images. Alternatively, if the initial images are frames obtained from video decomposition, then the individual frames are ordered, allowing the initial images to be arranged sequentially to form an initial image sequence.

[0111] For any initial image, the execution entity in Embodiment 2 can perform the image road sign region correction method described in Embodiment 1 on that initial image. That is, the initial image is used as image P in Embodiment 1, and the image road sign region correction method described in Embodiment 1 is executed.

[0112] In the initial images obtained in Example 2, Example 1 can be used to determine whether there are initial images that do not require correction of the road sign region of interest, whether there are initial images that can undergo correction of the road sign region of interest, and whether there are initial images that cannot undergo correction of the road sign region of interest. For the initial images that can undergo correction of the road sign region of interest, the road sign region correction operation is performed. For the initial images that have undergone road sign region correction, the edges of the regions occupied by the road signs of interest satisfy the regularity condition.

[0113] S204: (Executing entity) takes the initial image that has undergone road sign region correction and the initial image that does not need road sign region correction as target images to form a target image sequence; and determines whether the target image sequence is usable based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in the target image sequence.

[0114] In Example 2, the executing entity can use an initial image that has already undergone road sign region correction (or there may not be an initial image capable of road sign region correction) and an initial image that does not require road sign region correction (or there may not be an initial image that does not require road sign region correction) as target images to form a target image sequence. Since the initial images are already ordered, and the target images themselves are either initial images (if road sign region correction is not required) or initial images that have undergone road sign region correction (if road sign region correction is possible), the target images can follow the sequence number of the initial images or be ordered according to the sequence number of the initial images, thereby forming a target image sequence.

[0115] After obtaining the target image sequence, the execution entity in Embodiment 2 can determine whether the target image sequence is usable based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image.

[0116] The determination of whether the target image sequence is usable based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in the target image sequence may include: if the area difference between the area occupied by the road sign of interest in the first image (if a road sign region correction operation was performed on the first image, then the area occupied by the road sign of interest here refers to the area occupied by the road sign of interest after the road sign region correction operation) and the area occupied by the road sign of interest in the last image (if a road sign region correction operation was performed on the last image, then the area occupied by the road sign of interest here refers to the area occupied by the road sign of interest after the road sign region correction operation) in the target image sequence meets a preset condition, then the target image sequence is determined to be usable.

[0117] In Embodiment 2, the executing entity can perform IOU calculations on the regions occupied by the road signs of interest in the first image and the last image of the target image sequence, and determine the area difference between the regions occupied by the road signs of interest in the first image and the last image based on the IOU calculation results. Furthermore, the executing entity in Embodiment 2 can use the IOU calculation results as the area difference between the regions occupied by the road signs of interest in the first image and the last image.

[0118] The IOU calculation process may include: calculating the intersection area of ​​the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image; and calculating the union area of ​​the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image. The intersection area is then divided by the union area, and the result is the IOU calculation result. When calculating the intersection and union areas, the first and last images can be placed in the same coordinate system. For example... Figure 10 As shown, Figure 10 The upper part (molecule) represents the intersection area. Figure 10 The lower part (denominator) is the union area. Dividing the intersection area by the union area yields the IOU calculation result.

[0119] In Example 2, the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image may satisfy a preset condition, which may include: the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image is not greater than a preset value (for example, the preset value may be 80%).

[0120] In Example 2, under the condition that the computing power of the execution subject in Example 2 is relatively stable, if the area difference between the area occupied by the road sign of interest in the first image (if the road sign region correction operation was performed before the first image, then the area occupied by the road sign of interest here refers to the area occupied by the road sign of interest after the road sign region correction operation) and the area occupied by the road sign of interest in the last image (if the road sign region correction operation was performed before the last image, then the area occupied by the road sign of interest here refers to the area occupied by the road sign of interest after the road sign region correction operation) meets the preset condition, it means that if there is occlusion of the road sign of interest in the initial image corresponding to each target image, then the occlusion scene or occlusion form (including the number of occluded vertices) is basically consistent from beginning to end (i.e. from the initial image corresponding to the first target image to the initial image corresponding to the last target image), so the shape of the road sign of interest identified based on each target image will not be much different, and the target image sequence is usable.

[0121] S206: (Executing entity) If the target image sequence is available, then generate the recognition result of the road sign of interest based on the area occupied by the road sign of interest in each image of the target image sequence.

[0122] If the target image sequence is determined to be usable, the execution entity in Embodiment 2 can generate the recognition result of the road sign of interest based on the area occupied by the road sign of interest in each image of the target image sequence.

[0123] In Example 2, generating the recognition result of the road sign of interest based on the region occupied by the road sign of interest in each image of the target image sequence may include:

[0124] From the target image sequence, each pair of adjacent target images is grouped together, and the average coordinates of the area occupied by the road sign of interest in each group of target images are calculated. For example, if the target image sequence includes target images from x1 to xi, then x1 and x2 are grouped together, x3 and x4 are grouped together, and so on, with x(i-1) and xi being grouped together. If the number of target images is odd, then one group may contain only one target image.

[0125] The recognition result of the road sign of interest is generated based on the average coordinates of the area occupied by the road sign of interest in each group of target images. This generation of the recognition result can include averaging the average coordinates of the areas occupied by the road signs of interest in each group of target images, using this average as the recognition result. Specifically, after averaging the average coordinates of the areas occupied by the road signs of interest in each group of target images, the average value of the average coordinates of the areas occupied by the road signs of interest in each group of target images can be used as the final coordinate value of the road sign of interest. The final area occupied by the road sign of interest is then identified based on this final coordinate value, and this final area is used as the recognition result of the road sign of interest.

[0126] All the coordinates mentioned above can be three-dimensional coordinates in the same coordinate system.

[0127] The following is an example of the algorithm steps in Example 2:

[0128] Given a set of initial images, determine whether the edges of the polygonal region occupied by the road sign of interest in each initial image satisfy the regularity condition. If the edges of the polygonal region occupied by the road sign of interest in each initial image satisfy the regularity condition, return True; otherwise, return False.

[0129] If it returns True, it means that the road signs of interest in this initial set of images are relatively regular; if it returns False, it means that some road signs of interest in this set of images are irregular.

[0130] For any initial image, determine whether the edges of the polygon region occupied by the road sign of interest in the initial image satisfy the regularity condition. If the regularity condition is satisfied, return 1; otherwise, return 0. Then, store the judgment results in a list in order (for example, starting from the top left vertex, traverse each edge clockwise and store them in the list in order) to obtain the number of edges that do not satisfy the regularity condition.

[0131] Sum the values ​​in the list.

[0132] If the summation result is the same as the number of edges of the road sign of interest in reality, it means that the edges of the area occupied by the road sign of interest in the initial image are regular.

[0133] If the summation result is the number of sides of the road sign of interest minus 1, it indicates that one side of the polygonal region occupied by the road sign of interest in the initial image is irregular (since the sides in the list are stored in order, it can be determined which side is irregular), and a road sign region correction operation is needed. The point to be adjusted on the irregular side is identified, and the coordinates of the point to be adjusted are adjusted using the coordinates of two adjacent vertices of the point to be adjusted.

[0134] If the summation result is the number of sides of the road sign of interest minus 2, it indicates that two sides of the polygonal region occupied by the road sign of interest in the initial image are irregular. Determine the relative positions of the two irregular sides (since the sides in the list are stored in order, it's possible to identify which two sides are irregular). If the two irregular sides are adjacent, their common vertex is the point to be adjusted. The coordinates of the point to be adjusted are then adjusted using the coordinates of its two adjacent vertices. Alternatively, if the two irregular sides are not adjacent, road sign region correction cannot be performed. The initial image number for which road sign region correction cannot be performed can be added to the `not_regular` list.

[0135] If the summation result shows that the number of sides of the road sign of interest is reduced by 3 or less, it means that the road sign area correction operation cannot be performed. The initial image sequence number that cannot be used for road sign area correction operation can be added to the not_regular list.

[0136] If the number of elements in the not_regular list is equal to the number of images in the initial set, it means that the areas occupied by the road signs of interest in the initial set are not regular and none of them can be corrected. In this case, the initial set of images should be discarded.

[0137] If the number of elements in the not_regular list is less than the number of images in the initial image set, then it is determined whether the area difference between the area occupied by the road sign in the first image and the area occupied by the road sign in the last image in the target image sequence consisting of the initial image that has undergone road sign area correction and the initial image that does not need road sign area correction meets the preset condition.

[0138] If the preset conditions are met, it means that if there is occlusion of the road sign of interest in the initial image corresponding to each target image in the target image sequence, then the occlusion scene or occlusion form (including the number of occluded vertices) is similar from beginning to end (i.e., from the initial image corresponding to the first target image to the initial image corresponding to the last target image) (e.g., only one vertex is occluded in each image), and the target image sequence is usable; if the preset conditions are not met, it means that the occlusion scene or occlusion form is not similar in the initial image corresponding to each target image in the target image sequence (e.g., multiple vertices are occluded in the initial image corresponding to each target image, and the occluded vertices may be different, or the number of occluded vertices may be different), and the target image sequence is discarded.

[0139] If the target image sequence is available, the recognition result of the road sign of interest is generated based on the area occupied by the road sign of interest in each image of the target image sequence.

[0140] In Example 2, by correcting the area occupied by the road sign of interest in the image, the integrity and regularity of the area occupied by the road sign of interest in the image can be effectively improved. It also makes the area occupied by the adjusted road sign of interest in the image more consistent with the actual shape of the road sign of interest in reality, thereby improving the accuracy of the area occupied by the road sign of interest in the image.

[0141] For a set of initial images containing road signs of interest, both the initial image with road sign region correction and the initial image without road sign region correction are used as target images. This improves the accuracy of identifying the area occupied by the road sign of interest in each target image, and the road sign recognition result obtained based on the target image sequence more closely matches the actual shape of the road sign of interest in reality. By employing multiple methods, including determining whether the initial image can undergo road sign region correction, the road sign region correction operation itself, and the usability of the target image sequence, random errors between target images and random errors in road sign recognition can be effectively reduced, thereby improving the accuracy of road sign recognition.

[0142] like Figure 11 As shown, the third embodiment of this specification provides a road sign region correction device in an image corresponding to the road sign region correction method in Embodiment 1, comprising:

[0143] The region determination module 301 is used to determine whether the initial image can be corrected for the road sign region of interest based on whether the edges of the polygonal region occupied by the road sign of interest in the initial image meet the regularity condition after acquiring the initial image.

[0144] The region correction module 303 is used to perform a road sign region correction operation on the initial image if it is determined that the initial image can be corrected for the road sign region of interest.

[0145] The road sign region correction operation on the initial image includes: if only one edge of the polygonal region does not meet the regularization condition, then the adjustment point among the two vertices connected to that edge is adjusted so that all edges of the polygonal region meet the regularization condition; if only two adjacent edges of the polygonal region do not meet the regularization condition, then the common vertex of the two edges is the adjustment point, and the adjustment point is adjusted so that all edges of the polygonal region meet the regularization condition.

[0146] Optionally, based on whether the edges of the polygonal region occupied by the road sign of interest in the initial image satisfy the regularity condition, it is determined whether the initial image can be used for road sign region correction, including:

[0147] If the polygonal region has only one edge or only two adjacent edges that do not meet the regularity condition, then it is determined that the initial image can be used for the correction of the road sign region of interest.

[0148] And / or,

[0149] If all sides of the polygonal region satisfy the regularity condition, then it is determined that the initial image does not need to be corrected for the road sign region of interest.

[0150] And / or,

[0151] If at least two non-adjacent sides of the polygonal region do not meet the regularity condition, then the initial image is determined to be unable to perform the correction of the road sign region of interest.

[0152] Optionally, the region correction module 303 is further configured to use the vertex with the larger error among the two vertices connected by the edge as the point to be adjusted.

[0153] Optionally, adjusting the point to be adjusted among the two vertices connected by the edge includes:

[0154] For the point to be adjusted among the two vertices connected by the side, the adjacent vertices of the point to be adjusted are used to adjust the point to be adjusted;

[0155] or,

[0156] Adjusting the point to be adjusted includes:

[0157] The point to be adjusted is adjusted using the adjacent vertices of the point to be adjusted.

[0158] Optionally, the point to be adjusted is adjusted using its adjacent vertices, including:

[0159] The coordinates of the point to be adjusted are adjusted using the coordinates of two adjacent vertices of the point to be adjusted.

[0160] Optionally, the regularization condition includes the angle with the ground plane falling within a preset range.

[0161] like Figure 12 As shown, the fourth embodiment of this specification provides an image road sign recognition device corresponding to the image road sign recognition method described in Embodiment 2, comprising:

[0162] The region correction module 402 is used to obtain a set of initial images containing road signs of interest, and then, for any one of the initial images, execute the road sign region correction method in the image described in Embodiment 1.

[0163] The image filtering module 404 is used to select the initial image that has undergone road sign region correction and the initial image that does not need to undergo road sign region correction as target images to form a target image sequence; and to determine whether the target image sequence is usable based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in the target image sequence.

[0164] The image recognition module 406 is used to generate the recognition result of the road sign of interest based on the area occupied by the road sign of interest in each image of the target image sequence if the target image sequence is available.

[0165] Optionally, the usability of the target image sequence is determined based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image, including:

[0166] If the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image of the target image sequence meets a preset condition, then the target image sequence is determined to be usable.

[0167] Optionally, the image filtering module 404 is further configured to perform IOU calculation on the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in the target image sequence, and determine the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image based on the IOU calculation result.

[0168] Optionally, the area difference between the region of interest occupied by the road sign in the first image and the region of interest occupied by the road sign in the last image satisfies preset conditions, including:

[0169] The area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image is no greater than a preset value.

[0170] Optionally, based on the region occupied by the road sign of interest in each image of the target image sequence, the recognition result of the road sign of interest is generated, including:

[0171] From the target image sequence, each pair of adjacent target images is grouped together, and the average coordinates of the area occupied by the road sign of interest in each group of target images are calculated;

[0172] The recognition result of the road sign of interest is generated based on the average coordinate result of the area occupied by the road sign of interest in each group of target images.

[0173] Optionally, based on the average coordinates of the area occupied by the road sign of interest in each group of target images, the recognition result of the road sign of interest is generated, including:

[0174] The average coordinates of the regions occupied by the road signs of interest in each group of target images are taken as the recognition result of the road signs of interest.

[0175] The fifth embodiment of this specification provides a road sign region correction device in an image, comprising:

[0176] At least one processor;

[0177] as well as,

[0178] A memory that is communicatively connected to the at least one processor;

[0179] in,

[0180] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the image road sign region correction method described in Embodiment 1.

[0181] The sixth embodiment of this specification provides a road sign recognition device in an image, comprising:

[0182] At least one processor;

[0183] as well as,

[0184] A memory that is communicatively connected to the at least one processor;

[0185] in,

[0186] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the road sign recognition method in the image described in Embodiment 2.

[0187] The seventh embodiment of this specification provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image road sign region correction method described in Embodiment 1 or the image road sign recognition method described in Embodiment 2.

[0188] The above embodiments can be used in combination, and modules with the same name in different embodiments or within the same embodiment can be the same or different modules.

[0189] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0191] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and non-volatile computer storage medium will not be repeated here.

[0192] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0193] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0194] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0195] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0196] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0197] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0198] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0200] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0201] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0202] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0203] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0204] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0205] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0206] The above description is merely an embodiment of this specification and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for road sign region rectification in an image, comprising: after obtaining an initial image, determining whether the initial image can be subjected to road sign region rectification according to whether the edges of a polygonal region occupied by a road sign of interest in the initial image satisfy a regularity condition; if it is determined that the initial image can be subjected to road sign region rectification, performing road sign region rectification on the initial image; wherein the road sign region rectification on the initial image comprises: if only one edge of the polygonal region does not satisfy the regularity condition, adjusting a to-be-adjusted point among the two vertices connected by the edge so that all the edges of the polygonal region satisfy the regularity condition; if only two adjacent edges of the polygonal region do not satisfy the regularity condition, a common vertex of the two edges is a to-be-adjusted point, and the to-be-adjusted point is adjusted so that all the edges of the polygonal region satisfy the regularity condition. 2.The method of claim 1, wherein the determination of whether the initial image can be subjected to road sign region rectification according to whether the edges of a polygonal region occupied by a road sign of interest in the initial image satisfy a regularity condition comprises: if only one edge or only two adjacent edges of the polygonal region do not satisfy the regularity condition, it is determined that the initial image can be subjected to road sign region rectification; and / or, if all the edges of the polygonal region satisfy the regularity condition, it is determined that the initial image does not need to be subjected to road sign region rectification; and / or, if at least two non-adjacent edges of the polygonal region do not satisfy the regularity condition, it is determined that the initial image cannot be subjected to road sign region rectification. 3.The method of claim 1, wherein before adjusting the to-be-adjusted point among the two vertices connected by the edge, the method further comprises: taking the vertex with a greater error among the two vertices connected by the edge as the to-be-adjusted point. 4.The method of any one of claims 1 to 3, wherein the adjustment of the to-be-adjusted point among the two vertices connected by the edge comprises: adjusting the to-be-adjusted point among the two vertices connected by the edge using the adjacent vertices of the to-be-adjusted point; or, the adjustment of the to-be-adjusted point comprises: adjusting the to-be-adjusted point using the adjacent vertices of the to-be-adjusted point. 5.The method of claim 4, wherein the adjustment of the to-be-adjusted point using the adjacent vertices of the to-be-adjusted point comprises: adjusting the coordinates of the to-be-adjusted point using the coordinates of the two adjacent vertices of the to-be-adjusted point. 6.The method of claim 1, wherein the regularity condition comprises that the included angle with the ground plane belongs to a preset range. 7.A method for road sign recognition in an image, comprising: after obtaining a set of initial images containing a road sign of interest, performing the method for road sign region rectification in an image of any one of claims 1 to 6 on any initial image. The initial image subjected to the road sign region correction operation and the initial image not subjected to the road sign region correction are taken as target images to form a target image sequence; whether the target image sequence is available is determined according to the area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image in the target image sequence; If the target image sequence is available, the recognition result of the road sign of interest is generated according to the region occupied by the road sign of interest in each image in the target image sequence. 8.The method of claim 7, wherein whether the target image sequence is available is determined according to the area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image in the target image sequence, comprising: If the area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image in the target image sequence meets a preset condition, it is determined that the target image sequence is available. 9.The method of claim 8, further comprising: performing IOU calculation on the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image in the target image sequence, and determining the area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image according to the IOU calculation result. 10.The method of claim 8 or 9, wherein the area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image meets the preset condition, comprising: The area difference between the region occupied by the road sign of interest in the first image and the region occupied by the road sign of interest in the last image is not greater than a preset value. 11.The method of claim 7, wherein the recognition result of the road sign of interest is generated according to the region occupied by the road sign of interest in each image in the target image sequence, comprising: calculating the average coordinate result of the region occupied by the road sign of interest in each group of target images from the target image sequence, each group of target images being two adjacent target images; generating the recognition result of the road sign of interest according to the average coordinate result of the region occupied by the road sign of interest in each group of target images. 12.The method of claim 11, wherein the recognition result of the road sign of interest is generated according to the average coordinate result of the region occupied by the road sign of interest in each group of target images, comprising: averaging the average coordinate result of the region occupied by the road sign of interest in each group of target images to obtain the recognition result of the road sign of interest. 13.An apparatus for road sign region correction in an image, comprising: a region determination module configured to determine whether an initial image can be subjected to road sign region correction according to whether the edges of a polygonal region occupied by a road sign of interest in the initial image meet a regularity condition after the initial image is obtained; a region correction module configured to perform a road sign region correction operation on the initial image if it is determined that the initial image can be subjected to road sign region correction. If only one side of the polygonal region does not satisfy the regularity condition, the point to be adjusted among the two vertices connected by the side is adjusted so that each side of the polygonal region satisfies the regularity condition. If only two adjacent sides of the polygonal region do not satisfy the regularity condition, the common vertex of the two sides is the point to be adjusted, and the point to be adjusted is adjusted so that each side of the polygonal region satisfies the regularity condition.

14. An apparatus for recognizing a road sign in an image, comprising: a region correction module configured to, after obtaining a group of initial images containing a road sign of interest, perform the method for correcting a road sign region in an image according to any one of claims 1 to 6 on any initial image; an image screening module configured to, based on the area difference between the area occupied by the road sign of interest in the first image and the area occupied by the road sign of interest in the last image in a target image sequence composed of initial images on which the road sign region correction operation is performed and initial images on which the road sign region correction operation is not needed, determine whether the target image sequence is usable; an image recognition module configured to, if the target image sequence is usable, generate a recognition result of the road sign of interest based on the area occupied by the road sign of interest in each image in the target image sequence.

15. An apparatus for correcting a road sign region in an image, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for correcting a road sign region in an image according to any one of claims 1 to 6.

16. An apparatus for recognizing a road sign in an image, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for recognizing a road sign in an image according to any one of claims 7 to 12.

17. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method for correcting a road sign region in an image according to any one of claims 1 to 6 or the method for recognizing a road sign in an image according to any one of claims 7 to 12.

Images