Image multi-target information recognition method, system and medium

Abstract

The present invention discloses a method, system and medium for identifying multi-target information in an image. The method comprises: obtaining a grayscale image, drawing the minimum circumscribed rectangle of each area, selecting the minimum circumscribed rectangle that meets the preset aspect ratio and area ratio, and using the corresponding area as the target area; for each target area, performing angle correction on the image at its center, and intercepting a first rectangular area centered on the target area in the corrected image as an image to be processed; for each image to be processed, detecting its edge intersection points, performing perspective transformation on the image to be processed according to the edge intersection points, and obtaining a second rectangular area as an image to be identified; performing character recognition on each image to be identified, and obtaining the information identification result of each target area. The present invention can accurately identify multiple target areas in an image, and can effectively correct the image of each target area, thereby improving the accuracy of subsequent information recognition.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an image multi-target information recognition method, system and medium. Background Art

[0002] With the development of the industrial Internet and the continuous maturity of image recognition and processing technology, automated digital information character recognition based on image processing methods has gradually become an important part of industrial automation. For example, remote automatic meter reading technology uses a camera to obtain the dial image and recognizes the information on the dial through image processing. The meter can be a water meter or electricity meter in the form of a wheel, or a pressure gauge with a pointer test. Another example is license plate recognition technology, which recognizes license plate information by processing the monitoring screen.

[0003] The general steps of image recognition include grayscale processing, binarization, detection, correction and interception of recognition areas, segmentation of individual characters, and recognition of individual characters. In order to accurately recognize character information, detection, correction and interception of recognition areas are critical. Accurate detection, correction and interception of recognition targets are the prerequisites for accurate recognition of character information.

[0004] In the current solutions for image content recognition, generally only information recognition is performed on a single target in the image. For example, patent CN110414517A discloses an ID card text recognition algorithm, which recognizes the content of a single ID card image. However, in actual situations, there is often more than one target for which information needs to be obtained. For example, in a large-scale monitoring or production system, there are generally multiple display screens. If the information of a single screen image is recognized in turn according to the prior art, it will obviously cause a waste of time. In addition, since the installation position of each display screen is different, the angle deviations of these display screens in the image are different, which may lead to low character recognition accuracy during information recognition. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides an image multi-target information recognition method, system and medium, which can accurately identify multiple target areas in an image, and can effectively correct the image of each target area, thereby improving the accuracy of subsequent information recognition.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0007] A method for identifying multi-target information in an image comprises the following steps:

[0008] S1) obtaining a preprocessed image, dividing the preprocessed image into regions, and drawing a minimum bounding rectangle for each region, screening a minimum bounding rectangle that meets a preset aspect ratio and area ratio, and taking the region corresponding to the screened minimum bounding rectangle as a target region;

[0009] S2) for each target area, the image is angle-corrected with the center of the target area, and an image of a first rectangular area with the target area as the center is intercepted from the corrected image as the image to be processed of the target area;

[0010] S3) for each target area's image to be processed, detecting edge intersections in the image to be processed of the target area, performing perspective transformation on the image to be processed according to the edge intersections, and obtaining an image of a second rectangular area as the image to be identified of the target area;

[0011] S4) performing character recognition on each image to be recognized to obtain information recognition results for each target area.

[0012] Furthermore, the step of drawing the minimum bounding rectangle of each area in the image in step S1) specifically includes:

[0013] S111) performing Gaussian filtering and binarization processing on the image to obtain a first binarized image;

[0014] S112) performing morphological erosion and dilation processing on the first binary image to obtain a first morphologically processed image;

[0015] S113) performing edge detection on the first morphologically processed image to obtain edge data of each region;

[0016] S114) Draw the minimum circumscribed rectangle of each area according to the edge data.

[0017] Furthermore, the step of selecting the minimum bounding rectangle that meets the preset aspect ratio and area ratio in step S1) specifically includes:

[0018] S121) selecting a current minimum bounding rectangle, calculating the length and width of the current minimum bounding rectangle according to the number of pixels in the length direction and the width direction of the current minimum bounding rectangle, and calculating the area of ​​the current minimum bounding rectangle according to the length and the width;

[0019] S122) if the error between the aspect ratio of the current minimum bounding rectangle and the preset aspect ratio is less than the preset error, and the area proportion of the current minimum bounding rectangle in the image is greater than or equal to a preset threshold s, then the area corresponding to the current minimum bounding rectangle is the target area;

[0020] S123) Return to step S121) until all minimum bounding rectangles in the image are traversed.

[0021] Furthermore, step S123) also includes a step of adjusting the size of the threshold s, specifically: adjusting the value of the threshold s to the area ratio of the minimum enclosing rectangle corresponding to the actual target area with the smallest area in the current image, and screening and retaining the minimum enclosing rectangle whose area ratio is greater than or equal to the adjusted threshold s.

[0022] Furthermore, before step 2), a step of verifying the target area is also included, which specifically includes: intercepting the image surrounded by the minimum circumscribed rectangle corresponding to all target areas, performing text recognition on each intercepted image respectively, and if there is no recognition information or the recognition is wrong, the corresponding target area will be eliminated.

[0023] Furthermore, step S1) also includes: establishing a coordinate system with the horizontal direction as the horizontal axis and the vertical direction as the longitudinal axis, obtaining the coordinates of the center point of the minimum enclosing rectangle, and calculating the inclination direction and inclination angle of the minimum enclosing rectangle relative to the horizontal axis based on the long side of the minimum enclosing rectangle and the projection of the long side on the horizontal axis; step S2) performs angle correction on the image with the center of the target area, specifically including: taking the center point of the minimum enclosing rectangle corresponding to the target area as the rotation center, taking the value of the inclination angle of the minimum enclosing rectangle corresponding to the target area as the rotation angle value, and rotating the image in the opposite direction of the inclination direction according to the rotation center and the rotation angle value, so that the inclination angle of the minimum enclosing rectangle corresponding to the target area relative to the horizontal axis is 0.

[0024] Furthermore, in step S2), intercepting an image of a first rectangular area centered on the target area in the corrected image specifically includes: obtaining the center point of the minimum circumscribed rectangle corresponding to the target area, calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the center point and the preset expansion value, and then intercepting the image of the first rectangular area according to the length and width of the first rectangular area starting from the interception starting point.

[0025] Further, calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the center point and the preset expansion value specifically includes: calculating the coordinate value of the vertex of the first rectangular area according to the coordinate value of the center point of the minimum circumscribed rectangle and the preset expansion value, taking the vertex of the first rectangular area as the interception starting point, and the coordinate value function expression of the vertex of the first rectangular area is:

[0026]

[0027] In the above formula, x e ,y e are the horizontal and vertical coordinates of the vertices of the first rectangular area, respectively,c ,y c are the horizontal and vertical coordinates of the center point of the minimum bounding rectangle, W is the length of the minimum bounding rectangle, H is the width of the minimum bounding rectangle, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image;

[0028] The length and width of the first rectangular area are calculated according to the size of the minimum circumscribed rectangle and the preset expansion value. The length and width function expressions of the first rectangular area are:

[0029]

[0030] In the above formula, W e ,H e are the length and width of the vertices of the first rectangular area respectively, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image.

[0031] The present invention also proposes an image multi-target information recognition system, comprising:

[0032] The target screening unit is used to obtain the preprocessed image, divide the preprocessed image into regions, draw the minimum bounding rectangle of each region, screen the minimum bounding rectangle that meets the preset aspect ratio and area ratio, and use the area corresponding to the screened minimum bounding rectangle as the target area;

[0033] The first image correction unit is used to perform angle correction on the image with respect to each target area, respectively, with respect to the center of the target area, and intercept an image of a first rectangular area with respect to the target area as the image to be processed of the target area from the corrected image;

[0034] The second image correction unit is used to detect edge intersection points in the image to be processed of each target area, and perform perspective transformation on the image to be processed according to the edge intersection points to obtain an image of a second rectangular area as the image to be identified of the target area;

[0035] The information recognition unit is used to perform character recognition on each image to be recognized and obtain information recognition results for each target area.

[0036] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program programmed or configured to execute any of the above-mentioned image multi-target information recognition methods.

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] The present invention draws a minimum bounding rectangle for each area in an image, and screens out areas corresponding to the minimum bounding rectangle that meets the conditions for information recognition, thereby realizing the recognition of contents in multiple targets in one image, greatly improving work efficiency, and for each screened target area, performing image angle correction and perspective transformation in turn, thereby improving the recognition accuracy of information in each target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is an overall flow chart of an embodiment of the present invention.

[0040] Figure 2 Flow chart of step S1 in an embodiment of the present invention.

[0041] Figure 3 It is the image after preprocessing in the embodiment of the present invention.

[0042] Figure 4 It is the first binarized image in the embodiment of the present invention.

[0043] Figure 5 This is the first morphologically processed image in the embodiment of the present invention.

[0044] Figure 6 is the edge data of each region in the image in the embodiment of the present invention.

[0045] Figure 7 is the minimum bounding rectangle of each area in the image in the embodiment of the present invention.

[0046] Figure 8 It is a schematic diagram of calculating the minimum bounding rectangle related parameters in an embodiment of the present invention.

[0047] Fig. 9 This is the screening result of the minimum circumscribed rectangle and its corresponding area in the embodiment of the present invention.

[0048] Fig.10 Flow chart of step S2 in the embodiment of the present invention.

[0049] Fig.11 1 is the angle correction result of display screen 1 (left screen) and display screen 2 (right screen) in the embodiment of the present invention.

[0050] Fig.12 Schematic diagram of comparison between the first rectangular area and the minimum circumscribed rectangle in an embodiment of the present invention.

[0051] Fig.13These are the images to be processed of display screen 1 (left screen) and display screen 2 (right screen) in the embodiment of the present invention.

[0052] Fig.14 Flow chart of step S3 in the embodiment of the present invention.

[0053] Fig.15 It is the edge intersection point of display screen 1 (left screen) and display screen 2 (right screen) in the embodiment of the present invention.

[0054] Fig.16 These are the images to be identified on display screen 1 (left screen) and display screen 2 (right screen) in the embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0056] Embodiment 1

[0057] This embodiment designs an image multi-target information recognition method based on the OpenCV image processing tool, which can accurately detect, correct and capture multiple target areas in the same image, creating favorable conditions for information recognition, such as Figure 1 As shown, the following steps are included:

[0058] S1) obtaining a preprocessed image, dividing the preprocessed image into regions, and drawing a minimum bounding rectangle for each region, screening a minimum bounding rectangle that meets a preset aspect ratio and area ratio, and taking the region corresponding to the screened minimum bounding rectangle as a target region;

[0059] S2) for each target area, the image is angle-corrected with the center of the target area, and an image of a first rectangular area with the target area as the center is intercepted from the corrected image as the image to be processed of the target area;

[0060] S3) for each target area's image to be processed, detecting edge intersections in the image to be processed of the target area, performing perspective transformation on the image to be processed according to the edge intersections, and obtaining an image of a second rectangular area as the image to be identified of the target area;

[0061] S4) performing character recognition on each image to be recognized to obtain information recognition results for each target area.

[0062] Through the above steps, the method of this embodiment determines the target area based on the aspect ratio and area ratio of the minimum circumscribed rectangle, and at the same time, the image is intercepted after angle correction and perspective transformation based on the target area, thereby ensuring that information recognition can obtain accurate results.

[0063] The focus of this embodiment is how to accurately filter out the target area from the image. The object of information recognition in this embodiment is mainly the text on the display screen in the image. Generally speaking, there is a brightness difference between the screen area of ​​the display screen on the image and the border area. Therefore, in step S1) of this embodiment, the pre-processed image is first divided into regions based on the brightness difference, and the minimum bounding rectangle of each region is drawn, such as Figure 2 As shown, the steps specifically include:

[0064] S111) Gaussian filtering and binarization are performed on the image to obtain a first binarized image, such as Figure 3 As shown, the preprocessed image in this embodiment is a grayscale image ① obtained by grayscale conversion of a color image, which includes a display screen 1 on the left and a display screen 2 on the right. The number of display screens that can identify information in this solution is not limited to the case of two display screens in this embodiment. When the clarity is met, information can be identified for more display screens in the same image, such as Figure 4 As shown, the grayscale image ① is subjected to Gaussian filtering and binarization processing to obtain a first binarized image ②. Compared with the grayscale image ①, the difference between the screen area of ​​the display screen 1 and the screen area of ​​the display screen 2 is more obvious than the border area in the first binarized image ②. Gaussian filtering and binarization processing are both conventional means in image processing, and will not be described in detail here.

[0065] S112) performing morphological corrosion and dilation processing on the first binary image ② to obtain a first morphologically processed image ③, such as Figure 5 As shown, the first morphologically processed image ③ filters out noise compared to the first binary image ②, making the boundaries between the screen area and the frame area of ​​the display screen 1 and the display screen 2 clearer. The morphological erosion and dilation process is a conventional method in image processing, which will not be described in detail here.

[0066] S113) Perform edge detection on the first morphologically processed image ③ to obtain edge data of each region, such as Figure 6 As shown, this embodiment uses the OpenCV image processing tool to perform edge detection to obtain an edge detection image ④. From the edge detection image ④, it can be seen that in addition to the edges of the screen areas of the display screen 1 and the display screen 2, there are many interferences. For example, the edges of the windows, chandeliers, walls and other areas in the background are also in the edge detection image ④. Using the OpenCV image processing tool to perform edge detection is a conventional method in image processing, which will not be described in detail here.

[0067] S114) Draw the minimum bounding rectangle of each area according to the edge data, such as Figure 7 As shown, this embodiment uses the OpenCV image processing tool to draw the minimum bounding rectangle of each area in the grayscale image ① based on the edge data of the previous step to obtain an edge contour image ⑤, wherein the edge contour image ⑤ includes the minimum bounding rectangles of display screen 1 and display screen 2, and also includes the minimum bounding rectangles of windows, chandeliers, walls, etc. as interference. Using the OpenCV image processing tool to draw the minimum bounding rectangle is a conventional method in image processing and will not be described in detail here.

[0068] By observing the edge contour image ⑤, we find that the area framed by the minimum circumscribed rectangle of display screen 1 and display screen 2 is roughly the screen area of ​​display screen 1 and display screen 2. Since there is a unified standard for the aspect ratio of display device screens, the aspect ratio of the minimum circumscribed rectangle of display screen 1 and display screen 2 is basically the same, which is significantly different from the aspect ratio of the minimum circumscribed rectangle of other interferences such as windows, chandeliers, and walls. In addition, as the main observation objects, the proportion of display screen 1 and display screen 2 in the grayscale image ① is also greater than that of some other interferences. Therefore, the above observation results provide ideas for screening out the screen areas of display screen 1 and display screen 2 as the target area from the edge contour image ⑤, such as Figure 2 As shown, in step S1) of this embodiment, for all the minimum bounding rectangles in the edge contour image ⑤, the minimum bounding rectangle that meets the preset aspect ratio and area ratio is selected, and finally the following is obtained: Fig. 9 The screening result image ⑥ only includes the minimum bounding rectangle of display screen 1 and display screen 2. The steps of screening the minimum bounding rectangle that meets the preset aspect ratio and area ratio specifically include:

[0069] S121) Select the current minimum bounding rectangle, calculate the length and width of the current minimum bounding rectangle according to the number of pixels in the length direction and the width direction of the current minimum bounding rectangle, and calculate the area of ​​the current minimum bounding rectangle according to the length and width, such as Figure 8 As shown, in this embodiment, the XOY plane coordinate system is established with the horizontal direction as the X axis and the vertical direction as the Y axis. The center point coordinates p of the minimum circumscribed rectangle can be obtained by using the OpenCV image processing tool. c (x c ,y c ), the inclination angle between the minimum enclosing rectangle and the X-axis, and the length W of the minimum enclosing rectangle i and width value H i , where the length of the minimum enclosing rectangle is W i and width value H iThat is, the number of pixels in the length and width directions of the minimum circumscribed rectangle, and the length value W is calculated by i and width value H i The product of gets the area of ​​the minimum enclosing rectangle area i ;

[0070] S122) If the error between the aspect ratio of the current minimum enclosing rectangle and the preset aspect ratio is less than the preset error, and the area ratio of the current minimum enclosing rectangle in the grayscale image ① is greater than or equal to the preset threshold, then the area corresponding to the current minimum enclosing rectangle is the target area. In this embodiment, the function expression for determining whether the current minimum enclosing rectangle satisfies the preset aspect ratio and area ratio is as follows:

[0071]

[0072] In the above formula, i is the number of the current minimum bounding rectangle, R i is the aspect ratio of the current minimum circumscribed rectangle, R is the preset aspect ratio, α is the preset error, S i is the area ratio of the current minimum bounding rectangle in the grayscale image ①, s is the preset threshold, P i When it is 1, it means that the current minimum bounding rectangle meets the preset aspect ratio and area ratio, and the area corresponding to the current minimum bounding rectangle is the screen area of ​​display screen 1 or display screen 2;

[0073] In this embodiment, the functional expression of the area ratio of the current minimum bounding rectangle in the grayscale image ① is as follows:

[0074]

[0075] In the above formula, i is the number of the current minimum bounding rectangle, area i is the area of ​​the current minimum enclosing rectangle, H i is the width of the current minimum bounding rectangle, W i is the length of the current minimum bounding rectangle, row is the number of pixels in the width direction of the grayscale image ①, and col is the number of pixels in the length direction of the grayscale image ①;

[0076] In this embodiment, considering that the purpose of this solution is to obtain the information of the display screen in the image, subject to the current image resolution, when the display screen in the image is too small, the information displayed is too blurred and thus cannot be accurately identified. Therefore, a threshold s of the area ratio is set. In this embodiment, the preset value of the threshold s is 0.1, which means that a maximum of 9 display screens can exist simultaneously in an image. The threshold s can also be set to other values ​​according to the maximum number of all display screens expected in an image.

[0077] In this embodiment, the aspect ratio function expression of the current minimum bounding rectangle is as follows:

[0078] R i =H i / W i (3)

[0079] In the above formula, i is the number of the current minimum bounding rectangle, H i is the width of the current minimum bounding rectangle, W i is the length of the current minimum circumscribed rectangle. In this embodiment, the preset aspect ratio R is 16 / 9 because all current display devices use this ratio. At the same time, the value of the error α is set according to the actual situation. For example, when the image is distorted, the value of the error α can be appropriately increased. Generally, the value of the error α is set to 0.1;

[0080] S123) Return to step S121) until all minimum circumscribed rectangles in the edge contour image ⑤ are traversed.

[0081] like Fig. 9 As shown in FIG. 6 , the minimum bounding rectangle of display screen 1 and display screen 2 in the screening result image ⑥ is accurately selected, but there is still an angle deviation problem for the display screen areas of display screen 1 and display screen 2, and the minimum bounding rectangle cannot accurately select the display screen areas of display screen 1 and display screen 2. Figure 1 and Fig.10 As shown, this embodiment then corrects the image angle according to each target area, and intercepts the first rectangular area image centered on the target area. According to the content recorded in the previous text, in step S1) of this embodiment, an XOY plane coordinate system is established with the horizontal direction as the X axis and the vertical direction as the Y axis, and the center point coordinate p of the minimum circumscribed rectangle is obtained through the OpenCV image processing tool. c (x c ,y c ), and if Figure 8 As shown, the inclination direction and inclination angle of the minimum enclosing rectangle relative to the X-axis can be calculated based on the long side of the minimum enclosing rectangle and the projection of the long side on the X-axis, as shown in Fig. 9As shown, the minimum circumscribed rectangle of display screen 1 in the screening result image ⑥ is tilted to the left relative to the X-axis, and the tilt angle is A1, and the minimum circumscribed rectangle of display screen 2 is tilted to the right relative to the X-axis, and the tilt angle is A2; then in step S2) of this embodiment, the angle correction of the image with the center of the target area specifically includes: taking the center point of the minimum circumscribed rectangle corresponding to the target area as the rotation center, taking the value of the tilt angle of the minimum circumscribed rectangle corresponding to the target area as the rotation angle value, and rotating the image in the opposite direction of the tilt direction according to the rotation center and the rotation angle value, so that the tilt angle of the minimum circumscribed rectangle corresponding to the target area with respect to the X-axis is 0. In this embodiment, for the display screen 1 in the screening result image ⑥, since the corresponding minimum circumscribed rectangle is tilted to the left, the screening result image ⑥ or the grayscale image ① is rotated clockwise with the rotation center according to the rotation angle, that is, the tilt angle A1. The angle correction result image ⑦ is finally obtained as shown in Fig.11 As shown in the left picture of ; for the display screen 2 in the screening result image ⑥, since the corresponding minimum circumscribed rectangle is tilted to the right, the screening result image ⑥ or the grayscale image ① is rotated counterclockwise according to the rotation angle, that is, the tilt angle A2, with the rotation center as the final angle correction result image ⑦ is as shown Fig.11 As shown in the picture on the right.

[0082] After the angle correction, in order to obtain the information of the target area more accurately for identification, it is necessary to first intercept a rectangular area including the surrounding extension range with the target area as the center in the angle correction result image ⑦ to prepare for subsequent image processing, such as Fig.12 As shown, in step S2) of this embodiment, intercepting the image of the first rectangular area centered on the target area in the corrected image specifically includes: obtaining the center point of the minimum circumscribed rectangle corresponding to the target area, and calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the center point and the preset expansion value, that is, calculating the coordinate value of the vertex of the first rectangular area according to the coordinate value of the center point of the minimum circumscribed rectangle and the preset expansion value, and taking the vertex of the first rectangular area as the interception starting point. The coordinate value function expression of the vertex of the first rectangular area is:

[0083]

[0084] In the above formula, x e ,y e are the horizontal and vertical coordinates of the vertices of the first rectangular area, respectively, c ,y care the horizontal and vertical coordinates of the center point of the minimum bounding rectangle, W is the length of the minimum bounding rectangle, H is the width of the minimum bounding rectangle, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image;

[0085] The length and width of the first rectangular area are calculated according to the size of the minimum circumscribed rectangle and the preset expansion value. The length and width function expressions of the first rectangular area are:

[0086]

[0087] In the above formula, W e ,H e are the length and width of the vertices of the first rectangular area, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image;

[0088] Then, starting from the capture start point, the image of the first rectangular area is captured according to the length and width of the first rectangular area.

[0089] According to the above steps, the first rectangular area image is obtained by cutting the angle correction result image ⑦ of the display screen 1 and the display screen 2, and the first rectangular area image is used as the image to be processed ⑧. The images to be processed ⑧ of the display screen 1 and the display screen 2 are respectively as follows: Fig.13 As shown in the left and right pictures.

[0090] After obtaining the image to be processed⑧ of display screen 1 and display screen 2, the image to be processed⑧ can be further perspective transformed to more accurately extract the image for information recognition, such as Fig.14 As shown, in step S3), for each target area, edge intersections in the image to be processed of the target area are detected, which specifically includes the following steps:

[0091] S31) performing Gaussian filtering and binarization processing on the image to be processed to obtain a second binarized image;

[0092] S32) performing morphological erosion and dilation processing on the second binary image to obtain a second morphologically processed image;

[0093] S33) performing edge detection on the second morphologically processed image to obtain edge data of the target area;

[0094] S34) performing Hough line detection on the edge data to obtain edge lines of the target area;

[0095] S35) Calculate the intersection coordinates of the intersecting edge straight lines to obtain the edge intersection of the target area.

[0096] In the above steps, steps S31 to S33 are the same as the aforementioned steps S111 to S113, and will not be described in detail here. Steps S34 and S35 are also conventional operations in the image processing process. Currently, for the character recognition of ID cards, driver's licenses, etc., the processing of steps S31 to S35 is mostly used. The edge intersection images ⑨ of the display screen 1 and the display screen 2 obtained through the processing of steps S31 to S35 are respectively as follows: Fig.15 As shown in the left and right pictures.

[0097] After obtaining the edge intersection of display screen 1 and display screen 2, the perspective transformation can be performed using the OpenCV image processing tool to obtain the image of the second rectangular area as the image to be identified ⑩. The image to be identified ⑩ of display screen 1 and display screen 2 is as follows: Fig.16 As shown in the left and right pictures of . Information recognition can be performed on the images to be recognized ⑩ of display screen 1 and display screen 2 respectively to obtain the text content displayed on display screen 1 and display screen 2. The information recognition method used in this embodiment is the existing character recognition technology, which will not be described here.

[0098] This embodiment also provides an image multi-target information recognition system, including:

[0099] The target screening unit is used to obtain the preprocessed image, divide the preprocessed image into regions, draw the minimum bounding rectangle of each region, screen the minimum bounding rectangle that meets the preset aspect ratio and area ratio, and use the area corresponding to the screened minimum bounding rectangle as the target area;

[0100] The first image correction unit is used to perform angle correction on the image with respect to each target area, respectively, with respect to the center of the target area, and intercept an image of a first rectangular area with respect to the target area as the image to be processed of the target area from the corrected image;

[0101] The second image correction unit is used to detect edge intersection points in the image to be processed of each target area, and perform perspective transformation on the image to be processed according to the edge intersection points to obtain an image of a second rectangular area as the image to be identified of the target area;

[0102] The information recognition unit is used to perform character recognition on each image to be recognized and obtain information recognition results for each target area.

[0103] This embodiment also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute the image multi-target information recognition method.

[0104] Embodiment 2

[0105] This embodiment is basically the same as the first embodiment, except that, in step S121) to step S123) of the first embodiment, the preset threshold s and the aspect ratio R are fixed values, and interference that has not been eliminated may still exist. For example, the preset threshold s is much smaller than the area ratio corresponding to each display screen in the image, resulting in the minimum bounding rectangle of part of the non-display screen area being retained. Therefore, after step S123), this embodiment further includes a step of adjusting the size of the threshold s, specifically: adjusting the value of the threshold s to the area ratio of the display screen with the smallest area in the current image, screening and retaining the minimum bounding rectangle with an area ratio greater than or equal to the adjusted threshold s. In this embodiment, there are only display screen 1 and display screen 2 on the grayscale image ①, and display screen 1 is smaller than display screen 2. Therefore, in this embodiment, the area ratio of the minimum bounding rectangle of display screen 1 in the grayscale image ① is used as the threshold s, thereby eliminating the aforementioned possible interference.

[0106] Embodiment 3

[0107] This embodiment is basically the same as the first embodiment, except that, since the preset threshold s and the aspect ratio R in steps S121) to S123) of the first embodiment are fixed values, there may still be interference that has not been eliminated. For example, there is a minimum bounding rectangle with the same size and aspect ratio as the minimum bounding rectangle of the display screen 1 or the display screen 2 in the non-display screen area, which results in the minimum bounding rectangle of part of the non-display screen area being retained. Before step 2) of this embodiment, there is also a step of verifying the target area, which specifically includes: intercepting the images surrounded by the minimum bounding rectangles corresponding to all target areas, performing text recognition on each intercepted image respectively, and if there is no recognition information or the recognition is wrong, the corresponding target area is eliminated. Considering that the biggest difference between the display screen area and the non-display screen area is that there is display information in the display screen, and the text recognition result of the display screen area without angle adjustment and perspective transformation may only be affected in terms of accuracy, in this embodiment, text recognition is first performed on the images surrounded by all minimum bounding rectangles, and whether the area surrounded by the minimum bounding rectangle is a display screen is determined based on whether there is a recognition result, thereby improving the accuracy of target area recognition.

[0108] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for identifying multi-target information in an image, characterized in that: The following steps are involved: S1) obtaining the preprocessed image, dividing the preprocessed image into regions, and drawing the minimum bounding rectangle of each region, screening the minimum bounding rectangle that meets the preset aspect ratio and area ratio, and taking the region corresponding to the screened minimum bounding rectangle as the target region. The steps of screening the minimum bounding rectangle that meets the preset aspect ratio and area ratio specifically include: S121) selecting a current minimum bounding rectangle, calculating the length and width of the current minimum bounding rectangle according to the number of pixels in the length direction and the width direction of the current minimum bounding rectangle, and calculating the area of ​​the current minimum bounding rectangle according to the length and the width; S122) if the error between the aspect ratio of the current minimum bounding rectangle and the preset aspect ratio is less than the preset error, and the area proportion of the current minimum bounding rectangle in the image is greater than or equal to a preset threshold s, then the area corresponding to the current minimum bounding rectangle is the target area; S123) returning to step S121) until all minimum bounding rectangles in the image are traversed; S2) for each target area, the image is angle-corrected with the center of the target area, and an image of a first rectangular area with the target area as the center is intercepted from the corrected image as the image to be processed of the target area; S3) for each target area's image to be processed, detecting edge intersections in the image to be processed of the target area, performing perspective transformation on the image to be processed according to the edge intersections, and obtaining an image of a second rectangular area as the image to be identified of the target area; S4) performing character recognition on each image to be recognized to obtain information recognition results for each target area.

2. The image multi-target information recognition method according to claim 1, characterized in that: The step of drawing the minimum bounding rectangle of each area in the image in step S1) specifically includes: S111) performing Gaussian filtering and binarization processing on the image to obtain a first binarized image; S112) performing morphological erosion and dilation processing on the first binary image to obtain a first morphologically processed image; S113) performing edge detection on the first morphologically processed image to obtain edge data of each region; S114) Draw the minimum circumscribed rectangle of each area according to the edge data.

3. The image multi-target information recognition method according to claim 1, characterized in that: Step S123) also includes the step of adjusting the size of the threshold s, specifically: adjusting the value of the threshold s to the area ratio of the minimum bounding rectangle corresponding to the actual target area with the smallest area in the current image, and screening and retaining the minimum bounding rectangle whose area ratio is greater than or equal to the adjusted threshold s.

4. The image multi-target information recognition method according to claim 1, characterized in that: Before step 2), the step of verifying the target area is also included, which specifically includes: intercepting the image surrounded by the minimum circumscribed rectangle corresponding to all target areas, performing text recognition on each intercepted image respectively, and if there is no recognition information or the recognition is wrong, the corresponding target area is eliminated.

5. The image multi-target information recognition method according to claim 1, characterized in that: Step S1) also includes: establishing a coordinate system with the horizontal direction as the horizontal axis and the vertical direction as the longitudinal axis, obtaining the coordinates of the center point of the minimum enclosing rectangle, and calculating the inclination direction and inclination angle of the minimum enclosing rectangle relative to the horizontal axis based on the long side of the minimum enclosing rectangle and the projection of the long side on the horizontal axis; in step S2), the image is angle-corrected with the center of the target area, specifically including: taking the center point of the minimum enclosing rectangle corresponding to the target area as the rotation center, taking the value of the inclination angle of the minimum enclosing rectangle corresponding to the target area as the rotation angle value, and rotating the image in the opposite direction of the inclination direction according to the rotation center and the rotation angle value, so that the inclination angle of the minimum enclosing rectangle corresponding to the target area relative to the horizontal axis is 0.

6. The image multi-target information recognition method according to claim 1, characterized in that: In step S2), intercepting an image of a first rectangular area centered on the target area in the corrected image specifically includes: obtaining the center point of the minimum circumscribed rectangle corresponding to the target area, calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the center point and the preset expansion value, and then intercepting the image of the first rectangular area according to the length and width of the first rectangular area starting from the interception starting point.

7. The image multi-target information recognition method according to claim 6, characterized in that: Calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the center point and the preset expansion value specifically includes: calculating the coordinate value of the vertex of the first rectangular area according to the coordinate value of the center point of the minimum circumscribed rectangle and the preset expansion value, taking the vertex of the first rectangular area as the interception starting point, and the coordinate value function expression of the vertex of the first rectangular area is: In the above formula, x e ,y e are the horizontal and vertical coordinates of the vertices of the first rectangular area, respectively, c ,y c are the horizontal and vertical coordinates of the center point of the minimum bounding rectangle, W is the length of the minimum bounding rectangle, H is the width of the minimum bounding rectangle, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image; The length and width of the first rectangular area are calculated according to the size of the minimum circumscribed rectangle and the preset expansion value. The length and width function expressions of the first rectangular area are: In the above formula, W e ,H e are the length and width of the vertices of the first rectangular area respectively, col / 10 is the preset expansion value in the length direction, col is the number of pixels in the length direction of the image, row / 10 is the preset expansion value in the width direction, row is the number of pixels in the width direction of the image.

8. An image multi-target information recognition system, characterized in that: include: The target screening unit is used to obtain the preprocessed image, divide the preprocessed image into regions, draw the minimum bounding rectangle of each region, screen the minimum bounding rectangle that meets the preset aspect ratio and area ratio, and use the area corresponding to the screened minimum bounding rectangle as the target area. The steps of screening the minimum bounding rectangle that meets the preset aspect ratio and area ratio specifically include: S121) selecting a current minimum bounding rectangle, calculating the length and width of the current minimum bounding rectangle according to the number of pixels in the length direction and the width direction of the current minimum bounding rectangle, and calculating the area of ​​the current minimum bounding rectangle according to the length and the width; S122) if the error between the aspect ratio of the current minimum bounding rectangle and the preset aspect ratio is less than the preset error, and the area proportion of the current minimum bounding rectangle in the image is greater than or equal to a preset threshold s, then the area corresponding to the current minimum bounding rectangle is the target area; S123) returning to step S121) until all minimum bounding rectangles in the image are traversed; The first image correction unit is used to perform angle correction on the image with respect to each target area, respectively, with respect to the center of the target area, and intercept an image of a first rectangular area with respect to the target area as the image to be processed of the target area from the corrected image; The second image correction unit is used to detect edge intersection points in the image to be processed of each target area, and perform perspective transformation on the image to be processed according to the edge intersection points to obtain an image of a second rectangular area as the image to be identified of the target area; The information recognition unit is used to perform character recognition on each image to be recognized and obtain information recognition results for each target area.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program programmed or configured to execute the image multi-target information recognition method according to any one of claims 1 to 7.

Images