Text detection method, device, electronic device and medium

Abstract

The embodiments of the present application provide a text detection method, device, electronic device and medium, which obtain a first track point set generated by a user performing a circle selection operation on a screen, and obtain a minimum rectangular frame containing all track points of the first track point set; perform data cleaning on the track points in the first track point set, remove track points that meet preset conditions, and obtain a second track point set, thereby purifying the data set and reducing noise interference. Based on the second track point set, the text area that the user actually intends to circle can be identified; based on the second track point set and the minimum rectangular frame, a text mask image is obtained, and the text area is highlighted by the mask image, and interference from irrelevant background is eliminated; the text mask image is superimposed on the minimum rectangular frame to obtain a target text detection frame, so that the obtained target text detection frame is more accurate, thereby improving the accuracy of text detection.

Description

Technical Field

[0001] The present application belongs to the field of multimedia technology, and specifically relates to a text detection method, device, electronic device and medium. Background Art

[0002] In the field of multimedia technology, display devices are widely used in various occasions, such as commercial display, education, entertainment, etc. In these application occasions, it is often necessary to intelligently interact with the content displayed on the screen. How to accurately and intelligently circle and detect specific text areas on the screen is one of the current urgent needs. For example, in educational classroom presentations, information in a specific area of ​​the presentation can be circled and further marked.

[0003] In the prior art, for text on a display device, a closed area can usually be determined by detecting a circled selection track in a screen area to determine the selected text.

[0004] However, the accuracy of detecting the circled text area using the prior art is not high. Summary of the invention

[0005] Embodiments of the present application provide a text detection method, device, electronic device, and medium to solve the problem of low accuracy in detecting circled text areas in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a text detection method, comprising:

[0007] Acquire a first track point set generated by a user performing a circle selection operation on the screen, and acquire a minimum rectangular frame containing all track points of the first track point set;

[0008] Performing data cleaning on the trajectory points in the first trajectory point set, removing the trajectory points that meet the preset conditions, and obtaining a second trajectory point set;

[0009] Based on the second trajectory point set and the minimum rectangular frame, a text mask image is obtained;

[0010] The text mask image is superimposed on the minimum rectangular frame to obtain a target text detection frame.

[0011] Optionally, the preset condition includes at least one of the following:

[0012] Deviation from track trend graph;

[0013] At the non-closed end of the trajectory overlap intersection;

[0014] Any of two adjacent trajectory points whose distance is less than a preset threshold.

[0015] Optionally, obtaining a text mask image based on the second trajectory point set and the minimum rectangular frame includes:

[0016] Obtaining coordinate information of the second trajectory point set relative to the minimum rectangular frame;

[0017] Based on the coordinate information relative to the minimum rectangular frame, drawing a cleaning trajectory diagram;

[0018] A target closed area in the cleaning trajectory map is obtained, and a text mask map is obtained based on the target closed area.

[0019] Optionally, the acquiring a target closed area in the cleaning trajectory map and obtaining a text mask map based on the target closed area includes:

[0020] The target closed area in the cleaning trajectory map is obtained, the target closed area is transparently processed, and the area outside the target closed area is blocked to obtain the text mask map.

[0021] Optionally, obtaining the target closed area in the cleaning trajectory map includes:

[0022] Obtaining an initial closed area in the cleaning trajectory map;

[0023] The initial closed areas are merged and screened to obtain a target closed area.

[0024] Optionally, the merging and screening the initial closed areas to obtain a target closed area includes:

[0025] The initial closed areas contained in other initial closed areas are deleted, the initial closed areas whose areas are smaller than a preset threshold are deleted, and the remaining initial closed areas are determined as target closed areas.

[0026] Optionally, drawing a cleaning trajectory diagram based on the coordinate information relative to the minimum rectangular frame includes:

[0027] Connect two adjacent trajectory points with straight lines in sequence, and connect the last trajectory point with the first trajectory point with a straight line to obtain a cleaning trajectory diagram.

[0028] Optionally, superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame includes:

[0029] Superimposing the text mask image onto the minimum rectangular frame to obtain a first text detection image;

[0030] The first text detection image is input into a text detection model to obtain a target text detection frame.

[0031] Optionally, inputting the first text detection image into a text detection model to obtain a target text detection frame includes:

[0032] Inputting the first text detection image into a text detection model to obtain an initial text detection frame;

[0033] The initial text detection frame is expanded to obtain a target text detection frame.

[0034] Optionally, obtaining a text mask image based on the second trajectory point set and the minimum rectangular frame includes:

[0035] Expanding the minimum rectangular frame to obtain an expanded minimum rectangular frame;

[0036] Obtaining a text mask image based on the second trajectory point set and the expanded minimum rectangular frame;

[0037] The step of superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame includes:

[0038] The text mask image is superimposed on the expanded minimum rectangular frame to obtain a target text detection frame.

[0039] In a second aspect, an embodiment of the present application provides a text detection device, including:

[0040] An acquisition module, used to acquire a first track point set generated by a user performing a circle selection operation on the screen, and acquire a minimum rectangular frame containing all track points of the first track point set;

[0041] a cleaning module, configured to clean the track points in the first track point set, remove outlier track points, and obtain a second track point set;

[0042] A first processing module, configured to obtain a text mask image based on the second trajectory point set and the minimum rectangular frame;

[0043] The second processing module is used to superimpose the text mask image on the minimum rectangular frame to obtain a target text detection frame.

[0044] Optionally, the preset condition includes at least one of the following:

[0045] Deviation from track trend graph;

[0046] At the non-closed end of the trajectory overlap intersection;

[0047] Any of two adjacent trajectory points whose distance is less than a preset threshold.

[0048] Optionally, the first processing module is specifically used to obtain coordinate information of the second trajectory point set relative to the minimum rectangular box; draw a cleaning trajectory map based on the coordinate information relative to the minimum rectangular box; obtain a target closed area in the cleaning trajectory map, and obtain a text mask map based on the target closed area.

[0049] Optionally, the first processing module is specifically used to obtain a target closed area in the cleaning trajectory map, perform transparency processing on the target closed area, and perform occlusion processing on the outside of the target closed area to obtain the text mask map.

[0050] Optionally, the first processing module is specifically used to obtain an initial closed area in the cleaning trajectory map; merge and filter the initial closed area to obtain a target closed area.

[0051] Optionally, the first processing module is specifically configured to delete an initial closed area contained in other initial closed areas, delete an initial closed area whose area is smaller than a preset threshold, and determine the remaining initial closed areas as target closed areas.

[0052] Optionally, the first processing module is specifically used to connect two adjacent trajectory points in sequence with straight lines, and connect the last trajectory point and the first trajectory point with a straight line to obtain a cleaning trajectory diagram.

[0053] Optionally, the second processing module is specifically used to superimpose the text mask image on the minimum rectangular frame to obtain a first text detection image; and input the first text detection image into a text detection model to obtain a target text detection frame.

[0054] Optionally, the second processing module is specifically used to input the first text detection image into a text detection model to obtain an initial text detection frame; and perform expansion processing on the initial text detection frame to obtain a target text detection frame.

[0055] Optionally, the first processing module is specifically configured to expand the minimum rectangular frame to obtain an expanded minimum rectangular frame; and obtain a text mask image based on the second trajectory point set and the expanded minimum rectangular frame.

[0056] Optionally, the second processing module is specifically configured to superimpose the text mask image onto the expanded minimum rectangular frame to obtain a target text detection frame.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the text detection method described in the first aspect are implemented.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the text detection method described in the first aspect are implemented.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed by a processor of a vehicle or a cloud server, implements the steps of the text detection method described in the first aspect above.

[0060] The text detection method, device, electronic device and medium provided in the embodiments of the present application can capture the approximate range of the text area that the user intends to identify by acquiring a first track point set generated by a user performing a circle selection operation on the screen, and obtain a minimum rectangular frame containing all track points of the first track point set. The system determines a clear boundary of the range to be identified, and this boundary contains possible text information, thereby ensuring the comprehensiveness of the detection; the track points in the first track point set are cleaned, and the track points that meet the preset conditions are removed to obtain a second track point set, thereby purifying the data set and reducing noise interference. Based on the second track point set, the text area that the user actually intends to circle can be identified; based on the second track point set and the minimum rectangular frame, a text mask image is obtained, and the text area is highlighted by the mask image, and the interference of irrelevant background is eliminated; the text mask image is superimposed on the minimum rectangular frame to obtain a target text detection frame, so that the obtained target text detection frame is more accurate, thereby improving the accuracy of text detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a text detection method provided in an embodiment of the present application;

[0062] Figure 2A A schematic diagram of a scene of all track points of the first track point set applied on a screen provided in an embodiment of the present application;

[0063] Figure 2B A schematic diagram of a scene of a minimum rectangular frame including all trajectory points of the first trajectory point set applied on a screen provided in an embodiment of the present application;

[0064] Figure 2C A schematic diagram of a scene of an expanded minimum rectangular frame including all trajectory points of the first trajectory point set applied on a screen provided in an embodiment of the present application.

[0065] Figure 2D A schematic diagram of a scene of a minimum rectangular frame including all trajectory points of the second trajectory point set applied on a screen provided in an embodiment of the present application;

[0066] Figure 2E A schematic diagram of a cleaning trajectory diagram applied on a screen is provided for an embodiment of the present application;

[0067] Figure 2F A schematic diagram of a scene of a text mask image applied on a screen provided in an embodiment of the present application;

[0068] Figure 2G A schematic diagram of a scene of a first text detection graph applied on a screen is provided for an embodiment of the present application;

[0069] Figure 2H A schematic diagram of a scene of an initial text detection frame applied on a screen is provided for an embodiment of the present application;

[0070] Fig.2I A schematic diagram of a scene of a target text detection frame applied on a screen provided in an embodiment of the present application;

[0071] Figure 3A A schematic diagram of a scene of selecting a continuous area with a smooth trajectory applied on a screen provided in an embodiment of the present application;

[0072] Figure 3B A schematic diagram of a scene of selecting a continuous area with an uneven trajectory applied on a screen provided in an embodiment of the present application;

[0073] Figure 3C A schematic diagram of a scene of non-continuous area selection applied on a screen provided in an embodiment of the present application;

[0074] Figure 3D A schematic diagram of a scene in which circled areas are connected end to end on a screen provided in an embodiment of the present application;

[0075] Figure 4 A flowchart of another text detection method provided in an embodiment of the present application;

[0076] Figure 5 A flowchart of another text detection method provided in an embodiment of the present application;

[0077] Figure 6 A flowchart of another text detection method provided in an embodiment of the present application;

[0078] Figure 7 A flowchart of another text detection method provided in an embodiment of the present application;

[0079] Figure 8 A structural schematic diagram of a text detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0081] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0082] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0083] With regard to the prior art, a closed area is determined by detecting a circle selection track in a screen area to determine the selected text. However, due to the subjectivity of the circle selection user, the circle selection track is too arbitrary and may not accurately contain the text, resulting in a circle selection range that is too large or too small, resulting in the problem of low accuracy in detecting the circled text area. The present application provides a text detection method, which first obtains a minimum outer rectangular frame of the circle selection track points, and then removes the track points that affect the target circle selection area from the obtained circle selection track points according to preset conditions, and then combines a text mask image to obtain a text detection frame that only contains the text, thereby realizing the circle selection and detection of the target text area and improving the accuracy of text detection.

[0084] In an embodiment of the present application, the trajectory points generated by the user's circling operation on the screen are first obtained, and then data cleaning is performed. Then, the target text detection frame is obtained by combining the text mask image. For the convenience of description, in the following embodiments of the present application, the trajectory point set generated by the user's initial circling operation on the screen is described as a first trajectory point set, the trajectory point set obtained after data cleaning of the trajectory points in the first trajectory point set is described as a second trajectory point set, the text detection image obtained by superimposing the text mask image and the minimum rectangular frame is described as a first text detection image, the module for obtaining the text mask image based on the second trajectory point set and the minimum rectangular frame is described as a first processing module, and the module for superimposing the text mask image on the minimum rectangular frame to obtain the target text detection frame is described as a second processing module.

[0085] The following describes the technical solution of the text detection method of the present application by taking several embodiments as examples:

[0086] Figure 1 A flowchart of a text detection method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method of this embodiment is as follows:

[0087] S11: Acquire a first track point set generated by a user performing a circle selection operation on the screen, and acquire a minimum rectangular frame containing all track points of the first track point set.

[0088] The user performs a circle selection operation on the screen, that is, the user clicks on the screen to start circle selection, enters circle selection mode, touches the screen again to start recording the track, and leaves the screen again to end the circle selection mode. At this time, the set of circled track points on the screen is the first track point set. The coordinates of all track points in the first track point set are recorded, and the coordinate set is expressed as:

[0089] Points=[{x:x0,y:y0},{x:x1,y:y1},……{x:x n ,y:y n}] (1)

[0090] Points represents the coordinate set of all trajectory points in the first trajectory point set, {x:x n ,y:y n} represents the coordinates of a certain trajectory point among all trajectory points in the first trajectory point set, such as Figure 2A As shown, Figure 2A A schematic diagram of a scene of all the track points of the first track point set applied on a screen provided in an embodiment of the present application. A minimum outer bounding rectangle containing all the track points of the first track point set is obtained on the screen, and the upper left corner of the minimum outer bounding rectangle is its starting point on the screen, and the coordinates are marked as S = [X, Y], such as Figure 2B As shown, Figure 2B A schematic diagram of a scene of a minimum rectangular frame including all trajectory points of the first trajectory point set applied on a screen provided in an embodiment of the present application.

[0091] Optionally, in some scenarios, in order to improve the integrity of the selected track and content, the minimum outer bounding rectangle can be expanded, that is, expanded in four directions by 0.a times the original width, such as 0.2 times, and the expanded rectangular area does not exceed the screen area, that is, the minimum coordinates of the upper left corner of the expanded rectangle are [0,0], and the lower right corner does not exceed [ht bs ,height bs ], where weight bs is the width of the screen, hht bs is the height of the screen, such as Figure 2C As shown, Figure 2C A schematic diagram of a scene of an expanded minimum rectangular frame including all trajectory points of the first trajectory point set applied on a screen provided in an embodiment of the present application.

[0092] S12: performing data cleaning on the trajectory points in the first trajectory point set, removing the trajectory points that meet a preset condition, and obtaining a second trajectory point set.

[0093] Optionally, the preset condition includes at least one of the following: deviation from a trajectory trend graph; being at a non-closed end of a trajectory overlap intersection; or any two adjacent trajectory points whose distance is less than a preset threshold.

[0094] Among them, the discrete points that deviate from the trajectory trend graph and the trajectory points at the non-closed ends of the trajectory overlap intersection are all outliers, and these outliers need to be removed. The two adjacent trajectory points whose distance is less than the preset threshold mean that the distance between the two adjacent trajectory points is too close. In order to save the coordinate calculation amount of subsequent trajectory points, it is best to remove any one of the two adjacent trajectory points. The preset threshold is expressed as l min ,like Figure 2D As shown, Figure 2D A schematic diagram of a scene of a minimum rectangular frame containing all the track points of the second track point set applied on a screen provided in an embodiment of the present application, wherein the set of track points after data cleaning is the second track point set, and the coordinates of all the track points in the second track point set are recorded, and the coordinate set is expressed as:

[0095]

[0096] Points vlean represents the coordinate set of all trajectory points in the second trajectory point set obtained after data cleaning, Represents the coordinates of a certain trajectory point among all trajectory points in the second trajectory point set.

[0097] S13: Obtain a text mask image based on the second trajectory point set and the minimum rectangular frame.

[0098] Among them, before obtaining the text mask map, first, based on the minimum rectangular frame map containing all the trajectory points in the second trajectory point set, a trajectory map of the same size with cleaning trajectory points drawn on it is obtained, and then closed area detection is performed on the trajectory map, and all detected closed areas are merged and screened, and finally, a picture of the same size as the trajectory map with cleaning trajectory points drawn on it is obtained, and the merged and screened closed areas are drawn on the picture to obtain the text mask map.

[0099] Correspondingly, in S11, the minimum rectangular frame is expanded, and a text mask image is obtained based on the second trajectory point set and the expanded minimum rectangular frame.

[0100] S14: superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame.

[0101] Among them, after the text mask image is superimposed on the minimum rectangular frame, a text detection image is obtained, and then the text detection image is input into the text detection model, the text content in the text detection image is detected to obtain an initial text detection frame, and finally the initial text detection frame is post-processed to obtain the target text detection frame.

[0102] Correspondingly, in S11, the minimum rectangular frame is expanded, and the text mask image is superimposed on the expanded minimum rectangular frame to obtain a target text detection frame.

[0103] The text detection method provided in this embodiment supports text detection in any circled area, including continuous areas and discontinuous areas, wherein the trajectory of the circled text detection image of the continuous area is basically or approximately a closed area, including continuous text with smooth trajectories in the same row and column of the same paragraph and continuous text with uneven trajectories in different rows and columns of the same paragraph, such as Figure 3A and Figure 3B As shown, Figure 3A A schematic diagram of a scene of selecting a continuous area with a smooth trajectory applied on a screen provided in an embodiment of the present application, Figure 3B The following is a schematic diagram of a scene of a continuous area with an uneven trajectory on the screen provided by an embodiment of the present application, wherein the dotted line part is the trajectory point, and the mask part is the text detection area; the circled text detection diagram of the non-continuous area is a closed area circled multiple times in any direction, the text detection area is the circled closed area, and the continuous part of the non-closed area in the middle is not recognized, such as Figure 3C As shown, Figure 3C A schematic diagram of a scene of non-continuous area selection applied on a screen provided in an embodiment of the present application, wherein the dotted line portion is a trajectory point and the masked portion is a text detection area.

[0104] The text detection method provided in this embodiment can also reduce the length and difficulty of the circled text area by connecting the end to the end. Figure 3D As shown, Figure 3D A schematic diagram of a scene in which circled areas are connected end to end on a screen provided in an embodiment of the present application, wherein the dotted line portion is a trajectory point and the masked portion is a text detection area.

[0105] In this embodiment, by acquiring a first track point set generated by a user performing a circle operation on the screen, the system can capture the approximate range of the text area that the user intends to identify, and obtain a minimum rectangular frame containing all track points of the first track point set. The system determines a clear boundary of a range to be identified, and this boundary contains possible text information, ensuring the comprehensiveness of the detection; data cleaning is performed on the track points in the first track point set, and track points that meet preset conditions are removed to obtain a second track point set, thereby purifying the data set and reducing noise interference. Based on the second track point set, the text area that the user actually intends to circle can be identified; based on the second track point set and the minimum rectangular frame, a text mask image is obtained, and the text area is highlighted by the mask image, eliminating interference from irrelevant background; the text mask image is superimposed on the minimum rectangular frame to obtain a target text detection frame, so that the target text detection frame obtained is more accurate. At the same time, the text detection method provided in this embodiment supports text detection in any circled area and the circled text can be connected end to end, thereby improving the accuracy of text detection and the flexibility of circle selection.

[0106] Figure 4 A flowchart of another text detection method provided in an embodiment of the present application is shown below. Figure 4 is Figure 1 Based on the embodiment shown, a possible implementation of S13 is further described as follows: Figure 4 As shown, the method of this embodiment is as follows:

[0107] S131: Obtain coordinate information of the second trajectory point set relative to the minimum rectangular frame.

[0108] Wherein, the coordinate information of the second trajectory point set relative to the minimum rectangular frame is obtained by subtracting the coordinates of the starting point of the minimum rectangular frame on the screen from the coordinates of all trajectory points in the second trajectory point set. The coordinate set is expressed as:

[0109]

[0110] Points tra represents the coordinate set of all trajectory points in the second trajectory point set relative to the starting point of the minimum rectangular frame on the screen, X represents the horizontal coordinate of the starting point of the intercepted minimum rectangular frame on the screen, and Y represents the vertical coordinate of the starting point of the minimum rectangular frame on the screen, Indicates the coordinates of a certain track point in all track points in the second track point set after conversion and update.

[0111] S132: Draw a cleaning trajectory diagram based on the coordinate information relative to the minimum rectangular frame.

[0112] Optionally, two adjacent trajectory points are connected in sequence with straight lines, and the last trajectory point is connected with the first trajectory point with a straight line to obtain a cleaning trajectory diagram.

[0113] A new white-bottomed rectangular image of the same size as the minimum rectangular frame image is created, and all the track points in the second track point set are drawn on the white-bottomed rectangular image according to the coordinate information of all the track points in the second track point set relative to the starting point of the minimum rectangular frame on the screen, and two adjacent track points are connected in sequence with straight lines, and finally the last track point is connected with the first track point with a straight line to obtain the cleaning track image. Figure 2E As shown, Figure 2E A scene schematic diagram of a cleaning trajectory diagram applied on a screen is provided for an embodiment of the present application.

[0114] S133: Acquire a target closed area in the cleaning trajectory map, and obtain a text mask map based on the target closed area.

[0115] Optionally, the target closed area in the cleaning trajectory map is obtained, the target closed area is transparently processed, and the area outside the target closed area is blocked to obtain the text mask map. Figure 2F As shown, Figure 2F A schematic diagram of a scene of a text mask image applied on a screen provided in an embodiment of the present application.

[0116] Among them, the cleaning trajectory map is firstly detected using a closed area detection algorithm to obtain a number of closed area contours, and then the closed area contours are merged and screened to obtain the target closed area.

[0117] In this embodiment, by obtaining the coordinate information of the second trajectory point set relative to the minimum rectangular box, a cleaning trajectory map is drawn based on the coordinate information relative to the minimum rectangular box, the target closed area in the cleaning trajectory map is obtained, and a text mask map is obtained based on the target closed area, thereby further determining the precise range of the text area to be circled.

[0118] Figure 5 A flowchart of another text detection method provided in an embodiment of the present application is shown below. Figure 5 is Figure 4 Based on the embodiment shown, a possible implementation method of obtaining the target closed area in the cleaning trajectory map in S133 is further described, as follows: Figure 5 As shown, the method of this embodiment is as follows:

[0119] S51: Acquire an initial closed area in the cleaning trajectory map.

[0120] Among them, the closed area detection algorithm for closed area detection is the algorithm provided by the openCV library. First, the image is preprocessed to remove noise and binarization, and then the Canny edge detection algorithm is used to find the edge of the closed area in the clear trajectory map. Then, the cv.findContours function is used to find the contour of the closed area, and finally several initial closed areas are obtained, which are expressed as:

[0121] Contours=[Contour1, ontour2,……, ontour j ] (4)

[0122] Among them, Contours represents a set of several initial closed areas. j Represents any initial closed region in a set of initial closed regions.

[0123] S52: merging and screening the initial closed areas to obtain a target closed area.

[0124] Optionally, the initial closed areas included in other initial closed areas are deleted, the initial closed areas whose areas are smaller than a preset threshold are deleted, and the remaining initial closed areas are determined as target closed areas.

[0125] The merging is to delete the initial closed area contained in the other initial closed areas, that is, to merge the two initial closed areas into one initial closed area, and the screening is to delete the initial closed area whose area is smaller than a preset threshold, where the preset threshold is represented by S min , the target closed area is the initial closed area remaining after merging and screening, expressed as:

[0126]

[0127] Among them, Contours clean represents a set of several target closed regions, Represents any one target closed area in a set of several target closed areas.

[0128] S133': Obtain a text mask image based on the target closed area.

[0129] Among them, a new rectangular image of the same size as the minimum rectangular frame image is created, and the outline of the target closed area is drawn on the rectangular image according to the target closed area. The inside of the target closed area is processed transparently, and the outside of the target closed area is blocked, for example, set to black, to obtain the text mask image.

[0130] In this embodiment, an initial closed area in the cleaning trajectory map is obtained, the initial closed area is merged and screened to obtain a target closed area, and a text mask map is obtained based on the target closed area, thereby obtaining a more accurate target closed area.

[0131] Figure 6 A flowchart of another text detection method provided in an embodiment of the present application is shown below. Figure 6 is Figure 1 Based on the embodiment shown, a possible implementation of S14 is further described as follows: Figure 6 As shown, the method of this embodiment is as follows:

[0132] S141: superimposing the text mask image onto the minimum rectangular frame to obtain a first text detection image.

[0133] The minimum rectangular frame is the minimum rectangular frame captured on the original screen and expanded and containing all the track points in the first track point set. The text mask image is superimposed on the minimum rectangular frame to obtain the first text detection image. Figure 2G As shown, Figure 2G A schematic diagram of a scene of a first text detection graph applied on a screen is provided for an embodiment of the present application.

[0134] S142: Input the first text detection image into a text detection model to obtain a target text detection frame.

[0135] Among them, the text detection model is a model such as a differentiable binarization network (Differentiable Binarization Network, DBNet), the first text detection image is input into these text detection models to obtain an initial text detection frame, and then the initial text detection frame is expanded and post-processed to obtain the target text detection frame.

[0136] In this embodiment, the text mask image is superimposed on the minimum rectangular frame to obtain a first text detection image, and the first text detection image is input into a text detection model to obtain a target text detection frame, thereby improving the accuracy of text detection.

[0137] Figure 7 A flowchart of another text detection method provided in an embodiment of the present application is shown below. Figure 7 is Figure 6 Based on the embodiment shown, a possible implementation of S142 is further described as follows: Figure 7 As shown, the method of this embodiment is as follows:

[0138] S1421: Input the first text detection image into a text detection model to obtain an initial text detection frame.

[0139] Wherein, the initial text detection frame is expressed as:

[0140]

[0141] Among them, Boxes1 represents the set containing the initial text detection box, [x i 1,y i 1,x i 2,y i 2] represents any initial text detection box. For any initial text detection box, [ i 1,y i 1] represents the upper left corner coordinate of the initial text detection box, [ i 2,y i 2] represents the lower right corner coordinate of the initial text detection box. The initial text detection box may be a regular rectangular box or an irregular arbitrary quadrilateral box, such as Figure 2H As shown, Figure 2H A schematic diagram of a scene of an initial text detection frame applied on a screen is provided for an embodiment of the present application.

[0142] S1422: Expand the initial text detection frame to obtain a target text detection frame.

[0143] In order to prevent the circled position from only selecting half of the text, the left and right expansion can significantly increase the text recognition accuracy. Therefore, the initial text detection frame is expanded, that is, the left and right are proportionally expanded to x times the width. The text detection frame obtained after expansion is the target text detection frame, which is expressed as:

[0144]

[0145] Among them, Boxes2 represents the set containing the target text detection box, dh i =|y i 1- i 2| represents the height of the initial text detection box, D width Indicates the maximum width of the minimum rectangular box. Similarly, the target text detection box may be a regular rectangular box or an irregular arbitrary quadrilateral box, such as Fig.2I As shown, Fig.2I A schematic diagram of a scene of a target text detection frame applied on a screen provided in an embodiment of the present application.

[0146] In this embodiment, the first text detection image is input into a text detection model to obtain an initial text detection frame, and the initial text detection frame is expanded to obtain a target text detection frame, thereby improving the accuracy of text recognition.

[0147] Figure 8 A structural diagram of a text detection device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the device of this embodiment includes: an acquisition module 801, a cleaning module 802, a first processing module 803 and a second processing module 804, wherein the acquisition module 801 is used to acquire a first track point set generated by a user performing a circle selection operation on the screen, and to acquire a minimum rectangular frame containing all track points of the first track point set.

[0148] The cleaning module 802 is used to perform data cleaning on the trajectory points in the first trajectory point set, remove outlier trajectory points, and obtain a second trajectory point set.

[0149] The first processing module 803 is used to obtain a text mask image based on the second trajectory point set and the minimum rectangular frame.

[0150] The second processing module 804 is used to superimpose the text mask image on the minimum rectangular frame to obtain a target text detection frame.

[0151] Optionally, the preset condition includes at least one of the following:

[0152] Deviation from track trend graph;

[0153] At the non-closed end of the trajectory overlap intersection;

[0154] Any of two adjacent trajectory points whose distance is less than a preset threshold.

[0155] Optionally, the first processing module 803 is specifically used to obtain the coordinate information of the second trajectory point set relative to the minimum rectangular box; draw a cleaning trajectory map based on the coordinate information relative to the minimum rectangular box; obtain the target closed area in the cleaning trajectory map, and obtain a text mask map based on the target closed area.

[0156] Optionally, the first processing module 803 is specifically used to obtain a target closed area in the cleaning trajectory map, perform transparency processing on the target closed area, and perform occlusion processing on the outside of the target closed area to obtain the text mask map.

[0157] Optionally, the first processing module 803 is specifically used to obtain an initial closed area in the cleaning trajectory map; merge and filter the initial closed area to obtain a target closed area.

[0158] Optionally, the first processing module 803 is specifically configured to delete an initial closed area included in other initial closed areas, delete an initial closed area whose area is smaller than a preset threshold, and determine the remaining initial closed areas as target closed areas.

[0159] Optionally, the first processing module 803 is specifically configured to connect two adjacent trajectory points in sequence with straight lines, and connect the last trajectory point with the first trajectory point with a straight line to obtain a cleaning trajectory diagram.

[0160] Optionally, the second processing module 804 is specifically configured to superimpose the text mask image onto the minimum rectangular frame to obtain a first text detection image; and input the first text detection image into a text detection model to obtain a target text detection frame.

[0161] Optionally, the second processing module 804 is specifically configured to input the first text detection image into a text detection model to obtain an initial text detection frame; and perform expansion processing on the initial text detection frame to obtain a target text detection frame.

[0162] Optionally, the first processing module 803 is specifically configured to expand the minimum rectangular frame to obtain an expanded minimum rectangular frame; and obtain a text mask image based on the second trajectory point set and the expanded minimum rectangular frame.

[0163] Optionally, the second processing module 804 is specifically configured to superimpose the text mask image onto the expanded minimum rectangular frame to obtain a target text detection frame.

[0164] The device of this embodiment corresponds to the technical solution that can be used to execute the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0165] An embodiment of the present application also provides an electronic device, including: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of any of the above-mentioned text detection method embodiments are implemented.

[0166] An embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of any of the above-mentioned text detection method embodiments are implemented.

[0167] An embodiment of the present application also provides a computer program product, which, when executed by a processor of a vehicle or a cloud server, implements the steps of any of the above-mentioned text detection method embodiments.

[0168] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, disk, CD, etc.), including several instructions to enable a terminal or a network-side device to execute the methods described in each embodiment of the present application.

[0170] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of the present application and the scope of protection of the claims, and these implementation methods are all within the protection of the present application.

Claims

1. A text detection method, characterized in that: include: Acquire a first track point set generated by a user performing a circle selection operation on the screen, and acquire a minimum rectangular frame containing all track points of the first track point set; Performing data cleaning on the trajectory points in the first trajectory point set, removing the trajectory points that meet the preset conditions, and obtaining a second trajectory point set; Based on the second trajectory point set and the minimum rectangular frame, a text mask image is obtained; Superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame; Among them, before obtaining the text mask map, first, based on the minimum rectangular frame map containing all the trajectory points in the second trajectory point set, a trajectory map of the same size with cleaning trajectory points drawn on it is obtained, and then closed area detection is performed on the trajectory map, and all detected closed areas are merged and screened, and finally, a picture of the same size as the trajectory map with cleaning trajectory points drawn on it is obtained, and the merged and screened closed areas are drawn on the picture to obtain the text mask map.

2. The method according to claim 1, characterized in that The preset condition includes at least one of the following: Deviation from track trend graph; At the non-closed end of the trajectory overlap intersection; Any of two adjacent trajectory points whose distance is less than a preset threshold.

3. The method according to claim 2, characterized in that The obtaining of a text mask image based on the second trajectory point set and the minimum rectangular frame includes: Obtaining coordinate information of the second trajectory point set relative to the minimum rectangular frame; Based on the coordinate information relative to the minimum rectangular frame, drawing a cleaning trajectory diagram; A target closed area in the cleaning trajectory map is obtained, and a text mask map is obtained based on the target closed area.

4. The method according to claim 3, characterized in that The step of acquiring a target closed area in the cleaning trajectory map and obtaining a text mask map based on the target closed area includes: The target closed area in the cleaning trajectory map is obtained, the target closed area is transparently processed, and the area outside the target closed area is blocked to obtain the text mask map.

5. The method according to claim 4, characterized in that The obtaining of the target closed area in the cleaning trajectory map comprises: Obtaining an initial closed area in the cleaning trajectory map; The initial closed areas are merged and screened to obtain a target closed area.

6. The method according to claim 5, characterized in that The merging and screening of the initial closed areas to obtain a target closed area includes: The initial closed areas contained in other initial closed areas are deleted, the initial closed areas whose areas are smaller than a preset threshold are deleted, and the remaining initial closed areas are determined as target closed areas.

7. The method according to claim 6, characterized in that Drawing a cleaning trajectory diagram based on the coordinate information relative to the minimum rectangular frame includes: Connect two adjacent trajectory points with straight lines in sequence, and connect the last trajectory point with the first trajectory point with a straight line to obtain a cleaning trajectory diagram.

8. The method according to any one of claims 1 to 7, characterized in that: The step of superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame includes: Superimposing the text mask image onto the minimum rectangular frame to obtain a first text detection image; The first text detection image is input into a text detection model to obtain a target text detection frame.

9. The method according to claim 8, characterized in that The step of inputting the first text detection image into a text detection model to obtain a target text detection frame includes: Inputting the first text detection image into a text detection model to obtain an initial text detection frame; The initial text detection frame is expanded to obtain a target text detection frame.

10. The method according to any one of claims 1 to 7, characterized in that: The obtaining of a text mask image based on the second trajectory point set and the minimum rectangular frame includes: Expanding the minimum rectangular frame to obtain an expanded minimum rectangular frame; Obtaining a text mask image based on the second trajectory point set and the expanded minimum rectangular frame; The step of superimposing the text mask image onto the minimum rectangular frame to obtain a target text detection frame includes: The text mask image is superimposed on the expanded minimum rectangular frame to obtain a target text detection frame.

11. A text detection device, characterized in that: include: An acquisition module, used to acquire a first track point set generated by a user performing a circle selection operation on the screen, and acquire a minimum rectangular frame containing all track points of the first track point set; a cleaning module, configured to clean the track points in the first track point set, remove outlier track points, and obtain a second track point set; A first processing module, configured to obtain a text mask image based on the second trajectory point set and the minimum rectangular frame; A second processing module is used to superimpose the text mask image on the minimum rectangular frame to obtain a target text detection frame; Among them, before obtaining the text mask map, first, based on the minimum rectangular frame map containing all the trajectory points in the second trajectory point set, a trajectory map of the same size with cleaning trajectory points drawn on it is obtained, and then closed area detection is performed on the trajectory map, and all detected closed areas are merged and screened, and finally, a picture of the same size as the trajectory map with cleaning trajectory points drawn on it is obtained, and the merged and screened closed areas are drawn on the picture to obtain the text mask map.

12. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the text detection method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the text detection method according to any one of claims 1 to 10 are implemented.

Images