Target stroke extraction method and device, equipment and medium

Abstract

The invention discloses a target stroke extraction method and device, equipment and a medium. The invention belongs to the technical field of image processing. The method comprises the following steps: aiming at a writing target area, identifying an outer contour of a written character; wherein the outer contour is in a closed shape formed by line segments which are connected end to end; determining a minimum external regular shape of the external contour, and determining candidate strokes according to the minimum external regular shape; and extracting key points of the candidate strokes, and screening the key points by adopting a preset rule to obtain a target stroke. By adopting the technical scheme, the purpose of screening out the candidate strokes before target stroke extraction can be achieved, the calculation amount of subsequent target stroke extraction is reduced, and the efficiency of target stroke extraction and the accuracy of an extraction result are improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a method, device, equipment and medium for extracting target strokes. Background Art

[0002] With the widespread adoption of smart mobile devices like smartphones and tablets, the application scope of handwriting input continues to expand. Examples include electronic teaching blackboards, electronic lecture manuscripts, and electronic signatures. Handwriting input allows users to enter text in the most natural and convenient way. To improve the reliability of handwritten text recognition within the target area, target stroke extraction technology has become extremely important.

[0003] In related technologies, the method of extracting target strokes is mainly to input the text image into a pre-trained stroke morphology analysis model, determine the image morphology features of all the text strokes in the text based on the model output results, analyze the stroke direction based on the image features and determine all the strokes in the text, and then determine the target strokes in the target area by comparing the position of each stroke in the text image with the position relationship of the target area in the text image.

[0004] However, existing techniques primarily extract strokes through morphological analysis of text strokes, which requires a large amount of prior information and results in inaccurate stroke extraction. Furthermore, target strokes must be determined based on the position of all strokes in the text image relative to the target region. This results in cumbersome extraction steps and low extraction efficiency. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method, device, equipment and medium for extracting target strokes, which can solve the problems of low extraction efficiency and inaccurate extraction results when extracting target strokes. By identifying the outer contour of the written text in the writing target area, determining the minimum circumscribed regular shape of the outer contour, and determining the candidate strokes with the minimum circumscribed regular shape, the purpose of screening out candidate strokes can be achieved, and the amount of calculation for subsequent extraction of target strokes can be reduced. By extracting the key points of the candidate strokes and screening the key points using preset rules, the target strokes are obtained, thereby improving the efficiency of extracting the target strokes and the accuracy of the extraction results.

[0006] In a first aspect, an embodiment of the present application provides a method for extracting target strokes, the method comprising:

[0007] For the writing target area, identifying the outer contour of the written text; wherein the outer contour is a closed shape formed by connecting line segments at the beginning and end;

[0008] Determining a minimum circumscribed regular shape of the outer contour, and determining candidate strokes based on the minimum circumscribed regular shape;

[0009] The key points of the candidate strokes are extracted, and the key points are screened using preset rules to obtain the target strokes.

[0010] This solution determines candidate strokes by identifying the minimum circumscribed regular shape of the outer contour of the written text, extracts and filters the key points of the candidate strokes to determine the target strokes, and can achieve the purpose of filtering out candidate strokes, reducing the amount of calculation for subsequent extraction of the target strokes. By extracting the key points of the candidate strokes and filtering the key points using preset rules, the target strokes are obtained, thereby improving the efficiency of extracting the target strokes and the accuracy of the extraction results.

[0011] Furthermore, the minimum circumscribed regular shape includes a minimum circumscribed rectangle;

[0012] Accordingly, determining candidate strokes based on the minimum circumscribed regular shape includes:

[0013] The strokes in the writing interface image that are distributed within the minimum circumscribed rectangle are determined as candidate strokes.

[0014] The beneficial effect of this solution is that by using the minimum circumscribed rectangle as the minimum regular shape of the external contour, it can avoid the problems of large computational complexity caused by the minimum circumscribed shape being too complex and inaccurate candidate stroke screening caused by the minimum circumscribed shape being too simple, thereby further improving the efficiency of target mural extraction.

[0015] Furthermore, the process of determining the minimum bounding rectangle includes:

[0016] Identify the coordinates of the contour points of the outer contour, and extract the minimum value of the horizontal coordinate, the minimum value of the vertical coordinate, the maximum value of the horizontal coordinate, and the maximum value of the vertical coordinate from the coordinates of the contour points;

[0017] The minimum horizontal coordinate and the minimum vertical coordinate are used as the first vertex coordinates, the minimum horizontal coordinate and the maximum vertical coordinate are used as the second vertex coordinates, the maximum horizontal coordinate and the maximum vertical coordinate are used as the third vertex coordinates, and the maximum horizontal coordinate and the minimum vertical coordinate are used as the fourth vertex coordinates to construct a minimum enclosing rectangle.

[0018] The beneficial effect of this solution is that by identifying the coordinates of the contour points of the external contour and constructing the minimum bounding rectangle according to the maximum values ​​of the horizontal and vertical coordinates of the contour points, the reliability of the construction of the minimum bounding rectangle can be improved.

[0019] Furthermore, each stroke in the writing interface image that is distributed within the minimum bounding rectangle is determined as a candidate stroke, including:

[0020] Obtaining the coordinates of the path points of each stroke in the writing interface image, and obtaining the coordinates of the vertices of the minimum circumscribed rectangle;

[0021] Comparing the coordinates of the waypoints of the current stroke with the vertex coordinates of the minimum circumscribed rectangle, and if the coordinates of the waypoints of the current stroke are all within the vertex coordinate range, determining the current stroke as a candidate stroke;

[0022] All strokes in the writing interface image are traversed to obtain all candidate strokes of the traversed writing interface image.

[0023] The beneficial effect of this solution is that by obtaining the coordinates of the waypoints of each stroke in the writing interface image and the vertex coordinates of the minimum circumscribed rectangle, and comparing the waypoint coordinates with the vertex coordinates, all candidate strokes of the writing interface image can be determined, which can simplify the steps of determining the candidate strokes and improve the efficiency of determining the candidate strokes.

[0024] Furthermore, the key points are screened using preset rules to obtain target strokes, including:

[0025] For each key point of the current candidate stroke, a geo-fencing algorithm is used to identify the number of intersections between the ray drawn from the key point and the outer contour, and based on the number of intersections of the ray drawn from each key point, whether the current candidate stroke is the target stroke is identified;

[0026] All candidate strokes are traversed to obtain all target strokes within the outer contour.

[0027] The beneficial effect of this solution is that, by using the geo-fence algorithm to identify the number of intersections between the key points of the current candidate stroke and the outer contour, it is determined whether the current candidate stroke is the target stroke, and all candidate strokes are traversed to obtain all target strokes within the outer contour, thereby improving the efficiency of target stroke recognition.

[0028] Furthermore, extracting key points of the candidate strokes includes:

[0029] At the starting position, middle position and end position of each candidate stroke, three key points of each candidate stroke are extracted.

[0030] The beneficial effect of this solution is that by extracting the three key points of each candidate stroke at the starting position, middle position and end position of each candidate stroke, the number of candidate stroke key points can be reduced, thereby reducing the number of screening and calculation times for the candidate stroke key points, and further improving the efficiency of target stroke extraction.

[0031] Furthermore, the process of determining the writing target area includes:

[0032] The blackboard writing image is input into a pre-built semantic algorithm model, and the writing target area is determined according to the pixel feature expression of the semantic algorithm model.

[0033] The beneficial effect of this solution is that by inputting the blackboard writing image into a pre-built semantic algorithm model and determining the writing target area according to the pixel feature expression of the semantic algorithm model, the efficiency of segmenting the writing target area can be improved.

[0034] In a second aspect, an embodiment of the present application provides a target stroke extraction device, the device comprising:

[0035] An external contour recognition module, for a writing target area, identifies the external contour of the written text; wherein the external contour is a closed shape formed by connecting line segments at the beginning and end;

[0036] a candidate stroke determination module, which determines a minimum circumscribed regular shape of the outer contour and determines candidate strokes based on the minimum circumscribed regular shape;

[0037] The target stroke screening module extracts key points of the candidate strokes and screens the key points using preset rules to obtain target strokes.

[0038] This solution determines candidate strokes by identifying the minimum circumscribed regular shape of the outer contour of the written text, extracts and filters the key points of the candidate strokes to determine the target strokes, and can achieve the purpose of filtering out candidate strokes, reducing the amount of calculation for subsequent extraction of the target strokes. By extracting the key points of the candidate strokes and filtering the key points using preset rules, the target strokes are obtained, thereby improving the efficiency of extracting the target strokes and the accuracy of the extraction results.

[0039] Furthermore, the minimum circumscribed regular shape includes a minimum circumscribed rectangle;

[0040] Accordingly, the candidate stroke determination module is specifically configured to:

[0041] The strokes in the writing interface image that are distributed within the minimum circumscribed rectangle are determined as candidate strokes.

[0042] The beneficial effect of this solution is that by using the minimum circumscribed rectangle as the minimum regular shape of the external contour, it can avoid the problems of large computational complexity caused by the minimum circumscribed shape being too complex and inaccurate candidate stroke screening caused by the minimum circumscribed shape being too simple, thereby further improving the efficiency of target mural extraction.

[0043] Furthermore, the candidate stroke determination module is further configured to:

[0044] Identify the coordinates of the contour points of the outer contour, and extract the minimum value of the horizontal coordinate, the minimum value of the vertical coordinate, the maximum value of the horizontal coordinate, and the maximum value of the vertical coordinate from the coordinates of the contour points;

[0045] The minimum horizontal coordinate and the minimum vertical coordinate are used as the first vertex coordinates, the minimum horizontal coordinate and the maximum vertical coordinate are used as the second vertex coordinates, the maximum horizontal coordinate and the maximum vertical coordinate are used as the third vertex coordinates, and the maximum horizontal coordinate and the minimum vertical coordinate are used as the fourth vertex coordinates to construct a minimum enclosing rectangle.

[0046] The beneficial effect of this solution is that by identifying the coordinates of the contour points of the external contour and constructing the minimum bounding rectangle according to the maximum values ​​of the horizontal and vertical coordinates of the contour points, the reliability of the construction of the minimum bounding rectangle can be improved.

[0047] Furthermore, the candidate stroke determination module is specifically configured to:

[0048] Obtaining the coordinates of the path points of each stroke in the writing interface image, and obtaining the coordinates of the vertices of the minimum circumscribed rectangle;

[0049] Comparing the coordinates of the waypoints of the current stroke with the vertex coordinates of the minimum circumscribed rectangle, and if the coordinates of the waypoints of the current stroke are all within the vertex coordinate range, determining the current stroke as a candidate stroke;

[0050] All strokes in the writing interface image are traversed to obtain all candidate strokes of the traversed writing interface image.

[0051] The beneficial effect of this solution is that by obtaining the coordinates of the waypoints of each stroke in the writing interface image and the vertex coordinates of the minimum circumscribed rectangle, and comparing the waypoint coordinates with the vertex coordinates, all candidate strokes of the writing interface image can be determined, which can simplify the steps of determining the candidate strokes and improve the efficiency of determining the candidate strokes.

[0052] Furthermore, the target stroke screening module is specifically used to:

[0053] For each key point of the current candidate stroke, a geo-fencing algorithm is used to identify the number of intersections between the ray drawn from the key point and the outer contour, and based on the number of intersections of the ray drawn from each key point, whether the current candidate stroke is the target stroke is identified;

[0054] All candidate strokes are traversed to obtain all target strokes within the outer contour.

[0055] The beneficial effect of this solution is that, by using the geo-fence algorithm to identify the number of intersections between the key points of the current candidate stroke and the outer contour, it is determined whether the current candidate stroke is the target stroke, and all candidate strokes are traversed to obtain all target strokes within the outer contour, thereby improving the efficiency of target stroke recognition.

[0056] Furthermore, the target stroke screening module is specifically used to:

[0057] At the starting position, middle position and end position of each candidate stroke, three key points of each candidate stroke are extracted.

[0058] The beneficial effect of this solution is that by extracting the three key points of each candidate stroke at the starting position, middle position and end position of each candidate stroke, the number of candidate stroke key points can be reduced, thereby reducing the number of screening and calculation times for the candidate stroke key points, and further improving the efficiency of target stroke extraction.

[0059] Furthermore, the external contour recognition module is further configured to:

[0060] The blackboard writing image is input into a pre-built semantic algorithm model, and the writing target area is determined according to the pixel feature expression of the semantic algorithm model.

[0061] The beneficial effect of this solution is that by inputting the blackboard writing image into a pre-built semantic algorithm model and determining the writing target area according to the pixel feature expression of the semantic algorithm model, the efficiency of segmenting the writing target area can be improved.

[0062] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0063] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0064] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0065] In an embodiment of the present application, the outer contour of the written text is identified for the writing target area; wherein the outer contour is a closed shape formed by line segments connected at the beginning and the end; the minimum circumscribed regular shape of the outer contour is determined, and candidate strokes are determined based on the minimum circumscribed regular shape; key points of the candidate strokes are extracted, and the key points are screened using preset rules to obtain target strokes. The above-mentioned target stroke extraction method can solve the problems of low extraction efficiency and unreliable extraction results when extracting target strokes. By identifying the outer contour of the written text for the writing target area, determining the minimum circumscribed regular shape of the outer contour, determining candidate strokes based on the minimum circumscribed regular shape, and extracting the key points of the candidate strokes, and screening the key points using preset rules to obtain target strokes, the calculation of stroke key points can be reduced, thereby improving the extraction efficiency of target strokes. At the same time, the target strokes are directly extracted using stroke key points, thereby improving the reliability of stroke extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 1 is a flow chart of a target stroke extraction method provided in an embodiment of the present application;

[0067] Figure 2 Schematic diagram of the geo-fencing algorithm provided for this application;

[0068] Figure 3 1 is a flow chart of a target stroke extraction method provided in an embodiment of the present application;

[0069] Figure 4 Schematic diagram of the structure of the target stroke extraction device provided in an embodiment of the present application;

[0070] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only parts related to the present application, not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0072] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0073] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0074] The target stroke extraction method, device, equipment and medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0075] In the existing technology, a geo-fence algorithm is used to extract strokes for all texts within the entire blackboard range interface. In the process of traversing the trajectory points of all strokes in the entire blackboard, when the predicted external contour shape of the blackboard text is more complex, the stroke extraction speed is often very slow and cannot meet the real-time requirements.

[0076] The geofencing algorithm aims to determine whether a stroke's trajectory point is within the target area, primarily through the ray method. Specifically, a ray is drawn from left to right along a stroke's trajectory point, and the number of times the ray crosses the predicted polygonal outer contour is determined. If the number is odd, the point is considered inside the area; if the number is even, the point is considered outside. The outer contour of any shape predicted by segmentation algorithms and image processing techniques for all text within the blackboard range is often composed of numerous line segments, resulting in a lengthy extraction process for the target stroke.

[0077] In an embodiment of the present invention, after obtaining the outer contour of the target writing area, the minimum bounding rectangle of the outer contour is further determined through image processing technology. Non-target strokes in the writing are quickly filtered out by first determining whether each point of all strokes in the blackboard writing trajectory data is within the minimum bounding rectangle. Then, for each remaining stroke, the beginning, middle, and end points are extracted to construct a key point set, further reducing the number of points to be determined. Based on the key point set, a geo-fencing algorithm is used to determine whether the stroke is within the predicted outer contour, thereby accelerating stroke extraction.

[0078] Figure 1 FIG. 1 is a flow chart of a target stroke extraction method provided in an embodiment of the present application. Figure 1 As shown, the specific steps include:

[0079] S101, identifying the outer contour of the written text in the writing target area; wherein the outer contour is a closed shape formed by connecting line segments at the beginning and the end;

[0080] First, the application scenario of this solution can be a scenario of performing stroke extraction or character recognition on handwritten text, especially a scenario of recognizing target strokes or target characters in a specific area of ​​handwritten text.

[0081] Based on the above usage scenarios, it can be understood that the executor of this application can be an electronic device with image segmentation, image recognition and data processing capabilities, such as: smart terminals such as mobile phones, tablets, desktop computers and large conference display panels.

[0082] Among them, the writing target area can be a text area in one or more specific areas with a hierarchical relationship in the handwritten text. For example: each text box area in a mind map, each component structure area in the blackboard structured writing, and the numbered area in the bullet point, etc. The outer contour of the written text can be the overall outer contour of all the written text in the writing target area. The outer contour is a closed shape composed of line segments connected end to end. The shape and size of the outer contour are the same as the shape and size of the writing target area, and the shape can be an irregular shape, such as a closed pentagon, hexagon or polygon with more sides composed of multiple line segments connected end to end.

[0083] In one embodiment, for a predetermined writing target area, an image recognition algorithm can be used to perform grayscale processing, noise reduction, binarization, and normalization on the image of the writing target area, thereby converting the image of the writing target area into an image area composed of black and white pixels. The outer contour of each written character in the writing target area is determined by identifying the boundaries of pixels of different colors in the image area. The overall outer contour of the written characters in the writing target area is determined based on the range of the writing target area and the outer contour of each written character. The black and white pixels can be the text pixels representing the written characters in the writing target area and the background pixels representing the text background, respectively.

[0084] In a feasible embodiment, optionally, the process of determining the writing target area includes:

[0085] The blackboard writing image is input into a pre-built semantic algorithm model, and the writing target area is determined according to the pixel feature expression of the semantic algorithm model.

[0086] Among them, the blackboard image can be a picture composed of handwritten blackboard writing. The blackboard image can be displayed by a display on an electronic device. The semantic algorithm model can be a semantic segmentation model, which is used to segment the blackboard image. The semantic segmentation can be a segmentation method that takes plane images as input and converts them into mask blocks with highlighted regions of interest. The semantic segmentation can assign a category ID to each pixel in the image according to its object of interest, and is divided into standard semantic segmentation and instance-aware semantic segmentation. The standard semantic segmentation is also called full-pixel semantic segmentation, which is the process of classifying each pixel as belonging to an object class; the instance-aware semantic segmentation is a subtype of standard semantic segmentation or full-pixel semantic segmentation, which classifies each pixel as belonging to an object class and the entity ID of the class. The object of interest of the semantic algorithm model in this scheme can be the various structural areas in the blackboard image that constitute the blackboard structure. The pixel feature expression can be the category or ID to which each pixel in the blackboard image belongs.

[0087] In one embodiment, the blackboard image can be input into a pre-built semantic algorithm model, and the pixel feature expression of each pixel in the blackboard image can be obtained through the semantic algorithm model, and the writing target area can be determined based on the pixel feature expression. For example: based on the category or ID of each pixel in the blackboard image, pixels of the same category or ID are classified as pixels in the same structural area in the blackboard image, and based on the pixel classification results, one or more different structural areas in the blackboard image and the location of each structural area can be obtained.

[0088] This solution can improve the efficiency of segmenting the writing target area by inputting the blackboard writing image into a pre-built semantic algorithm model and determining the writing target area based on the pixel feature expression of the semantic algorithm model.

[0089] S102, determining a minimum circumscribed regular shape of the outer contour, and determining candidate strokes based on the minimum circumscribed regular shape;

[0090] The minimum circumscribed regular shape may be a minimum regular polygon that completely encloses the outer contour, such as a rectangle, circle, triangle, parallelogram, or regular polygon. The candidate strokes may be all strokes within the minimum circumscribed regular shape. Since the minimum circumscribed regular shape includes the outer contour, the candidate strokes include all strokes within the outer contour.

[0091] In one embodiment, the minimum circumscribed regular shape of the outer contour may be determined according to the shape and area of ​​the outer contour, and candidate strokes may be determined by identifying whether each stroke in the handwritten text is within the coverage of the minimum circumscribed regular shape.

[0092] For example, a coordinate system can be pre-established for a handwritten text image. The coordinate values ​​of each stroke trajectory point in the handwritten text image and the vertex coordinate values ​​of the minimum circumscribed regular shape can be determined based on the coordinate system. The strokes within the coverage of the minimum circumscribed regular shape can then be determined based on these coordinate values. If the coordinate values ​​of all trajectory points of a stroke are within the vertex coordinate value range of the minimum circumscribed regular shape, the stroke is determined to be a candidate stroke; otherwise, the stroke is determined not to be a candidate stroke.

[0093] In one embodiment, the text and strokes in the handwritten text image can be obtained using an OCR (Optical Character Recognition) algorithm. OCR refers to the process by which an electronic device (such as a scanner or digital camera) examines characters printed on paper, determines their shape by detecting dark and light patterns, and then uses character recognition methods to translate the shape into computer text. The text in a paper document is optically converted into a black and white dot matrix image file, and recognition software is used to convert the text in the image into a text format for further editing and processing by word processing software.

[0094] S103, extracting key points of the candidate strokes, and screening the key points using preset rules to obtain target strokes.

[0095] Among them, the key points of the candidate strokes can be the trajectory points of the candidate strokes obtained after sampling the candidate strokes at a certain sampling rate. The key points of the candidate strokes can be used to calculate the coordinate position, inclination angle, stroke direction and stroke shape of the candidate strokes. The number of key points of the candidate strokes can be determined according to the complexity of the candidate strokes. The preset rules can be rules for screening the key points of strokes within the external contour. For example: calculate the number of intersections of the ray with the key point of the candidate stroke as the endpoint and the external contour. If the number of intersections is an odd number, it is determined that the key point of the stroke is within the external contour. The target stroke may be a candidate stroke within the range of the external contour.

[0096] In one embodiment, the key points of the candidate strokes can be extracted by sampling, and the coordinate values ​​of the key points can be determined based on a coordinate system pre-constructed for the handwritten text image. The key points can be screened using preset rules based on the coordinate values ​​to determine the key points within the outer contour, and the target stroke can be determined based on the key points within the outer contour.

[0097] In a feasible embodiment, optionally, extracting key points of the candidate strokes includes:

[0098] At the starting position, middle position and end position of each candidate stroke, three key points of each candidate stroke are extracted.

[0099] The starting position of each candidate stroke may be the starting position of each candidate stroke. The ending position may be the ending position of each candidate stroke. The starting position and the ending position are generally the endpoints of the candidate stroke. The intermediate position may be the position between the endpoints of the candidate stroke. The heights and spacings of the starting position, intermediate position, and ending position may be the same or different.

[0100] In one embodiment, the trajectory points at the starting position, middle position and end position of each candidate stroke can be used as the key points of the candidate stroke, and three key points of each candidate stroke can be extracted based on the trajectory points at the starting position, middle position and end position.

[0101] The technical solution provided in the embodiment of the present application can reduce the number of candidate stroke key points by extracting three key points of each candidate stroke at the starting position, middle position and end position of each candidate stroke, thereby reducing the number of screening and calculation times for the candidate stroke key points, and further improving the efficiency of target stroke extraction.

[0102] In a feasible embodiment, optionally, the key points are screened using preset rules to obtain target strokes, including:

[0103] For each key point of the current candidate stroke, a geo-fencing algorithm is used to identify the number of intersections between the ray drawn from the key point and the outer contour, and based on the number of intersections of the ray drawn from each key point, whether the current candidate stroke is the target stroke is identified;

[0104] All candidate strokes are traversed to obtain all target strokes within the outer contour.

[0105] Among them, the geofence can be a virtual geographical boundary enclosed by a virtual fence, that is, a polygon composed of a large number of location points. For example: when a mobile phone enters, leaves a specific geographical area, or moves within the area, the mobile phone can automatically receive notifications and warnings. The geofence algorithm can be abstracted as a method to determine whether a point belongs to a polygonal area. Commonly used geofence algorithms include the ray method. The ray method can be performed by intersecting the ray with each edge of the polygon and determining the relationship between the point and the polygon based on the number of intersections. The key point-derived ray can be a ray with the key point as the endpoint, drawn horizontally to the right or vertically upward.

[0106] Figure 2 Schematic diagram of the geofencing algorithm provided for this application.

[0107] like Figure 2 As shown in the figure, key point 1 and key point 2 are key points in the current candidate strokes, which can be key points in the same candidate stroke or key points in different candidate strokes. The frame of the irregular figure is the outer contour. With key point 1 and key point 2 as endpoints, draw rays horizontally to the right, such as Figure 2As shown, the number of intersections between the ray derived from key point 1 and the outer contour is 2, and the number of intersections between the ray derived from key point 2 and the outer contour is 3. It can be understood that when the number of intersections between the ray derived from a key point and the outer contour is an odd number, the key point is within the range of the outer contour; when the number of intersections between the ray derived from a key point and the outer contour is an even number, the key point is outside the range of the outer contour.

[0108] In one embodiment, a geo-fencing algorithm can be used to draw rays in a uniform direction for each key point of the current candidate stroke. The number of intersections between the rays drawn from the key points and the outer contour is calculated. Based on the number of intersections, it is determined whether the key point is within the outer contour. All key points of the same candidate stroke are traversed to determine whether all key points of the candidate stroke are within the outer contour. If so, the current candidate stroke is identified as the target stroke. All candidate strokes are then traversed to obtain all target strokes within the outer contour.

[0109] In one embodiment, the line segment coordinate range of each edge of the external contour and the ray coordinate range of the key point derived ray can be calculated through a pre-constructed coordinate system, and the line segment coordinate range and the ray coordinate range are used to determine whether the key point intersects with each boundary of the external contour, thereby determining the number of intersections between the key point and the external contour. For example, if the coordinates of the key point are (3, 5), the horizontal coordinate range of the key point derived ray of the key point is [3, +∞), and the vertical coordinate range is y=5. If one edge of the external contour is expressed as y=x+1, and x∈[2,8], then when x=4, the vertical coordinate value of the external contour edge is 5, and the vertical coordinate value of the key point derived ray is also 5. At this time, the key point derived ray intersects with the external contour edge.

[0110] The technical solution provided in the embodiment of the present application uses a geo-fence algorithm to identify the number of intersections between the key points of the current candidate stroke and the external contour, determines whether the current candidate stroke is the target stroke, and traverses all candidate strokes to obtain all target strokes within the external contour, thereby improving the efficiency of identifying the target strokes.

[0111] The technical solution provided in the embodiment of the present application is to identify the outer contour of the written text in the writing target area; wherein the outer contour is a closed shape formed by line segments connected at the beginning and the end; determine the minimum circumscribed regular shape of the outer contour, and determine the candidate strokes based on the minimum circumscribed regular shape; extract the key points of the candidate strokes, and filter the key points using preset rules to obtain the target strokes. The above-mentioned target stroke extraction method can solve the problems of low extraction efficiency and unreliable extraction results in target stroke extraction. By identifying the outer contour of the written text in the writing target area, determining the minimum circumscribed regular shape of the outer contour, determining the candidate strokes using the minimum circumscribed regular shape, and extracting the key points of the candidate strokes, and filtering the key points using preset rules to obtain the target strokes, the calculation of the stroke key points can be reduced, thereby improving the extraction efficiency of the target strokes. At the same time, the target strokes can be directly extracted using the stroke key points, thereby improving the reliability of the stroke extraction.

[0112] Figure 3 FIG. 1 is a flow chart of a target stroke extraction method provided in an embodiment of the present application. Figure 3 As shown, the specific steps include:

[0113] S301, identifying the outer contour of the written text in the writing target area; wherein the outer contour is a closed shape formed by connecting line segments at the beginning and the end;

[0114] S302, determining a minimum circumscribed rectangle of the outer contour, and determining each stroke in the writing interface image that is distributed within the minimum circumscribed rectangle as a candidate stroke;

[0115] The minimum bounding rectangle may be the minimum rectangle that contains all boundaries of the outer contour. The writing interface image may be an image of handwritten text. The writing interface image may be an image of text written directly by a user online on a smart terminal screen, or an image of text written offline in a paper document.

[0116] In one embodiment, the minimum enclosing rectangle of the outer contour can be determined based on the shape and area of ​​the outer contour, and the coordinate range covered by the minimum enclosing rectangle and the coordinate range of each stroke in the writing interface image or the trajectory point coordinates of each stroke can be determined based on a coordinate system pre-constructed for the writing interface. Based on whether the coordinate range or the trajectory point coordinates are within the coordinate range covered by the minimum enclosing rectangle, it is determined whether the stroke is distributed inside the minimum enclosing rectangle, and the strokes in the writing interface image that are distributed inside the minimum enclosing rectangle are determined as candidate strokes.

[0117] In one embodiment, optionally, the process of determining the minimum bounding rectangle includes:

[0118] Identify the coordinates of the contour points of the outer contour, and extract the minimum value of the horizontal coordinate, the minimum value of the vertical coordinate, the maximum value of the horizontal coordinate, and the maximum value of the vertical coordinate from the coordinates of the contour points;

[0119] The minimum horizontal coordinate and the minimum vertical coordinate are used as the first vertex coordinates, the minimum horizontal coordinate and the maximum vertical coordinate are used as the second vertex coordinates, the maximum horizontal coordinate and the maximum vertical coordinate are used as the third vertex coordinates, and the maximum horizontal coordinate and the minimum vertical coordinate are used as the fourth vertex coordinates to construct a minimum enclosing rectangle.

[0120] The contour point coordinates may be the pixel point coordinates constituting the outer contour.

[0121] In one embodiment, the coordinates of the contour points of the external contour can be identified by image processing techniques, and the minimum abscissa value, the minimum ordinate value, the maximum abscissa value, and the maximum ordinate value of the contour point coordinates can be extracted. The minimum abscissa value and the minimum ordinate value are used as the coordinates of a first vertex, the minimum abscissa value and the maximum ordinate value are used as the coordinates of a second vertex, the maximum abscissa value and the maximum ordinate value are used as the coordinates of a third vertex, and the maximum abscissa value and the minimum ordinate value are used as the coordinates of a fourth vertex. The above four vertices are used as the vertices of the minimum enclosing rectangle to construct the minimum enclosing rectangle.

[0122] The technical solution provided in the embodiment of the present application can improve the reliability of constructing the minimum bounding rectangle by identifying the coordinates of the contour points of the external contour and constructing the minimum bounding rectangle based on the maximum values ​​of the horizontal and vertical coordinates of the contour points.

[0123] In one embodiment, optionally, determining the strokes in the writing interface image that are distributed within the minimum bounding rectangle as candidate strokes includes:

[0124] Obtaining the coordinates of the path points of each stroke in the writing interface image, and obtaining the coordinates of the vertices of the minimum circumscribed rectangle;

[0125] Comparing the coordinates of the waypoints of the current stroke with the vertex coordinates of the minimum circumscribed rectangle, and if the coordinates of the waypoints of the current stroke are all within the vertex coordinate range, determining the current stroke as a candidate stroke;

[0126] All strokes in the writing interface image are traversed to obtain all candidate strokes of the traversed writing interface image.

[0127] The coordinates of the path points of each stroke may be the coordinates of the trajectory points of each stroke, or the coordinates of the key points of each stroke. The path points of each stroke include at least the starting point, the middle point and the midpoint of each stroke.

[0128] In one embodiment, the coordinates of the path points of each stroke in the writing interface image and the vertex coordinates of the minimum bounding rectangle can be obtained based on a coordinate system pre-constructed for the writing interface image. The coordinates of all path points of the current stroke are compared with the vertex coordinates of the minimum bounding rectangle. If all path point coordinates of the current stroke are within the vertex coordinate range, the current stroke is determined to be a candidate stroke. All strokes in the writing interface image are traversed, and all candidate strokes in the writing interface image are obtained based on the relationship between the path point coordinates of the strokes and the vertex coordinates of the minimum bounding rectangle.

[0129] The technical solution provided in the embodiment of the present application determines all candidate strokes of the writing interface image by obtaining the coordinates of the path points of each stroke in the writing interface image and the vertex coordinates of the minimum circumscribed rectangle, and comparing the path point coordinates with the vertex coordinates. This can simplify the steps of determining the candidate strokes and improve the efficiency of determining the candidate strokes.

[0130] S303: extract key points of the candidate strokes, and filter the key points using preset rules to obtain target strokes.

[0131] The technical solution provided in the embodiment of the present application, by using the minimum circumscribed rectangle as the minimum regular shape of the external contour, can avoid the problems of large calculation amount caused by the minimum circumscribed shape being too complex and inaccurate screening of candidate strokes caused by the minimum circumscribed shape being too simple, thereby further improving the efficiency of target mural extraction.

[0132] Figure 4 Schematic diagram of the structure of the target stroke extraction device provided in the embodiment of the present application. Figure 4 As shown, specifically including the following:

[0133] The outer contour recognition module 401 recognizes the outer contour of the written text in the writing target area; wherein the outer contour is a closed shape formed by connecting line segments at the beginning and the end;

[0134] A candidate stroke determination module 402 determines a minimum circumscribed regular shape of the outer contour, and determines candidate strokes based on the minimum circumscribed regular shape;

[0135] The target stroke screening module 403 extracts key points of the candidate strokes and screens the key points using preset rules to obtain target strokes.

[0136] Furthermore, the minimum circumscribed regular shape includes a minimum circumscribed rectangle;

[0137] Accordingly, the candidate stroke determination module 402 is specifically configured to:

[0138] The strokes in the writing interface image that are distributed within the minimum circumscribed rectangle are determined as candidate strokes.

[0139] Furthermore, the candidate stroke determination module 402 is further configured to:

[0140] Identify the coordinates of the contour points of the outer contour, and extract the minimum value of the horizontal coordinate, the minimum value of the vertical coordinate, the maximum value of the horizontal coordinate, and the maximum value of the vertical coordinate from the coordinates of the contour points;

[0141] The minimum horizontal coordinate and the minimum vertical coordinate are used as the first vertex coordinates, the minimum horizontal coordinate and the maximum vertical coordinate are used as the second vertex coordinates, the maximum horizontal coordinate and the maximum vertical coordinate are used as the third vertex coordinates, and the maximum horizontal coordinate and the minimum vertical coordinate are used as the fourth vertex coordinates to construct a minimum enclosing rectangle.

[0142] Furthermore, the candidate stroke determination module 402 is specifically configured to:

[0143] Obtaining the coordinates of the path points of each stroke in the writing interface image, and obtaining the coordinates of the vertices of the minimum circumscribed rectangle;

[0144] Comparing the coordinates of the waypoints of the current stroke with the vertex coordinates of the minimum circumscribed rectangle, and if the coordinates of the waypoints of the current stroke are all within the vertex coordinate range, determining the current stroke as a candidate stroke;

[0145] All strokes in the writing interface image are traversed to obtain all candidate strokes of the traversed writing interface image.

[0146] Furthermore, the target stroke screening module 403 is specifically configured to:

[0147] For each key point of the current candidate stroke, a geo-fencing algorithm is used to identify the number of intersections between the ray drawn from the key point and the outer contour, and based on the number of intersections of the ray drawn from each key point, whether the current candidate stroke is the target stroke is identified;

[0148] All candidate strokes are traversed to obtain all target strokes within the outer contour.

[0149] Furthermore, the target stroke screening module 403 is specifically configured to:

[0150] At the starting position, middle position and end position of each candidate stroke, three key points of each candidate stroke are extracted.

[0151] Furthermore, the external contour recognition module 401 is further configured to:

[0152] The blackboard writing image is input into a pre-built semantic algorithm model, and the writing target area is determined according to the pixel feature expression of the semantic algorithm model.

[0153] The technical solution provided by the embodiment of the present application is as follows: an external contour recognition module, for a writing target area, identifies the external contour of the written text; wherein the external contour is a closed shape formed by line segments connected at the beginning and the end; a candidate stroke determination module, for determining the minimum circumscribed regular shape of the external contour, and determining the candidate strokes based on the minimum circumscribed regular shape; a target stroke screening module, for extracting the key points of the candidate strokes, and screening the key points using preset rules to obtain the target strokes. The above-mentioned target stroke extraction device can solve the problems of low extraction efficiency and unreliable extraction results in target stroke extraction. By identifying the external contour of the written text in the writing target area, determining the minimum circumscribed regular shape of the external contour, determining the candidate strokes using the minimum circumscribed regular shape, and extracting the key points of the candidate strokes, and screening the key points using preset rules to obtain the target strokes, the calculation of the stroke key points can be reduced, thereby improving the extraction efficiency of the target strokes. At the same time, the target strokes can be directly extracted using the stroke key points, thereby improving the reliability of the stroke extraction.

[0154] The target stroke extraction device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0155] The target stroke extraction device in the embodiment of the present application can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0156] The target stroke extraction device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0157] Figure 5 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a program or instruction stored in the memory 502 and executable on the processor 501. When the program or instruction is executed by the processor 501, each process of the above-mentioned target stroke extraction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0158] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0159] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned target stroke extraction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0160] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0161] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned target stroke extraction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0162] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0163] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0165] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0166] The above are only preferred embodiments of the present application and the technical principles employed. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that are possible for those skilled in the art will not depart from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include more other equivalent embodiments without departing from the concept of the present application. The scope of the present application is determined by the scope of the claims.

Claims

1. A method for extracting target strokes, characterized in that: The method comprises: For the writing target area, identifying the outer contour of the written text; wherein the outer contour is a closed shape formed by connecting line segments at the beginning and end; Determining a minimum circumscribed regular shape of the outer contour, and determining candidate strokes based on the minimum circumscribed regular shape; The key points of the candidate strokes are extracted, and the key points are screened using preset rules to obtain the target strokes.

2. The method according to claim 1, characterized in that The minimum circumscribed regular shape includes a minimum circumscribed rectangle; Accordingly, determining candidate strokes based on the minimum circumscribed regular shape includes: The strokes in the writing interface image that are distributed within the minimum circumscribed rectangle are determined as candidate strokes.

3. The method according to claim 2, characterized in that The process of determining the minimum circumscribed rectangle includes: Identify the coordinates of the contour points of the outer contour, and extract the minimum value of the horizontal coordinate, the minimum value of the vertical coordinate, the maximum value of the horizontal coordinate, and the maximum value of the vertical coordinate from the coordinates of the contour points; The minimum horizontal coordinate and the minimum vertical coordinate are used as the first vertex coordinates, the minimum horizontal coordinate and the maximum vertical coordinate are used as the second vertex coordinates, the maximum horizontal coordinate and the maximum vertical coordinate are used as the third vertex coordinates, and the maximum horizontal coordinate and the minimum vertical coordinate are used as the fourth vertex coordinates to construct a minimum enclosing rectangle.

4. The method according to claim 2, characterized in that Strokes in the writing interface image that are distributed within the minimum bounding rectangle are determined as candidate strokes, including: Obtaining the coordinates of the path points of each stroke in the writing interface image, and obtaining the coordinates of the vertices of the minimum circumscribed rectangle; Comparing the coordinates of the waypoints of the current stroke with the vertex coordinates of the minimum circumscribed rectangle, and if the coordinates of the waypoints of the current stroke are all within the vertex coordinate range, determining the current stroke as a candidate stroke; All strokes in the writing interface image are traversed to obtain all candidate strokes of the traversed writing interface image.

5. The method according to claim 1, wherein The key points are screened using preset rules to obtain target strokes, including: For each key point of the current candidate stroke, a geo-fencing algorithm is used to identify the number of intersections between the ray drawn from the key point and the outer contour, and based on the number of intersections of the ray drawn from each key point, whether the current candidate stroke is the target stroke is identified; All candidate strokes are traversed to obtain all target strokes within the outer contour.

6. The method according to claim 1, characterized in that Extracting key points of the candidate strokes includes: At the starting position, middle position and end position of each candidate stroke, three key points of each candidate stroke are extracted.

7. The method according to claim 1, characterized in that The process of determining the writing target area includes: The blackboard writing image is input into a pre-built semantic algorithm model, and the writing target area is determined according to the pixel feature expression of the semantic algorithm model.

8. A target stroke extraction device, characterized in that: The device comprises: An external contour recognition module, for a writing target area, identifies the external contour of the written text; wherein the external contour is a closed shape formed by connecting line segments at the beginning and end; a candidate stroke determination module, which determines a minimum circumscribed regular shape of the outer contour and determines candidate strokes based on the minimum circumscribed regular shape; The target stroke screening module extracts key points of the candidate strokes and screens the key points using preset rules to obtain target strokes.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the target stroke extraction method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the target stroke extraction method according to any one of claims 1 to 7 are implemented.

Images