Image processing methods and apparatuses, electronic devices, storage media

By using a coordinate attention mechanism to extract image features, the false detection problem in existing line detection methods is solved, achieving more accurate line detection, reducing interference from non-line pixels, and improving detection accuracy.

CN117523219BActive Publication Date: 2026-11-17BEIJING KINGSOFT OFFICE SOFTWARE INC +2
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210900443.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-28
Publication Date
2026-11-17
Estimated Expiration
2042-07-28

AI Technical Summary

Technical Problem

Existing line detection methods rely on binarization algorithms, which can easily lead to false line detection in documents with color blocks, PPTs, photographs, etc., and can also cause false underline detection when the page is blurry, resulting in insufficient accuracy.

Method used

A coordinate attention mechanism is used to extract features from the image. A second feature map is generated through layer-by-layer feature extraction and coordinate attention processing, which reduces interference from non-line pixels, avoids binarization, and accurately detects line objects.

Benefits of technology

It improves the accuracy of line detection, reduces false detections of documents with color blocks and underlines caused by page blur, and can more accurately locate line objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117523219B_ABST
    Figure CN117523219B_ABST
Patent Text Reader

Abstract

This application relates to the field of computer technology and discloses an image processing method, comprising: acquiring an image to be detected; performing layer-by-layer feature extraction on the image according to a preset feature extraction path to obtain multiple first feature maps; performing coordinate attention mechanism processing on at least one first feature map to obtain a second feature map; and determining line objects in the image based on the first feature map and the second feature map. This method can more accurately detect lines in an image. This application also discloses an image processing apparatus, an electronic device, and a storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, such as an image processing method and apparatus, electronic devices, and storage media. Background Technology

[0002] In daily office scenarios, image-based documents are ubiquitous. Users often need to extract content from document images, and this content often contains various line objects, such as underlines, decorative lines, and partial underlines.

[0003] Line object detection is an important topic in document image analysis and understanding research. In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related technology:

[0004] Existing line detection methods require first binarizing the document image, then opening the binarized image to obtain the contour, thus achieving preliminary line detection. Then, numerous logical operations are needed to filter out abnormal lines. Because current technologies heavily rely on binarization algorithms, they are prone to generating numerous false line detections in documents with color blocks, PPTs, photographs, and document titles. When the page is blurry, it easily generates numerous false underlines in text, with English text being the most severely affected. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This disclosure provides an image processing method, apparatus, electronic device, and storage medium that can more accurately detect line objects in an image.

[0007] In some embodiments, the image processing method includes: acquiring an image to be detected; performing layer-by-layer feature extraction on the image according to a preset feature extraction path to obtain multiple first feature maps; performing coordinate attention mechanism processing on at least one first feature map to obtain a second feature map; and determining line objects in the image based on the first feature map and the second feature map.

[0008] In some embodiments, performing coordinate attention mechanism processing on at least one first feature map to obtain a second feature map includes: performing coordinate attention mechanism processing on at least one first feature map as a reference feature map to obtain a first directional attention map and a second directional attention map; and weighting the first directional attention map and the second directional attention map onto the reference feature map to obtain the second feature map.

[0009] In some embodiments, performing coordinate attention mechanism processing on at least one first feature map as a reference feature map to obtain a first directional attention map and a second directional attention map includes: determining at least one first feature map as a reference feature map; processing each reference feature map, including: performing global pooling processing on each channel of the reference feature map along the vertical and horizontal directions of the reference feature map respectively to generate a first directional sensing feature map and a second directional sensing feature map; and obtaining the first directional attention map and the second directional attention map based on the first directional sensing feature map and the second directional sensing feature map.

[0010] In some embodiments, obtaining a first directional attention map and a second directional attention map based on the first directional sensing feature map and the second directional sensing feature map includes: concatenating the first directional sensing feature map and the second directional sensing feature map to obtain a concatenated image; performing a convolutional transformation on the concatenated image to generate an intermediate feature map that aggregates horizontal and vertical spatial information; segmenting the intermediate feature map into a first tensor in the horizontal direction and a second tensor in the vertical direction according to the spatial dimension; performing convolutional processing on the first tensor so that the number of channels in the first tensor is the same as the number of channels in the reference feature map; performing nonlinear activation on the convolutionally processed first tensor to obtain a first directional attention map; performing convolutional processing on the second tensor so that the number of channels in the second tensor is the same as the number of channels in the reference feature map; and performing nonlinear activation on the convolutionally processed second tensor to obtain a second directional attention map.

[0011] In some embodiments, the step of weighting the first directional attention map and the second directional attention map onto the reference feature map to obtain the second feature map includes: multiplying the first directional attention map and the second directional attention map with the reference feature map to obtain the second feature map.

[0012] In some embodiments, determining line objects in the image based on the first feature map and the second feature map includes: performing feature fusion on the first feature map and the second feature map to obtain image features of the image; and determining line objects in the image based on the image features.

[0013] In some embodiments, determining line objects in the image based on the image features includes: obtaining the probability that each pixel in the image belongs to a line pixel based on the image features; determining the pixels in the image with a probability greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

[0014] In some embodiments, determining line objects in the image based on target pixels includes: obtaining n regions in the image composed of consecutively connected target pixels; where n is a positive integer; obtaining the minimum bounding quadrilateral of each region; and determining the obtained minimum bounding quadrilateral as the line object in the image.

[0015] In some embodiments, the image processing apparatus includes: an acquisition module configured to acquire an image to be detected; a feature extraction module configured to perform layer-by-layer feature extraction on the image according to a preset feature extraction path to obtain a plurality of first feature maps; perform coordinate attention mechanism processing on at least one first feature map to obtain a second feature map; and a detection module configured to determine line objects in the image based on the first feature map and the second feature map.

[0016] In some embodiments, the feature extraction module is configured to perform coordinate attention mechanism processing on at least one first feature map to obtain a second feature map by: performing coordinate attention mechanism processing on at least one first feature map as a reference feature map to obtain a first directional attention map and a second directional attention map; and weighting the first directional attention map and the second directional attention map onto the reference feature map to obtain the second feature map.

[0017] In some embodiments, the feature extraction module is configured to perform coordinate attention mechanism processing on at least one first feature map as a reference feature map to obtain a first directional attention map and a second directional attention map by: determining at least one first feature map as a reference feature map; processing each reference feature map, including: performing global pooling processing on each channel of the reference feature map along the vertical and horizontal directions of the reference feature map respectively to generate a first directional sensing feature map and a second directional sensing feature map; and obtaining the first directional attention map and the second directional attention map based on the first directional sensing feature map and the second directional sensing feature map.

[0018] In some embodiments, the feature extraction module is configured to obtain a first directional attention map and a second directional attention map based on the first directional sensing feature map and the second directional sensing feature map in the following manner: concatenating the first directional sensing feature map and the second directional sensing feature map to obtain a concatenated image; performing a convolutional transformation on the concatenated image to generate an intermediate feature map that aggregates horizontal and vertical spatial information; segmenting the intermediate feature map into a first tensor in the horizontal direction and a second tensor in the vertical direction according to the spatial dimension; performing convolutional processing on the first tensor so that the number of channels of the first tensor is the same as the number of channels of the reference feature map; performing nonlinear activation on the convolutionally processed first tensor to obtain a first directional attention map; performing convolutional processing on the second tensor so that the number of channels of the second tensor is the same as the number of channels of the reference feature map; performing nonlinear activation on the convolutionally processed second tensor to obtain a second directional attention map.

[0019] In some embodiments, the feature extraction module is configured to weight the first directional attention map and the second directional attention map onto the reference feature map to obtain a second feature map by multiplying the first directional attention map and the second directional attention map with the reference feature map to obtain the second feature map.

[0020] In some embodiments, the detection module is configured to determine line objects in the image based on the first feature map and the second feature map by performing feature fusion on the first feature map and the second feature map to obtain image features of the image; and determining line objects in the image based on the image features.

[0021] In some embodiments, the detection module is configured to determine line objects in the image based on the image features by: obtaining the probability that each pixel in the image belongs to a line pixel based on the image features; determining the pixels in the image with a probability greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

[0022] In some embodiments, the detection module is configured to determine line objects in the image based on target pixels by: acquiring n regions in the image consisting of consecutively connected target pixels; where n is a positive integer; acquiring the minimum bounding quadrilateral of each region; and determining the acquired minimum bounding quadrilateral as the line object in the image.

[0023] In some embodiments, the electronic device includes a processor and a memory storing program instructions, the processor being configured to perform the image processing method described above when the program instructions are executed.

[0024] In some embodiments, the storage medium stores program instructions that, when executed, perform the image processing method described above.

[0025] The image processing method, apparatus, electronic device, and storage medium provided in this disclosure can achieve the following technical effects: Since the coordinate attention mechanism can capture long-term dependencies, it can acquire long-range dependencies in one spatial direction while retaining precise positional information in another spatial direction, which helps to more accurately locate objects of interest. Therefore, by applying the coordinate attention mechanism to at least one obtained first feature map during feature extraction of the image to be detected, the obtained second feature map can take into account the positional information of line pixels, adding the representation of the object of interest—pixels belonging to the line—so that the method of determining lines in the image using the first and second feature maps can reduce the generation of interference points such as non-line pixels. Simultaneously, since binarization of the image is not required, it can avoid generating a large number of false line detections in documents with color blocks, PPTs, photographs, document titles, etc., and can also avoid false detections of underlines generated by binarization adhesion due to page blur. Therefore, the above method can more accurately detect line objects in images.

[0026] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0027] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0028] Figure 1 This is a schematic diagram of an image processing method provided in an embodiment of this disclosure;

[0029] Figure 2 This is a schematic diagram of another image processing method provided in an embodiment of this disclosure;

[0030] Figure 3 This is a schematic diagram of the structure of a line detection model;

[0031] Figure 4 This is a schematic diagram of the structure of a first encoder;

[0032] Figure 5 This is a schematic diagram of the structure of a coordinate attention mechanism;

[0033] Figure 6(a) is a schematic diagram of an image to be detected;

[0034] Figure 6(b) is a schematic diagram of the lines detected by binarization processing of the image in Figure 6(a);

[0035] Figure 6(c) is a schematic diagram of the lines detected in the image of Figure 6(a) using a line detection model that incorporates a coordinate attention mechanism.

[0036] Figure 7(a) is a schematic diagram of another image to be detected;

[0037] Figure 7(b) is a schematic diagram of the lines detected by binarization processing of the image in Figure 7(a);

[0038] Figure 7(c) is a schematic diagram of the lines detected in the image of Figure 7(a) using a line detection model that incorporates a coordinate attention mechanism.

[0039] Figure 8 This is a schematic diagram of an image processing apparatus provided in an embodiment of the present disclosure;

[0040] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0041] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0042] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0043] This application can be applied to detect lines in images. The executing entity of this application can be an application running in a browser, a client, an electronic device, etc. An application is a web application running on a browser on a terminal. Furthermore, the electronic devices involved in this application can include, but are not limited to, mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers, personal computers (PCs), and PDAs (personal digital assistants), etc.

[0044] Combination Figure 1 As shown, this disclosure provides an image processing method, including:

[0045] Step S101: Obtain the image to be detected.

[0046] Step S102: The image to be detected is subjected to layer-by-layer feature extraction according to a preset feature extraction path to obtain multiple first feature maps; and at least one first feature map is processed by a coordinate attention mechanism during the feature extraction process to obtain a second feature map.

[0047] Step S103: Determine the line objects in the image to be detected based on the first feature map and the second feature map.

[0048] The image processing method provided in this disclosure, by employing the coordinate attention mechanism, can capture long-term dependencies, obtaining long-range dependencies in one spatial direction while retaining precise positional information in another, thus facilitating more accurate localization of objects of interest. Therefore, by applying the coordinate attention mechanism to at least one obtained first feature map during feature extraction of the image to be detected, the resulting second feature map can take into account the positional information of line pixels, adding a representation of the object of interest—pixels belonging to the line—resulting in a reduction of interference points such as non-line pixels. Simultaneously, since binarization of the image is unnecessary, it avoids numerous false detections of lines in documents with color blocks, PPTs, photographs, document titles, etc., and also avoids false detections of underlines generated by binarization adhesion due to page blur. Thus, the above method can more accurately detect line objects in images.

[0049] In this embodiment, the image to be detected is subjected to layer-by-layer feature extraction according to a preset feature extraction path, including: preprocessing the image to be detected to obtain the features of the image to be detected. The features of the image to be detected are extracted layer-by-layer according to the preset feature extraction path. Optionally, the preprocessing of the image to be detected includes: normalizing the image to be detected.

[0050] In this embodiment, a preset line detection model is used to extract features from the image to be detected, resulting in multiple first feature maps and at least one second feature map.

[0051] Optionally, a preset line detection model is used to extract features from the image to be detected, including: normalizing the image to be detected using the preset line detection model to obtain features of the image to be detected; and extracting features from the image to be detected using the line detection model.

[0052] Optionally, the preset line detection model is obtained by inputting samples into U. 2 The Net model is trained to obtain a line detection model. The samples include images with line markings. 2 The advantage of the Net model lies in its fusion of features from receptive fields of different sizes, allowing it to capture more contextual information at different scales. It also increases the depth of the entire network structure without significantly increasing computational cost. In U 2 The line detection model trained on the Net model includes an encoding path for progressive downsampling to extract local features and a decoding path for progressive upsampling to increase resolution. The line detection model progressively improves the semantics of feature extraction through the encoding path and progressively generates a confidence map of the same size as the input image through the decoding path. In this embodiment, the preset feature extraction path is the encoding and decoding path of the line detection model. Since images often contain many pixels similar to lines, this method introduces a coordinate attention mechanism to enable the model to learn the positional information of line pixels during training, thereby reducing the generation of interference points. Specifically, the image marked with lines is an image marked with horizontal lines. Or, the image marked with lines is an image marked with vertical lines. Or, the image marked with lines is an image marked with both horizontal and vertical lines.

[0053] Furthermore, during feature extraction, at least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map. This includes: using at least one first feature map as a reference feature map during feature extraction to process it using a coordinate attention mechanism, obtaining a first-direction attention map and a second-direction attention map. The first-direction attention map and the second-direction attention map are then weighted onto the reference feature map to obtain the second feature map. In this way, applying the coordinate attention mechanism to the reference feature map yields weight matrices in two directions, namely the first-direction attention map and the second-direction attention map. Each element in the first-direction attention map and the second-direction attention map reflects whether the object of interest exists in the corresponding row and column. The first-direction attention map and the second-direction attention map are then weighted onto the reference feature map, enhancing its representational power. Simultaneously, this increases the effectiveness of training, making the model more focused on regions with horizontal and vertical lines, thus enhancing the confidence of that region in the output confidence map.

[0054] In this embodiment, a first feature map can be selected as a reference feature map for coordinate attention mechanism processing to obtain a first directional attention map and a second directional attention map corresponding to the reference feature map. Alternatively, multiple first feature maps can be selected as reference feature maps for coordinate attention mechanism processing. In this case, each selected first feature map is used as a reference feature map for coordinate attention mechanism processing to obtain a first directional attention map and a second directional attention map corresponding to each reference feature map.

[0055] In this embodiment, in the encoding path of the line detection model, the features of the image to be detected are encoded multiple times, with downsampling performed step by step during the encoding process to obtain multiple first feature maps. In the decoding path, the first feature maps obtained in the encoding process are decoded multiple times, with upsampling performed step by step during the decoding process. Since the pixel width of lines in the image is very thin, the pixel and feature information will be significantly reduced after multiple scaling. Therefore, the first feature map obtained in the first encoding can be used as the reference feature map for coordinate attention mechanism processing only in the feature extraction path. Alternatively, the first feature map input during the first decoding can be used as the reference feature map for coordinate attention mechanism processing only in the feature extraction path. Alternatively, the first feature map obtained in the first encoding and the first feature map input during the first decoding can be used as reference feature maps for coordinate attention mechanism processing respectively. Alternatively, the first feature maps obtained in the first and second encodings, and the first feature maps input during the first and second decodings can be used as reference feature maps for coordinate attention mechanism processing respectively.

[0056] Further, during the feature extraction process, at least one first feature map is used as a reference feature map for coordinate attention mechanism processing to obtain a first directional attention map and a second directional attention map. This includes: determining at least one first feature map as a reference feature map during feature extraction; processing each reference feature map, including: performing global pooling on each channel of the reference feature map along the vertical and horizontal directions respectively to generate a first directional perceptual feature map and a second directional perceptual feature map; and obtaining the first directional attention map and the second directional attention map based on the first directional perceptual feature map and the second directional perceptual feature map. The global pooling process is a one-dimensional global pooling process.

[0057] The coordinate attention mechanism encodes each channel of the reference feature map using one-dimensional global pooling in both the vertical (Y-axis) and horizontal (X-axis) directions, generating corresponding Y-axis and X-axis perceptual feature maps. By decomposing channel attention into two feature encoding processes, coordinate information is embedded along the X and Y axes respectively, generating perceptual feature maps in these directions. This allows the application of the coordinate attention mechanism to the reference feature map, yielding a weight matrix along the X-axis (first-direction attention map) and a weight matrix along the Y-axis (second-direction attention map). This enables the capture of long-range dependencies along one spatial direction while preserving precise positional information along the other, helping the network to more accurately locate objects of interest. By fully utilizing the positional information of the reference feature map along the X and Y axes and weighting each channel, the effectiveness of training is increased, making the model more focused on regions with horizontal and vertical lines, thus enhancing the confidence of these regions in the output confidence map. Among them, the first direction perception feature map is the X-axis direction perception feature map, and the second direction perception feature map is the Y-axis direction perception feature map.

[0058] Meanwhile, since the image to be detected may contain both horizontal and vertical lines, a coordinate attention mechanism is applied to at least one first feature map. This involves weighting the probability that each pixel in the first feature map is a horizontal or vertical line. If it's a horizontal line, there will be many pixels along the X-axis. Through one-dimensional global pooling along the X-axis, the features of the horizontal line can be integrated. If it's a vertical line, there will be many pixels along the Y-axis. Through one-dimensional global pooling along the Y-axis, the features of the vertical line can be integrated. Therefore, regardless of whether it's a horizontal or vertical line, the weight of the area containing the line will increase, thus enhancing the attention. Therefore, the coordinate attention mechanism allows the algorithm to better identify horizontal and vertical pixels, that is, to better identify horizontal and vertical line objects in the image.

[0059] Further, obtaining a first-direction attention map and a second-direction attention map based on the first-direction perceptual feature map and the second-direction perceptual feature map includes: concatenating the first-direction perceptual feature map and the second-direction perceptual feature map to obtain a concatenated image; performing a convolutional transformation on the concatenated image to generate an intermediate feature map that aggregates horizontal and vertical spatial information; segmenting the intermediate feature map along the spatial dimension into a first tensor in the horizontal direction and a second tensor in the vertical direction; performing convolutional processing on the first tensor to make the number of channels in the first tensor the same as the number of channels in the reference feature map; then performing nonlinear activation on the convolutional first tensor to obtain the first-direction attention map; performing convolutional processing on the second tensor to make the number of channels in the second tensor the same as the number of channels in the reference feature map; then performing nonlinear activation on the convolutional second tensor to obtain the second-direction attention map. Here, the first tensor in the horizontal direction is the first tensor in the X-axis direction of the intermediate feature map. The second tensor in the vertical direction is the second tensor in the Y-axis direction of the intermediate feature map. The spatial dimension includes the horizontal dimension and the vertical dimension. This allows for the capture of long-range dependencies along one spatial direction while simultaneously acquiring precise location information along another, thus encoding a pair of direction-aware and location-sensitive directional attention maps. Furthermore, each element in these two directional attention maps reflects the confidence that the object of interest exists in the corresponding row and column. The coordinate attention mechanism can more accurately locate the precise position of the object of interest, thereby helping the entire model to better recognize line objects.

[0060] Furthermore, a convolution transformation is performed on the stitched image, including transforming the stitched image using a 1×1 convolution.

[0061] Furthermore, the following detailed description further explains how to obtain a second feature map by weighting the first directional attention map and the second directional attention map onto a reference feature map: The first directional attention map and the second directional attention map are compared with the reference feature map... Figure 3 Multiplying the first and second tensors yields the second feature map. After convolution of the first and second tensors respectively, non-linear activation is applied to obtain the weight matrices for horizontal and vertical lines. The first directional attention map, i.e., the X-axis attention map, contains the weights along the X-axis, which constitute the weight matrix for horizontal lines. The second directional attention map, i.e., the Y-axis attention map, contains the weights along the Y-axis, which constitute the weight matrix for vertical lines. Multiplying the X-axis and Y-axis weights with the reference feature map allows the line pixels to receive larger activation values. Furthermore, considering the weights of each pixel coordinate in the X and Y axes of the reference feature map enhances the target features of the reference feature map, improving the accuracy of the line detection model in acquiring line pixels from the feature map.

[0062] Furthermore, the first-direction attention map and the second-direction attention map are compared with the reference features. Figure 3 The multiplication process includes multiplying the value of each pixel in the reference feature map by the weights of the pixel's coordinates in the X-axis direction and the Y-axis direction to obtain the second feature map.

[0063] In this embodiment, a preset line detection model is used to determine the lines in the image based on the first feature map and the second feature map.

[0064] Further, step S103, determining line objects in the image based on the first and second feature maps, includes: fusing features from the first and second feature maps to obtain image features. Line objects in the image are then determined based on these image features. Since the second feature map is obtained through a coordinate attention mechanism, this mechanism helps the network more accurately locate objects of interest and enhances the representation ability of image features for these objects. This allows the second feature map to consider the positional information of line pixels, increasing the representation of pixels belonging to the line as objects of interest and reducing the generation of interference points such as non-line pixels, making the line pixels in the image features fused with the second feature map more accurate. Thus, the line detection model, by introducing a coordinate attention mechanism, can perform more accurate line detection. Furthermore, even when there are many pixels in the image to be detected that are similar to line pixels, the line detection model can more accurately identify the line pixels.

[0065] Furthermore, determining line objects in the image based on image features includes: obtaining the probability that each pixel in the image belongs to a line pixel based on image features; identifying pixels in the image with a probability greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

[0066] Furthermore, the probability of each pixel in the image belonging to a line pixel is obtained based on image features, including: performing convolution processing on the image features to obtain the probability of each pixel belonging to a line pixel. Specifically, the probability of each pixel belonging to a line pixel includes: the probability of each pixel belonging to a horizontal line pixel, or the probability of each pixel belonging to a vertical line pixel, or the probability of each pixel belonging to both horizontal and vertical line pixels. Thus, by obtaining the probability of each pixel belonging to a horizontal line pixel, horizontal lines in the image can be identified. By obtaining the probability of each pixel belonging to a vertical line pixel, vertical lines in the image can be identified. By obtaining the probabilities of each pixel belonging to both horizontal and vertical lines, both horizontal and vertical lines in the image can be identified.

[0067] Optionally, the image features are convolved using a preset line detection model to output a segmentation map. The pixel value of the first pixel in the segmentation map is a first value, and the pixel value of the second pixel in the segmentation map is a second value. The first pixel is a pixel with a probability greater than or equal to a preset confidence threshold, and the second pixel is a pixel with a probability less than the preset confidence threshold. For example, if the preset confidence threshold is 0.5, the first pixel is a pixel with a probability greater than or equal to 0.5, which is a line pixel. If the first value is 255, then the second value is 0. Or, if the first value is 0, then the second value is 255.

[0068] In this embodiment, a pre-defined line detection model is used to process image features through convolution, outputting the probability that each pixel in the image belongs to a line pixel. The probability of belonging to a line pixel includes the probability of belonging to a horizontal line pixel, or the probability of belonging to a vertical line pixel, or the probability of belonging to both horizontal and vertical line pixels respectively. This line detection method is pixel-level accurate, resulting in more accurate probabilities of each pixel belonging to a line. Furthermore, it can detect horizontal and vertical lines separately, and utilizes a coordinate attention mechanism, making the identified target pixels more accurate and the line detection more precise.

[0069] Optionally, if the probability of each pixel in the image belonging to a line pixel is less than a preset confidence threshold, then none of the pixels belong to a line pixel, and it is determined that there is no target pixel in the image, thus determining that there is no line in the image.

[0070] Optionally, determining line objects in the image based on target pixels includes: obtaining n regions in the image composed of consecutively connected target pixels; where n is a positive integer; obtaining the minimum bounding quadrilateral of each region; and determining the obtained minimum bounding quadrilateral as the line object in the image. Because a coordinate attention mechanism is used during image feature extraction, the identified target pixels as line pixels in the image are more accurate, thus leading to more accurate line detection.

[0071] In some embodiments, when target pixels exist in the image, a first simple bounding quadrilateral is obtained for the region formed by consecutively connected target pixels. A simple bounding quadrilateral is a bounding quadrilateral with sides parallel to the X-axis or Y-axis. The region formed by consecutively connected target pixels is rotated by at least one preset degree. After rotation by each degree, a second simple bounding quadrilateral of the rotated region is obtained, and the area, vertex coordinates, and corresponding rotation degree of each second simple bounding quadrilateral are obtained. The simple bounding quadrilateral with the smallest area is selected from the first and second simple bounding quadrilaterals, and this is determined as the third simple bounding quadrilateral. The third simple bounding quadrilateral is rotated in the opposite direction by the corresponding rotation degree to obtain the smallest bounding quadrilateral of the region formed by the consecutively connected target pixels.

[0072] Optionally, determining line objects in the image based on target pixels includes: acquiring n regions in the image composed of consecutively connected target pixels; where n is a positive integer; and determining the acquired regions as line objects.

[0073] In some embodiments, when target pixels exist in an image, the region formed by consecutively connected target pixels is defined as a line object.

[0074] In this embodiment, feature extraction is first performed on the image to be detected, resulting in multiple first feature maps. During the feature extraction process, at least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map. Then, the probability that each pixel in the image belongs to a line pixel is obtained using the first and second feature maps. This determines whether the pixels in the image belong to horizontal or vertical lines and their positions. Furthermore, lines in the image can be obtained based on the pixels that belong to horizontal or vertical lines. The introduction of a coordinate attention mechanism during feature extraction reduces false detections of decorative lines caused by creases, ink stains, and boundaries, and removes false detections of underlines generated by binarization adhesion due to page blurring. This allows for more accurate determination of line outlines and positions, facilitating subsequent processing such as line type determination, filtering, and color judgment.

[0075] Optionally, after determining the line objects in the image based on the target pixels, the method further includes: displaying the line objects identified by the line detection model.

[0076] Optionally, displaying line objects identified by the line detection model includes: in response to a document conversion operation on the image, determining the target document corresponding to the document conversion operation, and displaying the line objects identified by the line detection model in the target document.

[0077] Optionally, displaying the line objects identified by the line detection model in the target document includes: displaying the coordinates of the corresponding target pixel in black and the coordinates of the corresponding pixel with a probability less than a preset confidence threshold in white.

[0078] Optionally, displaying the line objects identified by the line detection model in the target document includes: displaying the coordinates of the corresponding target pixel in white and the coordinates of the corresponding pixel with a probability less than a preset confidence threshold in black.

[0079] Furthermore, in response to a document conversion operation on an image, determining the target document corresponding to the document conversion operation includes: in response to a document conversion operation on an image, creating a new document and identifying the new document as the target document.

[0080] The image processing method provided in this embodiment can be applied to scenarios involving image format conversion, such as converting an image to a streaming document like Word. Content recognition is performed on the original image, and the recognized content is displayed within the new document type.

[0081] Optionally, displaying the lines identified by the line detection model includes: displaying the identified lines using a segmentation map. The segmentation map is generated based on the probability that each pixel belongs to a line pixel. The segmentation map contains two confidence maps. The pixel value of the first pixel in the confidence map is set to a first value, and the pixel value of the second pixel in the confidence map is set to a second value, thus obtaining the segmentation map; wherein, the first pixel is a pixel in the confidence image with a probability greater than or equal to a preset confidence threshold, and the second pixel is a pixel in the confidence map with a probability less than the preset confidence threshold. The preset confidence threshold is 0.5, and the first pixel is a pixel with a probability greater than or equal to 0.5, i.e., a line pixel. Wherein, if the first value is set to 255, then the second value is 0, that is, the target pixel is white, and the other pixels are black. Or, the first value is set to 0, and the second value is set to 255, that is, the target pixel is black, and the other pixels are white.

[0082] Combination Figure 2 As shown, this disclosure provides another image processing method, including:

[0083] Step S201: Obtain the image to be detected.

[0084] Step S202: Preprocess the image to be detected to obtain the features of the image to be detected. Optionally, the preprocessing of the image to be detected includes: normalizing the image to be detected.

[0085] Step S203: Input the features of the image to be detected into a preset line detection model to obtain a first segmentation image. The first segmentation image contains the probability that each pixel belongs to a line pixel. The probability of belonging to a line pixel includes the probability of belonging to a horizontal line pixel, or the probability of belonging to a vertical line pixel, or the probability of belonging to both horizontal and vertical line pixels respectively.

[0086] Step S204: Select pixels with a confidence level greater than or equal to a preset confidence threshold in the first segmentation image to obtain the second segmentation image.

[0087] Step S205: Obtain the smallest bounding quadrilateral of the second segmentation image to obtain the line object in the image.

[0088] Since lines occupy a small proportion of an image, traditional neural network model architectures employ a series of downsampling operations, such as max pooling or convolutions with a stride greater than 1, to reduce the resolution of the feature map and thus lower computational complexity. However, after multiple scaling operations, the features of the lines become very indistinct. This application utilizes the idea of ​​object segmentation, processing horizontal and vertical lines separately by segmenting the pixels of each line. This application utilizes U... 2 The Net model is trained on samples to obtain a line detection model, allowing the network to maintain high resolution while deepening, thus more effectively extracting multi-scale features of lines within each stage and multi-level features of the aggregation stage during training. Furthermore, in U... 2 The Net model incorporates a coordinate attention mechanism to reduce the influence of interfering pixels that resemble lines with less computation. After inputting the image to be detected into the trained line detection model, the probability of each pixel belonging to a horizontal or vertical line is obtained. Connectivity analysis is performed on the horizontal and vertical line pixels, and the minimum bounding quadrilateral of each connected component is calculated to obtain the lines in the image.

[0089] In some embodiments, combined with Figure 3 As shown, Figure 3This diagram illustrates the structure of a line detection model. The pre-defined line detection model includes: a five-level encoder, a four-level decoder, a feature map fusion model, and a sixth convolution. The five-level encoder consists of a first encoder, a second encoder, a third encoder, a fourth encoder, and a fifth encoder. The four-level decoder consists of a first decoder, a second decoder, a third decoder, and a fourth decoder. Each encoder is a residual U-shaped encoder. Each decoder is a residual U-shaped decoder. The first, second, third, fourth, and fifth convolutions constitute the feature map fusion model. The first encoder and first decoder have identical and symmetrical structures. The second encoder and second decoder have identical and symmetrical structures. The third encoder and third decoder have identical and symmetrical structures. The fourth encoder and fourth decoder have identical and symmetrical structures. The fourth and fifth encoders are the same encoder. Each encoder and decoder has a different number of channels. The number of channels refers to the number of input or output channels. For example, the number of channels for each encoder and decoder is a multiple of 8, and the number of channels gradually increases during encoding and gradually decreases during decoding. The number of output channels for each decoder is the same as the number of input channels for the subsequent convolution. That is, the number of output channels for the first decoder is the same as the number of input channels for the first convolution, the number of output channels for the second decoder is the same as the number of input channels for the second convolution, the number of output channels for the third decoder is the same as the number of input channels for the third convolution, the number of output channels for the fourth decoder is the same as the number of input channels for the fourth convolution, and the number of output channels for the fifth encoder is the same as the number of input channels for the fifth convolution. The features of the image to be detected are input to the first encoder. From the first encoder to the fifth encoder, progressive downsampling is performed to form the encoding path; from the fourth decoder to the first decoder, progressive upsampling is performed to form the decoding process. The encoding path mainly improves the semantics of feature extraction through each encoder stage and progressive downsampling. Each encoder and decoder outputs the first feature map. The decoding process mainly generates a confidence map of the same size as the input image through each decoder stage and progressive upsampling. The first feature map output from each encoder stage is downsampled and used as the input for the next encoder stage. The fourth decoder stage takes as input a spliced ​​image of the first feature map output from the fifth encoder stage and the first feature map output from its symmetric fourth encoder stage. Each of the remaining decoder stages takes as input a spliced ​​image of the first feature map upsampled from the previous stage and the first feature map from its symmetric encoder stage.The feature map fusion model is used to generate image features. In this model, the first feature map output from the first decoder is processed by a first convolution. The first feature map output from the second decoder is upsampled and concatenated with the first feature map output from the first encoder, then processed by a second convolution. The first feature map output from the third decoder is upsampled and concatenated with the first feature map output from the second encoder, then processed by a third convolution. The first feature map output from the fourth decoder is upsampled and concatenated with the first feature map output from the third encoder, then processed by a fourth convolution. The first feature map output from the fifth encoder is upsampled and concatenated with the first feature map output from the fourth encoder, then processed by a fifth convolution. Finally, the feature maps processed by the first, second, third, fourth, and fifth convolutions are concatenated to obtain the image features. A sixth convolution is then applied to the image features to output a confidence map. The output confidence map contains a confidence map of the probability that each pixel belongs to a horizontal line pixel, and / or, a confidence map of the probability that each pixel belongs to a vertical line pixel. Figure 3 In the middle, ○ indicates splicing.

[0090] In this embodiment of the disclosure, for U 2 The Net model is trained on samples to obtain a line detection model. This model extracts features through a feature extraction path, resulting in multiple first feature maps. During feature extraction, at least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map. After fusing the first and second feature maps, the fused image features are convolved to output the probability that each pixel in the image belongs to a line pixel. Thus, using U... 2 The Net model detects lines and achieves accurate line detection by introducing a coordinate attention mechanism.

[0091] In some embodiments, combined with Figure 4 As shown, Figure 4This is a schematic diagram of the structure of the first encoder. The first encoder includes: a first convolutional layer, three levels of second convolutional layers, a dilated convolutional layer, three levels of third convolutional layers, a fourth convolutional layer, and a coordinate attention mechanism. The three levels of second convolutional layers are: the first second convolutional layer, the second second convolutional layer, and the third second convolutional layer. The three levels of third convolutional layers are: the first third convolutional layer, the second third convolutional layer, and the third third convolutional layer. The first to third second convolutional layers are symmetrical to the first to third third convolutional layers, respectively. The output of each convolutional layer is a first feature map. The features of the image to be detected are input to the first convolutional layer. In the first encoder, the first feature map output by the first convolutional layer is processed by the coordinate attention mechanism to obtain a second feature map. Simultaneously, the first feature map output by the first convolutional layer is downsampled and used as the input to the first second convolutional layer. The remaining second convolutional layers downsample the first feature map output by the previous level of the second convolutional layer and use it as their input. The first feature map output by the third second convolutional layer is used as the input to the dilated convolutional layer. The third convolutional layer concatenates the first feature map output from the dilated convolutional layer with the first feature map output from the symmetrical third convolutional layer as input. The remaining third convolutional layers take the concatenated image of the first feature map output from the previous convolutional layer (upsampled) and the first feature map output from the symmetrical second convolutional layer as input. The fourth convolutional layer takes the concatenated image of the first feature map output from the first convolutional layer (upsampled) and the first feature map output from the first convolutional layer as input. The first feature map output from the fourth convolutional layer is then added to the second feature map to obtain the output of the first encoder, i.e., the first feature map output by the first encoder. Figure 4 In the middle, ○ indicates splicing. It indicates addition.

[0092] The second encoder includes: a first convolutional layer, a second-level second convolutional layer, a dilated convolutional layer, a second-level third convolutional layer, a fourth convolutional layer, and a coordinate attention mechanism. The second-level second convolutional layers consist of a first and a second second convolutional layer. The second-level third convolutional layers consist of a first and a second third convolutional layer. The first and second second convolutional layers are symmetrical to the first and second third convolutional layers, respectively. The output of each convolutional layer is a first feature map. The first feature map output by the first encoder is downsampled and input into the first convolutional layer of the second encoder. In the second encoder, the first feature map output by the first convolutional layer is processed by the coordinate attention mechanism to obtain a second feature map. Simultaneously, the first feature map output by the first convolutional layer is downsampled and used as the input to the first second convolutional layer. The second second convolutional layer downsamples the first feature map output by the first second convolutional layer and uses it as its input. The first feature map output by the second second convolutional layer is used as the input to the dilated convolutional layer. The second convolutional layer concatenates the first feature map output from the dilated convolutional layer with the first feature map output from the second convolutional layer, and uses this as input. The first convolutional layer upsamples the first feature map output from the second convolutional layer and concatenates it with the first feature map output from the first convolutional layer, and uses this concatenation as input. The fourth convolutional layer upsamples the first feature map output from the first convolutional layer and concatenates it with the first feature map output from the first convolutional layer, and uses this concatenation as input. Finally, the first feature map output from the fourth convolutional layer is added to the second feature map to obtain the output of the second encoder, i.e., the first feature map output by the second encoder.

[0093] The third encoder comprises: a first convolutional layer, a second convolutional layer, a dilated convolutional layer, a third convolutional layer, a fourth convolutional layer, and a coordinate attention mechanism. The second and third convolutional layers are symmetrical. The output of each convolutional layer is a first feature map. The first feature map output by the second encoder is downsampled and input into the first convolutional layer of the third encoder. In the third encoder, the first feature map output by the first convolutional layer is processed by the coordinate attention mechanism to obtain the second feature map. Simultaneously, the first feature map output by the first convolutional layer is downsampled and used as input to the second convolutional layer. The first feature map output by the second convolutional layer is used as input to the dilated convolutional layer. The third convolutional layer concatenates the first feature map output by the dilated convolutional layer with the first feature map output by the second convolutional layer and uses this as input. The fourth convolutional layer upsamples the first feature map output by the third convolutional layer and uses this concatenated image with the first feature map output by the first convolutional layer as input. The first feature map output by the fourth convolutional layer is added to the second feature map to obtain the output of the third encoder, i.e., the first feature map output by the third encoder.

[0094] The fourth encoder comprises a first convolutional layer, a dilated convolutional layer, a fourth convolutional layer, and a coordinate attention mechanism. The output of each convolutional layer is a first feature map. The first feature map output by the third encoder is downsampled and input into the first convolutional layer of the fourth encoder. In the fourth encoder, the first feature map output by the first convolutional layer is processed by the coordinate attention mechanism to obtain a second feature map. Simultaneously, the first feature map output by the first convolutional layer is used as the input to the dilated convolutional layer. The fourth convolutional layer takes as input the concatenated image of the first feature map output by the dilated convolutional layer and the first feature map output by the first convolutional layer. The first feature map output by the fourth convolutional layer is added to the second feature map to obtain the output of the fourth encoder, i.e., the first feature map output by the fourth encoder.

[0095] Because the image contains a large number of pixels similar to the pixels of lines, this application introduces a coordinate attention mechanism to make U 2 During training, the Net model learns the positional information of line pixels, reducing the generation of interference points. By segmenting lines using the trained line detection model, the model obtains the line pixels, reducing false detections of decorative lines caused by creases, ink smudges, and boundaries. It also removes false detections of underlines generated by binarization adhesion due to page blurring, and accurately locates the line outline and position. This facilitates subsequent post-processing such as line type determination, filtering, and color judgment.

[0096] In some embodiments, combined with Figure 5 As shown, Figure 5The network structure diagram for the coordinate attention mechanism is shown below. The coordinate attention mechanism includes: a global pooling layer, a concatenation layer, a fifth convolutional layer, a batch normalization layer, a non-linear activation function, a sixth convolutional layer, a numerical normalization layer, and a merging operation layer, connected sequentially. Numerical normalization normalizes the output of the sixth convolutional layer to the (0,1) interval using the sigmoid function. The merging operation layer multiplies the features input to it. The global pooling layer includes parallel global pooling layers along the X and Y axes, both connected to the concatenation layer. The sixth convolutional layer also includes parallel convolutional layers along the X and Y axes, each connected to its corresponding numerical normalization layer. Both numerical normalization layers are connected to the merging operation layer. The first feature map input is also directly connected to the merging operation layer. The X-axis and Y-axis global pooling layers globally pool each channel of the input first feature map along its horizontal (X-axis) and vertical (Y-axis) directions, respectively, to obtain X-axis and Y-axis perceptual feature maps. These X-axis and Y-axis perceptual feature maps are then concatenated by a concatenation layer to obtain a stitched image, which is then input into the fifth convolutional layer. The fifth convolutional layer performs a 1×1 convolution transformation, followed by batch normalization, and then uses a non-linear activation function to generate an intermediate feature map that aggregates horizontal and vertical spatial information. This intermediate feature map is then segmented along the spatial dimension into a first tensor along the X-axis and a second tensor along the Y-axis. Then, a convolutional layer along the X-axis is used to convolve the first tensor, and the resulting tensor is numerically normalized to obtain the X-axis attention map. Similarly, a convolutional layer along the Y-axis is used to convolve the second tensor, and the resulting tensor is numerically normalized to obtain the Y-axis attention map. A merging layer multiplies the X-axis and Y-axis attention maps input to it with the first input feature map. The output of the merging layer is the output feature of the coordinate attention mechanism, i.e., the second output feature map. The coordinate attention mechanism introduces batch normalization, which not only accelerates model convergence but, more importantly, alleviates gradient vanishing (dispersed feature distribution) in deep networks, making training deep network models easier and more stable.

[0097] The image processing method provided in this disclosure utilizes a line detection model incorporating a coordinate attention mechanism to detect lines in an image. Since image binarization is unnecessary, it avoids incomplete or excessive line detection, and reduces the probability of numerous false line detections in documents with color blocks, PPTs, photographs, and document titles. It also avoids false detections of underlines generated due to page blurring and binarization adhesion. Therefore, this method can detect lines in images more accurately. Furthermore, it eliminates the need for extensive logic to eliminate interference factors such as creases, boundaries, and ink stains, improving the efficiency of line detection. Simultaneously, the line detection model incorporating a coordinate attention mechanism does not require limiting the operator length of the opening operation, improving the accuracy of detecting short and vertical lines.

[0098] In some embodiments, referring to Figures 6(a), 6(b), and 6(c), Figure 6(a) is the image to be detected. Line detection is performed on the image to be detected in Figure 6(a) using binarization processing, resulting in a schematic diagram of the detected lines as shown in Figure 6(b). Line detection is then performed on the image to be detected in Figure 6(a) using the line detection model with a coordinate attention mechanism provided in this embodiment, resulting in a schematic diagram of the detected lines as shown in Figure 6(c). It can be seen that the lines detected using the line detection model with the coordinate attention mechanism are more accurate than those detected using binarization processing.

[0099] In some embodiments, referring to Figures 7(a), 7(b), and 7(c), Figure 7(a) is the image to be detected. Line detection is performed on the image to be detected in Figure 7(a) using binarization processing, resulting in a schematic diagram of the detected lines as shown in Figure 7(b). Line detection is then performed on the image to be detected in Figure 7(a) using the line detection model with a coordinate attention mechanism provided in this embodiment, resulting in a schematic diagram of the detected lines as shown in Figure 7(c). It can be seen that the lines detected using the line detection model with the coordinate attention mechanism are more accurate than those detected using binarization processing.

[0100] Combination Figure 8 As shown, this disclosure provides an image processing apparatus, including: an acquisition module 1, a feature extraction module 2, and a detection module 3. The acquisition module 1 is configured to acquire an image to be detected. The feature extraction module 2 is configured to perform layer-by-layer feature extraction on the image to be detected according to a preset feature extraction path to obtain multiple first feature maps; and to perform coordinate attention mechanism processing on at least one first feature map during the feature extraction process to obtain a second feature map. The detection module 3 is configured to determine line objects in the image based on the first and second feature maps.

[0101] Furthermore, the feature extraction module is configured to perform coordinate attention mechanism processing on at least one first feature map during the feature extraction process to obtain a second feature map in the following manner: during the feature extraction process, at least one first feature map is used as a reference feature map to perform coordinate attention mechanism processing to obtain a first direction attention map and a second direction attention map; the first direction attention map and the second direction attention map are weighted onto the reference feature map to obtain the second feature map.

[0102] Furthermore, the feature extraction module is configured to use at least one first feature map as a reference feature map for coordinate attention mechanism processing during the feature extraction process to obtain a first directional attention map and a second directional attention map: determining at least one first feature map as a reference feature map during the feature extraction process; processing each reference feature map, including: performing global pooling processing on each channel of the reference feature map along the vertical and horizontal directions respectively to generate a first directional perceptual feature map and a second directional perceptual feature map; and obtaining the first directional attention map and the second directional attention map based on the first directional perceptual feature map and the second directional perceptual feature map.

[0103] Furthermore, the feature extraction module is configured to obtain a first-direction attention map and a second-direction attention map based on the first-direction perceptual feature map and the second-direction perceptual feature map in the following manner: concatenating the first-direction perceptual feature map and the second-direction perceptual feature map to obtain a concatenated image; performing a convolutional transformation on the concatenated image to generate an intermediate feature map that aggregates horizontal and vertical spatial information; dividing the intermediate feature map into a first tensor in the horizontal direction and a second tensor in the vertical direction according to the spatial dimension; performing convolutional processing on the first tensor to make the number of channels in the first tensor the same as the number of channels in the reference feature map; performing nonlinear activation on the convolutional first tensor to obtain the first-direction attention map; performing convolutional processing on the second tensor to make the number of channels in the second tensor the same as the number of channels in the reference feature map; performing nonlinear activation on the convolutional second tensor to obtain the second-direction attention map.

[0104] Furthermore, the feature extraction module is configured to weight the first directional attention map and the second directional attention map onto the reference feature map to obtain the second feature map by multiplying the first directional attention map and the second directional attention map with the reference feature map.

[0105] Furthermore, the detection module is configured to determine line objects in the image based on the first feature map and the second feature map in the following manner: performing feature fusion on the first feature map and the second feature map to obtain image features of the image; and determining line objects in the image based on the image features.

[0106] Furthermore, the detection module is configured to determine line objects in an image based on image features by: obtaining the probability that each pixel in the image belongs to a line pixel based on the image features; identifying pixels in the image with a probability greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

[0107] Furthermore, the detection module is configured to determine line objects in an image based on target pixels by: acquiring n regions in the image consisting of consecutively connected target pixels; where n is a positive integer; acquiring the minimum bounding quadrilateral of each region; and determining the acquired minimum bounding quadrilateral as a line object in the image.

[0108] Combination Figure 9 As shown, this disclosure provides an electronic device including a processor 4 and a memory 5 storing program instructions. Optionally, the electronic device may further include a communication interface 6 and a bus 7. The processor 4, communication interface 6, and memory 5 can communicate with each other via the bus 7. The communication interface 6 can be used for information transmission. The processor 4 can call logical instructions in the memory 5 to execute the image processing method of the above embodiment.

[0109] Furthermore, the program instructions in the aforementioned memory 5 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0110] The memory 5, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 4 executes functional applications and data processing by running the program instructions / modules stored in the memory 5, that is, it implements the image processing method in the above embodiments.

[0111] The memory 5 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 5 may include high-speed random access memory and may also include non-volatile memory.

[0112] This disclosure provides a storage medium storing program instructions that, when executed, perform the image processing method described above.

[0113] The aforementioned storage media can be transient computer-readable storage media or non-transitory computer-readable storage media, including: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store program code.

[0114] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0116] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. An image processing method, characterized in that, include: Acquire the image to be detected; the image is a document image; The image is subjected to layer-by-layer feature extraction according to a preset feature extraction path to obtain multiple first feature maps; At least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map; The line objects in the image are determined based on the first feature map and the second feature map; Determining line objects in the image based on the first feature map and the second feature map includes: performing feature fusion on the first feature map and the second feature map to obtain image features of the image; and determining line objects in the image based on the image features. The step of determining line objects in the image based on the image features includes: obtaining the probability that each pixel in the image belongs to a line pixel based on the image features; determining the pixels in the image whose probability is greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

2. The method according to claim 1, characterized in that, At least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map, including: At least one first feature map is used as a reference feature map to perform coordinate attention mechanism processing to obtain a first direction attention map and a second direction attention map. The first directional attention map and the second directional attention map are weighted and added to the reference feature map to obtain the second feature map.

3. The method according to claim 2, characterized in that, At least one first feature map is used as a reference feature map for coordinate attention mechanism processing to obtain a first-direction attention map and a second-direction attention map, including: At least one first feature map is identified as a reference feature map; Processing each of the reference feature maps includes: Global pooling is performed on each channel of the reference feature map along both the vertical and horizontal directions to generate a first-direction perceptual feature map and a second-direction perceptual feature map. Based on the first direction perception feature map and the second direction perception feature map, obtain the first direction attention map and the second direction attention map.

4. The method according to claim 3, characterized in that, Obtaining a first directional attention map and a second directional attention map based on the first directional perception feature map and the second directional perception feature map includes: The first direction perception feature map and the second direction perception feature map are stitched together to obtain a stitched image. The stitched image is subjected to a convolutional transformation to generate an intermediate feature map that aggregates horizontal and vertical spatial information; The intermediate feature map is divided into a first tensor in the horizontal direction and a second tensor in the vertical direction according to the spatial dimension. The first tensor is convolved so that the number of channels in the first tensor is the same as the number of channels in the reference feature map. The first tensor after convolution is non-linearly activated to obtain a first-direction attention map. The second tensor is convolved so that the number of channels in the second tensor is the same as the number of channels in the reference feature map; The second tensor after convolution is nonlinearly activated to obtain the second directional attention map.

5. The method according to claim 2, characterized in that, The step of weighting the first directional attention map and the second directional attention map onto the reference feature map to obtain the second feature map includes: The first directional attention map and the second directional attention map are multiplied with the reference feature map to obtain the second feature map.

6. The method according to claim 1, characterized in that, Determining line objects in the image based on target pixels includes: In the image, obtain n regions consisting of consecutively connected target pixels; where n is a positive integer. Find the minimum bounding quadrilateral for each region; The smallest bounding quadrilateral obtained is determined as the line object in the image.

7. An image processing apparatus, characterized in that, include: The acquisition module is configured to acquire an image to be detected; the image is a document image. The feature extraction module is configured to extract features from the image layer by layer according to a preset feature extraction path to obtain multiple first feature maps; At least one first feature map is processed using a coordinate attention mechanism to obtain a second feature map; The detection module is configured to determine objects in the image based on the first feature map and the second feature map; The step of determining line objects in the image based on the first feature map and the second feature map includes: performing feature fusion on the first feature map and the second feature map to obtain image features of the image; and determining line objects in the image based on the image features. The step of determining line objects in the image based on the image features includes: obtaining the probability that each pixel in the image belongs to a line pixel based on the image features; determining the pixels in the image whose probability is greater than or equal to a preset confidence threshold as target pixels; and determining line objects in the image based on the target pixels.

8. An electronic device comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the image processing method as described in any one of claims 1 to 6 when executing the program instructions.

9. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the image processing method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Lane line detection method based on attention mechanism and feature aggregation

    CN114550135A