Text detection method, device and product for network freight scene

Abstract

This application discloses a text detection method, device, and product for online freight scenarios, relating to the field of text detection. The method includes text detection in online freight scenarios using a deep learning network. The deep learning network includes a first network module, a second network module, and a detection head stacked sequentially. Both the first and second network modules include multiple network units stacked sequentially. Each network unit includes an image patch embedding module, a query-key attention module with impulse decomposition, and a normalized integer impulse firing module stacked sequentially. The step of calculating the attention score in the query-key attention module with impulse decomposition includes: calculating the interactions between the active part of the query and the active part of the key, between the active part of the query and the inactive part of the key, between the inactive part of the query and the active part of the key, and between the inactive part of the query and the inactive part of the key to calculate the attention score.

Description

Technical Field

[0001] This application relates to the field of text detection, and in particular to a text detection method, device and product for online freight scenarios. Background Technology

[0002] In the digital transformation of the online freight industry, text inspection is a core supporting technology, widely used in key audit scenarios such as waybill information extraction, contract clause recognition, and qualification certificate verification. With the explosive growth of business volume, traditional text inspection methods are no longer sufficient to meet the demands of efficient auditing. The platform's operations management and customer service departments continue to expand, yet they still face enormous auditing pressure, necessitating the use of technological means to overcome efficiency bottlenecks.

[0003] Text detection in online freight scenarios faces unique challenges. Waybills, contracts, and qualification certificates often coexist with interfering elements such as watermarks, logos, and background patterns. Furthermore, the text formats are diverse and the layouts complex, making existing technologies prone to incomplete information extraction and insufficient recognition accuracy. Specifically, traditional methods tend to overlook contextual information in low-salience text areas, leading to omissions in extracting key fields such as weight and route on waybills and contract terms. When faced with blurry documents or documents with complex backgrounds, detection accuracy drops significantly, increasing the cost of manual review. Summary of the Invention

[0004] The purpose of this application is to provide a text detection method, device, and product for online freight scenarios. When faced with images with complex backgrounds and many interfering elements, the model can more comprehensively and deeply understand the entire picture, thereby more accurately identifying blurry or inconspicuous text and solving the problems of incomplete information extraction and model instability.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a text detection method for online freight scenarios, which realizes text detection in online freight scenarios through a deep learning network, wherein the deep learning network includes a first network module, a second network module and a detection head stacked in sequence; Both the first network module and the second network module include a plurality of network units stacked sequentially, and each network unit includes an image patch embedding module, a query-key attention module for pulse decomposition, and a normalized integer pulse firing module stacked sequentially. The steps for calculating the attention score in the query-key attention module of the impulse decomposition include: The query and key are decomposed into active and inactive parts respectively; Calculate a first interaction between the active portion of the query and the active portion of the key; calculate a second interaction between the active portion of the query and the inactive portion of the key; calculate a third interaction between the inactive portion of the query and the active portion of the key; calculate a fourth interaction between the inactive portion of the query and the inactive portion of the key. The attention score is calculated based on the first interaction, the second interaction, the third interaction, and the fourth interaction.

[0006] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the text detection method for online freight scenarios described above.

[0007] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the text detection method for the online freight scenario described above.

[0008] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the text detection method for online freight scenarios described above.

[0009] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a text detection method for online freight scenarios. It utilizes a deep learning network to perform text detection in online freight scenarios. The deep learning network comprises a first network module, a second network module, and a detection head stacked sequentially. The first network module extracts the backbone features of the document to be detected. The second network module further extracts features based on the backbone features. Finally, the detection head outputs the text detection result. Both the first and second network modules include multiple network units stacked sequentially. Each network unit includes an image patch embedding module, a query-key attention module with impulse decomposition, and a normalized integer impulse firing module stacked sequentially. Crucially, in the query-key attention module with impulse decomposition, the query and key are first decomposed into active and inactive parts. Then, the first interaction between the active part of the query and the active part of the key, the second interaction between the active part of the query and the inactive part of the key, the third interaction between the inactive part of the query and the active part of the key, and the fourth interaction between the inactive part of the query and the inactive part of the key are calculated. Finally, an attention score is calculated based on the first, second, third, and fourth interactions. This significantly expands the information range used by the model to calculate attention. This allows the model to not only capture the most obvious features when analyzing images, but also to pick up the correlation information in inconspicuous and easily overlooked background or detail areas. Therefore, when faced with images with complex backgrounds and many interfering elements, the model can understand the entire image more comprehensively and deeply, thus more accurately identifying blurry or inconspicuous text and solving the problems of incomplete information extraction and model instability. Furthermore, in traditional spiking networks, the neuronal membrane potential continuously accumulates and eventually saturates, causing its output to be insensitive to subtle differences in spatial features, impairing spatial resolution and training stability. NISF neurons address this problem by introducing a normalized integer pulse firing strategy. Therefore, using NISF neurons in the model can further improve the model's spatial resolution and training stability. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is an architectural diagram of a deep learning network in one embodiment of this application; Figure 2 This is an architectural diagram of a network unit in one embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] The text detection method for online freight scenarios provided in this application can be applied to computer devices such as terminals and servers. The terminal can be, but is not limited to, various desktop computers, laptops, and tablets. The server can be a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0015] In a specific embodiment of this application, a text detection method for online freight scenarios is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server.

[0016] This text detection method for online freight scenarios utilizes deep learning networks to perform text detection in such scenarios. Figure 1 The deep learning network includes a first network module (referred to as the backbone network in this embodiment), a second network module (referred to as the neck network in this embodiment), and a detection head, which are stacked sequentially. Both the first and second network modules include multiple network units stacked sequentially, as shown in the figure. Figure 2 Each network unit includes, in sequence, an image patch embedding (PE) module, a query-key attention module with spike decomposition (SDQK-A) module, and a normalized integer spike firing module (NISF neuron). The output of the image patch embedding module is also used as the input of the normalized integer spike firing module through a residual connection. The steps of the query-key attention module with spike decomposition to calculate the attention score include: decomposing the query and the key into active and inactive parts respectively; calculating a first interaction between the active part of the query and the active part of the key; calculating a second interaction between the active part of the query and the inactive part of the key; calculating a third interaction between the inactive part of the query and the active part of the key; calculating a fourth interaction between the inactive part of the query and the inactive part of the key; and calculating the attention score based on the first, second, third, and fourth interactions.

[0017] In the above specific implementation, the deep learning network adopts the DeQKFormer detection framework, which includes a two-stage encoding-decoding architecture of feature extraction, detection head and loss function. The core components include the Spike-Decomposed Q-KAttention (SDQK-A) module and the Normalized Integer Spike-Fire (NISF) neuron.

[0018] The feature extraction stage consists of a backbone network (first network module) and a neck network (second network module). Each network unit of the backbone network and the neck network contains a Patch Embedding (PE) module and an SDQK-A module, and NISF neurons are embedded as basic computing units in the computational processes of each layer of the backbone network and the neck network.

[0019] Backbone network processing: The computer processes the pre-processed input feature map (T is the time step, H is the image height, W is the image width, and C is the number of feature channels) is input into the backbone network. First, convolutional layers and spiking convolutions are integrated through the PE modules of each network unit (the input of the network unit serves as the input of the convolutional layer, the output of the convolutional layer serves as the input of the spiking convolution, and the output of the spiking convolution serves as the input of the PE module). This encodes the input feature map with local spatial patterns, capturing detailed features such as text edges and textures. Then, the encoded features (output of the PE module) are input into the SDQK-A module. The SDQK-A module expands the attention coverage and mines the contextual information of low-saliency text regions by explicitly modeling four neuron interaction types (activation-activation AA, activation-inactivation AI, inactivation-activation IA, and inactivation-inactivation II), solving the problem of incomplete information extraction in traditional methods. Finally, the features are quantized by combining the normalized firing strategy of NISF neurons, stabilizing the gradient flow and improving the robustness of feature representation. The intermediate feature map output by the backbone network (output of the last network unit in the backbone network) is concatenated by channel dimension and scaled before being transmitted to the neck network.

[0020] The SDQK-A module addresses the sparse attention problem inherent in traditional spiking self-attention mechanisms, which only consider activated neurons. By performing complementary decomposition of the spiking mechanism, the SDQK-A module models four types of neuronal interactions, thus expanding the scope of attention. Its process includes the following steps: (1) Input and decomposition: Given a query matrix in discrete impulse form after discrete normalization. Bond matrix ,in First, it is broken down into activating components and inactive (complementary) components: ; in, and These are the query and the key, respectively. and They are respectively and The activation part, and They are respectively and The inactive part, For activation function, This is a truncation function.

[0021] and This represents the activated neuronal components, while and This represents the complementary information provided by neurons that are not activated or have low firing values.

[0022] (2) Interaction Modeling: Based on the above decomposition, the SDQK-A module explicitly constructs the interactions between four neuron pairs: First Interaction :Depend on and Interaction capture corresponds to the core information that traditional attention focuses on.

[0023] Second Interaction :Depend on and Interaction capture models how active regions are affected by the surrounding inactive context.

[0024] Third Interaction :Depend on and Interaction trapping reflects how inactive regions are modulated by key activation features.

[0025] Fourth Interaction :Depend on and Interaction capture uncovers potential associations between low-significance regions.

[0026] (3) Attention Calculation: The SDQK-A module considers the state of all neurons in the network when calculating the attention score. It not only focuses on those neurons that are actively working, but also on the relationships and influences between seemingly inactive neurons, that is, it simultaneously considers the first, second, third, and fourth interactions. This comprehensive approach greatly expands the range of information on which the model calculates attention. This allows the model to not only grasp the most obvious features when analyzing an image, but also capture the related information of inconspicuous and easily overlooked background or detail areas. Therefore, when faced with images with complex backgrounds and many interfering elements, the model can understand the entire image more comprehensively and deeply, thereby more accurately identifying blurry or inconspicuous text and solving the problems of incomplete information extraction and model instability.

[0027] Specifically, the attention score can be obtained by weighting the first, second, third, and fourth interactions, and the calculation formula is as follows: ; in, , , and All are weights.

[0028] After obtaining the attention score, it is normalized using Softmax to obtain the attention weight: in, , Let be the attention weight and attention score between the i-th query and the j-th key, respectively. The attention score between the i-th query and the k-th key. n Let i be the total number of keys corresponding to the i-th query. This is the temperature coefficient.

[0029] The neck network and the backbone network have the same network architecture, both consisting of multiple network units stacked sequentially. The number of network units in both can be the same or different.

[0030] The neck network processing procedure is as follows: The neck network receives multi-scale intermediate feature maps output by the backbone network, and further optimizes the spatial resolution and channel correlation of the features through the PE modules of each network unit to adapt to the requirements of subsequent detection tasks; then the SDQK-A module enhances the global interaction of cross-scale features and integrates text feature information at different levels (such as local character features and global text layout features); finally, the NISF neurons complete the normalization and discretization of features, outputting a unified scale, high-recognition fused feature map, which is then transmitted to the detection head.

[0031] In the feature extraction stage, the output processing of each network unit can be represented as follows: ; in, This is the output of the i-th network unit. X This is the input to the network unit.

[0032] Specifically, the detection head receives the fused feature map output by the neck network to classify and regress the text target, providing accurate target localization and category determination results for subsequent text information extraction. For example, the detection head is a YOLOv3-tiny detection head. The YOLOv3-tiny detection head directly receives the fused feature map output by the neck network, and uses its lightweight convolutional and pooling layers to reduce the dimensionality and semantically enhance the features, outputting the classification confidence and position coordinate prediction results for each anchor box. This result is then passed to the loss function for model optimization and serves as the core output data during the inference stage.

[0033] In summary, the above specific implementation uses a deep learning network to achieve text detection in a network freight scenario. The deep learning network includes a first network module, a second network module, and a detection head stacked sequentially. The first network module extracts the backbone features of the document to be detected, the second network module further extracts features based on the backbone features, and finally the detection head outputs the text detection result. Both the first and second network modules include multiple network units stacked sequentially. Each network unit includes an image patch embedding module, a query-key attention module with pulse decomposition, and a normalized integer pulse firing module stacked sequentially. Crucially, in the query-key attention module with pulse decomposition, the query and key are first decomposed into active and inactive parts. Then, the first interaction between the active part of the query and the active part of the key, the second interaction between the active part of the query and the inactive part of the key, the third interaction between the inactive part of the query and the active part of the key, and the fourth interaction between the inactive part of the query and the inactive part of the key are calculated. Finally, an attention score is calculated based on the first, second, third, and fourth interactions. This significantly expands the range of information used by the model to calculate attention. This allows the model to not only capture the most obvious features when analyzing images, but also to pick up the correlation information in inconspicuous and easily overlooked background or detail areas. Therefore, when faced with images with complex backgrounds and many interfering elements, the model can understand the entire image more comprehensively and deeply, thus more accurately identifying blurry or inconspicuous text and solving the problems of incomplete information extraction and model instability. Furthermore, in traditional spiking networks, the neuronal membrane potential continuously accumulates and eventually saturates, causing its output to be insensitive to subtle differences in spatial features, impairing spatial resolution and training stability. NISF neurons address this problem by introducing a normalized integer pulse firing strategy. Therefore, using NISF neurons in the model can further improve the model's spatial resolution and training stability.

[0034] Specifically, the calculation process of the normalized integer pulse firing module is as follows: ; in, For normalized integer pulse firing rate, This represents the number of virtual time steps. Let be the membrane potential at time t. This is the nearest integer rounding function. Indicates will Restricted to closed intervals .

[0035] Its working mechanism is as follows: The formula first rounds the cumulative sum of membrane potentials over a period of time to obtain an integer firing count, and then normalizes it by dividing by D. This makes the neuron's output no longer limited to binary (0 or 1), but has multiple discrete levels. Multi-bit pulse rate.

[0036] In some embodiments, the training steps of the deep learning network include: performing feature extraction training on a first network module using sample text of all types individually, and freezing the parameters of the first network module after training; with the parameters of the first network module frozen, performing end-to-end text detection training on the deep learning network using sample text of different types respectively, to obtain the deep learning network trained for the corresponding different types.

[0037] In this training method, the first network module serves as the backbone network, used to implement basic backbone feature extraction, and acts as a shared feature extraction module for different types of text. The second network module and the detection head, in this embodiment, serve as type-specific interaction modules; that is, a separate second network module and detection head are trained for each type of text. Since the dedicated interaction module is only for one type of text, a lightweight design can be adopted.

[0038] Correspondingly, text detection in online freight scenarios is achieved through deep learning networks. Specifically, this includes: identifying the type of text to be detected; and then calling a trained deep learning network corresponding to that type to perform text detection on the document. Text types include waybills, contracts, and qualification certificates. The text detection results include the classification results and location information of the text target, providing accurate data support for platform review and realizing text detection in online freight scenarios.

[0039] In some embodiments, the deep learning network employs binary cross-entropy loss during end-to-end training. The specific formula is: ; Wherein, represents the binary cross-entropy loss. The actual label of the i-th anchor box (1 represents the text target, 0 represents the background). To predict confidence scores, N This represents the total number of anchor frames.

[0040] For example, for end-to-end training of a deep learning network, the optimizer is stochastic gradient descent, and the learning rate is set to 10. −2 .

[0041] The model was trained for 300 epochs on both real business datasets and public datasets, and then iteratively optimized to bring it to a stable state.

[0042] The following is a detailed explanation of how to obtain the dataset.

[0043] We selected real business data from online freight scenarios and publicly available benchmark datasets, covering three core text types: First, real business text data accumulated by the platform, such as waybills, contracts, and qualification documents (ID cards, driver's licenses, business licenses), including samples with varying degrees of ambiguity and complex backgrounds; second, publicly available COCO-Text Dataset and ICDAR Dataset, supplementing with diverse text layout samples. Both datasets together cover key information on waybills (cargo weight, transportation route, freight amount), contract terms, and key fields of qualification documents, ensuring the datasets' business adaptability and diversity.

[0044] After obtaining the dataset, the data is preprocessed before being provided to the deep learning model. The specific operations are as follows: (1) Adjust the image size uniformly to adapt to the model input specifications; (2) Use the discrete normalization method to convert the text feature mapping into a discrete pulse form query matrix Q and key matrix K in the interval [0,1] to enhance data adaptability; (3) Divide the training set and test set according to the business scenario. The real business data is used for model training in batches of 32, and the public dataset is used for testing and verification in batches of 128 to ensure the stability of the training and testing process.

[0045] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0046] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0047] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0049] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0050] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0051] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0052] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A text detection method for online freight scenarios, which achieves text detection in online freight scenarios through deep learning networks, characterized in that... The deep learning network includes a first network module, a second network module, and a detection head stacked sequentially; Both the first network module and the second network module include multiple network units stacked sequentially. Each network unit includes an image patch embedding module, a query-key attention module for pulse decomposition, and a normalized integer pulse firing module stacked sequentially. The output of the image patch embedding module is also used as the input of the normalized integer pulse firing module through a residual connection. The steps for calculating the attention score in the query-key attention module of the impulse decomposition include: The query and key are decomposed into active and inactive parts respectively; Calculate a first interaction between the active portion of the query and the active portion of the key; calculate a second interaction between the active portion of the query and the inactive portion of the key; calculate a third interaction between the inactive portion of the query and the active portion of the key; calculate a fourth interaction between the inactive portion of the query and the inactive portion of the key. The attention score is calculated based on the first interaction, the second interaction, the third interaction, and the fourth interaction.

2. The text detection method for online freight scenarios according to claim 1, characterized in that, The formulas for decomposing the query and the key are as follows: ； in, and These are the query and the key, respectively. and They are respectively and The activation part, and They are respectively and The inactive part, For activation function, This is a truncation function; The first interaction The second interaction The third interaction The fourth interaction ; The formula for calculating the attention score is as follows: ； in, , , and All are weights.

3. The text detection method for online freight scenarios according to claim 1, characterized in that, The training steps of the deep learning network include: The first network module is trained by extracting features from sample texts of all types individually, and the parameters of the first network module are frozen after training. With the parameters of the first network module frozen, the deep learning network is trained end-to-end using different types of sample texts to obtain the deep learning network trained for the corresponding different types.

4. The text detection method for online freight scenarios according to claim 3, characterized in that, Text detection in online freight scenarios is achieved using deep learning networks, specifically including: The type of the text to be detected is identified, and the deep learning network trained according to the type of the text to be detected is invoked to perform text detection on the text to be detected.

5. The text detection method for online freight scenarios according to claim 1, characterized in that, The calculation process of the normalized integer pulse delivery module is as follows: ； in, For normalized integer pulse firing rate, This represents the number of virtual time steps. Let be the membrane potential at time t. This is the nearest integer rounding function. Indicates will Restricted to closed intervals .

6. The text detection method for online freight scenarios according to claim 1, characterized in that, The deep learning network employs binary cross-entropy loss during end-to-end training.

7. The text detection method for online freight scenarios according to claim 6, characterized in that, For end-to-end training of the deep learning network, stochastic gradient descent is used as the optimizer, and the learning rate is set to 10. −2 .

8. The text detection method for online freight scenarios according to claim 1, characterized in that, The detection head is a yolov3-tiny detection head.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the text detection method for online freight scenarios according to any one of claims 1-8.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the text detection method for online freight scenarios as described in any one of claims 1-8.

Images