A method for identifying a bid document envelope

Abstract

The application discloses a kind of tender file surrounding string identification method, it is related to surrounding string identification technical field, including the following steps: obtaining tender file and extracting text content and layout structure element, according to layout structure element set generation structure symbol sequence, and the structure symbol sequence of different tender file is compared with sequence to identify layout structure homologous region;Identification template anchor point and according to this filter candidate tender file set, detect shared text segment and identify variable field;According to layout structure homologous region proportion, shared text segment density and variable field distribution stability calculate template binding stability, and determine template copy scale and template diffusion rate, to identify surrounding string document set.The present application is used to solve the problems that existing method is difficult to identify structure homologous relationship under the condition of field replacement and local rewriting, cannot distinguish template fixed content and variable field, and is difficult to quantitatively analyze and propagate traceability to template copy behavior.

Description

Technical Field

[0001] This invention relates to the field of bid-rigging identification technology, and more specifically, to a method for identifying bid-rigging in tender documents. Background Technology

[0002] In engineering construction and bidding management, bid rigging typically manifests as multiple bidding entities generating bid documents by sharing templates, copying text content, or only replacing certain fields, thereby circumventing supervision and manipulating bidding results. This type of behavior is highly concealed and difficult to identify, severely disrupting fair market competition. Therefore, it is urgent to use technical means to effectively identify and analyze the correlation between bid documents.

[0003] To identify anomalous similarities between bid documents, existing technologies typically employ methods such as text similarity calculation, hash fingerprint comparison, or simple layout matching. For example, some methods compare different bid documents based on keyword matching, edit distance, or vector similarity to determine whether there is duplicate or highly similar content. However, these methods mainly rely on surface-level text features. When bid documents are disguised through synonym substitution, word order adjustment, or partial rewriting, the accuracy of similarity identification can easily decrease, making it difficult to effectively identify bid-rigging behavior generated based on a uniform template.

[0004] Meanwhile, some technical solutions attempt to analyze tender documents through layout structure or document format information, such as matching paragraph structure, heading levels, or table layout. However, such methods typically only perform coarse-grained comparisons of structure, lacking in-depth modeling of the correspondence between structure and content, making it difficult to identify template copying characteristics of "same structure but localized content variations." Furthermore, differences in format conversion and export methods among different tender documents can cause variations in layout structure, further reducing the stability of such methods.

[0005] In existing technologies, bid document similarity analysis often employs a pairwise comparison of all documents, resulting in high computational complexity and difficulty in efficiently filtering potentially related documents from large-scale bid document sets. Furthermore, the lack of stable and reusable structural features as screening criteria leads to low efficiency in the analysis process. In addition, for template-generated bid documents, existing methods typically struggle to distinguish between fixed template content and variable field content, failing to accurately identify field replacement patterns and their distribution, thus making it difficult to determine whether multiple bid documents originate from the same template.

[0006] Furthermore, most existing technologies only focus on similarity detection and lack the ability to quantitatively analyze template duplication behavior. For example, it is difficult to assess the degree of template reuse across different tender documents, to characterize the template diffusion process and its scope, or to identify the template's source file or propagation path. This makes it difficult to form a convincing basis for judgment based solely on similarity results in actual regulatory scenarios. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for identifying bid rigging in tender documents, thereby addressing the problems that existing methods struggle to identify structural homology relationships in cases of field substitution and partial rewriting, cannot distinguish between fixed and variable fields in templates, and are difficult to quantify and trace the spread of template copying behavior.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A method for identifying bid rigging in tender documents includes the following steps: Multiple bid documents are acquired, and their text content and layout structure elements, including table and list structures, are extracted. A structural symbol sequence is generated based on the set of layout structure elements. Continuous symbol subsequences with a frequency exceeding a preset frequency threshold are detected within this sequence and used as template anchors. Bid documents containing the same template anchor are selected to construct a candidate bid document set. Within the candidate bid document set, shared text segments are determined based on the matching relationship between structural symbol sequences from different bid documents within the layout structure homogeneous region, and variable fields are identified. The layout structure homogeneous region is determined by identifying continuous matching segments through sequence alignment of the structural symbol sequences. The structural symbol length of the layout structure homogeneous region and the corresponding bid document structural symbol sequence are calculated. The ratio of the total lengths yields the proportion of the common source region in the layout structure; the ratio of the total number of words in the shared text fragments to the length of the corresponding text word sequence in the common source region of the layout structure yields the density of the shared text fragments; the template binding stability is calculated based on the proportion of the common source region in the layout structure, the density of the shared text fragments, and the stability of the variable field distribution; the template replication propagation relationship is determined based on the template binding stability and the structural position changes of the variable fields; and the template replication propagation sequence is extracted based on the propagation relationship to identify the set of tag-rigging files; wherein, the stability of the variable field distribution is determined based on the ratio of the number of template replacement positions to the total number of words in the shared text fragments, and the template replacement positions are the difference segments whose positional offset relative to the common source region of the layout structure is less than a preset offset threshold.

[0009] The technical effects and advantages of the bid-rigging identification method in this invention are as follows: This invention encodes the layout structural elements of tender documents into a sequence of structural symbols and performs sequence alignment to identify common structural regions. It then combines this with the identification of template anchor points within the structural symbol sequence to filter a set of candidate tender documents, thereby achieving a discretized mapping and initial feature comparison of cross-document layout relationships. Furthermore, by detecting shared text fragments and identifying variable fields within the common structural regions of the candidate tender documents, and calculating template binding stability based on the proportion of common structural regions, the density of shared text fragments, and the distribution stability of variable fields, this invention helps to accurately quantify and characterize the binding relationship and co-evolutionary features between fixed structures and filler text during template copying. By further determining the template copying scale and template diffusion rate based on the template binding stability to identify sets of collusive bidding documents, this invention helps to identify the diffusion range and evolution path of template copying behavior. This effectively solves the problems of existing methods, such as difficulty in identifying structural commonalities in the presence of field replacement and partial rewriting, inability to distinguish between fixed template content and variable fields, and difficulty in quantitative analysis and tracing the propagation of template copying behavior. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the bid rigging and collusion identification method provided in an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of the local alignment score matrix of the structural symbol sequence provided in an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of the distribution of shared text fragments and variable fields provided in an embodiment of the present invention.

[0013] Figure 4 This is a schematic diagram illustrating the relationship between template binding stability, template replication scale, and template diffusion rate provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0015] Example 1, Figure 1 This invention provides a method for identifying bid rigging in tender documents, comprising the following steps: S1. Obtain multiple tender documents and extract text content and layout structure elements including table structure and list structure; generate a structural symbol sequence based on the layout structure element set; determine the layout structure homogeneous area by identifying continuous matching segments through sequence comparison of the structural symbol sequence. S2, in the structural symbol sequence, a continuous subsequence of symbols with a frequency higher than a preset frequency threshold is detected as a template anchor point, and bid documents containing the same template anchor point are selected to construct a candidate bid document set; in the layout structure homogeneous area within the candidate bid document set, shared text fragments are determined and variable fields are identified based on the matching relationship of structural symbol sequences between different bid documents; S3, calculate the ratio of the structural symbol length of the common source area of ​​the layout structure to the total length of the corresponding structural symbol sequence of the tender document to obtain the proportion of the common source area of ​​the layout structure; calculate the ratio of the total number of words in the shared text fragments to the length of the corresponding text word sequence in the common source area of ​​the layout structure to obtain the density of the shared text fragments; calculate the template binding stability based on the proportion of the common source area of ​​the layout structure, the density of the shared text fragments, and the stability of the variable field distribution; determine the template replication propagation relationship based on the template binding stability and the structural position change of the variable fields; and extract the template replication propagation sequence based on the propagation relationship to identify the set of collusive bidding documents; wherein, the stability of the variable field distribution is determined according to the ratio of the number of template replacement positions to the total number of words in the shared text fragments, and the template replacement position is the difference segment with a position offset of less than a preset offset threshold relative to the common source area of ​​the layout structure.

[0016] This invention addresses the problem of identifying bid-rigging in tender documents by constructing a comprehensive technical solution that progresses step-by-step from structural analysis and content modeling to propagation analysis. By performing structured analysis on the tender documents and establishing a sequence of structural symbols, a unified expression of document format features is achieved. Based on this, structural relationships between different documents are mined, providing a stable structural reference framework for subsequent analysis. Furthermore, a template anchoring mechanism is introduced based on these structural relationships to cluster and filter the tender documents. Combined with the collaborative analysis of shared text fragments and variable fields, this effectively characterizes templated text and its variation patterns. Furthermore, by constructing a template binding stability fusion multi-dimensional index and introducing template replication scale and diffusion rate, the expansion behavior of templates in the document set is quantitatively described. Ultimately, this results in a comprehensive ability to identify bid-rigging document sets, which helps improve the accuracy and stability of identifying bid-rigging behavior in complex spoofing scenarios.

[0017] In this embodiment, step S1 is specifically implemented as follows: S101, Obtain multiple tender documents and extract text content and layout structure element information; When there are template reuses among bid documents but differences in text content, it is difficult to identify structural consistency based solely on text comparison. Therefore, we first obtain multiple bid documents and extract text content and layout structure element information.

[0018] Specifically, for all tender documents submitted for the same tender project, the file format can be PDF, DOC, or DOCX.

[0019] A document parsing engine runs in a server environment, integrating a parsing module based on the Apache Tika framework and a PDFBox parsing component. For electronic tender documents whose text streams can be directly read, the document parsing engine directly extracts their text content and layout structure element information; wherein, the layout structure element information includes structural features such as paragraph levels, heading levels, table structure, list structure, headers and footers, page number positions, chart areas, and font styles.

[0020] Furthermore, to uniformly process various complex format bid documents, including scanned documents and images, a document layout parsing algorithm is introduced to divide the page into several logical regions. For each region, its structural category, hierarchical relationship, and relative position coordinates are recorded to form a complete set of structural elements. In this embodiment, the document layout parsing algorithm specifically adopts the LayoutLMv3 model.

[0021] In the specific analysis, if the input tender document page is an image or scanned copy, a lightweight OCR engine extracts the text content and original absolute coordinates of each text unit on the page. If the input is a non-scanned electronic document, the text content and original absolute coordinates are directly extracted using a document parsing engine. Subsequently, the page image features, text semantic features, and two-dimensional positions after page size normalization are embedded into the joint input LayoutLMv3 model. The model performs multimodal feature fusion and layout analysis, and outputs the bounding box coordinates of each logical area on the page and its corresponding structural category labels (e.g., Title, Text, Table, Header). Then, based on the coordinate inclusion relationship and vertical relative distance of each bounding box, a tree-like hierarchical topology is constructed, thus forming a complete set of structural elements.

[0022] S102, Generate a sequence of structural symbols based on the set of layout structural elements; First, a unique structural symbol code is assigned to each different type of layout element. This structural symbol code can be represented by specific character, number, or string labels. For example, in this embodiment, a first-level heading is coded as the symbol label "H1", a second-level heading as the symbol label "H2", a regular paragraph as the symbol label "P", a table as the symbol label "T", a list as the symbol label "L", and a header as the symbol label "". The footer is coded as the symbol label " ”; Then, following the conventional reading order of the document page from top to bottom and from left to right, all layout structural elements are linearly expanded, and structural symbol sequences are generated in sequence according to their appearance order, thereby transforming the original two-dimensional page structure into a one-dimensional symbol sequence representation. To preserve hierarchical information, hierarchical identifiers are introduced between symbols. By recording parent-child node relationships, structural hierarchy markers are formed, ensuring that the structural symbol sequence not only reflects the layout element types but also preserves structural nesting relationships. In this embodiment, the hierarchical relationship is represented as a tree structure; and when converting to a one-dimensional symbol sequence, a hierarchical depth prefix is ​​embedded in each structural symbol code, or a closed bracket label (such as "...") is used. <h1>...< / h1> The tree structure is linearly serialized using a sequence to preserve the multi-dimensional hierarchical topological information in a one-dimensional sequence.

[0023] Therefore, each tender document can be represented as a sequence of structural symbols. ,in Indicates the first One tender document.

[0024] S103, perform sequence comparison of structural symbol sequences of different tender documents to identify the common source area of ​​the layout structure; To discover structural similarities among different tender documents, sequence alignment of structural symbol sequences from different documents is performed to identify common areas in the layout structure, specifically including: First, the structural symbol sequences of any two tender documents are matched and scanned. Continuous matching segments between the two sequences are identified through sequence comparison. Based on the length of the continuous matching segments and their distribution in the sequences, the common structural regions are determined. The longer the continuous matching segments and the more frequently they appear in multiple documents, the higher the consistency of the template structure in that region.

[0025] When performing sequence comparison, matching weights are first assigned based on the type of layout structural elements and their hierarchical relationships. Different types of structural elements exhibit varying stability during template copying. For example, heading and table structures typically have higher stability, while ordinary paragraph structures are more variable; therefore, different weight values ​​are assigned to them. The formula for calculating the matching weights is as follows: (1) in, Representing layout structure elements With layout structural elements Matching weights; Indicates the first The type of each layout structural element; Indicates the first The type of each layout structural element; This is a structure type consistency function; it takes a value of 1 when the two structure types are the same, and 0 otherwise. Representing layout structure elements The depth of the hierarchy; Representing layout structure elements The depth of the hierarchy; This represents the structure type matching weight coefficient, with a preferred value of 0.7. This represents the attenuation coefficient for hierarchical differences, with a preferred value of 0.3. Using this formula, even if two elements have different absolute hierarchical depths, as long as their types are consistent and the hierarchical difference is small, a positive matching score can still be obtained, thereby improving the tolerance for cross-document layout fine-tuning.

[0026] After obtaining the structural matching weights, a sequence alignment score matrix is ​​constructed based on these weights. Specifically, assume that the two structural symbol sequences are... and Their lengths are respectively and Construct a dimension as Score matrix Each element in the matrix represents the structural matching score at the corresponding position; the formula for calculating the sequence alignment score matrix is ​​as follows: (2) in, Indicates sequence position The cumulative score for alignment; Indicates the matching weights between layout structural elements; This represents the gap penalty coefficient, used to penalize insertion or deletion operations caused by paragraph additions, deletions, or format misalignments in the structural symbol sequence. Its preferred value is 0.4. This represents the cumulative score for the alignment of the previous pair of layout structure elements; and These represent the cumulative alignment scores when a structural element is missing or inserted, respectively. Subsequently, local sequence alignment calculation is performed based on the score matrix. Specifically, the point with the global maximum cumulative alignment score is found by scanning the entire score matrix as the starting point for tracking. Starting from this point, according to the calculation formula of the score matrix for sequence alignment, the cumulative alignment score at the current position is deduced in reverse: is it calculated from the cumulative alignment score of the previous pair of layout structural elements, or from the cumulative alignment score when structural elements are missing or inserted? The state is then backtracked in reverse along the path corresponding to the source of the calculation towards the upper left corner of the score matrix. During the backtracking process, the corresponding cumulative alignment score gradually decreases until the cumulative alignment score drops to 0, at which point the backtracking stops. The corresponding reverse state backtracking path is the continuous matching segment mapped in the two symbol sequences. The longer the continuous matching segment and the higher the cumulative score, the higher the probability of homology of the corresponding structural region. Further analysis of the distribution of continuous matching segments across multiple tender documents reveals that when a continuous matching segment appears repeatedly in at least three documents and its length exceeds a preset threshold (preferably, the length of the structural symbol sequence corresponding to the continuous matching segment is greater than 15), it is determined to be a homology region of the layout structure. For example... Figure 2 As shown, Figure 2 The image displays a local alignment score matrix of structural symbol sequences from two bid documents. The horizontal axis represents the position of structural symbols in bid document B, and the vertical axis represents the position of structural symbols in bid document A. The values ​​in the matrix reflect the matching scores between the structural symbols at corresponding positions. In the score matrix, high-scoring regions are presented as continuous segments distributed along the diagonal, indicating a high degree of consistency between the two bid documents in that structural segment. The continuous high-scoring segments corresponding to the boxed area in the image are the identified structural homogeneous regions, used to characterize the parts with high structural consistency between different bid documents.

[0027] The shared structural areas typically correspond to fixed template regions in tender documents, such as company qualification description pages, project implementation plan frameworks, or quotation templates. This structural modeling method allows for calculable measurement of structural consistency between different tender documents, providing a unified reference basis for subsequent template content analysis and serving as a prerequisite for cross-document template recognition.

[0028] Through the above steps, addressing the problems of ineffective utilization of bid document layout structure information and difficulty in identifying template structure consistency, this invention, in step S1, encodes layout structure elements into a sequence of structural symbols and introduces a weighted sequence comparison method based on structural type and hierarchical relationship to identify common areas in the layout structure, thereby achieving explicit modeling and precise positioning of the consistency of the bid document's structural skeleton. By transforming the original two-dimensional layout structure into a computable one-dimensional sequence of structural symbols, it helps to transform the template layout from an "interference factor" into an "analysis object," effectively improving the ability to identify template structure reuse behavior and solving the technical problem that existing methods struggle to utilize layout structure features for bid rigging analysis.

[0029] In this embodiment, step S2 is specifically implemented as follows: S201, Identify template anchor points in the structural symbol sequence and filter the candidate tender document set accordingly; Specifically, a sliding window scan is performed on the structural symbol sequence of all bid documents to detect consecutive symbol subsequences with an occurrence frequency higher than a frequency threshold, which are then used as template anchors. Template anchors typically correspond to highly stable structural combinations within the bid document template, such as fixed chapter structures like "Project Overview—Technical Solution—Implementation Plan—Quotation Table." The sliding window length is preferably 5 to 15 structural symbols, more preferably 8 to 12 structural symbols; the occurrence frequency threshold can be determined based on the total number of bid documents, preferably appearing in at least 3 bid documents, or accounting for more than 20% of the total number of bid documents.

[0030] When a continuous subsequence of symbols is detected to appear repeatedly in multiple bid documents, it is considered that the subsequence may represent a shared template structure. Bid documents containing the same continuous subsequence of symbols are located based on template anchor points, and these bid documents containing the same template anchor point are identified as a set of candidate bid documents. This process allows for rapid filtering of bid documents that may have been generated using the same template, thus significantly reducing the computational scale of subsequent analysis. Physically, template anchor points are equivalent to the "fingerprint" of the template structure; their existence implies a high degree of consistency in layout structure among multiple documents, thus highly likely originating from the same template copying behavior.

[0031] It should be noted that, according to the template anchor point, the tender documents containing the same continuous symbol subsequence refer to the numerous tender documents containing the same continuous symbol subsequence. The template anchor point includes various subsequences with different symbols. This embodiment does not mention the "screening or sorting rules when multiple anchor points are hit", but considering the case of multiple anchor points, when there are multiple template anchor points, the template anchor point with the highest frequency of occurrence is selected as the template anchor point used in subsequent steps.

[0032] In one implementation, to ensure the reliability of anchor points, the structural stability of the template anchor points needs to be calculated before determining the tender documents containing the same template anchor point as the candidate tender document set. Specifically, firstly, the occurrence positions of the template anchor points in different tender documents and their structural context, such as preceding and following structural symbols, hierarchical relationships, and page positions, are statistically analyzed. Then, the structural stability is calculated based on the statistical data. The formula for calculating the structural stability is as follows: (3) in, Indicates template anchor point Structural stability; Indicates the number of tender documents containing this template anchor point; Indicates the degree of difference in the context structure of template anchors; and These represent the position offset weighting coefficient and the context difference weighting coefficient, respectively, with preferred values ​​of 0.6 and 0.4. Indicates the first The normalized offset of the template anchor point position relative to its average position in each tender document, according to Calculation, where For the first Structural location index of template anchor points in each tender document This is the average of the anchor point structure location indices from all the bid documents. The length of the text word sequence in the cognate region of the layout structure; The context structure difference is calculated using the normalized edit distance formula, which is as follows: (4) in, This represents a function that calculates the standard edit distance (such as LevenshteinDistance) between two input sequences; and These represent consecutive symbol subsequences of a preset length (e.g., 5 structural symbols) preceding the template anchor point in the two tender documents to be compared. and Each represents a continuous subsequence of symbols of a preset length following the template anchor point (e.g., 5 structural symbols). This represents a function that retrieves the length of a corresponding consecutive symbol subsequence.

[0033] When structural stability If the value exceeds a preset threshold, the template anchor point is considered a valid template anchor point. The preset threshold is preferably set between 0.75 and 0.9, and more preferably 0.82. When the structural stability reaches this range, it indicates that the position of the structural subsequence in multiple tender documents maintains a high degree of consistency with the context structure, thus possessing high template indicative significance. This method can effectively eliminate misjudgments caused by accidental structural similarity.

[0034] S202, detect shared text fragments and identify variable fields in structurally similar regions within the set of candidate tender documents; Specifically, firstly, the text located within the common area of ​​the layout structure is extracted from the bid documents corresponding to the candidate bid document set, and then represented as... Sequence. The... A sequence refers to a series of consecutive... Each word forms a text segment unit, the... The value is set according to the text granularity requirements, preferably between 3 and 8, more preferably 5, for example when In k-gram sequences, every five consecutive words form a text unit. This representation improves matching efficiency while maintaining semantic continuity. Compared to full-text matching, k-gram sequences preserve text structural continuity while allowing for local differences, making them more suitable for identifying text patterns with field substitutions in template duplication.

[0035] Then, a search is conducted among the candidate bid documents for consecutive matches. sequence. when a certain If the sequence appears in the same position across different bid documents, or if its relative position offset is less than a preset sharing threshold, then the sequence is considered valid. The sequence is determined to be a shared text segment. The preset sharing threshold is preferably set between 0.05 and 0.12, more preferably 0.08. When the offset is lower than the preset sharing threshold, it indicates that the text segments maintain a basically consistent position within the template structure. The formula for calculating the relative position offset is as follows: (5) in, Indicates the relative position offset; Indicates the first The bid documents The position index of the sequence; Indicates another tender document China The position index of the sequence; This indicates the length of the text word sequence in the cognate region of the layout structure.

[0036] The shared text fragments represent template text directly copied or slightly modified from multiple bid documents. They typically correspond to fixed descriptive content within the bid template, such as company introduction paragraphs or technical description paragraphs. In one practical application, analysis of 18 bid documents for a bidding project revealed a 32-character (or similar) shared text fragment within the shared layout area. The text sequence was identified as a shared text fragment because its relative position offset was less than 0.06 in the 14 files. The content of this fragment corresponds to the enterprise qualification statement template text.

[0037] The variable fields are then identified, including the following steps: 1) When constructing shared text fragments, statistics are performed. The system counts the number of differing words in the sequence and identifies their field types, such as company name, project amount, and contact information. The identification of the field types of differing words utilizes a pre-defined business key feature rule base. Regular expressions are used to perform character pattern matching on the differing words: for company names, specific industry suffixes (such as "Limited Company" or "Group") are used for tail feature reduction; for project amounts, combinations of numbers and currency units (such as "ten thousand yuan" or "yuan") are used for capture. The system locks and outputs the corresponding field type label if and only if the differing word completely matches any specific text pattern in the rule base.

[0038] 2) Calculate the proportion of differing words. When the proportion of differing words to the length of the k-gram sequence is less than a preset difference ratio, it is determined as a variable field. The preset difference ratio is preferably between 0.15 and 0.25, and more preferably 0.2. The formula for calculating the proportion of differing words is as follows: (6) in, Indicates the proportion of words with differences; Indicates the correspondence in different bid documents The number of differing words (i.e., non-matching words) found in the sequence; This indicates that Total number of words in the sequence.

[0039] Furthermore, a field type recognition model identifies the field type to which the differing words belong, such as an amount field, a date field, or a company name field. This field type recognition model can be implemented based on a preset field rule base or a named entity recognition model. In this embodiment, the named entity recognition model specifically adopts a deep learning-based BERT-BiLSTM-CRF model. The preset field rule base is used to match fields with fixed format characteristics (such as matching the unified social credit code or a date in a specific format using regular expressions), and the named entity recognition model is used to identify entity types such as company names, amounts, dates, and personnel information in the text through contextual semantic feature serialization annotation.

[0040] To prevent key crosstalk clues from being mistakenly removed due to a large number of modified words, a relaxed filtering mechanism based on field type is introduced: when the difference word belongs to a variable field type, even if its difference word ratio is slightly higher than the preset difference ratio (for example, within the preset range of 0.2 to 0.35), it is still identified as a variable field.

[0041] The variable field types include, but are not limited to: bidder entity identification fields, mainly based on bidder name and unified social credit code; project business association fields, mainly based on total bid price, amount in words, and delivery period; and industry technical characteristic fields dynamically loaded according to the industry to which the bidding project belongs (such as server configuration parameters for software projects or building material topology indicators for civil engineering projects).

[0042] The identified variable fields were then incorporated. In the sequence, this maintains the continuity of the shared text structure, allowing for local field variations while preserving the overall consistency of the template structure, thus accurately characterizing template copying behavior. For example... Figure 3 As shown in the figure, the text distribution of multiple bid documents within the same source area of ​​the same layout structure is illustrated. The horizontal axis represents the relative positions within the source area, and different horizontal bars correspond to different bid documents. Solid bars represent shared text segments, while segments filled with diagonal lines represent variable fields. It can be seen that shared text segments maintain consistent relative positions across different bid documents, while variable fields are concentrated in several specific locations. Through the collaborative identification of shared text segments and variable fields, the combination pattern of fixed template content and field replacement content can be fully depicted, thereby achieving a structured expression of template anchor points.

[0043] For example, as shown in Table 1, the table presents exemplary data on template anchor point structure, structural stability, the location of shared text fragments, and variable field identification results for different bid documents. Specifically, template anchor points with high structural consistency are selected based on structural stability, and shared text fragments and variable fields are identified within the corresponding source area of ​​the layout structure. Table 1 shows that the template anchor point structures within the candidate bid document set exhibit high consistency, shared text fragments are mainly concentrated in the same structural sections, and variable fields correspond to field types such as company name, amount, or date, thus verifying the combined characteristic of "fixed text + variable fields" during template copying.

[0044] Table 1. Example data from different bid documents

[0045] It should be noted that template anchors are used to determine the candidate range at the structural level, shared text fragments are used to characterize the fixed content in the template, and variable fields are used to describe the replacement parts in the template. Together, the three constitute a complete expression of the template copying behavior.

[0046] Through the above steps, addressing the problems of being unable to distinguish between fixed content and variable fields in templates and the difficulty in identifying "template copying + field replacement" behavior, this invention, in S2, filters the set of candidate tender documents based on template anchor points and constructs k-gram sequences within the common source area of ​​the layout structure to identify shared text fragments. Simultaneously, it identifies variable fields by combining the proportion of differing words and field types, thereby achieving unified modeling of fixed text and variable fields in the template. By characterizing field replacement behavior while maintaining structural continuity, it helps to accurately identify the change patterns during template copying, effectively improving the accuracy and robustness of string identification, and solving the problem that existing methods struggle to distinguish between template copying and normal text similarity.

[0047] In this embodiment, the specific implementation of step S3 includes: After obtaining information on the common source areas of the layout structure, shared text fragments, and variable fields, to avoid misjudgment due to relying solely on local similarity, template binding stability is further calculated based on the proportion of common source areas of the layout structure, the density of shared text fragments, and the distribution stability of variable fields. Based on the template binding stability, the template replication scale and template diffusion rate are determined to identify the set of collusion files, as detailed below: S301, Calculate template binding stability; Specifically, among the identified shared text fragments, continuously matching structurally fixed sections are first identified. These structurally fixed sections refer to continuous text regions that remain completely consistent across multiple tender documents. These regions typically originate from unmodified fixed descriptions within a template, such as tender response instructions or general qualification descriptions. In this embodiment, by analyzing the shared text fragments... Perform consistency checks on the sequences, and select completely consistent consecutive sequences. Sequence (consistent) Sequence length and Sequence segments whose ratio to the total length exceeds a preset fixed ratio threshold are identified as structurally fixed segments.

[0048] Subsequently, text areas where differences exist between adjacent fixed-structure sections are divided into candidate field replacement sections. These sections typically correspond to locations in the template where specific information is allowed to be entered, such as company name, contact information, or quoted amount.

[0049] To determine whether a candidate field replacement section constitutes a genuine template replacement, the structural position offset of the candidate field replacement section relative to the source area of ​​the layout structure is further calculated in different bid documents. The formula for calculating the structural position offset is as follows: (7) in, Indicates the first The structural position offset of the candidate field replacement section in each bid document; Indicates the first The starting position of the candidate segment in the same source area of ​​the layout structure in each tender document; This represents the average starting position of the candidate segment across all relevant bid documents; This indicates the length of the text word sequence in the corresponding cognate region of the layout structure.

[0050] When the structural position offset is less than a preset offset threshold, the segment is identified as a template replacement position. The preset offset threshold is typically set between 0.03 and 0.1, preferably 0.06. When the structural position offset is below this range, it indicates that the field's position in the template remains essentially fixed, and therefore it can be considered a template field position.

[0051] Then, the stability of the variable field distribution is calculated based on the ratio of the number of template replacement bits to the length of the shared text fragment. The stability of the variable field distribution reflects the structural consistency of template fields within the document, and its calculation formula is as follows: (8) in, Indicates the first Stability of variable field distribution in individual tender documents; Indicates the first Number of template replacement positions in each tender document; Indicates the first The total number of words in the shared text segments across all tender documents. Assuming the template replacements have passed positional consistency screening, a higher index indicates a more concentrated and regular distribution of replacements within the shared text structure.

[0052] After obtaining the stability of the variable field distribution, the proportion of the common source area of ​​the layout structure is further calculated. This proportion is used to measure the percentage of the template structure in the entire tender document, and its calculation formula is as follows: (9) in, Indicates the first The proportion of common source areas in the layout structure of each tender document; Indicates the first The length of structural symbols in the same source area of ​​the layout structure in each tender document; Indicates the first The total length of the sequence of structural symbols in each tender document. The higher this proportion, the more likely the overall document structure is to originate from a uniform template.

[0053] Simultaneously, the density of shared text fragments is calculated to represent the degree of template text duplication. The formula for calculating the density of shared text fragments is as follows: (10) in, Indicates the first Density of shared text fragments in individual tender documents; Indicates the first The total number of words in the shared text segments of each tender document within the same source area of ​​the layout structure; This indicates the length of the text word sequence corresponding to the common source area of ​​the layout structure.

[0054] The word count of the shared text segment can be obtained by summing the word counts of all shared text segments. For shared segments with overlap, it is preferable to perform deduplication to avoid duplicate counting.

[0055] After obtaining the above three indicators, the template binding stability is calculated by integrating them. Template binding stability describes the binding strength between the bid document and the template, and its calculation formula is as follows: (11) in, Indicates the first Stability of template binding for individual tender documents; , , These represent the structural consistency weight, text duplication weight, and field stability weight, respectively, with preferred values ​​of 0.4, 0.35, and 0.25. ; Indicates the first The proportion of common source areas in the layout structure of each tender document; Indicates the first Density of shared text fragments in individual tender documents; Indicates the first Stability of variable field distribution in individual tender documents.

[0056] In this step, since a single indicator is difficult to reflect structural consistency, text reuse, and field replacement patterns simultaneously, the fusion of the above-mentioned multiple indicators can avoid misjudgments caused by relying solely on local features, thereby improving the overall accuracy of tag collusion identification.

[0057] S302, Determine the template replication scale and template diffusion rate based on template binding stability; Specifically, the template replication scale is determined by statistically analyzing the number of bid documents that simultaneously meet several conditions. First, the template binding stability difference between any two bid documents is calculated using the following formula: (12) in, Indicates the first The tender document and the first The difference in template binding stability between individual tender documents; Indicates the first Stability of template binding for individual tender documents; Indicates the first The template binding stability of two tender documents is determined. When the difference is less than a preset threshold, it indicates that the two documents have a highly consistent template structure. The threshold is typically set between 0.05 and 0.15, preferably 0.1.

[0058] Furthermore, it simultaneously determines whether the two documents share the same template anchor point and checks whether their template replacement positions are consistent within the same source area of ​​the layout structure. When all the above conditions are met, the two documents are considered to belong to the same template copy set. The template copy scale is determined by counting the number of bid documents that meet the conditions, and the calculation formula is as follows: (13) in, Indicates the template copy size; Indicates the total number of tender documents; As an indicator variable, when the first The value is 1 when each tender document meets the following three template replication scale conditions: stability difference, sharing the same template anchor point, and the template replacement position maintaining the same structural position in the same source area of ​​the layout structure; otherwise, it is 0.

[0059] The template diffusion rate is then calculated. The template diffusion rate describes the speed at which the template spreads between different bid documents, and its calculation formula is as follows: (14) in, Indicates the template diffusion rate; This indicates the number of tender documents that meet the template replication scale condition and are newly added to the template replication set in the current analysis period; This indicates the scale of template replication. This metric reflects the level of activity in template propagation.

[0060] The number of newly added bid documents to the template copy set is counted based on a preset time window. The time window can be the time interval for bid document submission or a set rolling time period. By continuously receiving new bid documents and dynamically updating the template copy set, the template diffusion rate can be dynamically calculated.

[0061] It should be noted that template binding stability characterizes the degree of consistency between a single bid document and the template, template replication scale describes the scope of template reuse, template diffusion rate reflects the dynamic characteristics of template propagation, and template propagation path reflects the evolution of template replication relationships. For example... Figure 4 As shown in the figure, the distribution of different candidate file sets across three dimensions—template binding stability, template replication scale, and template diffusion rate—is illustrated. The horizontal axis represents template binding stability, the vertical axis represents template replication scale, and the bubble size represents the template diffusion rate. The figure reveals that candidate sets with high template binding stability and large template replication scale typically have larger bubbles, indicating widespread propagation of the template across multiple bid documents and a high likelihood of bid rigging. By comprehensively analyzing these multi-dimensional characteristics, it is possible to effectively distinguish between normal bid documents and abnormal file sets exhibiting template replication behavior.

[0062] The above analysis quantifies the scale and spread trend of template duplication in bidding documents, thereby identifying document groups that may engage in bid rigging. This step, by constructing template binding stability and template propagation indicators, achieves a quantitative characterization of template duplication behavior in bidding documents, solving the technical challenge of traditional methods failing to identify covert template duplication.

[0063] To illustrate the results of bid-rigging identification based on the above indicators, Table 2 shows the average template binding stability, template duplication scale, template diffusion rate, examples of template duplication propagation paths, template source files, and final judgment results for different candidate file sets. The average template binding stability reflects the consistency between the template structure and the text content; the template duplication scale represents the number of bid documents involved in template duplication; and the template diffusion rate characterizes the degree of template propagation within the file set. Through comprehensive analysis of these indicators, candidate file sets with high template binding stability, large duplication scale, and high diffusion rate can be identified, thus determining that they exhibit bid-rigging behavior. As shown in Table 2, candidate sets A and B meet the preset thresholds on multiple indicators and are therefore identified as bid-rigging file sets, while candidate sets C and D are not identified as bid-rigging sets because they do not meet the corresponding threshold conditions.

[0064] Table 2 Results of Markup Co-operation Identification

[0065] S303, Identify the set of collusive bidding files based on the template replication and propagation relationship; Building upon the above, further identification of collusive bidding documents is performed. First, the template replication propagation relationship is determined based on the template binding stability and changes in the structural positions of variable fields in each bid document. When the absolute value of the template binding stability difference between two bid documents is less than a preset stability difference threshold and the structural positions of the template replacement bits remain consistent, a propagation relationship is determined between the two nodes, i.e., a replication propagation edge exists, and a template replication propagation relationship is established between them. When determining the direction of the propagation relationship, the submission time order of the bid documents or the degree of template evolution can be considered, preferentially selecting the bid document with an earlier submission time or a lower template evolution degree as the propagation source node.

[0066] Subsequently, a template replication propagation sequence is constructed based on the template replication propagation relationship, and the template replication propagation path is further identified. The template replication propagation path can be realized by constructing a propagation relationship graph between bid documents, where bid documents are nodes and propagation relationships are directed edges. Each propagation path is further identified through graph traversal (such as depth-first search (DFS) or breadth-first search (BFS)) or path search algorithms.

[0067] If multiple bids appear consecutively on the same propagation path, these bids are identified as a set of candidate bid-rigging documents.

[0068] To identify the template source file, the number of field replacements and structural changes in each bid document along the template copying and propagation path is further counted, and the template evolution degree is calculated. Before determining the template evolution degree, an initial reference template is first generated. Specifically, based on the set of candidate bid documents sharing the same valid template anchor point, the common structural region corresponding to each bid document is extracted. The structural symbol sequences and shared text fragments in the common structural region are matched accordingly. The structural symbol sequences that exist in multiple bid documents are identified as the structural part of the reference template, the shared text fragments that exist in corresponding positions are identified as the text part of the reference template, and the difference areas between text fragments are identified as replaceable field positions, thereby forming an initial reference template that includes the structural part of the reference template, the text part of the reference template, and the replaceable field positions. The formula for calculating the template evolution degree is as follows: (15) in, Indicates the first The degree of template evolution of each tender document; Indicates the first The number of field replacements in each bid document relative to the reference template; Indicates the first The number of structural changes of each bid document relative to the reference template, wherein the number of structural changes represents the number of structural insertion, deletion or structural type replacement operations generated when performing structural symbol sequence comparison between different bid documents, preferably obtained by statistics based on the number of gap operations and the number of structural type inconsistencies during the sequence comparison process; Indicates the first The total number of words in the shared text segments across all tender documents; and These represent the weights for field changes and structure changes, respectively, with preferred values ​​of 0.6 and 0.4.

[0069] It is important to note that the number of field replacements and the number of structural changes correspond to change events at the text content level and the layout structure level, respectively. Therefore, the total number of words in the shared text segment is used as the normalization benchmark to characterize the degree of change occurring within a single shared text segment. This processing is not intended to represent a direct quantitative relationship between the number of structural changes and the number of words in the text, but rather to break down the template evolution process into two dimensions: (1) Content evolution dimension: represented by the number of field replacements; (2) Structural evolution dimension: characterized by the number of insertions, deletions and type replacements in the structural symbol sequence.

[0070] Both types of changes occur within shared text fragments. Therefore, the total number of words in the shared text fragments is used as the template reuse range normalization parameter to obtain comparable evolutionary degrees between shared text fragments of different sizes.

[0071] When the evolution degree of a bid document template is lower than a preset evolution threshold, it indicates that the document is closest to the original template and is therefore identified as the template source document. The threshold is preferably set between 0.08 and 0.2.

[0072] The above analysis allows us to identify template replication and propagation paths and locate the template source file, thereby reconstructing the template propagation sequence. This step addresses the problem of traditional detection methods' difficulty in determining template origins by establishing template propagation relationships and evolutionary analysis mechanisms.

[0073] In the final identification stage, it is further determined whether the conditions for identifying bid rigging are met. When the number of bids in the candidate set of bid rigging reaches the template replication scale, and the template diffusion rate exceeds a preset diffusion threshold, the bids in the candidate set can be identified as a set of bid rigging documents. The diffusion threshold is usually set between 0.2 and 0.35, preferably 0.28. If there are multiple template replication propagation paths, and there are shared template anchor points or shared structural homogeneous regions between different paths, these propagation paths are merged, and all bids in the merged path are uniformly identified as a set of bid rigging documents.

[0074] Through the above steps, addressing the problem that similarity can only be judged between single files or file pairs, and that it is impossible to characterize the group behavior and propagation process of template copying, this invention constructs template binding stability in S3 by integrating the proportion of homogeneous regions in the layout structure, the density of shared text fragments, and the stability of variable field distribution. On this basis, it introduces template copying scale and template diffusion rate to quantitatively model the propagation range and diffusion trend of templates in the set of tender documents, thereby realizing the characterization of template copying behavior from "local similarity" to "group propagation". Furthermore, by constructing template copying propagation paths and identifying template source files based on template evolution, it helps to reveal the propagation relationship and evolution law between tender documents, effectively improving the integrity, relevance, and traceability of collusion identification, and solving the problem that existing methods are difficult to identify template propagation paths and sources.

[0075] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0076] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0077] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0080] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying bid rigging in tender documents, characterized in that, include: Multiple tender documents are obtained and their text content and layout structure elements, including table structure and list structure, are extracted. A structural symbol sequence is generated based on the layout structure element set. In the structural symbol sequence, a continuous subsequence of symbols with a frequency higher than a preset frequency threshold is detected as a template anchor point. Tender documents containing the same template anchor point are selected to construct a candidate tender document set. Within the candidate bid document set, the shared text segments are determined and variable fields are identified based on the matching relationship between the structural symbol sequences of different bid documents in the same structural source area; wherein, the same structural source area is determined by the structural symbol sequences through sequence alignment to identify continuous matching segments; The ratio of the length of the structural symbols in the common source region of the layout structure to the total length of the corresponding structural symbol sequence in the tender document is used to obtain the proportion of the common source region of the layout structure; the ratio of the total number of words in the shared text fragments to the length of the corresponding text word sequence in the common source region of the layout structure is used to obtain the density of the shared text fragments. Template binding stability is calculated based on the proportion of homogeneous areas in the layout structure, the density of shared text fragments, and the stability of variable field distribution. Template replication propagation relationship is determined based on the template binding stability and the structural position changes of variable fields. Template replication propagation sequence is extracted based on the propagation relationship to identify the set of collusive bidding files. The stability of the variable field distribution is determined based on the ratio of the number of template replacement positions to the total number of words in the shared text segment. The template replacement position is a difference segment whose position offset relative to the homogeneous area of ​​the layout structure is less than a preset offset threshold.

2. The method according to claim 1, characterized in that, The sequence alignment includes: Set matching weights based on the type and hierarchical relationship of layout structural elements; A sequence alignment score matrix is ​​constructed based on the matching weights, and local sequence alignment is calculated based on the score matrix.

3. The method according to claim 1, characterized in that, Before constructing the candidate bid set by filtering bids containing the same template anchor point, the process also includes: The occurrence positions and structural contexts of the template anchor points in different tender documents are statistically analyzed to calculate the structural stability of the template anchor points; When the structural stability is higher than a preset threshold, the corresponding template anchor point is determined as a valid template anchor point.

4. The method according to claim 1, characterized in that, Shared text fragments are determined based on the matching relationships of structural symbol sequences between different tender documents, including: Extract the text located in the homologous region of the layout structure from the bid documents corresponding to the candidate bid document set and represent it as a k-gram sequence; Search for consecutively matching k-gram sequences among the candidate bid documents; When the k-gram sequence appears in the same position in different tender documents or the relative position offset is less than a preset threshold, the k-gram sequence is identified as a shared text fragment.

5. The method according to claim 4, characterized in that, When a text fragment is identified as a shared text fragment, it also includes: Count the number of differential words in the k-gram sequence and identify the field types of the differential words; When the proportion of the number of differing words to the length of the k-gram sequence is less than the preset difference proportion, or when the differing words belong to the variable field type, they are determined as variable fields; The variable field is incorporated into the k-gram sequence.

6. The method according to claim 1, characterized in that, Before extracting the template replication propagation sequence based on the propagation relationship to identify the set of collusion files, the method further includes determining the template replication scale and template diffusion rate based on the template binding stability: The number of tender documents that meet the replication scale conditions is used as the template replication scale. The template diffusion rate is obtained by combining the number of newly added tender documents that meet the preset replication scale conditions within the preset analysis period. The template replication scale conditions include: The difference in template binding stability between the bid documents is less than a preset threshold; The tender documents share the same template anchor point; The template replacement position in the tender document maintains the same structural position in the same source area of ​​the layout structure.

7. The method according to claim 6, characterized in that, The step of extracting template copy propagation sequences based on the propagation relationship to identify the set of smuggling files includes: The template replication propagation path is identified based on the template replication propagation sequence, and the bid documents that are in the same template replication propagation path are identified as a set of candidate bid-rigging documents; the template replication propagation path is realized by constructing a propagation relationship graph between bid documents; The number of field replacements and structural changes of each bid document relative to the preset reference template in the template copying and propagation path are counted, and the template evolution degree, which characterizes the degree of change of the bid document relative to the reference template, is determined by combining the total number of words in the shared text fragments. When the template evolution degree of a certain bid document is lower than the preset evolution threshold, the bid document is determined as the template source file.

8. The method according to claim 7, characterized in that, The step of extracting template copy propagation sequences based on the propagation relationship to identify a set of collusive bidding files further includes: When the number of bid documents in the bid-rigging candidate document set reaches the template replication scale, and the template diffusion rate exceeds the preset diffusion threshold, the corresponding bid documents in the bid-rigging candidate document set are identified as bid-rigging document set. If there are multiple template copying and propagation paths, and there are shared template anchor points or structural homogeneous areas between different paths, then the multiple template copying and propagation paths will be merged, and all bid documents on the merged path will be identified as a set of bid-rigging documents.

Images