Retrieval enhancement large model graph discrimination method based on target detection and complexity prior

Abstract

The invention discloses a retrieval enhancement large model graph discrimination method based on target detection and complexity priori, which adopts'detection frame driving, retrieval enhancement and single-round structured decision 'to replace'block retrieval and multi-round generation', and reduces interaction links and uncertainty; complexity priori and guard calibration (high-risk one-ticket negative, evidence consistency check and thresholding gating) are introduced, excessive confidence in a complex scene is effectively inhibited, and no-basis release is avoided; a structured result (including judgment conclusion, confidence coefficient, reason and evidence index) in a fixed format is output, index-level highlighting and full-link auditing of a detection box and text evidence are achieved, and law enforcement redisk, responsibility tracing and continuous learning are facilitated; retrieval enhancement is compatible with multiple languages, multiple aliases, new brands and new packages, and the manual maintenance pressure of the rule base is remarkably reduced; a unified context and output mode enables a system interface to be stable, easy to integrate and easy to expand, and can be rapidly deployed in an existing security check workflow to form a reusable engineering closed loop.

Description

Technical Field

[0001] This invention relates to the fields of security inspection technology and image processing technology, specifically to a large-scale image judgment method based on target detection and complexity prior. Background Technology

[0002] Existing cargo security inspection image interpretation technology has the following drawbacks:

[0003] First, the usual approach is "image segmentation retrieval + two rounds of large model-generated responses," which lacks compliance-oriented "qualified / with review / unqualified" three-state criteria, structured JSON, and evidence indexes, making auditing and review inconvenient.

[0004] Second, the lack of prior knowledge of scene complexity and external fusion of confidence results in poor stability of judgment in complex scenarios such as occlusion and overlap.

[0005] Third, the lack of protection mechanisms such as high-risk veto, evidence consistency check and automatic downgrade means that the safety net is insufficient.

[0006] Fourth, no solution for index-level linked interface highlighting and full-link archiving is provided, making it difficult to achieve traceable human-computer interaction and continuous learning loop. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention aims to provide a large-scale image discrimination method for retrieval enhancement based on object detection and complexity prior.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] The image discrimination method for large-scale retrieval models based on object detection and complexity prior includes the following steps:

[0010] S1, Receive Input:

[0011] It receives X-ray images of goods and their corresponding cargo manifest numbers, as well as text information obtained from image tags, barcodes, and outer packaging, and generates a unique task number.

[0012] S2, Target Detection:

[0013] The target detection model is called to perform target detection on the security inspection X-ray image, and the detection bounding box, target category and confidence score are output.

[0014] S3. Complexity Assessment:

[0015] The complexity assessment model is used to determine the scene complexity of the entire security inspection X-ray image, and the result and probability value of whether the security inspection X-ray image belongs to a "complex scene" or a "regular scene" are output.

[0016] S4. Evidence Collection and Retrieval:

[0017] Using the target detection results output by the target detection model and the freight manifest number as clues, a combination of keyword retrieval and semantic vector retrieval is employed to search for relevant evidence in the business database, brand database, historical disposal database, and knowledge base. The business database stores structured business records including the freight manifest number, declared product name, quantity, weight, and business labels. The brand database stores the brand name, product specifications, multilingual aliases, packaging text features, and related knowledge entries. The historical disposal database stores feature summaries of past security inspection X-ray images, manual review conclusions, disposal reasons, evidence indexes, and audit records. The knowledge base stores a set of textual knowledge for risk assessment and semantic comparison, including dangerous goods classification rules, item attribute knowledge, synonyms / aliases / language variants / misspelling dictionaries, policy clause summaries, and semantic knowledge items for vector retrieval.

[0018] The evidence includes three types of information:

[0019] Image text information: including text entries based on the image label of the goods or the barcode recognition results of the goods;

[0020] Search results information: Relevant records matched from business databases, brand databases, and knowledge bases. Records matched from the knowledge base include text content, source, and similarity.

[0021] Historical records: Historical disposal cases retrieved from the historical disposal database that are similar to the current security X-ray image or the corresponding goods;

[0022] S5. Context Summary:

[0023] The current security X-ray image number, cargo manifest number, target detection result, complexity assessment result, and evidence information obtained in step S4 are integrated to generate a standardized context object Context.json;

[0024] S6, Structured Decision Making for Large Language Models:

[0025] Input the context object Context.json obtained in step S5 into the constrained large language model. The large language model is required to output only a structured JSON file. The output content includes: (1) the judgment result, including qualified, pending review, and unqualified; (2) the confidence score; (3) the reason for the judgment; (4) the evidence index; and (5) the remarks.

[0026] The judgment result is obtained by the large language model based on the context object in one inference, and the confidence score is the semantic consistency score within the context object; the evidence index corresponds to each piece of evidence information found in step S4; the remarks information is used to provide supplementary explanations.

[0027] S7, JSON compliance validation and controlled retries:

[0028] The output of the large language model is formatted. If it does not conform to the specifications, the large language model is prompted to re-output, and the number of retries shall not exceed the set N. If it still does not conform to the specifications after more than N retries, the judgment result is automatically set to "pending review" and "output format is non-compliant" is marked in the judgment reason.

[0029] S8. Execute the following protection mechanism on the output of the large language model after step S7:

[0030] Protection Mechanism 1: High-Risk Check

[0031] Based on a pre-set high-risk rule base, the target category and evidence are compared; when content belonging to the high-risk category appears in the target category or evidence, the judgment result is forcibly set to "unqualified" or "pending review" to ensure security;

[0032] Guardian Mechanism 2: Consistency Check

[0033] Check whether the judgment result is consistent with the evidence index; if there is a judgment result but the corresponding evidence index is missing, it means that the judgment result lacks basis, and the judgment result will be forcibly adjusted to "pending review", and the explanation of "insufficient evidence" will be added to the judgment reason.

[0034] Furthermore, in step S4, the found evidence is further deduplicated and denoised, and the number of entries is limited to ensure input stability.

[0035] Furthermore, the above method also includes S9, external confidence calculation, and secondary judgment:

[0036] After the large language model outputs the initial judgment result, the external confidence score conf_final is calculated based on multi-source information, and a second judgment is made on the "qualified" judgment result;

[0037] The external confidence score integrates the following three types of information:

[0038] (1) Aggregate confidence value of the referenced detection box

[0039] The confidence scores of the bounding boxes corresponding to the target detection results referenced by the large language model in step S6 are statistically aggregated, and the aggregated result is denoted as S_det;

[0040] (2) Probability value of complexity assessment

[0041] Take the probability value corresponding to the "rule scenario" in the complexity evaluation result in step S3, denoted as P_complex;

[0042] (3) Relevance density of evidence

[0043] Calculate the semantic relevance between the evidence cited by the large language model and the information of the current task, and aggregate the semantic relevance of the cited text evidence to obtain a normalized support metric between 0 and 1, denoted as D_evi;

[0044] (4) External confidence fusion formula

[0045] conf_final = α·S_det + β·P_complex + γ·D_evi

[0046] The external confidence conf_final is calculated by the following linear weighted method: where α, β, and γ are preset weights, and the weights can be manually set through experimental verification and are allowed to be configured according to the scenario, satisfying α + β + γ = 1;

[0047] When the judgment result output by the large language model is "qualified", perform a secondary judgment according to conf_final:

[0048] If conf_final ≥ T_high, then maintain the "qualified" conclusion of the large language model; if conf_final < T_high, then automatically downgrade the judgment result to "pending review;

[0049] For the cases where the judgment result output by the large language model is "pending review" or "unqualified", do not perform the secondary judgment operation; when hitting the high-risk category or triggering the guardian mechanism, take the result of the guardian mechanism in step S8 as the standard, regardless of the value of conf_final.

[0050] Furthermore, in step (1), the aggregation value is any one of the following:

[0051] Average value: Take the average of the confidence levels of the reference detection boxes, which can reflect the overall detection quality;

[0052] Weighted average: Perform weighting according to the detection box area, category weight, or confidence distribution;

[0053] Top-K average value: Take the average of the K detection boxes with the highest confidence levels to reduce the influence of outliers;

[0054] Statistical combination value: Simultaneously use the maximum and minimum values of the confidence levels to generate a stable confidence metric.

[0055] Furthermore, the calculation process of D_evi includes the following steps:

[0056] Step A: Text Vectorization Processing

[0057] All cited evidence, including image and text information, search results information, and historical information, are uniformly converted into vector form using a preset text encoding model to obtain evidence_vector[i].

[0058] Step B: Task Context Vectorization

[0059] Encode the key text information of the current task into a task vector context_vector; Step C: Calculate the semantic relevance of each piece of evidence.

[0060] For each evidence vector and task vector, calculate the relevance score sim_i, and normalize the value of relevance sim_i to the interval of 0 to 1;

[0061] Step D: Obtain the overall density by polymer correlation.

[0062] Aggregate the sim_i of all cited evidence to obtain the final evidence relevance density D_evi.

[0063] Furthermore, in step D, the aggregation method is any of the following:

[0064] average value:

[0065] Weighted average: The average is calculated based on the weight of the evidence source. D_evi = Σw i ·sim i , Σw i =1;

[0066] Top-K average: Take the K highest similarity scores and calculate the average: D_evi = mean(topK(sim)).

[0067] Furthermore, the above method also includes step S10, result presentation and archiving:

[0068] The results and confidence values ​​are displayed in the form of decision bars on the graphical interface, and the relevant detection boxes and evidence are highlighted according to the evidence index for easy manual verification. At the same time, the context object, the output of the large language model, the guard mechanism record, the external confidence and the final conclusion are generated into archived data, and a unique audit number is generated for subsequent backtracking, review and continuous learning of the large language model.

[0069] The beneficial effects of this invention are as follows: Without reducing the level of dangerous goods recalls, this invention significantly improves the consistency, robustness, and feasibility of security inspection judgments. Specifically:

[0070] 1) This invention uses "detection box driven + retrieval enhancement + single-round structured decision" instead of "block retrieval + multi-round generation" to reduce interaction steps and uncertainties;

[0071] 2) This invention introduces complexity prior and safeguard calibration (high-risk veto, evidence consistency check, threshold gating) to effectively suppress overconfidence in complex scenarios and avoid unfounded approval;

[0072] 3) This invention outputs structured results in a fixed format (including judgment conclusions, confidence levels, reasons, and evidence indexes), enabling index-level highlighting and full-link auditing of detection boxes and text evidence, facilitating law enforcement review, accountability tracing, and continuous learning;

[0073] 4) In this invention, the retrieval is enhanced to be compatible with multiple languages, multiple aliases, and new brands and packaging, significantly reducing the manual maintenance burden on the rule base;

[0074] 5) In this invention, the unified context and output mode make the system interface stable, easy to integrate and easy to expand, and can be quickly deployed in the existing security inspection workflow to form a reusable engineering closed loop. Attached Figure Description

[0075] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention. Detailed Implementation

[0076] The present invention will be further described below with reference to the accompanying drawings. It should be noted that this embodiment is based on the present technical solution and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to this embodiment.

[0077] Example 1

[0078] This embodiment provides a large-scale image judgment method for retrieval enhancement based on object detection and complexity prior, which can complete image-text consistency verification without reducing security recall and output a structured decision (qualified / pending review / unqualified) with evidence index. It achieves an auditable and robust engineering closed loop through guardian calibration and external fusion of confidence.

[0079] like Figure 1 As shown, the method in this embodiment includes the following steps:

[0080] S1, Receive Input:

[0081] It receives X-ray images of goods and the corresponding cargo manifest number (or invoice number), and also receives text information identified from the image label, the goods' barcode, and the outer packaging of the goods, and generates a unique task number (e.g., it can be synthesized by timestamp + location number).

[0082] S2, Target Detection:

[0083] The target detection model is invoked to perform target detection on the security inspection X-ray image, and the target detection box, target category and confidence score are output for subsequent evidence localization and visualization.

[0084] S3, Complexity Assessment

[0085] The complexity assessment model is used to determine the scene complexity of the entire security inspection X-ray image, and outputs the result and probability value of whether the security inspection X-ray image belongs to a "complex scene" or a "regular scene", which serves as the prior basis for subsequent confidence fusion and protection calibration.

[0086] S4. Evidence Collection and Retrieval (Enhanced Retrieval):

[0087] Using the target detection results output by the target detection model and the cargo manifest number as clues, a combination of keyword retrieval and semantic vector retrieval is employed to search for relevant evidence in the business database, brand database, historical disposal database, and knowledge base. The business database stores structured business records such as cargo manifest number, declared product name, quantity, weight, and business labels. The brand database stores brand names, product specifications, multilingual aliases, packaging text features, and related knowledge entries. The historical disposal database stores feature summaries of past security inspection X-ray images, manual review conclusions, disposal reasons, evidence indexes, and audit records. The knowledge base stores a set of textual knowledge for risk assessment and semantic comparison, including dangerous goods classification rules, item attribute knowledge, synonyms / aliases / language variants / misspelling dictionaries, policy clause summaries, and semantic knowledge items for vector retrieval.

[0088] The evidence includes three types of information:

[0089] Image text information (OCR): includes text entries based on the image label of the goods or the recognition result of the goods barcode;

[0090] Retrieval information: Relevant records matched from the business database, brand information database, and knowledge base. Records matched from the knowledge base include text content, source, and similarity.

[0091] Historical records (history[]): Historical disposal cases retrieved from the historical disposal database that are similar to the current security X-ray image or its corresponding cargo.

[0092] In this embodiment, to facilitate enhanced retrieval, a business database, a brand database, and a historical handling database are used to store different types of evidence data. These three databases provide textual evidence support for the large model from three dimensions: business information, brand knowledge, and similar cases, improving the consistency and interpretability of the judgments. The knowledge base assists in enhanced retrieval, enabling matching using rules and semantic knowledge beyond brand and business information.

[0093] In this embodiment, the found evidence is further deduplicated and denoised, and the number of entries is limited to ensure input stability.

[0094] More specifically, the business database contains structured tabular records (JSON, SQL tables). The brand database is a text-based knowledge base that can be vectorized for semantic retrieval, supports multilingual and multi-alias retrieval, and improves text evidence matching capabilities. The historical processing database contains image feature vectors + text labels + conclusion records.

[0095] More specifically, the knowledge base includes dangerous goods classification rules such as standardized descriptions, keywords, embargoes, and combined risk statements for items like fireworks, explosives, knives, liquids, and batteries. Item attribute knowledge includes common forms, packaging descriptions, and functional characteristics of such goods, such as "canned beer usually corresponds to 'can / 330ml / beer'". The thesaurus / aliases / language variants / misspellings is a fault-tolerant dictionary built to improve matching stability. Policy clause summaries mainly include extracted summaries of security inspection-related policy documents and general rule texts, used to understand semantics such as "permitted," "prohibited," and "requires review" during large-scale model judgment. Semantic knowledge items for vector retrieval are used for semantic vector retrieval, forming enhanced retrieval input together with the brand database and business database.

[0096] S5. Context Summary (Fixed Format)

[0097] The current security X-ray image number, cargo manifest number, target detection result, complexity assessment result, and evidence information found in step S4 are integrated to generate a standardized context object Context.json, providing standardized input for the subsequent structured adjudication of the large language model.

[0098] S6, Large Language Model Structured Adjudication (Single Round, Strict JSON Output)

[0099] Input the context object Context.json obtained in step S5 into the constrained large language model. The large language model is required to output only a structured JSON file. The output content includes: (1) judgment result (qualified, pending review, unqualified); (2) confidence score; (3) judgment reason; (4) evidence index (evidence_refs, corresponding to the serial number of the input evidence); (5) remarks.

[0100] The judgment result is obtained by the large language model based on the context object in one-time reasoning, and the confidence value is the semantic consistency score within the context object; the evidence index corresponds to each piece of evidence information found in step S4; the remarks information is used to provide supplementary explanations, such as uncertainty prompts, evidence status descriptions, etc.

[0101] Specifically, under constraint hints, the large language model performs a one-time structured decision based on the context object in Context.json. Its judgment criteria include: target detection results, evidence matching, whether the business declaration information in the business database matches the target detection results, historical records of similar cases, and complexity assessment (scenario complexity / rules). Based on this information, the large language model outputs three types of structured decision results:

[0102] "Qualified": The evidence information is fully consistent;

[0103] "Pending review": Insufficient evidence or minor conflict exists;

[0104] "Unqualified": Hits a high-risk category or there is obvious conflict in the evidence.

[0105] Furthermore, the confidence score output by the large language model is its internal confidence level generated based on context objects. It is a numerical field in the large language model output, used to reflect the degree of consistency in the decision. The confidence score can be generated through: semantic consistency scoring of the current judgment, support for evidence (such as the number and consistency of evidence items), and normalized probability values ​​generated by the internal scoring mechanism. The large language model generates a confidence score (between 0 and 1) based on factors such as textual consistency analysis of context objects, the degree of evidence support, and the strength of historical case matching, representing the large language model's confidence in the judgment conclusion.

[0106] Large language models prohibit the output of free text and can only generate structured data that meets the required format.

[0107] S7, JSON compliance validation and controlled retries

[0108] The output of the large language model is formatted. If it does not conform to the specifications, the large language model is prompted to re-output, and the number of retries shall not exceed the set N. If it still does not conform to the specifications after more than N retries, the judgment result is automatically set to "pending review" and "output format is non-compliant" is marked in the judgment reason.

[0109] S8. Execute the following protection mechanism on the output of the large language model after step S7:

[0110] Protection Mechanism 1: High-Risk Check

[0111] Based on a pre-defined high-risk rule base, the target category and evidence are compared. When the target category or evidence contains content belonging to the high-risk category (such as fireworks, ammunition, oxidizers, etc.), the judgment result is forcibly set to "unqualified" or "pending review" to ensure safety.

[0112] Guardian Mechanism 2: Consistency Check

[0113] Check whether the judgment result is consistent with the evidence index. If a judgment result exists but the corresponding evidence index is missing, it means that the judgment result lacks basis. The judgment result will be forcibly adjusted to "pending review", and the explanation of "insufficient evidence" will be added to the judgment reason.

[0114] S9. External Confidence Calculation and Secondary Judgment

[0115] To improve the stability and interpretability of the judgment results, this embodiment calculates the external confidence level (conf_final) based on multi-source information after the large language model outputs the initial judgment results, and performs a second judgment on the "qualified" judgment results.

[0116] External confidence scores incorporate the following three types of information:

[0117] (1) Aggregate confidence value of the referenced detection box

[0118] The confidence scores of the bounding boxes corresponding to the target detection results referenced by the large language model in step S6 are statistically aggregated, and the aggregated result is denoted as S_det to reflect the overall reliability of the evidence on which the large language model is based.

[0119] It should be noted that in step S1, the object detection model will output a set of detection boxes and their confidence scores. In step S6, the large language model will reference a portion of these detection boxes, and their indexes are recorded in evidence_refs.detections.

[0120] The aggregated value may include:

[0121] Mean: The average confidence score of the referenced detection boxes is taken as a whole, which can reflect the overall detection quality.

[0122] Weighted mean: Weighted based on the area of ​​the detection box, class weights, or confidence distribution;

[0123] Top-K mean: The average of the K detection boxes with the highest confidence is taken to reduce the impact of outliers;

[0124] Statistical composite: This method uses both the maximum and minimum confidence levels to generate a stable confidence index.

[0125] (2) Probability value of complexity assessment

[0126] Take the probability value corresponding to the "rule scenario" from the complexity evaluation results in step S3, and denote it as P_complex, which reflects the prior influence of scenario complexity on the reliability of the judgment.

[0127] (3) Relevance density of evidence

[0128] The semantic relevance between the evidence cited by the large language model (including image and text information, retrieval results information, and historical information) and the information of the current task is calculated. The semantic relevance of the cited text evidence is aggregated to obtain a normalized support index between 0 and 1, denoted as D_evi.

[0129] The relevance density of evidence is used to quantify the degree of semantic support between evidence cited by the large language model and information related to the current task (including cargo manifest number, declared product name, target category, etc.). Its calculation process includes the following steps:

[0130] Step A: Text Vectorization Processing

[0131] All cited evidence, including image and text information, retrieval results information, and historical information, are uniformly converted into vector form using a preset text encoding model (such as word vectors, sentence vectors, or other feature encoders) to obtain evidence_vector[i].

[0132] Step B: Task Context Vectorization

[0133] Encode the key text information of the current task (such as the declared product name corresponding to the freight manifest number, the inspection box category label, brand name, etc.) into a task vector context_vector.

[0134] Step C: Calculate the semantic relevance of each piece of evidence.

[0135] Calculate the relevance score sim_i for each evidence vector and the task vector. The relevance function can be cosine similarity, normalized Euclidean distance, keyword overlap rate (TF-IDF matching coefficient), or multi-feature fusion similarity. The value of the relevance sim_i is normalized to the range of 0 to 1.

[0136] Step D: Aggregate the relevance to obtain the overall density

[0137] Aggregate the sim_i of all the cited evidence to obtain the final evidence relevance density D_evi.

[0138] The aggregation methods can include but are not limited to:

[0139] Mean value (mean):

[0140] Weighted mean: Weight according to the evidence source weight (e.g., brand database > OCR > historical record) D_evi = ∑w i ·sim i , ∑w i = 1

[0141] Top-K mean: Take the top K similarities and calculate the average D_evi = mean(topK(sim)) to suppress noise. [[ID=二十六]]

[0142] (4) External confidence fusion formula

[0143] conf_final = α · S_det + β · P_complex + γ · D_evi

[0144] The external confidence conf_final is calculated through the following linear weighting method: where α, β, and γ are preset weights. The weights can be manually set through experimental verification and are allowed to be configured according to the scenario, satisfying α + β + γ = 1.

[0145] When the judgment result output by the large language model is "qualified", perform a secondary judgment based on conf_final:

[0146] If conf_final ≥ T_high, maintain the "qualified" conclusion of the large language model; if conf_final < T_high, automatically downgrade the judgment result to "pending review".

[0147] For the cases where the judgment result output by the large language model is "pending review" or "unqualified", do not perform the secondary judgment operation. When a high-risk category is hit or the guardian mechanism is triggered, the result of the guardian mechanism in step S8 shall prevail regardless of the value of conf_final.

[0148] S10. Results Presentation and Archiving

[0149] The results and confidence values ​​are displayed in the form of decision bars on the graphical interface, and the relevant detection boxes and evidence are highlighted according to the evidence index for easy manual verification. At the same time, the context object, the output of the large language model, the guard mechanism record, the external confidence and the final conclusion are generated into archived data, and a unique audit number is generated for subsequent backtracking, review and continuous learning of the large language model.

[0150] Example 2

[0151] This embodiment provides an application of the method described in Embodiment 1 in the compliance determination of drinkable liquids (beer), including the following steps:

[0152] Step 1: Receive Input: Obtain the security X-ray image and the corresponding cargo manifest number (manifest_id), and simultaneously receive the text entries (such as "Budweiser", "beer", "330ml") obtained from image label / barcode recognition.

[0153] Step 2, Target Detection: Output candidate bounding boxes such as bottle / can, their confidence scores, and their positions in the security inspection X-ray image; the highest confidence score is recorded as 0.86.

[0154] Step 3: Complexity assessment: Output the entire security inspection X-ray image and determine whether it is a "complex scene" with a probability vector of [0.72, 0.28] (complex / regular).

[0155] Step 4, Evidence Collection and Retrieval (Retrieval Enhancement): Using the candidate box and the freight manifest number as clues, a joint search using keywords and semantic vectors was conducted. The search for "330ml beer (Budweiser)" was found in the brand database and business database, with a relevance score of 0.91. At the same time, a historical disposal record of "drinkable beer → qualified" was also found.

[0156] Step 5: Context Summary: Generate a standardized context object Context.json (containing image ID, detection result, complexity result, image text information, retrieval results, and historical records).

[0157] Step Six, Structured Decision (Single Round): Submit Context.json to the large language model, requiring only the following JSON output:

[0158] {

[0159] "decision":"qualified",

[0160] "confidence":0.78,

[0161] "reasons":["Detected as a liquid container, the text matches the beer information found in the search, and the packaging is independent"],

[0162] "evidence_refs":{"detections":[0,1],"ocr":[0,1],"retrieval":[0],"history":[0]},

[0163] "notes":""}

[0164] Step 7: JSON validation: If it does not conform to the specifications, it will be retried under controlled conditions; this example passed on the first try.

[0165] Step 8: Guardian Mechanism

[0166] ① High-risk inspection: No high-risk clusters such as "fireworks, ammunition, and oxidizers" were detected.

[0167] ② Consistency check: The ruling contains a valid evidence index and passes.

[0168] Step Nine: External Confidence Fusion and Secondary Judgment

[0169] According to the formula conf_final=α·maxS_det+β·P_complex+γ·D_evi, taking α=0.45, β=0.15, γ=0.40, and substituting maxS_det=0.86, P_complex=0.72, D_evi=0.91, we get conf_final≈0.86. When the threshold is set to T_high=0.80 and T_mid=0.50, "qualified" is adopted.

[0170] Step 10, Presentation and Archiving: The interface displays the conclusion and confidence level with a green decision bar; the detection boxes and text evidence are highlighted by the evidence index; the context, model output, guard records and final conclusions are archived at the same time, and an audit number is generated.

[0171] In this embodiment, the Top-K search entries can be set to 5 to 10; the complexity evaluation model can be replaced; the fusion weights (α, β, γ) can be configured or learned through historical annotations.

[0172] Example 3

[0173] This embodiment provides an application of the method described in Embodiment 1 in the determination of suspected fireworks interception.

[0174] Step 1: Receive Input: Obtain the security X-ray image and cargo manifest number, and use text recognition OCR to obtain text fragments such as "festival pack" and "sparkler" on the outer packaging of the goods.

[0175] Step 2, Object Detection: Output Fireworks candidate boxes with a maximum confidence score of 0.82.

[0176] Step 3, Complexity Assessment: Output "Rule Scenario", probability vector [0.31, 0.69] (complexity / rule).

[0177] Step 4: Evidence collection and retrieval: The search query for "fireworks assortmentpack" was found in the business database / brand database with a relevance of 0.88; there were no clear precedents of "qualification" in the historical processing database.

[0178] Step 5: Context summary: Generate a standardized Context.json file.

[0179] Step Six, Structured Adjudication (Single Round): The large language model outputs a "Pending Review" conclusion based on the context (reason: "Suspected fireworks kit requires confirmation of components"), and provides an evidence index. The output JSON file content is as follows:

[0180] "decision":"Pending review",

[0181] "confidence":0.73,

[0182] "reasons":["Suspected fireworks kit; composition and ignition device need to be confirmed"],

[0183] "evidence_refs":{"detections":[0],"ocr":[1],"retrieval":[0],"history":[]},

[0184] "notes":""}

[0185] Step 7, JSON validation: Passed.

[0186] Step 8: Guardian Mechanism

[0187] ① High-risk inspection: Fireworks is classified as high-risk, triggering a veto and forcibly setting the judgment result to "unqualified" or, under policy requirements, "pending review".

[0188] ② Consistency check: Valid evidence index exists, pass.

[0189] Step 9, External Confidence Fusion and Secondary Judgment: Substituting maxS_det = 0.82, P_complex = 0.31, and D_evi = 0.88, we get conf_final ≈ 0.76. Since a high-risk result has been hit, the conclusion remains "unqualified"; for ease of display, the minimum display confidence level can be set to 0.85 (this does not change the substantive decision, but is only used for consistency of the interface prompt).

[0190] Step 10, Presentation and Archiving: The interface displays the conclusion with a red judgment bar; the detection boxes and referenced text of suspected fireworks are highlighted by index; the entire chain of data is archived to generate an audit number for review and law enforcement evidence collection.

[0191] In this embodiment, the high-risk rule base can be maintained according to the airport / port requirements in terms of version and severity level; when there is a "combined risk" (such as "battery + wire + ignition device"), protection can be triggered through combined rules; for OCR noise, synonym / misspelling tolerance and whitelist suppression of false alarms can be enabled.

[0192] For those skilled in the art, various corresponding changes and modifications can be made based on the above technical solutions and concepts, and all such changes and modifications should be included within the protection scope of the claims of this invention.

Claims

1. A large-scale image discrimination method for retrieval enhancement based on object detection and complexity prior, characterized in that, Includes the following steps: S1, Receive input: It receives X-ray images of goods and their corresponding cargo manifest numbers, as well as text information obtained from image tags, barcodes, and outer packaging, and generates a unique task number. S2, Target Detection: The target detection model is called to perform target detection on the security inspection X-ray image, and the detection bounding box, target category and confidence score are output. S3. Complexity Assessment: The complexity assessment model is used to determine the scene complexity of the entire security inspection X-ray image, and the result and probability value of whether the security inspection X-ray image belongs to "complex scene" or "regular scene" are output. S4. Evidence Collection and Retrieval: Using the target detection results output by the target detection model and the freight manifest number as clues, a combination of keyword retrieval and semantic vector retrieval is employed to search for relevant evidence in the business database, brand database, historical disposal database, and knowledge base. The business database stores structured business records including the freight manifest number, declared product name, quantity, weight, and business labels. The brand database stores the brand name, product specifications, multilingual aliases, packaging text features, and related knowledge entries. The historical disposal database stores feature summaries of past security inspection X-ray images, manual review conclusions, disposal reasons, evidence indexes, and audit records. The knowledge base stores a set of textual knowledge for risk assessment and semantic comparison, including dangerous goods classification rules, item attribute knowledge, synonyms / aliases / language variants / misspelling dictionaries, policy clause summaries, and semantic knowledge items for vector retrieval. The evidence includes three types of information: Image text information: including text entries based on the image label of the goods or the barcode recognition results of the goods; Search results information: Relevant records matched from business databases, brand databases, and knowledge bases. Records matched from the knowledge base include text content, source, and similarity. Historical records: Historical disposal cases retrieved from the historical disposal database that are similar to the current security X-ray image or the corresponding goods; S5. Context Summary: The current security X-ray image number, cargo manifest number, target detection result, complexity assessment result, and evidence information obtained in step S4 are integrated to generate a standardized context object Context.json; S6, Structured Decision Making for Large Language Models: Input the context object Context.json obtained in step S5 into the constrained large language model. The large language model is required to output only a structured JSON file. The output content includes: (1) the judgment result, including qualified, pending review, and unqualified; (2) the confidence score; (3) the reason for the judgment; (4) the evidence index; and (5) the remarks. The judgment result is obtained by the large language model based on the context object in one inference, and the confidence score is the semantic consistency score within the context object; the evidence index corresponds to each piece of evidence information found in step S4; the remarks information is used to provide supplementary explanations. S7, JSON compliance validation and controlled retries: Perform format verification on the output of the large language model. If it does not meet the specifications, prompt the large language model to re-output, and the number of retries does not exceed the set N times; if it still does not meet the specifications after N retries, automatically set the determination result to "pending review", and mark "output format non-compliant" in the determination reason; S8. Execute the following guardian mechanism on the output of the large language model after step S7: Guardian mechanism 1. High-risk inspection: Compare the target category and evidence according to the preset high-risk rule library; when the content belonging to the high-risk category appears in the target category or evidence, forcibly set the determination result to "unqualified" or "pending review" to ensure safety; Guardian mechanism 2. Consistency check: Check whether the determination result is consistent with the evidence index; if there is a determination result but the corresponding evidence index is missing, it means that the determination result lacks basis, and the determination result is forcibly adjusted to "pending review", and an explanation of "insufficient evidence" is added to the determination reason.

2. The method according to claim 1, characterized in that, In step S4, further remove duplicates and noise from the found evidence, and limit the number of entries to ensure stable input.

3. The method according to claim 1, characterized in that, It also includes S9. External confidence calculation and secondary determination: After the large language model outputs a preliminary determination result, calculate the external confidence conf_final based on multi-source information, and perform secondary determination on the determination result of "qualified"; The external confidence integrates the following three types of information: (1) Confidence aggregation value of the detected frame being cited Statistically aggregate the confidence of the detection frames corresponding to the target detection results cited by the large language model in step S6, and the aggregation result is denoted as S_det; (2) Probability value of complexity assessment Take the probability value corresponding to the "rule scenario" in the complexity assessment result in step S3, and denote it as P_complex; (3) Relevance density of evidence Calculate the semantic relevance between the evidence cited by the large language model and the information of the current task, and aggregate the semantic relevance of the cited text evidence to obtain a normalized support index between 0 and 1, denoted as D_evi; (4) External confidence fusion formula conf_final = α·S_det + β·P_complex + γ·D_evi The external confidence conf_final is calculated by the following linear weighting method: where α, β, and γ are preset weights, and the weights can be manually set through experimental verification and are allowed to be configured according to the scenario, satisfying α + β + γ = 1; When the determination result output by the large language model is "qualified", perform secondary determination according to conf_final: If conf_final ≥ T_high, maintain the "qualified" conclusion of the large language model; if conf_final < T_high, automatically downgrade the determination result to "pending review; For the cases where the determination result output by the large language model is "pending review" or "unqualified", no secondary determination operation is performed; when a high-risk category is hit or the guardian mechanism is triggered, the result of the guardian mechanism in step S8 shall prevail regardless of the value of conf_final.

4. The method according to claim 3, characterized in that, In step (1), the aggregation value is any one of the following: Average value: The average confidence level of the referenced detection boxes is taken, which can reflect the overall detection quality; Weighted average: Weighted based on the area of ​​the detection box, class weights, or confidence distribution; Top-K average: The average of the K detection boxes with the highest confidence is taken to reduce the impact of outliers; Statistical combination value: A stable confidence index is generated by using both the maximum and minimum confidence values.

5. The method according to claim 3, characterized in that, The calculation process of D_evi includes the following steps: Step A: Text Vectorization Processing All cited evidence, including image and text information, search results information, and historical information, are uniformly converted into vector form using a preset text encoding model to obtain evidence_vector[i]. Step B: Task Context Vectorization Encode the key text information of the current task into a task vector context_vector; Step C: Calculate the semantic relevance of each piece of evidence. For each evidence vector and task vector, calculate the relevance score sim_i, and normalize the value of relevance sim_i to the interval of 0 to 1; Step D: Obtain the overall density by polymer correlation. Aggregate the sim_i of all cited evidence to obtain the final evidence relevance density D_evi.

6. The method according to claim 5, characterized in that, In step D, the aggregation method is any of the following: average value: Weighted average: The average is calculated based on the weight of the evidence source. D_evi = Σw i ·sim i , Σw i =1; Top-K average: Take the K highest similarity scores and calculate the average: D_evi = mean(topK(sim)).

7. The method according to claim 1, characterized in that, It also includes step S10, result presentation and archiving: The results and confidence values ​​are displayed in the form of decision bars on the graphical interface, and the relevant detection boxes and evidence are highlighted according to the evidence index for easy manual verification. At the same time, the context object, the output of the large language model, the guard mechanism record, the external confidence and the final conclusion are generated into archived data, and a unique audit number is generated for subsequent backtracking, review and continuous learning of the large language model.

Images