A remote merchant inspection method

Abstract

The application discloses a kind of remote merchant inspection methods, belong to mobile internet technical field, comprising: by obtaining and analyzing the to-be-audited inspection task package, after splitting data, relevance check is carried out, and time-space abnormal data is screened out;Recognize image scene and call corresponding exclusive model, and trigger cascade call when multiple scene features;Image feature detection quality and compliance are extracted, and the result is generated by fusion score;According to priority, artificial audit is pushed, and difference information is recorded;The output result is integrated in multiple dimensions, and is synchronized to risk control system;Daily sample collection, regularly update model;Through remote data analysis and relevance check, replace traditional artificial store inspection, greatly reduce the cost of manpower and travel in cross-regional, high-frequency scene, while saving scheduling, journey and other time-consuming links, significantly improve inspection efficiency, rely on multidimensional data correlation check, scene AI quality inspection and double-layer priority audit, effectively prevent on-site photo fraud, missed shot and other problems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile Internet, in particular to a remote merchant inspection method. BACKGROUND

[0002] In order to ensure the compliance of the receiving terminal, promotional materials and store image, banks, payment institutions or brand parties need to conduct merchant inspection regularly. The traditional manual store inspection has the following defects: the cost of manpower and travel is extremely high in the cross-regional and high-frequency scenarios; the execution efficiency is low due to scheduling and travel; the data authenticity cannot be guaranteed because the problems such as on-site photo fraud and missed shooting cannot be verified in real time; the headquarters cannot grasp the abnormalities in time and guide the rectification, and the real-time performance and risk control capability are weak.

[0003] The existing market photographing and clocking type remote inspection tools only have a single picture uploading function, lack the task automatic distribution, multi-dimensional authenticity verification, double-layer audit, abnormality closed loop and deep integration capability with the operation and management platform, and cannot meet the comprehensive needs of financial institutions for authenticity, compliance and large-scale operation. SUMMARY

[0004] In order to solve the problems of the prior art, the present application provides a remote merchant inspection method, comprising:

[0005] Step 1, obtaining an unverified inspection task package, performing analysis and processing on the unverified inspection task package, and splitting to obtain basic attribute data, scenario image data and space-time verification data; based on a preset association rule, the basic attribute data, the scenario image data and the space-time verification data are associated and verified, if the deviation of the shooting positioning information and the store registered address exceeds the preset threshold, the space-time abnormal data is marked and pushed to the abnormality processing process;

[0006] Step 2, performing label recognition on the scenario image data to determine the target inspection scene corresponding to each image; according to the target inspection scene, a corresponding scene-specific verification model in a preset scene-adaptive verification model cluster is called; if a single image contains multiple target inspection scene features, a model cascade calling mechanism is triggered, and the corresponding scene-specific verification model is called in a preset order;

[0007] Step 3, extracting the general visual features and scene-specific features of the image through the scene-specific verification model, and performing quality and compliance detection respectively; based on a preset weight, the two types of detection results are fused to obtain a comprehensive verification score, and the verification result and the corresponding abnormal identification code are output according to the comprehensive verification score;

[0008] Step 4, generating an audit priority according to the verification result and the abnormal identification code, and pushing the scenario image data and the verification result to a manual audit terminal for audit based on the audit priority;

[0009] Step 5, obtaining the relevance verification score and the merchant historical compliance score, fusing and calculating the comprehensive verification score, the relevance verification score and the merchant historical compliance score based on a preset weight ratio to obtain a final comprehensive score, outputting an audit result according to a comparison result of the final comprehensive score and a preset passing threshold, and synchronizing the audit result to a payment organization risk control system;

[0010] Further, in step 1, the basic attribute data includes a merchant registration name, a terminal registration model and a store registration address, the scenario image data includes a storefront photo, a cash register terminal photo, an SN code close-up photo and an operating place photo classified according to a preset inspection scene, and the space-time verification data includes photo shooting positioning information, a timestamp and a terminal device online state.

[0011] Further, in step 2, the scene-adaptive audit model cluster includes a storefront compliance verification model, a terminal device compliance verification model, an SN code recognition verification model and an operating scene compliance verification model.

[0012] Further, in step 3, the quality and compliance detection process includes: performing double-dimensional feature extraction and verification on the corresponding scenario image data through the scene-specific verification model; extracting general image features and performing quality detection;

[0013] The general image features include blurriness, reflectivity and exposure, the blurriness detection uses a Laplacian threshold combined with a MobileNetV3 algorithm, the reflectivity detection uses HSV color space screening combined with a U-Net network algorithm, and the exposure detection uses a brightness histogram analysis algorithm;

[0014] Extracting scene-specific features and performing compliance detection, the scene-specific features include storefront brand logo integrity, terminal device model matching, SN code character clarity and operating format matching;

[0015] Based on a preset weight, the general image feature detection score and the scene-specific feature detection score are weighted and calculated to obtain a comprehensive verification score, and a qualified or unqualified or recheck verification result and a corresponding abnormal identification code are output according to the comprehensive verification score.

[0016] Further, in step 5, if the final comprehensive score is in a preset rectification threshold interval, a re-audit result after rectification is output, the re-audit result after rectification is accompanied by unqualified item disassembly instructions; if the final comprehensive score is less than a preset risk threshold, a high-risk early warning result is output, and the high-risk early warning result is synchronized to the payment organization risk control system.

[0017] Further, the method further comprises: collecting an artificial correction sample, a new violation scene sample and a merchant appeal success sample into a verification sample library every day, the artificial correction sample is generated based on the difference reason and the correction information, and the new violation scene sample is a first appearing violation scene image and corresponding annotation information; and iteratively updating the scene adaptation audit model cluster according to a preset period;

[0018] The iteratively updating the scene adaptation audit model cluster according to a preset period comprises: fine-tuning parameters of each scene-specific verification model based on new samples every week, and performing full-volume retraining on each scene-specific verification model based on full-volume samples in the verification sample library every month, to update scene-specific feature extraction algorithms and detection rules in the model.

[0019] Further, in step 1, the preset association rule comprises a space-time association rule and an attribute consistency rule.

[0020] The space-time association rule is used to verify whether a deviation of space-time verification data from space reference information in the basic attribute data exceeds a preset space-time deviation threshold, and the space reference information is geographic association information of the target inspection object recorded in the system;

[0021] The attribute consistency rule is used to verify whether appearance features and identification features of a target object in the scene image data meet a preset matching degree with recorded features of a corresponding object in the basic attribute data; the preset association rule pre-configures a judgment threshold and an abnormal trigger condition of the two types of rules based on a business compliance standard of a target inspection scene, a historical inspection abnormal data statistical result and a risk control demand, and the preset association rule supports iteration and abnormal mode update according to a subsequent inspection business scene, and dynamically adjusts the judgment threshold and the trigger condition.

[0022] Further, the construction process of the scene-specific verification model comprises:

[0023] Collecting historical compliance images, violation images and corresponding compliance judgment results under each target inspection scene, labeling general visual defect labels and scene-specific compliance labels to the samples, and forming a scene-labeled sample set;

[0024] Based on the feature complexity of the target inspection scene, an adaptive initial model architecture is selected, and the initial model architecture comprises a general feature backbone network for extracting basic image features and a specific feature branch network for capturing scene-specific compliance features;

[0025] The initial model architecture is input with a set of contextualized labeled samples. General visual features are extracted through a general feature backbone network and general quality detection capabilities are trained. Contextual compliance features are learned through a specific feature branch network and compliance judgment capabilities are trained. A joint loss function that combines general feature loss and specific feature loss is used to iteratively adjust the model parameters until the matching degree between the model output verification results and the sample labeled results reaches a preset performance threshold.

[0026] The trained model is evaluated by using the validation sample set in the validation sample library. If there are any missed or misjudged features specific to a particular scenario, incremental samples for the corresponding scenario are added for secondary fine-tuning, ultimately creating a scenario-specific validation model that meets the compliance validation requirements of each scenario.

[0027] Furthermore, in step 4, if the scene-specific feature corresponding to the abnormal identification code is unqualified, it is marked as high priority; if the corresponding general image feature is unqualified, it is marked as medium priority; if the verification result needs to be reviewed, it is marked as low priority. The scene-specific image data and verification results are pushed to the manual review terminal in the order of high, medium and low priority.

[0028] The manual review terminal is equipped with a scenario-based review assistance module. The reviewer first uses the module's automatic association display unit to view the comparison display of the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants. Then, the reviewer uses the intelligent annotation prompt unit to view the suspected violation areas automatically marked based on the abnormal identification code and the accompanying compliance standard explanation. The reviewer uses this auxiliary information to judge the image quality and compliance, and draws a manual review conclusion of whether the review is passed, failed, or requires further review and enters it into the terminal.

[0029] The automatic association display unit is used to compare and display the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants, highlighting the differences.

[0030] The intelligent annotation and prompting unit automatically marks suspected violation areas in the image based on the anomaly identification code output by the scene-specific verification model, and attaches a compliance standard explanation.

[0031] The scenario-based review assistance module also includes a feedback closed-loop unit;

[0032] After the manual review conclusion is entered, the system automatically associates the corresponding model verification results to form a comparison record. The comparison record is synchronously stored in the verification sample library as a priority sample for model iteration.

[0033] Furthermore, the model cascading invocation mechanism also includes a scene feature conflict resolution module;

[0034] When there are correlational conflicts in the features of multiple target inspection scenes in a single image, the scene feature conflict resolution module automatically triggers an operation:

[0035] The operation includes: extracting feature weight values ​​of conflict scenarios, the feature weight values ​​being pre-configured based on the compliance importance of each scenario in the current inspection task; sorting the feature weight values ​​from high to low, prioritizing the retention of verification results for high-weight scenarios, and marking the verification results of low-weight scenarios as requiring secondary verification; and attaching the conflict scenarios and priority ranking results to the manual review task as an auxiliary basis for manual judgment.

[0036] The beneficial effects of this invention are:

[0037] By replacing traditional manual on-site inspections with remote data analysis and correlation verification, the system significantly reduces manpower and travel costs in cross-regional, high-frequency scenarios, while eliminating time-consuming steps such as scheduling and travel, thus significantly improving inspection efficiency. Relying on multi-dimensional data correlation verification, scenario-based AI quality inspection, and two-layer priority review, it effectively prevents issues such as falsified on-site photos and missed shots, ensuring the authenticity of inspection data. The review results can be synchronized to the acquiring institution's risk control system in real time, allowing headquarters to promptly grasp anomalies and guide rectification, strengthening the real-time nature of risk control. In addition, the scenario-adaptive model cluster and regular iteration mechanism make up for the shortcomings of the single function of existing photo check-in tools, realizing capabilities such as automatic task assignment, multi-dimensional verification, and anomaly closure, meeting the comprehensive needs of financial institutions for compliance and large-scale operations. Attached Figure Description

[0038] Fig. 1 This is a schematic diagram of the inspection process principle framework provided by the present invention;

[0039] Fig. 2 A schematic diagram illustrating the construction principle framework of the preset association rules provided by this invention;

[0040] Fig. 3 This is a schematic diagram illustrating the construction principle framework of the scenario-specific verification model provided by the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Please see Figs. 1-3 This invention provides a remote merchant inspection method, comprising the following steps:

[0043] Step 1, obtaining a to-be-audited inspection task package, performing analysis processing on the to-be-audited inspection task package, and splitting to obtain basic attribute data, scene image data, and space-time verification data; performing correlation verification on the basic attribute data, scene image data, and space-time verification data based on a preset correlation rule, and if the deviation of the shooting positioning information and the store registered address exceeds a preset threshold, marking as space-time abnormal data, and pushing the space-time abnormal data to an abnormal processing process;

[0044] The basic attribute data includes a merchant registered name, a terminal record type, and a store registered address, the scene image data includes a storefront photo, a cash register terminal photo, an SN code close-up photo, and a business place photo classified according to a preset inspection scene, and the space-time verification data includes photo shooting positioning information, a timestamp, and a terminal device online state.

[0045] In the step, the merchant record information, such as the store name, the address, and the required photos, such as the storefront, the terminal, the positioning when shooting, and the time, are automatically split, and then checked according to the rules defined in advance. Specifically, for example, the positioning when shooting and the store registered address are checked for the distance. If the distance is too far, the abnormality is directly marked, and the processing process is immediately entered. In this way, the problem of photo fraud and incorrect positioning can be found on the spot without manual checking, and the problems of traditional inspection data disorder, difficulty in verification, and low efficiency are solved. Compared with the existing tools, the data authenticity verification is added.

[0046] Now the information is clearly and completely displayed, including the merchant's name, terminal type, and address recorded in the system; necessary photos, such as the storefront photo, the cash register terminal photo, the SN code close-up photo, and the business place photo; and the positioning, time, and terminal online state when shooting.

[0047] Step 2, performing label recognition on the scene image data to determine the target inspection scene corresponding to each image; calling a corresponding scene-specific verification model in a preset scene-adaptive audit model cluster according to the target inspection scene; if a single image contains multiple target inspection scene features, triggering a model cascade calling mechanism, and sequentially calling corresponding scene-specific verification models in a preset order;

[0048] In step 2, the scene-adaptive audit model cluster includes a storefront compliance verification model, a terminal device compliance verification model, an SN code recognition verification model, and a business scene compliance verification model.

[0049] The inspection process involves four tools: one specifically for checking the storefront, mainly for inspecting the logo; one specifically for checking the terminal equipment, mainly for checking the consistency between the model and the registration; one specifically for checking the serial number, mainly for checking whether the handwriting is clear and recognizable; and one specifically for checking the business scenario, for checking the consistency between the actual products sold and the registered business type.

[0050] Traditionally, inspectors have to manually determine whether to check the storefront or the terminal, which can easily lead to omissions. Existing tools, even when receiving photos, cannot distinguish the scene, making targeted inspections impossible. This application automatically identifies whether each photo shows the storefront, the cashier terminal, or the serial number (SN), and then calls the tool specifically for that scene. If a photo contains both a terminal and a SN, it checks each scene sequentially using the corresponding tool. This eliminates the need for manual judgment, ensuring that every scene is checked, thus solving the problems of missed checks, low efficiency, and weak scene recognition capabilities of existing tools in traditional inspections.

[0051] Step 3: Extract the general visual features and scene-specific features of the image through the scene-specific verification model, and perform quality and compliance checks respectively; fuse the two types of detection results based on preset weights to obtain a comprehensive verification score, and output the verification result and the corresponding anomaly identification code according to the comprehensive verification score;

[0052] In step 3, the quality and compliance testing process includes: extracting and verifying dual-dimensional features of the corresponding scene-specific image data using the scene-specific verification model; extracting general image features and performing quality testing; the general image features include blurriness, reflectivity, and exposure. Blurriness detection uses Laplacian thresholding combined with the MobileNetV3 algorithm, reflectivity detection uses HSV color space filtering combined with the U-Net network algorithm, and exposure detection uses a brightness histogram analysis algorithm; extracting scene-specific features and performing compliance testing; the scene-specific features include the integrity of the storefront brand logo, the matching of the terminal device model, the clarity of the SN code characters, and the matching of the business format; weighting the general image feature detection score and the scene-specific feature detection score based on preset weights to obtain a comprehensive verification score; and outputting the verification result (pass / fail or requiring review) and the corresponding abnormal identification code based on the comprehensive verification score.

[0053] Among them, "qualified" means that the comprehensive verification score reaches the preset qualified threshold (≥80 points), and there are no serious problems with general image features (blur, reflection, exposure), and scene-specific features (completeness of storefront logo, terminal model matching, etc.) meet the compliance requirements.

[0054] If the overall verification score is lower than the preset failure threshold (≤60 points), it is usually due to serious problems, such as serious failure of general image features (serious blurring, serious reflection, etc.), or failure of key items of scene-specific features (complete mismatch of terminal model, serious incomplete SN code, etc.).

[0055] If a review is required, and the overall verification score falls between the acceptable and unacceptable thresholds (60 < score < 80), it usually indicates minor issues, such as minor deficiencies in general image features (slight blurring, slight reflection, etc.), or non-critical scene-specific features that require further confirmation (slightly incomplete SN code, partial association with business types, etc.). Further verification is needed to determine the final result.

[0056] This step involves a thorough inspection from two aspects: photo quality and compliance. Following the principle that compliance is more important than photo quality, the results of both inspections are scored, a total score is calculated, and a result is given as either "qualified," "unqualified," or "requires further review," along with the specific reasons for any discrepancies.

[0057] Traditional manual photo review has limitations, such as approximating photos as clear as possible and overlooking discrepancies between the device model and the registered specifications. Existing tools primarily focus on photo clarity, disregarding compliance issues. This application's step involves a two-pronged approach: firstly, checking photo quality (e.g., blurriness, glare, brightness); secondly, verifying compliance (e.g., completeness of the storefront logo, correct device model, clear serial number). A total score is calculated based on the principle that compliance is more important than photo quality, resulting in a pass / fail or require further review assessment. Anomalies, such as blurry serial numbers or incorrect device models, are clearly marked. This method ensures that neither poor-quality photos nor compliance issues are overlooked, making it more accurate than manual review based on intuition or the limited functionality of existing tools.

[0058] Step 4: Generate an audit priority based on the verification result and the anomaly identification code, and push the scene-based image data and verification result to the manual audit terminal for auditing based on the audit priority;

[0059] Previously, manual review was conducted sequentially, neglecting to consider the urgency of issues and thus delaying processing. This new step prioritizes tasks, for example, compliance violations are marked as high priority for manual processing; minor issues like blurry photos are marked as medium priority; and issues requiring further confirmation are marked as low priority. If the model makes an error after manual review, it will be recorded and marked, ensuring that serious issues are addressed promptly.

[0060] Step 5: Obtain the correlation verification score and the merchant's historical compliance score. Based on the preset weight ratio, perform a fusion calculation on the comprehensive verification score, the correlation verification score, and the merchant's historical compliance score to obtain the final comprehensive score. Based on the final comprehensive score and the comparison result with the preset pass threshold, output the audit result and synchronize the audit result to the acquiring institution's risk control system.

[0061] If the final comprehensive score is within the preset rectification threshold range, the re-examination result after rectification is output, and the re-examination result after rectification is accompanied by a breakdown of the non-conforming items; if the final comprehensive score is less than the preset risk threshold, a high-risk warning result is output, and the high-risk warning result is synchronized to the acquiring institution's risk control system.

[0062] It should be noted that step 4 generates an audit priority based on the comprehensive verification score and anomaly identification code from step 3, pushes the scenario-based image data to the manual audit terminal, and outputs the manual audit conclusion through the scenario-based audit assistance module (automatically compares historical / standard images and intelligently marks violation areas). This not only corrects the possible deviations in the model verification in step 3, but also generates a comparison record of the model verification results and the manual audit conclusions, providing a reliable comprehensive verification score data source for step 5. Step 5, based on this manual verification data, further integrates the correlation verification score from step 1 and the merchant's historical compliance score to calculate the final comprehensive score and output the audit result (qualified / rectification and re-audit / high-risk warning). At the same time, the result is synchronized to the acquiring institution's risk control system. The local image compliance audit in step 4 eliminates data deviation interference for the global merchant compliance judgment in step 5. The two work together to ensure the accuracy of the inspection results and the real-time nature of risk control, forming a complete judgment link of manual correction - multi-dimensional integration - risk control synchronization.

[0063] Currently, the inspection only considers the current inspection results, regardless of whether the merchant has violated regulations in the past, and does not take into account factors such as whether the location is correct. This step of the application will calculate a total score based on the importance of data correlation, the merchant's previous compliance records, and the score of this photo review: if the total score meets the requirements, it will be approved; otherwise, the merchant will be told how to rectify the situation; if the total score is too low, it will be directly marked as high risk and automatically transmitted to the risk control system.

[0064] The correlation verification score is obtained from the verification results based on preset correlation rules in step 1. In step 1, the system performs dual verification on the data according to the spatiotemporal correlation rules and the attribute consistency rules. The spatiotemporal correlation rules verify whether the deviation between the spatiotemporal verification data (shooting location, timestamp, terminal online status) and the basic attribute data (spatial reference information such as store registration address) exceeds a preset threshold. The attribute consistency rules verify whether the matching degree between the appearance / identification features of the target object in the scened image data (images of storefronts, terminals, etc.) and the filing features in the basic attribute data meets the standard. The system scores the verification results according to preset quantitative standards (such as whether the deviation exceeds the threshold and whether the matching degree meets the requirements) based on the verification results of the two types of rules (such as whether the deviation exceeds the threshold and whether the matching degree meets the requirements). Finally, the system integrates the scores of the two types of rules to generate the correlation verification score.

[0065] The merchant's historical compliance score is derived from the merchant's past inspection records stored in the system. The system automatically retrieves the final audit results of each previous inspection (such as whether it is qualified, whether there are violations, and whether rectification is successful), historical comprehensive verification score, frequency of violations, and rectification completion status, etc.; and according to preset calculation logic (such as the percentage of qualified inspections in the past 6 / 12 months, the duration of no high-risk warning records, and the timeliness of rectification after violations, etc.), it summarizes and weights the past compliance data to generate a merchant historical compliance score that reflects the merchant's long-term compliance level.

[0066] The comprehensive verification score, correlation verification score, and merchant historical compliance score are fused and calculated based on preset weight ratios to obtain the final comprehensive score. The audit result is then output based on a comparison between the final comprehensive score and a preset pass threshold: if the final comprehensive score is greater than or equal to the preset pass threshold, the audit is passed; if the final comprehensive score is within the preset rectification threshold range (below the pass threshold but above the risk threshold), a re-audit result after rectification is output, accompanied by a breakdown of non-compliance items; if the final comprehensive score is less than the preset risk threshold, a high-risk warning result is output. Finally, the audit results are synchronized to the acquiring institution's risk control system.

[0067] In addition, the core of fusion computing is the preset weight allocation and step-by-step weighted summation. All scores and weights are set around the inspection compliance logic and risk control requirements defined in the document. The specific process is as follows:

[0068] First, clarify the source and basic attributes of the three types of scores: all three types of scores are quantitative results of 0-100 points. The comprehensive verification score comes from step 3 (weighted result of image quality and scene-specific compliance), the correlation verification score comes from step 1 (correlation matching degree score of spatiotemporal data and basic attribute data), and the merchant's historical compliance score comes from the merchant's past inspection compliance records stored in the system (such as the comprehensive score of the percentage of qualified inspections in the past 6-12 months, rectification completion rate, etc.).

[0069] Next, the fusion calculation is performed in two steps: The first step is to preset the weight allocation, which needs to be set in advance according to the compliance priorities of the inspection business. For example, common preset weights are: comprehensive verification score accounting for 50%-60% (current image compliance is the core), correlation verification score accounting for 20%-30% (spatiotemporal / attribute matching is the basis of data authenticity), and merchant historical compliance score accounting for 10%-20% (past performance helps to judge overall compliance); The second step is to calculate the final score by weighted summation, multiplying each type of score by its corresponding weight, and then adding the three products together to obtain the final comprehensive score.

[0070] For example, if a merchant's overall verification score is 80 (weight 50%), relevance verification score is 90 (weight 30%), and historical compliance score is 70 (weight 20%), then the final overall score = 80×50%+90×30%+70×20%=40+27+14=81 points.

[0071] Finally, the final comprehensive score is compared with the preset thresholds (pass threshold, rectification threshold, and risk threshold) to output the corresponding results: if the score is ≥ the pass threshold, the review is approved; if it is in the rectification threshold range, rectification and re-review are required; if it is < the risk threshold, a high-risk warning is triggered and the risk control system is synchronized. The entire integrated calculation process always serves the goal of accurately determining the merchant's current overall compliance.

[0072] In some embodiments, in step 1, the preset association rules include spatiotemporal correlation rules and attribute consistency rules; the spatiotemporal correlation rules are used to verify whether the deviation between the spatiotemporal verification data and the spatial reference information in the basic attribute data exceeds a preset spatiotemporal deviation threshold, wherein the spatial reference information is the geographic association information of the target inspection object registered in the system; the attribute consistency rules are used to verify whether the appearance features and identification features of the target object in the scene-based image data and the registration features of the corresponding object in the basic attribute data meet a preset matching degree; the preset association rules are based on the business compliance standards of the target inspection scenario, the statistical results of historical inspection anomaly data, and risk control requirements, and pre-configure the judgment thresholds and anomaly triggering conditions of the two types of rules, and the preset association rules support dynamic adjustment of the judgment thresholds and triggering conditions according to subsequent inspection business scenario iterations and anomaly mode updates.

[0073] Among them, two verification rules are set: the first is the spatiotemporal correlation rule, which checks the location, time, and distance of the photo to the registered store address; if the difference is too large, it is marked as abnormal; the second is the attribute consistency rule, which checks whether the appearance and logo of the device in the photo are consistent with the registration; if the difference is too large, it is also marked as abnormal. These rules are set according to business requirements and past abnormal situations, and can be modified if the situation changes in the future.

[0074] In some embodiments, the construction process of the scenario-specific verification model includes: collecting historical compliance images, violation images, and corresponding compliance judgment results for each target inspection scenario; labeling the samples with general visual defect labels and scenario-specific compliance labels to form a scenario-specific labeled sample set; selecting an appropriate initial model architecture based on the feature complexity of the target inspection scenario, wherein the initial model architecture includes a general feature backbone network for extracting basic image features and a specific feature branch network for capturing scenario-specific compliance features; inputting the scenario-specific labeled sample set into the initial model architecture; extracting general visual features and training general quality detection capabilities through the general feature backbone network; learning scenario-specific compliance features and training compliance judgment capabilities through the specific feature branch network; iteratively adjusting the model parameters using a joint loss function that integrates general feature loss and specific feature loss until the matching degree between the model output verification result and the sample labeling result reaches a preset performance threshold; evaluating the performance of the trained model using the verification sample set in the verification sample library; if there are scene-specific feature omissions or misjudgments, supplementing the incremental samples of the corresponding scenario for secondary fine-tuning, ultimately creating a scenario-specific verification model that meets the compliance verification requirements of each scenario.

[0075] In building the inspection tool, we first collected previous compliance and non-compliance photos, clearly marking "where it was blurry and where it was non-compliant." Then, we had the tool learn in two parts: one part learned how to check photo quality, and the other part learned how to check compliance points in each scenario. After learning, we tested it with previous cases. If it was inaccurate, we added more photos and adjusted it again until the tool could make accurate judgments like a veteran employee.

[0076] In some embodiments, in step 4, if the scene-specific feature corresponding to the abnormal identification code is unqualified, it is marked as high priority; if the corresponding general image feature is unqualified, it is marked as medium priority; if the verification result needs to be reviewed, it is marked as low priority. The scene-specific image data and verification results are pushed to the manual review terminal in the order of high, medium and low priority.

[0077] The manual review terminal is equipped with a scenario-based review assistance module. The reviewer first uses the module's automatic association display unit to view the comparison display of the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants. Then, the reviewer uses the intelligent annotation prompt unit to view the suspected violation areas automatically marked based on the abnormal identification code and the accompanying compliance standard explanation. The reviewer uses this auxiliary information to judge the image quality and compliance, and draws a manual review conclusion of whether the review is passed, failed, or requires further review and enters it into the terminal.

[0078] The automatic association display unit is used to compare and display the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants, highlighting the differences.

[0079] The intelligent annotation and prompting unit automatically marks suspected violation areas in the image based on the anomaly identification code output by the scene-specific verification model, and attaches a compliance standard explanation.

[0080] The scenario-based review assistance module also includes a feedback closed-loop unit;

[0081] After the manual review conclusion is entered, the system automatically associates the corresponding model verification results to form a comparison record. The comparison record is synchronously stored in the verification sample library as a priority sample for model iteration.

[0082] The audit terminal has three auxiliary functions. First, it automatically puts the current photos together with the merchant's previous compliant photos and standard photos of similar merchants, and marks the differences. Second, it marks the places where violations may occur and tells the auditor what the compliance requirements are. Third, after the manual audit is completed, it automatically records the differences between the manual and the model and saves them as teaching materials to optimize the model.

[0083] In some embodiments, the model cascading call mechanism further includes a scene feature conflict resolution module; when there is a correlation conflict between the features of multiple target inspection scenes in a single image, the scene feature conflict resolution module automatically triggers an operation: the operation includes: extracting the feature weight values ​​of the conflicting scenes, the feature weight values ​​being pre-configured based on the compliance importance of each scene in the current inspection task; sorting the feature weight values ​​from high to low, prioritizing the retention of the verification results of high-weight scenes, and marking the verification results of low-weight scenes as requiring secondary verification; and attaching the conflicting scenes and priority ranking results to the manual review task as an auxiliary basis for manual judgment.

[0084] When encountering a photo containing both a terminal and a serial number (SN) that contradict each other, making it difficult to determine which to check first and potentially causing omissions, the system prioritizes the photos based on their importance, retaining the results of checks on the most important scenes first, marking the less important ones as needing further investigation, and informing the human operator of the conflicting information and which to check first.

[0085] In some embodiments, the method further includes: collecting manual error correction samples, new violation scenario samples, and successful merchant appeal samples daily and storing them in a verification sample library; wherein the manual error correction samples are generated based on the reasons for the differences and the error correction information; and the new violation scenario samples are images of violation scenarios that appear for the first time and their corresponding annotation information; and iteratively updating the scenario adaptation review model cluster according to a preset cycle.

[0086] The iterative update of the scenario-adaptive audit model cluster according to the preset cycle includes: fine-tuning the parameters of each scenario-specific verification model weekly based on new samples, and retraining each scenario-specific verification model monthly based on all samples in the verification sample library, updating the scenario-specific feature extraction algorithm and detection rules in the model.

[0087] This involves regularly updating the inspection tool, collecting new error correction cases weekly to make it more accurate, and retraining the tool monthly with all cases to further improve its precision.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A remote merchant inspection method, characterized in that, Includes the following steps: Step 1: Obtain the inspection task package to be reviewed, parse and process the inspection task package to be reviewed, and split it into basic attribute data, scene-based image data and spatiotemporal verification data. Based on preset association rules, the basic attribute data, scene-based image data, and spatiotemporal verification data are subjected to association verification. If the deviation between the shooting location information and the store registration address exceeds a preset threshold, it is marked as spatiotemporal abnormal data and the spatiotemporal abnormal data is pushed to the abnormal handling process. The preset association rules include spatiotemporal association rules and attribute consistency rules; The spatiotemporal correlation rule is used to verify whether the deviation between the spatiotemporal verification data and the spatial reference information in the basic attribute data exceeds the preset spatiotemporal deviation threshold. The spatial reference information is the geographic correlation information of the target inspection object registered in the system. The attribute consistency rule is used to verify whether the appearance features and identification features of the target object in the scene-based image data and the filing features of the corresponding object in the basic attribute data meet the preset matching degree. The preset association rules are based on the business compliance standards of the target inspection scenario, the statistical results of historical inspection anomaly data, and risk control requirements. They pre-configure the judgment thresholds and anomaly triggering conditions for two types of rules. Furthermore, the preset association rules support dynamic adjustment of the judgment thresholds and triggering conditions according to subsequent inspection business scenario iterations and anomaly mode updates. Step 2: Perform label recognition on the scene-based image data to determine the target inspection scene corresponding to each image; According to the target inspection scenario, the corresponding scenario-specific verification model in the preset scenario adaptation audit model cluster is called; if a single image contains multiple target inspection scenario features, the model cascading call mechanism is triggered, and the corresponding scenario-specific verification models are called in sequence according to the preset order. Step 3: Extract the general visual features and scene-specific features of the image through the scene-specific verification model, and perform quality and compliance checks respectively; Based on the preset weights, the two types of detection results are fused to obtain a comprehensive verification score. The verification result and the corresponding anomaly identification code are then output according to the comprehensive verification score. The model cascading invocation mechanism also includes a scene feature conflict resolution module; When there are correlation conflicts in the features of multiple target inspection scenes in a single image, the scene feature conflict resolution module automatically triggers an operation: The operation includes: extracting feature weight values ​​for conflict scenarios, the feature weight values ​​being pre-configured based on the compliance importance of each scenario in the current inspection task; sorting the feature weight values ​​from high to low, prioritizing the retention of verification results for high-weight scenarios, and marking the verification results for low-weight scenarios as requiring secondary verification; and attaching the conflict scenarios and priority ranking results to the manual review task as an auxiliary basis for manual judgment. Step 4: Generate an audit priority based on the verification result and the anomaly identification code, and push the scene-based image data and verification result to the manual audit terminal for auditing based on the audit priority; Step 5: Obtain the correlation verification score and the merchant's historical compliance score. Based on the preset weight ratio, perform a fusion calculation on the comprehensive verification score, the correlation verification score, and the merchant's historical compliance score to obtain the final comprehensive score. Based on the final comprehensive score and the comparison result with the preset pass threshold, output the audit result and synchronize the audit result to the acquiring institution's risk control system.

2. The remote merchant inspection method according to claim 1, characterized in that, In step 1, the basic attribute data includes the merchant's registered name, the terminal's registered model, and the store's registered address. The scenario-based image data includes photos of the storefront, the cash register terminal, close-up photos of the SN code, and photos of the business premises, categorized according to preset inspection scenarios. The spatiotemporal verification data includes photo shooting location information, timestamps, and the online status of the terminal device.

3. The remote merchant inspection method according to claim 1, characterized in that, In step 2, the scenario adaptation audit model cluster includes a storefront compliance verification model, a terminal device compliance verification model, an SN code recognition verification model, and an operational scenario compliance verification model.

4. The remote merchant inspection method according to claim 1, characterized in that, In step 3, the quality and compliance testing process includes: performing two-dimensional feature extraction and verification on the corresponding scene-specific image data using the scene-specific verification model; extracting general image features and performing quality testing; The general image features include blur, reflectivity, and exposure. Blur detection uses a Laplacian threshold combined with the MobileNetV3 algorithm, reflectivity detection uses HSV color space filtering combined with the U-Net network algorithm, and exposure detection uses a brightness histogram analysis algorithm. Extract scene-specific features and conduct compliance checks. These scene-specific features include the integrity of the storefront brand logo, the matching of terminal device models, the clarity of SN code characters, and the matching of business formats. The general image feature detection score and the scene-specific feature detection score are weighted and calculated based on preset weights to obtain a comprehensive verification score. The verification result and the corresponding abnormal identification code are output according to the comprehensive verification score, indicating whether the verification is qualified or not or needs to be reviewed.

5. The remote merchant inspection method according to claim 1, characterized in that, In step 5, if the final comprehensive score is within the preset rectification threshold range, the re-examination result after rectification is output, and the re-examination result after rectification is accompanied by a breakdown of the non-conforming items. If the final comprehensive score is less than the preset risk threshold, a high-risk warning result will be output and synchronized to the acquiring institution's risk control system.

6. The remote merchant inspection method according to claim 1, characterized in that, The method also includes: collecting manual error correction samples, new violation scenario samples, and successful merchant appeal samples daily and storing them in a verification sample library. The manual error correction samples are generated based on the reasons for the differences and error correction information. The new violation scenario samples are images of violation scenarios that appear for the first time and their corresponding annotation information. The scenario adaptation review model cluster is iteratively updated according to a preset cycle. The iterative update of the scenario-adaptive audit model cluster according to the preset cycle includes: fine-tuning the parameters of each scenario-specific verification model weekly based on new samples, and retraining each scenario-specific verification model monthly based on all samples in the verification sample library, updating the scenario-specific feature extraction algorithm and detection rules in the model.

7. The remote merchant inspection method according to claim 1, characterized in that, The construction process of the scenario-specific verification model includes: Collect historical compliance images, violation images, and corresponding compliance judgment results for each target inspection scenario; label the samples with general visual defect labels and scenario-specific compliance labels to form a scenario-specific labeled sample set. Based on the feature complexity of the target inspection scenario, an appropriate initial model architecture is selected. The initial model architecture includes a general feature backbone network for extracting basic image features and a dedicated feature branch network for capturing scenario-specific compliance features. The initial model architecture is input with a set of contextualized labeled samples. General visual features are extracted through a general feature backbone network and general quality detection capabilities are trained. Contextual compliance features are learned through a specific feature branch network and compliance judgment capabilities are trained. A joint loss function that combines general feature loss and specific feature loss is used to iteratively adjust the model parameters until the matching degree between the model output verification results and the sample labeled results reaches a preset performance threshold. The trained model is evaluated by using the validation sample set in the validation sample library. If there are any missed or misjudged features specific to a particular scenario, incremental samples for the corresponding scenario are added for secondary fine-tuning, ultimately creating a scenario-specific validation model that meets the compliance validation requirements of each scenario.

8. The remote merchant inspection method according to claim 1, characterized in that, In step 4, if the scene-specific feature corresponding to the abnormal identification code is unqualified, it is marked as high priority; if the corresponding general image feature is unqualified, it is marked as medium priority; if the verification result needs to be reviewed, it is marked as low priority. The scene-specific image data and verification results are pushed to the manual review terminal in the order of high, medium and low priority. The manual review terminal is equipped with a scenario-based review assistance module. The reviewer first uses the module's automatic association display unit to view the comparison display of the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants. Then, the reviewer uses the intelligent annotation prompt unit to view the suspected violation areas automatically marked based on the abnormal identification code and the accompanying compliance standard explanation. The reviewer uses this auxiliary information to judge the image quality and compliance, and draws a manual review conclusion of whether the review is passed, failed, or requires further review and enters it into the terminal. The automatic association display unit is used to compare and display the current image to be reviewed with the merchant's historical compliant images and standard images of similar merchants, highlighting the differences. The intelligent annotation and prompting unit automatically marks suspected violation areas in the image based on the anomaly identification code output by the scene-specific verification model, and attaches a compliance standard explanation. The scenario-based review assistance module also includes a feedback closed-loop unit; After the manual review conclusion is entered, the system automatically associates the corresponding model verification results to form a comparison record. The comparison record is synchronously stored in the verification sample library as a priority sample for model iteration.

Images