Order screenshot detection method and device, detection model training method, and logistics request processing method

Abstract

The application discloses an order screenshot detection method and device, a detection model training method and a logistics request processing method, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: recognizing text content in an order screenshot and position information of the text content; determining a comprehensive determination result of the order screenshot according to the order screenshot, the text content and the position information; determining a format determination result of the order screenshot according to the position information; and judging the authenticity of the order screenshot according to the comprehensive determination result and the format determination result. The embodiment reduces detection cost and data and template dependence, and improves detection efficiency and detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, training method for a detection model, and processing method for logistics requests for detecting order screenshots. Background Technology

[0002] Currently, when logistics e-commerce platforms offer services such as "return and shipping discounts" based on order screenshots to individual order users, there is a phenomenon of users using edited or AI-generated fake order screenshots to fraudulently obtain discounts. Existing order screenshot detection methods on logistics e-commerce platforms have the following problems: Single-image binary classification schemes based on CNN / ViT (Convolutional Neural Network / VisionTransformer) ignore core features such as text information and layout details in the screenshot, making it difficult to identify edited fake screenshots; schemes based on OCR (Optical Character Recognition) key field comparison and template consistency verification suffer from strong template dependency, making it difficult to identify AI-generated fake order screenshots; existing solutions struggle to balance accuracy and recognition efficiency, resulting in high data annotation costs and weak generalization ability. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus, training method for a detection model, and processing method for logistics requests for detecting order screenshots, which can reduce detection costs and data / template dependencies, and improve detection efficiency and accuracy.

[0004] To achieve the above objectives, according to one aspect of the present invention, a method for detecting order screenshots is provided, comprising: Identify the text content and location information within the order screenshot; Based on the order screenshot, text content, and location information, determine the comprehensive judgment result of the order screenshot; Based on the location information, determine the layout of the order screenshot; The authenticity of the order screenshot is determined based on the comprehensive judgment results and the layout judgment results.

[0005] Optionally, a comprehensive judgment result for the order screenshot is determined based on the order screenshot, text content, and location information, including: Extract image features from the order screenshot to obtain image feature encoding; The text content is segmented into multiple tokens, and the tokens are mapped to text token codes; Map the location information of the text content to spatial location codes; By fusing image feature encoding, text lexical encoding, and spatial location encoding, a fused feature representation is obtained. Based on the fusion feature representation, the comprehensive judgment result of the order screenshot is determined.

[0006] Optionally, based on the fusion feature representation, a comprehensive judgment result for the order screenshot is determined, including: Extract global and local feature vectors from the fused feature representation; Based on the global feature vector, the global probability that the order screenshot has been tampered with is obtained; Based on the local feature vectors, the independent probability of each word in the text content of the order screenshot being tampered with is obtained; The overall judgment result of the order screenshot is determined based on the global probability and the independent probability.

[0007] Optionally, the layout determination result of the order screenshot is determined based on the location information, including: Based on location information, calculate the geometric relationship parameters between the positions of multiple text contents; The geometric relationship parameters are encoded to obtain the layout feature vector; The layout judgment result of the order screenshot is determined based on the layout feature vector.

[0008] Secondly, embodiments of the present invention provide a training method for an order screenshot detection model, the order screenshot detection model including a first branch neural network and a second branch neural network, and the training method including: Identify the text content and its location information in each order screenshot in the labeled dataset; Input the order screenshot, text content, and location information into the first branch neural network to obtain a comprehensive judgment vector; The location information is input into the second branch neural network to obtain the layout determination vector; Based on the comprehensive judgment vector, the layout judgment vector, and the labeled dataset, the parameters of the first branch neural network and the second branch neural network are determined through backpropagation, and the first branch neural network and the second branch neural network with determined parameters are used as the teacher model. The teacher model is distilled to obtain the detection model.

[0009] Optionally, the teacher model is distilled to obtain the detection model, including: The labeled dataset is input into the teacher model to obtain the intermediate features of the labeled dataset, which are used as soft labels; The soft labels are merged with the labels in the labeled dataset to obtain a hybrid labeled dataset; The mixed labeled dataset is input into the student model for training to obtain the detection model.

[0010] Thirdly, embodiments of the present invention provide a method for processing logistics requests, including: Extract the corresponding order screenshot from the received logistics request; The detection result of the order screenshot can be determined by using any of the methods in the first aspect; or the detection model obtained by calling any of the training methods in the second aspect can be used to determine the detection result of the order screenshot. In response to the test results indicating that the order screenshot is genuine, the system provides the user with the logistics service corresponding to the logistics request; In response to the detection results indicating that the order screenshot is fake, the logistics request is blocked.

[0011] Fourthly, embodiments of the present invention provide an order screenshot detection device, comprising: The visual recognition module is used to identify the text content and location information of the text content in the order screenshot; The comprehensive judgment module is used to determine the comprehensive judgment result of the order screenshot based on the order screenshot, text content, and location information; The layout determination module is used to determine the layout determination result of the order screenshot based on the location information; The fusion calibration module is used to determine the authenticity of order screenshots based on the comprehensive judgment result and the layout judgment result.

[0012] Fifthly, embodiments of the present invention provide an electronic device, comprising: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the methods of any of the above embodiments.

[0013] In a sixth aspect, embodiments of the present invention provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method of any of the above embodiments.

[0014] In a seventh aspect, embodiments of the present invention provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method of any of the above embodiments.

[0015] One embodiment of the above invention has the following advantages or beneficial effects: by identifying the text content and its location information in the order screenshot, key text detail features in the screenshot are obtained; based on the overall image features of the order screenshot and the detail features of the text content and location information, a comprehensive judgment result for the order screenshot is determined; for geometric layouts, the layout judgment result determined by the location information avoids dependence on interface templates from different e-commerce platforms. Combining the above comprehensive judgment result and layout judgment result, the authenticity judgment result of the order screenshot is obtained, thus balancing accuracy and detection efficiency. Therefore, through the order screenshot detection method, device, detection model training method, and logistics request processing method of the above embodiments, detection costs and data / template dependence can be reduced, while detection efficiency and accuracy can be improved.

[0016] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0017] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the main flow of an order screenshot detection method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for detecting order screenshots according to another embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the comprehensive determination of order screenshots provided in one embodiment of the present invention; Figure 4 This is a geometric determination diagram of an order screenshot provided in one embodiment of the present invention; Figure 5 This is a flowchart illustrating the training method of an order screenshot detection model provided in one embodiment of the present invention; Figure 6 This is an engineering implementation diagram of an order screenshot detection method provided in one embodiment of the present invention; Figure 7 This is a flowchart illustrating a logistics request processing method according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the main modules of an order screenshot detection device provided in one embodiment of the present invention; Figure 9 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0018] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security. The collection, use, storage, sharing, and transfer of user personal information all comply with relevant laws and regulations, and require notification to users and obtaining their consent or authorization. Where applicable, user personal information has undergone de-identification and / or anonymization and / or encryption technical processing.

[0020] For example, after collecting personal information from order screenshots, we will use technical means to de-identify the data.

[0021] Figure 1 This is an embodiment of the order screenshot detection method provided by the present invention, such as... Figure 1 As shown, it includes the following steps: Step S101: Identify the text content and location information of the text content in the order screenshot.

[0022] The order screenshot in this embodiment of the invention can be a static image of a graphical interactive interface with a specific geometric layout, containing necessary transaction information, such as a purchase order, batch order, or pre-purchase order from a (logistics) e-commerce platform. Provided that the detection accuracy is met, it can also be a high-resolution photograph of the order screenshot, or an order screenshot containing hidden (e.g., mosaic masking) non-essential transaction information.

[0023] The text content of the order screenshot, i.e., the necessary transaction information, may include: order number, traded items, traded quantity, traded amount, sender information / address, recipient information / address, etc. Information regarding the text location may include: the vertices (absolute coordinates), area, and perimeter of the smallest bounding rectangle or outline of the text content; the relative coordinates between the vertices of the smallest bounding rectangle or outline of the text content and specific markers such as screenshot edges and vertices.

[0024] The text content and location information in order screenshots can be recognized using technologies such as OCR, CNN, and YOLO (You-Only-Look-Once object detection algorithm). Preferably, OCR technology can be used to improve the real-time speed of recognition and save on data training and computing power costs.

[0025] Step S102: Determine the comprehensive judgment result of the order screenshot based on the order screenshot, text content, and location information.

[0026] In some embodiments, determining the comprehensive judgment result of the order screenshot based on the order screenshot, text content, and location information includes: extracting image features from the order screenshot to obtain image feature encoding; segmenting the text content into multiple words and mapping the words to text word encoding; mapping the location information of the text content to spatial location encoding; fusing the image feature encoding, text word encoding, and spatial location encoding to obtain a fused feature representation; and determining the comprehensive judgment result of the order screenshot based on the fused feature representation.

[0027] Before extracting image features from order screenshots, the order screenshots can be preprocessed, such as by normalization / standardization, channel adjustment, filtering, geometric adjustment, and color adjustment.

[0028] Extracting image features from order screenshots to obtain image feature encoding can include: extracting features from the pixel information of the input order screenshot using a convolutional neural network (CNN)-based encoder (such as ResNet, VGG, MobileNet) or an attention-based visual encoder (ViT), resulting in a matrix encoding after channel transformation and cross-level feature concatenation, which serves as the image feature encoding. Preferably, the order screenshot can first be processed by a CNN network to extract a compact feature map, and then the feature map can be input into a ViT network to be segmented into a patch sequence, thereby obtaining an image feature encoding containing local detailed features.

[0029] The process of segmenting text content into multiple tokens and mapping these tokens to text token codes can include: segmenting text content into multiple tokens (smallest semantic units) using a token segmentation algorithm (segmenter); and mapping the resulting tokens to text token codes using preset encoding rules (such as context-based positional encoding, one-hot encoding, etc.).

[0030] Mapping the location information of text content to spatial location encoding can include: mapping the coordinates (absolute coordinates / relative coordinates) of the smallest bounding rectangle of the text content to spatial location encoding through preset encoding rules (such as context-based location encoding, one-hot encoding, etc.).

[0031] By fusing image feature encoding, text lexical encoding, and spatial location encoding, a fused feature representation can be obtained. This can include mapping image feature encoding, text lexical encoding, and spatial location encoding to the same vector space and concatenating them to obtain the fused feature representation. The fused feature representation, which combines image and text modalities, can characterize deep multimodal logic, such as "what state the visual texture (background color, texture) should present when a specific word appears in a specific position."

[0032] Based on the fused feature representation, the comprehensive judgment result of the order screenshot is determined, including: extracting global feature vectors and local feature vectors from the fused feature representation; obtaining the global probability that the order screenshot has been tampered with based on the global feature vectors; obtaining the independent probability that each word in the text content of the order screenshot has been tampered with based on the local feature vectors; and determining the comprehensive judgment result of the order screenshot based on the global probability and the independent probability.

[0033] The global feature vector represents the authenticity information at the overall screenshot level, while the local feature vector represents the authenticity information at the text level. Based on the global feature vector, the global probability of the order screenshot being tampered with can be obtained by inputting the global feature vector into a Multi-Layer Perception (MLP). Based on the local feature vector, the independent probability of each word in the text content of the order screenshot being tampered with can be obtained by inputting the local feature vector into a fully connected layer (Linear Layer).

[0034] The comprehensive judgment result of the order screenshot is determined based on the global probability and the independent probability. This can include: concatenating the global probability and the independent probability to obtain the comprehensive judgment result of the order screenshot; or fusing the global probability and the independent probability through a fully connected layer or a multilayer perceptron to obtain the comprehensive judgment result. Preferably, the former method is used to retain global and local multimodal judgment information.

[0035] Step S103: Determine the layout of the order screenshot based on the location information.

[0036] In some embodiments, determining the layout judgment result of the order screenshot based on location information includes: calculating geometric relationship parameters between the positions of multiple text contents based on location information; encoding the geometric relationship parameters to obtain a layout feature vector; and determining the layout judgment result of the order screenshot based on the layout feature vector.

[0037] Calculating the geometric relationship parameters between the positions of multiple text contents based on location information refers to using a pre-set geometric algorithm to calculate these parameters. For example, it involves calculating purely geometric scalars such as the horizontal deviation, vertical spacing, height ratio, and intersection over union (IoU) between the minimum bounding rectangles of adjacent text contents.

[0038] Encoding the geometric relationship parameters to obtain a layout feature vector can include: inputting the calculated geometric relationship parameters into a lightweight graph neural network (GNN) or a simplified self-attention network (with one layer) to encode them into a fixed-length layout feature vector. Optionally, text content or the category of text content (such as order number) can be used as supplementary information to be encoded together with the geometric relationship parameters to obtain the layout feature vector.

[0039] Based on the layout feature vector, the layout judgment result of the order screenshot can be determined, which may include: mapping the layout feature vector to a specific vector space through a fully connected layer / multilayer perceptron to obtain the layout judgment result (vector).

[0040] Step S104: Determine the authenticity of the order screenshot based on the comprehensive judgment result and the layout judgment result.

[0041] In some embodiments, the authenticity of an order screenshot is determined based on the comprehensive judgment result and the layout judgment result. The comprehensive judgment result and the layout judgment result can be input into a fusion calibration network (such as a fully connected layer, a multilayer perceptron, etc.) to output the authenticity probability of the order screenshot. In response to the authenticity probability meeting a preset threshold, the authenticity of the order screenshot is determined.

[0042] In some embodiments, the authenticity of an order screenshot is determined based on the comprehensive judgment result and the layout judgment result. This determination can be made according to preset rules based on the global probability included in the comprehensive judgment result, the local probability of each text word, and the layout judgment result. For example, a veto rule might be used, where any judgment result of "false" indicates the order screenshot is considered "false"; or a complementary rule might be used, where at least two judgment results of "false" indicate the order screenshot is considered "false". These preset rules can also be dynamically selected based on real-time conditions during deployment (such as data throughput).

[0043] Figure 2 This is another embodiment of the order screenshot detection method provided by the present invention, such as... Figure 2 As shown, it includes the following steps: Step S201: Input the received order screenshot into the pre-data extraction module to identify the text content and the location information of the text content in the order screenshot.

[0044] This invention implements an order screenshot detection method by calling an order screenshot detection model. The detection model adopts a unified architecture based on the Layout Language Model. Before calling the core detection model, a pre-data extraction module first receives the original return screenshot uploaded by the user and calls the OCR engine to extract the full text content in the screenshot and the vertex coordinates of its corresponding minimum bounding rectangles. The extracted full text content can be segmented into multiple tokens using a token segmentation algorithm; text tokens can also refer to each text box detected by the OCR, and these tokens are further segmented into sub-tokens using the token segmentation algorithm.

[0045] Step S202: Determine the comprehensive judgment result of the order screenshot based on the order screenshot, text content, and location information.

[0046] In this embodiment of the invention, steps S202-S204 are completed by calling the order screenshot detection model. Specifically, the order screenshot detection model includes a multimodal joint discrimination branch network (first branch neural network), a layout geometric consistency branch network (second branch neural network), and a fusion calibration network.

[0047] By inputting multimodal information such as order screenshots, text content, and location information into a multimodal joint discriminant branch network, and deeply fusing image pixels, text semantics, and spatial coordinates, it can not only output the authenticity of the entire screenshot, but also pinpoint the specific words that have been tampered with (such as modifying prices or recipient information) in a fine-grained manner, thereby solving the problem of difficulty in identifying edit-type forged order screenshots.

[0048] like Figure 3 As shown, the multimodal joint discriminant branch network includes a visual encoder for extracting image features, a text spatial encoder (Text & LayoutEncoder) for extracting text content and location information features, a multimodal fusion layer, and a decoder / task head.

[0049] The visual encoder uses CNN or ViT to capture image feature encoding of order screenshots. The text spatial encoder receives the text content or text words extracted in step S201, along with the vertex coordinates of their minimum bounding rectangles. The text content or text words are converted into text word encoding; the vertex coordinates are converted into spatial positional embeddings, which precisely contain the absolute physical x-axis and y-axis coordinates of the top-left and bottom-right corners of each minimum bounding rectangle.

[0050] The multimodal fusion layer employs a Transformer Encoder structure relying on a self-attention mechanism. It maps and concatenates the extracted image feature encoding, text word encoding, and spatial location encoding features into the same vector space for deep interaction. This layer enables the model to learn deep logic, such as "when a specific word appears in a specific position, what state should its underlying visual texture (background color, texture) present?" The fused feature representation obtained after the multimodal fusion layer is a set of contextualized vectors, for example: H = [h_CLS, h_1, h_2,…, h_N]; Where h_CLS is the global feature vector at the whole screenshot level, used to determine the authenticity of the entire order screenshot; h_i is the local feature vector of the i-th text word, containing the text semantics, spatial location, visual background, and relationship with other words of the word, used to determine whether the word has been tampered with; N represents the number of text words, which changes dynamically with the number of text words detected in step S201. For example, if the text words of the order screenshot are ["refund amount", "¥199.00", "recipient", "Zhang San"], then N=4.

[0051] In some embodiments, ViT is used to capture local patch-level image feature encoding of order screenshots, such as compression artifacts, edge abrupt changes, or uneven noise distribution caused by image editing. The patches obtained by ViT are input together with image feature encoding, text lexical encoding, and spatial location encoding features into a multimodal fusion layer. The resulting fused feature representation also includes the tampering probability feature representation of each patch.

[0052] The following example illustrates a method for obtaining fused feature representations: For a screenshot of an order, the batch size of data processed each time is set to 2, and a maximum of 80 text terms and 196 ViT patches are processed (for example, for a 224x224 pixel image, each patch is 16x16 pixels in size). The size of the (Transformer) hidden layer is 768 neurons.

[0053] Take screenshots of two batches of orders of size [3, 224, 224] pixels, 80 text words, and the vertex coordinates [x1, y1, x2, y2] of the minimum bounding rectangle of each text word.

[0054] The visual encoder outputs image feature codes (visual_emb), and the text spatial encoder outputs text word codes (word_emb) and spatial location codes (layout_emb). These codes are concatenated with a vector parameter (CLS) of size [2, 1, 768] to form a vector sequence X. This process can be represented by the following formula: X=concat([CLS, visual_emb, word_emb+layout_emb], dim=1); The size of the vector sequence X obtained after splicing is [2, 277, 768].

[0055] After inputting the vector sequence X into the multimodal fusion layer, the fused feature representation H can be obtained. Among them, the sequence at position 0 of the first dimension of H (H[:, 0, :]) is the global feature vector; the sequence at positions 1-197 of the first dimension of H (H[:, 1:197, :]) is the local feature vector of each patch; and the sequence at positions 197-277 of the first dimension of H (H[:, 197:277, :]) is the local feature vector of each text word.

[0056] The decoder outputs the global probability and the total probability of an order screenshot being tampered with, respectively, based on the fused feature representation through two task heads. The global classification head, employing a multilayer perceptron, receives the global feature vector and outputs the global probability that the entire image has been tampered with; the fine-grained localization head, using a fully connected layer, receives the local feature vectors of text words and outputs the independent probability of whether each specific token in the sequence has been tampered with, achieving "spot the difference" anomaly localization.

[0057] The global probability can be defined as an output of 0 for a genuine screenshot and an output of 1 for a fake screenshot that has been tampered with. The independent probability is a sequence of length N (corresponding to the number of text words in the screenshot), for example, an output of [0, 1, 0, 0] indicates that the second word word has been tampered with. The global probability and the independent probability are concatenated to obtain the comprehensive judgment result of the order screenshot.

[0058] Step S203: Determine the layout of the order screenshot based on the location information.

[0059] This invention overcomes the hard-coded dependence of existing technologies on platform UI templates and defends against advanced AI-generated fake images (generative imitation) by discarding the geometric consistency branch network of text content and image pixels and focusing on extracting features of order layout and relative geometric rules (such as uniform spacing between multiple address lines).

[0060] like Figure 4 As shown, the layout geometry consistency branch network includes an encoder and a decoder.

[0061] The encoder is designed with two parts: feature engineering for geometric relation parameters and a graph neural network (or self-attention network) for obtaining layout feature vectors. Feature engineering for geometric relation parameters is used to compute the relative relation feature set of the minimum bounding rectangle of text content or text words. For example, it explicitly computes purely geometric scalars such as horizontal deviation, vertical spacing, height ratio, and overlap (IoU) between adjacent boxes.

[0062] The calculated geometric relationship parameters are input into an encoder, such as a lightweight graph neural network (GNN) or a single-layer simplified self-attention network, and encoded into a fixed-length layout feature vector to extract geometric relationship features.

[0063] The layout feature vector is mapped to a specific vector space through a fully connected layer to obtain the layout judgment result (vector). The layout judgment result (vector) is not represented by absolute 0 and 1, but by a triple form. That is, it is represented by the vector distance between the constructed set of coordinates of the smallest bounding rectangle of real, normally and reasonably formatted text content / text words as the baseline data (Anchor) and the normal layout (Positive) / abnormal layout (Negative). Here, normal layout refers to the set of coordinates of the smallest bounding rectangle of real text content / text words on the same platform, or even with slight differences in UI version, but the layout logic still conforms to normal physical laws; abnormal layout refers to the set of coordinates of the smallest bounding rectangle of text content / text words that is deliberately shifted vertically by several pixels (simulating misalignment in manual image editing), or the coordinate space is stretched or shortened (simulating the collapse of the text skeleton when AI generates fake images).

[0064] Step S204: Determine the authenticity of the order screenshot based on the comprehensive judgment result and the layout judgment result.

[0065] The comprehensive judgment result and the layout judgment result are input into the fusion calibration network (such as a fully connected layer, multilayer perceptron, etc.), and the output is the authenticity probability of the order screenshot. In response to the authenticity probability meeting the preset threshold, the authenticity of the order screenshot is determined.

[0066] Figure 5This invention provides a training method for an order screenshot detection model, which includes a first branch neural network and a second branch neural network, as shown in one embodiment of the invention. Figure 5 As shown, the training method includes the following steps: Step S501: Identify the text content and location information of the text content in each order screenshot in the labeled dataset.

[0067] The pre-trained order screenshot detection models used in the above embodiments are often high-precision, high-parameter models, requiring significant computing resources. In practical engineering implementation, alternative methods can be used... Figure 6 The method shown accelerates knowledge distillation, quantification, and inference to meet industrial-grade standards for high concurrency and low latency while reducing computational overhead for the model.

[0068] Specifically, a large number of order screenshots are first collected (which can be from real order screenshots or order screenshots automatically synthesized by the platform's code program), and then manually labeled to obtain a labeled dataset. For example, it can include: whole image labels, labeled with 1 (true) or 0 (false); tampering labels for each text word, labeled with 1 (true) or 0 (false); positive / negative format labels, etc.

[0069] The labeled dataset can identify the text content and location information of the text content in each order screenshot using a method similar to step S101.

[0070] Step S502: Input the order screenshot, text content and location information into the first branch neural network to obtain the comprehensive judgment vector.

[0071] In some embodiments, the first branch neural network and the second branch neural network can be trained separately using a labeled dataset or by end-to-end training.

[0072] For example, the first branch neural network employs a multi-task learning mechanism, receiving order screenshots, text content, and location information, and outputting a comprehensive decision vector (including global probability and independent probability). The first branch network uses a binary cross-entropy (BCE Loss) loss function. The formula is as follows: ; in, These are the full image label and the predicted global probability, respectively. These are the tampering label and the predicted independent probability of the i-th text word, respectively; These are hyperparameters, calculated based on the statistical results of the labeled dataset.

[0073] Step S503: Input the position information into the second branch neural network to obtain the layout determination vector.

[0074] In some embodiments, the second branch neural network employs a triplet data structure and uses a triplet margin loss function. Training is performed to measure the distance between the Anchor and the Positive in the vector space. At the same time, increase the distance between the Anchor and the Negative. The formula is as follows: ; in, The base layout was obtained by (randomly) selecting from the Positive tags; These are standard layout tags; This is an abnormal layout tag; The safety margin limits are set to ensure that features of different categories have sufficient boundaries in the metric space.

[0075] Step S504: Based on the comprehensive decision vector, the layout decision vector, and the labeled dataset, determine the parameters of the first branch neural network and the second branch neural network through backpropagation, and use the first branch neural network and the second branch neural network with determined parameters as the teacher model.

[0076] Step S505: Distill the teacher model to obtain the detection model.

[0077] In some embodiments, distilling the teacher model to obtain a detection model includes: inputting a labeled dataset into the teacher model to obtain intermediate features of the labeled dataset as soft labels; fusing the soft labels with the labels in the labeled dataset to obtain a hybrid labeled dataset; and inputting the hybrid labeled dataset into the student model for training to obtain a detection model.

[0078] In step S501, the labels can be used as hard labels to represent absolute objective facts. By inputting the labeled dataset into the teacher model, intermediate features of the labeled dataset (such as the output probability distribution Logits or intermediate layer feature maps) are obtained as soft labels to obtain information on inter-class differences and image discrepancies (i.e., "how much this image resembles a fake image"), specifically presented as continuous floating-point numbers. By combining a hybrid labeled dataset of hard and soft labels to guide the student model in knowledge distillation training, the student model can be forced to fit the teacher's probability distribution, which can significantly improve the generalization robustness of the lightweight model and effectively prevent overfitting of small models.

[0079] Loss function of knowledge distillation By hard label ( Cross-entropy and soft tags The Kullback-Leibler divergence (KL) is used to ensure that the smaller model can maintain accuracy on deterministic samples while inheriting the generalization ability of the larger model on fuzzy boundary samples. The formula is shown below: ; in, For the predicted values ​​of the student model, These are hyperparameters, either preset or calculated.

[0080] The trained student model is quantized and compressed into low-bit int8 using Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) techniques. Structured pruning / trimming techniques are then used to further reduce computational overhead and slim the model down. The final detection model will be deployed to a high-concurrency production environment to achieve millisecond-level response times in real-time review processes (such as...). Figure 6 (As shown).

[0081] Figure 7 This is a method for processing logistics requests provided in one embodiment of the present invention, such as... Figure 7 As shown, it includes the following steps: Step S701: Parse the corresponding order screenshot from the received logistics request.

[0082] In some embodiments, a logistics request may be a shipment request, a compensation request, or a request to modify logistics information (such as modifying the recipient or delivery address); the order screenshot corresponding to the logistics request may be a screenshot of a platform order such as a purchase order or a mail order (bulk order or pre-order).

[0083] Step S702: Determine the detection result of the order screenshot using any of the methods in the first aspect; or call the detection model obtained by any of the training methods in the second aspect to determine the detection result of the order screenshot.

[0084] Step S703: In response to the detection result indicating that the order screenshot is true, provide the user with the logistics service corresponding to the logistics request.

[0085] In some embodiments, logistics services may include providing users with benefits such as discounts, coupons, points, and interest-free installments; or providing supply services such as priority pickup, expedited transit, dedicated air cargo space reservation, address change interception, and insurance claims; or providing after-sales services such as fast refunds and trade-ins.

[0086] In some embodiments, the text content identified in step S702 can be directly entered into the internal system as a record of logistics service fulfillment.

[0087] In some embodiments, the credit rating of a user's account may be improved for users whose order screenshots are genuine, based on the test results.

[0088] Step S704: In response to the detection result indicating that the order screenshot is fake, intercept the logistics request.

[0089] In some embodiments, tiered blocking can be implemented by combining prior information such as user level, risk weight, and user profile, or further secondary verification (such as manual review) can be requested. For users whose verification results still show fake order screenshots, the risk level of the user account can be increased, and this can be used as the basis for blocking in the next detection.

[0090] Figure 8 This is a schematic diagram of the main modules of an order screenshot detection device provided in one embodiment of the present invention, as shown below. Figure 8 As shown, the order screenshot detection device 800 includes: The visual recognition module 801 is used to recognize the text content and the location information of the text content in the order screenshot; The comprehensive judgment module 802 is used to determine the comprehensive judgment result of the order screenshot based on the order screenshot, text content and location information; The layout determination module 803 is used to determine the layout determination result of the order screenshot based on the location information. The fusion calibration module 804 is used to determine the authenticity of order screenshots based on the comprehensive judgment result and the format judgment result.

[0091] Figure 9 An exemplary system architecture 900 is shown, in which the order screenshot detection method, apparatus, detection model training method, or logistics request processing method of embodiments of the present invention can be applied.

[0092] like Figure 9 As shown, system architecture 900 may include terminal devices 901, 902, and 903, network 904, and server 905. Network 904 is used as a medium to provide a communication link between terminal devices 901, 902, and 903 and server 905. Network 904 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0093] Users can use terminal devices 901, 902, and 903 to interact with server 905 via network 904 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 901, 902, and 903, such as shopping applications, web browser applications, logistics applications, instant messaging tools, email clients, social media platform software, etc.

[0094] Terminal devices 901, 902, and 903 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0095] Server 905 can be a server providing various services, such as a backend management server supporting shopping websites browsed by users using terminal devices 901, 902, and 903. The backend management server can analyze and process received data such as order screenshots and feed back the processing results (e.g., the authenticity of the order screenshots) to the terminal devices.

[0096] It should be noted that the order screenshot detection method, the detection model training method, or the logistics request processing method provided in the embodiments of the present invention are generally executed by the server 905. Correspondingly, the order screenshot detection device is generally set in the server 905.

[0097] It should be understood that Figure 9 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0098] The following is for reference. Figure 10 It shows a schematic diagram of the structure of a computer system 1000 suitable for implementing a terminal device of the present invention. Figure 10 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0099] like Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage section 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the system 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0100] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0101] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined above in the system of this invention.

[0102] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a visual recognition module, a comprehensive judgment module, a layout judgment module, and a fusion calibration module. The names of these modules do not necessarily limit the module itself; for example, the visual recognition module may also be described as "a module for recognizing text content and the location information of text content in an order screenshot."

[0105] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to perform the detection of the order screenshot in the first aspect, including: Identify the text content and location information within the order screenshot; Based on the order screenshot, text content, and location information, determine the comprehensive judgment result of the order screenshot; Based on the location information, determine the layout of the order screenshot; The authenticity of the order screenshot is determined based on the comprehensive judgment results and the layout judgment results.

[0106] Alternatively, when one or more of the above procedures are executed by the device, the training method for the order screenshot detection model of the second aspect, including: Identify the text content and its location information in each order screenshot in the labeled dataset; Input the order screenshot, text content, and location information into the first branch neural network to obtain a comprehensive judgment vector; The location information is input into the second branch neural network to obtain the layout determination vector; The detection results of the order screenshot are obtained based on the comprehensive judgment vector and the layout judgment vector. Based on the detection results of the order screenshot and the labeled dataset, the parameters of the first branch neural network and the second branch neural network are determined through backpropagation, and the first branch neural network and the second branch neural network with determined parameters are used as the teacher model; The teacher model is distilled to obtain the detection model.

[0107] Alternatively, when one or more of the aforementioned procedures are executed by the device, causing the device to process a logistics request from a third party, the method includes: Extract the corresponding order screenshot from the received logistics request; The detection result of the order screenshot can be determined by using any of the methods in the first aspect; or the detection model obtained by calling any of the training methods in the second aspect can be used to determine the detection result of the order screenshot. In response to the test results indicating that the order screenshot is genuine, the system provides the user with the logistics service corresponding to the logistics request; In response to the detection results indicating that the order screenshot is fake, the logistics request is blocked.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting order screenshots, characterized in that, include: Identify the text content in the order screenshot and the location information of the text content; Based on the order screenshot, text content, and location information, a comprehensive judgment result for the order screenshot is determined; Based on the location information, determine the layout of the order screenshot; Based on the comprehensive judgment result and the layout judgment result, the authenticity of the order screenshot is determined.

2. The method according to claim 1, characterized in that, Based on the order screenshot, text content, and location information, a comprehensive judgment result for the order screenshot is determined, including: Extract the image features from the order screenshot to obtain the image feature encoding; The text content is segmented into multiple word units, and the word units are mapped to text word unit codes; Map the location information of the text content to a spatial location code; By fusing the image feature encoding, text lexical encoding, and spatial location encoding, a fused feature representation is obtained; Based on the fusion feature representation, a comprehensive judgment result for the order screenshot is determined.

3. The method according to claim 2, characterized in that, Based on the fusion feature representation, the comprehensive judgment result of the order screenshot is determined, including: From the fused feature representation, extract the global feature vector and the local feature vector; Based on the global feature vector, the global probability that the order screenshot has been tampered with is obtained; Based on the local feature vector, the independent probability of each word in the text content of the order screenshot being tampered with is obtained; Based on the global probability and the independent probability, a comprehensive judgment result for the order screenshot is determined.

4. The method according to claim 1, characterized in that, Based on the location information, the layout determination result of the order screenshot is made, including: Based on the location information, calculate the geometric relationship parameters between the positions of multiple text contents; The geometric relationship parameters are encoded to obtain the layout feature vector; The layout determination result of the order screenshot is determined based on the layout feature vector.

5. A training method for an order screenshot detection model, characterized in that, The order screenshot detection model includes a first-branch neural network and a second-branch neural network, and the training method includes: Identify the text content and location information of each order screenshot in the labeled dataset; The order screenshot, text content, and location information are input into the first branch neural network to obtain a comprehensive judgment vector; The location information is input into the second branch neural network to obtain the layout determination vector; Based on the comprehensive decision vector, the layout decision vector, and the labeled dataset, the parameters of the first branch neural network and the second branch neural network are determined through backpropagation, and the first branch neural network and the second branch neural network with determined parameters are used as the teacher model. The teacher model is distilled to obtain the detection model.

6. The method according to claim 5, characterized in that, Distillation is performed on the teacher model to obtain the detection model, including: The labeled dataset is input into the teacher model to obtain the intermediate features of the labeled dataset, which are used as soft labels; The soft labels are then merged with the labels in the labeled dataset to obtain a hybrid labeled dataset; The hybrid labeled dataset is input into the student model for training to obtain the detection model.

7. A method for processing logistics requests, characterized in that, include: Extract the corresponding order screenshot from the received logistics request; The detection result of the order screenshot is determined by any one of the methods described in claims 1-4; Alternatively, the detection model obtained by the training method described in any of claims 5-6 can be used to determine the detection result of the order screenshot; In response to the detection result indicating that the order screenshot is genuine, the user is provided with the logistics service corresponding to the logistics request; In response to the detection result indicating that the order screenshot is fake, the logistics request is intercepted.

8. A device for detecting order screenshots, characterized in that, include: The visual recognition module is used to identify the text content in the order screenshot and the location information of the text content; The comprehensive judgment module is used to determine the comprehensive judgment result of the order screenshot based on the order screenshot, text content, and location information; The layout determination module is used to determine the layout determination result of the order screenshot based on the location information; The fusion calibration module is used to determine the authenticity of the order screenshot based on the comprehensive judgment result and the layout judgment result.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images