Express anti-theft method and device, electronic equipment and storage medium

Abstract

The invention provides an express anti-theft method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. According to the specific implementation scheme, the method comprises the steps of obtaining multi-modal data when a user picks up an express, wherein the multi-modal data comprises at least one type of biological characteristic data, express pickup behavior data and express pickup environment data of the user; on the basis of a pre-trained large model, deciding whether pickup of the multi-modal data is abnormal or not, and obtaining a pickup verification result corresponding to the user; in response to the fact that the pickup verification result is normal, executing a pickup process; and in response to the abnormity of the pickup verification result, sending pickup abnormity alarm information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for preventing theft of express delivery. Background Art

[0002] With the rapid development of express delivery business, the problems of lost and mis-collected express delivery are becoming more and more common in express delivery stations and collection points, which brings inconvenience to users' lives and increases the difficulty of couriers' work. At present, most express delivery stations use mobile phone numbers and verification codes as the main means of identity verification, which allows users to pick up other people's express delivery at will, whether intentionally or unintentionally, and further leads to the increasing number of disputes caused by the lack of effective prevention and control measures. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and storage medium for express delivery theft prevention.

[0004] According to one aspect of the present disclosure, a method for preventing express delivery from being stolen is provided, comprising: obtaining multimodal data when a user picks up a package, wherein the multimodal data comprises at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data; making a decision on whether the package pickup is abnormal based on a pre-trained large model on the multimodal data, and obtaining a package pickup verification result corresponding to the user; in response to the package pickup verification result being normal, executing a package pickup process; in response to the package pickup verification result being abnormal, issuing a package pickup abnormality alarm message.

[0005] According to another aspect of the present disclosure, there is provided an express delivery anti-theft device, comprising: an acquisition module, for acquiring multimodal data when a user picks up a package, wherein the multimodal data comprises at least one type of biometric data of the user, package pickup behavior data and package pickup environment data; a verification module, for deciding whether the package pickup is abnormal based on a pre-trained large model on the multimodal data, and acquiring a package pickup verification result corresponding to the user; a package pickup module, for executing a package pickup process in response to a normal package pickup verification result; and an alarm module, for issuing an abnormal package pickup alarm message in response to an abnormal package pickup verification result.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the express delivery anti-theft method described in the above-mentioned one aspect embodiment.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program / instructions are stored, and the computer instructions are used to enable the computer to execute the express delivery anti-theft method described in the above-mentioned embodiment.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the express delivery anti-theft method described in the above-mentioned first embodiment is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 A schematic diagram of a flow chart of a method for preventing theft of express delivery provided by an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of a process for collecting multimodal data provided by an embodiment of the present disclosure;

[0013] Figure 3 A flowchart of another express delivery anti-theft method provided by an embodiment of the present disclosure;

[0014] Figure 4 A flowchart of another express delivery anti-theft method provided by an embodiment of the present disclosure;

[0015] Figure 5 A schematic diagram of a process for verifying a user's identity and behavior provided in an embodiment of the present disclosure;

[0016] Figure 6 A flowchart of another express delivery anti-theft method provided by an embodiment of the present disclosure;

[0017] Figure 7 A schematic diagram of a process for storing and tracing data provided in an embodiment of the present disclosure;

[0018] Figure 8 A schematic diagram of a process for abnormal alarming provided in an embodiment of the present disclosure;

[0019] Fig. 9 A timing diagram of express delivery theft prevention provided by an embodiment of the present disclosure;

[0020] Fig.10 A schematic diagram of the structure of a courier anti-theft device provided in an embodiment of the present disclosure;

[0021] Fig.11 The present invention is a block diagram of an electronic device used to implement the express delivery anti-theft method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Artificial Intelligence (AI) is a discipline that studies how computers can simulate certain thought processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technologies and software-level technologies. AI hardware technologies generally include computer vision technology, speech recognition technology, natural language processing technology, as well as learning / deep learning, big data processing technology, knowledge graph technology, and other aspects.

[0024] The following describes the express delivery theft prevention method, device, electronic device and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0025] Figure 1 A flowchart of a method for preventing theft of express delivery provided by an embodiment of the present disclosure is provided.

[0026] like Figure 1 As shown, the express delivery anti-theft method may include:

[0027] S101, obtaining multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data.

[0028] It should be noted that the execution subject of the express delivery anti-theft method in the embodiment of the present disclosure may be a hardware device with data processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a server, a user terminal and other intelligent devices. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, etc. Optionally, the server includes but is not limited to a network server, an application server, and may also be a server of a distributed system, or a server combined with a blockchain, etc. The embodiment of the present disclosure is not specifically limited.

[0029] In some implementations, multimodal data of a user picking up a package can be collected based on multiple sensors. Optionally, multimodal data of a user picking up a package can be collected based on one or more image sensors and sound sensors. For example, when a user picks up a package in a courier cabinet, the video, facial image, voice data, pickup behavior data, and pickup environment data of the user picking up the package can be collected as multimodal data based on the courier cabinet's own camera, microphone, and other devices. Among them, facial images and voice data can be used as biometric data.

[0030] S102, based on the pre-trained large model, a decision is made on whether the pickup is abnormal based on the multimodal data, and the pickup verification result corresponding to the user is obtained.

[0031] In some implementations, the user's identity can be verified based on biometric data in the multimodal data, and the user's behavior can be verified based on the pickup behavior data and pickup environment data in the multimodal data, and the pickup verification result corresponding to the user can be determined based on the verification result.

[0032] In some implementations, a pre-trained large model can extract a user's identity feature vector and behavior feature vector from multimodal data, and verify the user's identity and behavior based on the identity feature vector and the behavior feature vector to determine whether the user has any abnormalities.

[0033] Optionally, if the user fails the verification, it is determined that the pickup verification result corresponding to the user is abnormal; if the user passes the verification, it is determined that the pickup verification result corresponding to the user is normal.

[0034] Optionally, the user's identity and legitimacy can be verified based on the identity feature vector, for example, the user can be verified by face recognition, etc. Further, based on the behavior feature vector, it is verified whether the user's behavior matches the historical behavior. If they match, it is determined that the user has passed the verification, and the corresponding pickup verification result of the user is determined to be normal.

[0035] In some implementations, the pre-trained large model can be a neural network model. The large model can learn multimodal data and associate the relationship between each modality through a cross-modal attention mechanism to generate a deeper feature vector, that is, to obtain an identity feature vector and a behavior feature vector.

[0036] S103, in response to the pickup verification result being normal, executing the pickup process.

[0037] In some implementations, when the pickup verification result is normal and the user's identity and behavior are confirmed to have passed the verification, the user can execute the pickup process and take out the stored express. For example, if the express is stored in an express cabinet, the large model carried by the express cabinet itself will verify the user, and if the pickup verification result is normal, the cabinet door will be opened to facilitate the user to pick up the package.

[0038] S104: In response to an abnormal pickup verification result, an abnormal pickup alarm message is issued.

[0039] In some implementations, when the pickup verification result is abnormal, the alarm process is triggered immediately. Optionally, based on the degree of abnormality, an abnormal pickup alarm message can be sent to the user. If the abnormality degree is high risk, the abnormal pickup alarm message is sent to the user via SMS; if the abnormality degree is low risk, the abnormal pickup alarm message is sent to the user via a pop-up window.

[0040] Furthermore, after receiving the abnormal pickup alarm information, the user can confirm the abnormality. If the abnormality is confirmed, the user can be prompted to contact the courier to confirm whether the courier is delivered correctly; if there is no abnormality, it will be marked as a false alarm, and the big model will be updated based on the false alarm to reduce the occurrence of similar false alarms.

[0041] According to the express delivery anti-theft method provided by the embodiment of the present disclosure, the user's multimodal data is obtained, and the multimodal data is analyzed using a large model to verify the user's identity and behavior, and determine the user's corresponding pickup verification result. It can then be determined whether the pickup is abnormal based on the pickup verification result to execute the corresponding process. By verifying the user's pickup, it can effectively prevent the express from being picked up by mistake or stolen, to ensure that the express can be accurately delivered to the real recipient, and reduce delivery delays or failures caused by information errors, thereby improving user satisfaction. The large model can automatically perform face recognition and behavior analysis, reducing reliance on manual review and improving the operational efficiency of express delivery stations.

[0042] like Figure 2 The schematic diagram of the process of collecting multimodal data is shown. When the user picks up the package, the express delivery pick-up process is triggered and multimodal data collection is performed. Figure 2 In the process, multimodal data can be collected based on cameras, microphones, and sensors such as temperature sensors and optical sensors. By turning on the camera, the camera captures images and videos, and processes the images and videos to detect whether the face and the front of the user are captured, and then stores the collected images and videos. By starting the microphone, the user's voice data is recorded, and the voice is processed by noise filtering, voiceprint recognition, etc., and the voice data is stored. By starting the sensor, environmental data is collected, and temperature, light, location and other data in the environmental data are obtained, and the environmental data is stored.

[0043] Furthermore, multimodal data fusion is performed on the collected data to obtain multimodal data: biometric data, pickup behavior data and pickup environment data, and the multimodal data is analyzed using a large model to generate a pickup verification result, and the corresponding process is executed based on the pickup verification result.

[0044] Figure 3 A flowchart of a method for preventing theft of express delivery provided by an embodiment of the present disclosure is provided.

[0045] like Figure 3 As shown, the express delivery anti-theft method may include:

[0046] S301, obtaining multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data.

[0047] The relevant contents of step S301 can be found in the above embodiment and will not be described again here.

[0048] S302, extracting features from the multimodal data based on the large model, and performing identity verification and pickup behavior verification on the user based on the extracted feature information to obtain a pickup verification result.

[0049] In some implementations, the large model can perform feature extraction on biometric data in multimodal data to obtain an identity feature vector corresponding to the user, and perform feature extraction on pickup behavior data and pickup environment data in multimodal data to obtain a behavior feature vector corresponding to the user.

[0050] Optionally, the identity feature vector may include: facial features, voiceprint features, and the behavior feature vector may include: pickup time features, pickup frequency features, pickup location features, behavior action features, pickup action features, etc.

[0051] For example, the large model can extract features from multimodal data based on convolutional neural networks (CNN). For example, for an input face image, CNN can be used to extract a feature vector of 128 dimensions or higher to represent the face features. For example, CNN can be used to extract user action features from video data.

[0052] Furthermore, based on the identity feature vector and the behavior feature vector, the user's identity and pickup behavior are verified respectively. If the identity authentication and pickup behavior verification pass, the pickup verification result is determined to be normal; if the identity authentication and pickup behavior verification fail, the pickup verification result is determined to be abnormal.

[0053] S303: In response to the pickup verification result being normal, executing the pickup process.

[0054] S304: In response to an abnormal pickup verification result, an abnormal pickup warning message is issued.

[0055] The relevant contents of steps S303 - S304 can be found in the above embodiment and will not be described again here.

[0056] According to the express delivery anti-theft method provided by the embodiment of the present disclosure, the multimodal data of the user is obtained, and the feature extraction of the multimodal data is performed using a large model, and the identity and behavior of the user are verified by the extracted features to determine the pickup verification result corresponding to the user. It can then be determined whether the pickup is abnormal based on the pickup verification result to execute the corresponding process. By verifying the user's pickup, it is possible to effectively prevent the express delivery from being picked up by mistake or stolen, to ensure that the express delivery can be accurately delivered to the real recipient, and to reduce delivery delays or failures caused by incorrect information, thereby improving user satisfaction. The large model can automatically perform face recognition and behavior analysis, reducing reliance on manual review and improving the operational efficiency of express delivery stations.

[0057] Figure 4 A flowchart of a method for preventing theft of express delivery provided by an embodiment of the present disclosure is provided.

[0058] like Figure 4 As shown, the express delivery anti-theft method may include:

[0059] S401, obtaining multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data.

[0060] The relevant contents of step S401 can be found in the above embodiment and will not be described again here.

[0061] S402, extracting features from the multimodal data based on the large model to obtain identity feature vectors and behavior feature vectors.

[0062] In some implementations, the large model can extract identity feature vectors and behavior feature vectors from multimodal data. By extracting identity feature vectors from multimodal data, the comprehensiveness and accuracy of identity features can be improved, so that accurate verification can be performed based on the identity feature vectors.

[0063] Optionally, by performing feature extraction on modal data in the multimodal data, multi-dimensional modal features of the modal data are obtained, wherein the modal data may be any type of modal data in the multimodal data, that is, feature extraction may be performed on each type of modal data in the multimodal data to obtain multi-dimensional modal features of each type of modal data. For example, facial features of a user may be extracted from a face image as a type of modal feature, and voiceprint features may be extracted from voice data as a type of modal feature.

[0064] Furthermore, the multi-dimensional modal features of the modal data can be fused to obtain the fused modal features of the modal data, and the fused feature modalities of the multi-modal data can be cross-modally focused to obtain the identity feature vector. For example, facial features and voiceprint features can be fused and cross-modally focused to obtain the identity feature vector.

[0065] It is understandable that the behavioral feature vector mainly describes the user's behavioral habits and time series data, usually including the user's movement trajectory, time pattern, frequency and location, etc. The behavioral feature vector can further confirm the user and help improve the response capabilities to potential security risks such as theft.

[0066] Optionally, by extracting features from the multimodal data, the user's pickup posture, action features, and spatiotemporal information of the pickup are obtained, and a behavior feature vector is obtained based on the user's pickup posture, action features, and spatiotemporal information of the pickup. In other words, the user's pickup posture, action features, and spatiotemporal information of the pickup can be fused to obtain a behavior feature vector.

[0067] S403: Verify the identity of the user according to the identity feature vector to obtain a first verification result.

[0068] In some implementations, based on the identity feature vector, it is possible to verify whether the user's features match the stored reference features to improve the accuracy and reliability of identity authentication, effectively prevent theft, and enhance the security of the pickup process. Optionally, the similarity of the feature vectors can be calculated. If the similarity is greater than a set threshold, it is determined that the user has passed the identity authentication, and the first verification result is normal.

[0069] That is, the identification information of the recipient associated with the target express to be picked up can be obtained, and the identity reference feature vector corresponding to the biometric data of the recipient can be obtained according to the identification information. The first verification result can be obtained by performing similarity matching between the identity feature vector and the identity reference feature vector.

[0070] Optionally, if the similarity is greater than a set similarity threshold, the first verification result is determined to be normal; if the similarity is less than or equal to the set similarity threshold, the first verification result is determined to be abnormal.

[0071] S404: Verify the user's pickup behavior according to the behavior feature vector to obtain a second verification result.

[0072] In some implementations, the behavior feature vector can be used to verify whether the user behavior matches the historical behavior data, so as to further enhance the security of the pickup based on the identity verification. Optionally, the second verification result can be determined to be normal based on the similarity between the behavior feature vector and the historical feature vector.

[0073] Optionally, identification information of a recipient associated with a target express to be picked up is obtained, and historical pickup behavior data of the recipient is obtained based on the identification information, and a historical behavior feature vector of the recipient is obtained based on the historical pickup behavior data.

[0074] Further, the similarity between the behavior feature vector and the historical feature vector is calculated, and if the similarity is greater than a similarity threshold, it is determined that the behavior feature vector and the historical feature vector match. In other words, in response to the behavior feature vector and the historical feature vector matching, it is determined that the user's pickup behavior is not abnormal, which is used as the second verification result.

[0075] If the similarity is less than or equal to the similarity threshold, it is determined that the behavior feature vector and the historical feature vector do not match. In other words, in response to the behavior feature vector and the historical feature vector not matching, it is determined that the user's pickup behavior is abnormal as the second verification result. For example, the user's pickup behavior is abnormal if the user picks up the item at an unusual time period, the action is too hasty or slow, or the pickup posture is abnormal.

[0076] In some implementations, if the behavior feature vector and the historical feature vector do not match, the user's behavior can also be predicted, and whether the user's behavior is abnormal can be determined based on the prediction result to enhance the accuracy and reliability of behavior verification.

[0077] It is understandable that behavior prediction requires the time dependency of behavior, and behavior can be analyzed and predicted based on time series. For example, behavior can be predicted based on the Long Short-Term Memory (LSTM) network.

[0078] That is, in response to the behavior feature vector and the historical feature vector not matching, the time series feature of the recipient is obtained based on the historical pickup behavior data. The time series feature may be the pickup time of the recipient. Then, based on the pickup environment information, context information related to time and space is obtained, such as weather, holidays, etc.

[0079] Furthermore, based on the behavior feature vector, time series features and context information, a prediction is made as to whether the user's pickup behavior is abnormal to obtain a second verification result. For example, the behavior feature vector, time series features and context information can be input into the LSTM to predict the user's behavior to determine whether the user's behavior is affected by external factors. If the difference in user behavior is related to the context information, it is determined that the user's pickup behavior is not abnormal, which is used as the second verification result. If it is not related, it is determined that the user's pickup behavior is abnormal, which is used as the second verification result.

[0080] S405: Generate a pickup verification result based on the first verification result and the second verification result.

[0081] In some implementations, if both the first verification result and the second verification result are normal, the pickup verification result is determined to be normal; if there is an abnormality in the first verification result and the second verification result, the pickup verification result is determined to be abnormal.

[0082] S406: In response to the pickup verification result being normal, executing the pickup process.

[0083] S407: In response to an abnormal pickup verification result, an abnormal pickup warning message is issued.

[0084] The relevant contents of steps S406-S407 can be found in the above embodiment and will not be repeated here.

[0085] According to the express delivery anti-theft method provided by the embodiment of the present disclosure, by obtaining the multimodal data of the user, and using a large model to extract features from the multimodal data, and verifying the identity and behavior of the user with the extracted features, the corresponding pickup verification result of the user is determined. It can then be determined whether the pickup is abnormal based on the pickup verification result to execute the corresponding process. By verifying the identity feature vector, it can be ensured that only verified users can pick up the items, effectively preventing the express delivery from being picked up by mistake or stolen; by verifying the behavior feature vector, it can accurately identify abnormal pickup behavior, and then alert the user through abnormal alarm information, thereby enhancing security.

[0086] like Figure 5 The flowchart for verifying the identity and behavior of the user is shown. The multimodal data can be preprocessed first, such as standardization, denoising and other operations, and the preprocessed multimodal data can be input into the large model. The large model extracts and fuses the features of the multimodal data to obtain the identity feature vector and the behavior feature vector. Then, the identity of the user is verified using the identity feature vector to obtain a first verification result, and the behavior feature vector is used to verify the user's behavior to obtain a second verification result. If both the first verification result and the second verification result are normal, the pickup verification result is determined to be normal; if there is an abnormality in the first verification result and the second verification result, the pickup verification result is determined to be abnormal.

[0087] In response to the pickup verification result being normal, the pickup process can be executed. In response to the pickup verification result being abnormal, an abnormal pickup alarm message is issued.

[0088] Figure 6 A flowchart of a method for preventing theft of express delivery provided by an embodiment of the present disclosure is provided.

[0089] like Figure 6 As shown, the express delivery anti-theft method may include:

[0090] S601, obtaining multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data.

[0091] S602, based on the pre-trained large model, a decision is made on whether the pickup is abnormal based on the multimodal data, and the pickup verification result corresponding to the user is obtained.

[0092] The relevant contents of steps S601-S602 can be found in the above embodiment and will not be repeated here.

[0093] In some implementations, in order to ensure the integrity and accuracy of the data and facilitate subsequent user verification, the acquired multimodal data and pickup verification results may be stored. Optionally, a pickup log file may be generated based on the multimodal data and the pickup verification results, and the pickup log file may be stored in the first database, and the multimodal data may be stored in the second database. The pickup log file includes information such as pickup time, facial images, and videos. The first database may be a log database, which is used to store log files and supports efficient query and retrieval. The second database may be a data storage server, which is used to store all collected data in a database server in the cloud to ensure data security and persistence.

[0094] Furthermore, after the data is stored, it can be traced and retrieved based on the query interface, which enhances the trust and transparency of the pickup process. A data tracing request can be received based on the query interface, and data can be extracted from at least one of the first database and the second database according to the data tracing request to obtain a data tracing result.

[0095] like Figure 7 The flowchart of data storage and traceability is shown in FIG. By collecting data, the multimodal data of the user is obtained, and the pickup verification result of the pickup verification based on the multimodal data is obtained, and the pickup log file is generated based on the multimodal data and the pickup verification result, and then the pickup log file can be stored in the log database, and the multimodal data can be stored in the data storage server.

[0096] Furthermore, data tracing and retrieval are achieved by receiving data tracing requests and extracting data from stored data based on a query interface.

[0097] S603, in response to the pickup verification result being normal, executing the pickup process.

[0098] The relevant contents of step S603 can be found in the above embodiment and will not be described again here.

[0099] S604, in response to an abnormal pickup verification result, obtain the predicted risk level of the abnormal pickup output by the large model, and generate alarm information according to the predicted risk level.

[0100] In some implementations, when it is determined that the pickup verification result is abnormal, the large model will predict the risk level of the abnormality and obtain the predicted risk level of the pickup abnormality, where the predicted risk level includes high risk and low risk. Furthermore, corresponding alarm information can be generated according to the predicted risk level.

[0101] For example, if the predicted risk level is high risk, an alarm message is generated based on a text message; if the predicted risk level is low risk, an alarm message is generated based on a pop-up window.

[0102] S605, determining the contact information of the recipient associated with the target express to be picked up.

[0103] In some implementations, in order to warn users of abnormal pickup, ensure the safety of the pickup process, and avoid the occurrence of mispickup or theft of couriers, the recipient information of the target courier to be picked up can be determined and an alarm message can be sent to the recipient.

[0104] Optionally, the courier information of the target courier can be determined based on the pickup code input by the user. Alternatively, the courier information currently associated with the courier identification can be determined based on the cabinet identification of the courier cabinet where the user currently picks up the item. The courier information at least includes contact information.

[0105] S606: Send an alarm message to the recipient's terminal device based on the contact information.

[0106] Optionally, after the contact information of the recipient is obtained, an alarm message is sent to the terminal device of the recipient through the contact information. Optionally, the contact information may be information such as a telephone number and a terminal device identifier.

[0107] In some implementations, in order to clarify whether an abnormality actually exists and to reduce the possibility of false alarms, after sending the alarm information, the user may confirm the alarm information to obtain abnormality confirmation information fed back by the terminal device.

[0108] Optionally, in response to the abnormal confirmation information being normal, the alarm information is marked as a false alarm, and multimodal data corresponding to the false alarm information is obtained as sample data for model update. Further, based on the sample data for model update, the large model is optimized to improve the accuracy of model detection.

[0109] Optionally, in response to the abnormal confirmation information being abnormal, an alarm message is sent to the courier sender associated with the target courier so that the courier sender can promptly discover and correct errors, such as address errors, inconsistent recipient information, etc., thereby improving the accuracy of the courier service.

[0110] According to the express delivery anti-theft method provided by the embodiment of the present disclosure, the user's multimodal data is obtained, and the multimodal data is analyzed using a large model to verify the user's identity and behavior, and determine the user's corresponding pickup verification result. It can then be determined whether the pickup is abnormal based on the pickup verification result. In the event of an abnormality, the user is alerted to ensure the safety of the pickup process and avoid the occurrence of express delivery mispickup or theft. By verifying the user's pickup, the express delivery can be effectively prevented from being mispickup or theft, to ensure that the express delivery can be accurately delivered to the real recipient, and to reduce delivery delays or failures caused by information errors, thereby improving user satisfaction. The large model can automatically perform face recognition and behavior analysis, reducing reliance on manual review and improving the operational efficiency of express delivery stations.

[0111] like Figure 8 The flowchart of abnormal alarm is shown in FIG. By collecting multimodal data and analyzing the multimodal data by the large model, it is determined whether there is an abnormality in the user's pickup. If there is no abnormality, the pickup process is executed. If there is an abnormality, the alarm process is triggered. By generating an alarm message and sending emergency alarms and general alarms to the user based on the predicted risk level of the pickup abnormality. When sending an emergency alarm, by receiving the user's abnormal confirmation information on the alarm information, if the abnormal confirmation information is not abnormal, the alarm information is marked as a false alarm to update the large model; if the abnormal confirmation information is abnormal, the alarm information is sent to the courier sender associated with the target courier.

[0112] like Fig. 9 The timing diagram of express delivery anti-theft is shown in the figure. By collecting multimodal data when the user picks up the package, and uploading the multimodal data to the cloud. By inputting the multimodal data into the big model, the big model verifies the identity and behavior of the user, obtains the pickup verification result, and if the pickup verification result is normal, executes the pickup process; if the pickup verification result is abnormal, sends an alarm message, and predicts the risk level of the pickup abnormality. If the predicted risk level is high risk, an emergency alarm is issued to the user, and the user's abnormal confirmation information is received. If the abnormal confirmation information is not abnormal, the alarm information is marked as a false alarm to update the big model; if the abnormal confirmation information is abnormal, an alarm message is sent to the express sender associated with the target express. If the predicted risk level is low risk, a general alarm is sent to the user, and the user's behavior data is recorded. By storing multimodal data and pickup log files, data query and retrieval services can be realized.

[0113] Corresponding to the express anti-theft methods provided in the above-mentioned embodiments, an embodiment of the present disclosure further provides an express anti-theft device. Since the express anti-theft device provided in the embodiment of the present disclosure corresponds to the express anti-theft methods provided in the above-mentioned embodiments, the implementation methods of the above-mentioned express anti-theft methods are also applicable to the express anti-theft device provided in the embodiment of the present disclosure, and will not be described in detail in the following embodiments.

[0114] Fig.10 A schematic structural diagram of an express delivery anti-theft device provided in an embodiment of the present disclosure.

[0115] like Fig.10 As shown, the express delivery anti-theft device 1000 of the embodiment of the present disclosure includes an acquisition module 1001, a verification module 1002, a pickup module 1003 and an alarm module 1004.

[0116] The acquisition module 1001 is used to acquire multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data;

[0117] Verification module 1002, used to make a decision on whether the pickup is abnormal based on the pre-trained large model, and obtain the pickup verification result corresponding to the user;

[0118] The pickup module 1003 is used to execute the pickup process in response to the pickup verification result being normal;

[0119] The alarm module 1004 is used to issue an abnormal pickup alarm message in response to the abnormal pickup verification result.

[0120] In one embodiment of the present disclosure, the verification module 1002 is also used to: extract features from the multimodal data based on the large model, and perform identity authentication and pickup behavior verification on the user based on the extracted feature information to obtain the pickup verification result.

[0121] In one embodiment of the present disclosure, the verification module 1002 is also used to: extract features from the multimodal data based on the large model to obtain an identity feature vector and a behavior feature vector; verify the identity of the user according to the identity feature vector to obtain a first verification result; verify the user's pickup behavior according to the behavior feature vector to obtain a second verification result; and generate the pickup verification result based on the first verification result and the second verification result.

[0122] In one embodiment of the present disclosure, the verification module 1002 is also used to: extract features from the modal data in the multimodal data to obtain multi-dimensional modal features of the modal data; fuse the multi-dimensional modal features of the modal data to obtain fused modal features of the modal data; and perform cross-modal attention on the fused feature modality of the multimodal data to obtain the identity feature vector.

[0123] In one embodiment of the present disclosure, the verification module 1002 is also used to: obtain identification information of a recipient associated with a target express to be picked up; obtain an identity reference feature vector corresponding to the biometric data of the recipient based on the identification information; and perform similarity matching between the identity feature vector and the identity reference feature vector to obtain the first verification result.

[0124] In one embodiment of the present disclosure, the verification module 1002 is also used to: perform feature extraction on the multimodal data to obtain the user's pickup posture, action characteristics and pickup time and space information; obtain the behavior feature vector based on the user's pickup posture, action characteristics and pickup time and space information.

[0125] In one embodiment of the present disclosure, the verification module 1002 is further used to: obtain identification information of a recipient associated with a target express to be picked up; obtain historical pickup behavior data of the recipient based on the identification information, and obtain a historical behavior feature vector of the recipient based on the historical pickup behavior data; in response to a match between the behavior feature vector and the historical feature vector, determine that the user's pickup behavior is not abnormal as the second verification result; in response to a mismatch between the behavior feature vector and the historical feature vector, determine that the user's pickup behavior is abnormal as the second verification result.

[0126] In one embodiment of the present disclosure, the verification module 1002 is also used to: in response to the behavior feature vector and the historical feature vector not matching, obtain the timing characteristics of the recipient based on the historical pickup behavior data; obtain context information related to time and space based on the pickup environment information; and predict whether the user's pickup behavior is abnormal based on the behavior feature vector, the timing characteristics and the context information to obtain the second verification result.

[0127] In one embodiment of the present disclosure, the alarm module 1004 is also used to: obtain the predicted risk level of the abnormal pickup output by the large model, and generate the alarm information according to the predicted risk level; determine the contact information of the recipient associated with the target express to be picked up; and send the alarm information to the terminal device of the recipient based on the contact information.

[0128] In one embodiment of the present disclosure, the alarm module 1004 is also used to: determine the express delivery information of the target express based on the pickup code entered by the user; or, based on the cabinet identifier of the express cabinet where the user is currently picking up the package, determine the express delivery information currently associated with the cabinet identifier; wherein the express delivery information includes at least the contact information.

[0129] In one embodiment of the present disclosure, the alarm module 1004 is also used to: obtain abnormality confirmation information fed back by the terminal device; in response to the abnormality confirmation information being normal, mark the alarm information as a false alarm; in response to the abnormality confirmation information being abnormal, send an alarm message to the courier sender associated with the target courier.

[0130] In one embodiment of the present disclosure, the alarm module 1004 is further used to: obtain multimodal data corresponding to the alarm information of the false alarm as sample data for model update; and optimize the large model based on the sample data for model update.

[0131] In one embodiment of the present disclosure, the verification module 1002 is further used to: generate a pickup log file based on the multimodal data and the pickup verification result, store the pickup log file in a first database, and store the multimodal data in a second database.

[0132] In one embodiment of the present disclosure, the verification module 1002 is also used to: receive a data tracing request based on a query interface, and extract data from at least one of the first database and the second database according to the data tracing request to obtain a data tracing result.

[0133] According to the express delivery anti-theft device provided by the embodiment of the present disclosure, the user's multimodal data is obtained, and the multimodal data is analyzed using a large model to verify the user's identity and behavior, and determine the user's corresponding pickup verification result. It can then be determined whether the pickup is abnormal based on the pickup verification result to execute the corresponding process. By verifying the user's pickup, it can effectively prevent the express delivery from being picked up by mistake or stolen, to ensure that the express delivery can be accurately delivered to the real recipient, and reduce delivery delays or failures caused by incorrect information, thereby improving user satisfaction. The large model can automatically perform facial recognition and behavioral analysis, reducing reliance on manual review and improving the operational efficiency of express delivery stations.

[0134] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0135] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0136] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0137] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program / instruction stored in a read-only memory (ROM) 1102 or a computer program / instruction loaded from a storage unit 1106 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0138] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106 such as a keyboard, a mouse, etc.; an output unit 1107 such as various types of displays, speakers, etc.; a storage unit 1108 such as a disk, an optical disk, etc.; and a communication unit 1109 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0139] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the express anti-theft method. For example, in some embodiments, the express anti-theft method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1106. In some embodiments, part or all of the computer program / instructions may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program / instructions are loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the express anti-theft method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the express theft prevention method in any other appropriate manner (eg, by means of firmware).

[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0142] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0145] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs / instructions running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the disclosure can be achieved, and this document does not limit them here.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for preventing express delivery from being stolen, wherein: The method comprises: Acquire multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, package pickup behavior data, and package pickup environment data; Based on the pre-trained large model, a decision is made on whether the pickup is abnormal for the multimodal data, and a pickup verification result corresponding to the user is obtained; In response to the pickup verification result being normal, executing the pickup process; In response to the abnormal pickup verification result, an abnormal pickup alarm message is issued.

2. The method according to claim 1, wherein: The decision of whether the pickup is abnormal based on the pre-trained large model on the multimodal data and obtaining the pickup verification result corresponding to the user includes: Feature extraction is performed on the multimodal data based on the large model, and identity authentication and pickup behavior verification are performed on the user based on the extracted feature information to obtain the pickup verification result.

3. The method according to claim 2, wherein: The extracting features of the multimodal data based on the large model, and verifying the identity and pickup behavior of the user based on the extracted feature information to obtain the pickup verification result, includes: Extracting features from the multimodal data based on the large model to obtain an identity feature vector and a behavior feature vector; Verifying the identity of the user according to the identity feature vector to obtain a first verification result; Verifying the user's pickup behavior according to the behavior feature vector to obtain a second verification result; The pickup verification result is generated according to the first verification result and the second verification result.

4. The method according to claim 3, wherein: The process of obtaining the identity feature vector includes: Extracting features of modal data in the multimodal data to obtain multi-dimensional modal features of the modal data; fusing the multi-dimensional modal features of the modal data to obtain fused modal features of the modal data; A cross-modal focus is performed on the fused feature modality of the multimodal data to obtain the identity feature vector.

5. The method according to claim 4, wherein: The step of verifying the identity of the user according to the identity feature vector to obtain a first verification result includes: Obtain identification information of the recipient associated with the target express to be picked up; Acquire an identity reference feature vector corresponding to the biometric data of the recipient according to the identification information; The identity feature vector and the identity reference feature vector are matched for similarity to obtain the first verification result.

6. The method according to claim 3, wherein: The process of acquiring the behavior feature vector includes: Extracting features from the multimodal data to obtain the user's pickup posture, action features, and spatiotemporal information of the pickup; The behavior feature vector is obtained according to the user's picking-up posture, action features, and picking-up spatiotemporal information.

7. The method according to claim 6, wherein: The verifying the user's pickup behavior according to the behavior feature vector to obtain a second verification result includes: Obtain identification information of the recipient associated with the target express to be picked up; Acquire the historical pickup behavior data of the recipient according to the identification information, and acquire the historical behavior feature vector of the recipient according to the historical pickup behavior data; In response to the behavior feature vector and the history feature vector matching, determining that the user's pickup behavior is not abnormal, as the second verification result; In response to the behavior feature vector and the historical feature vector not matching, determining that the user's pickup behavior is abnormal is used as the second verification result.

8. The method according to claim 7, wherein: The method further comprises: In response to the behavior feature vector and the historical feature vector not matching, acquiring the time series feature of the pickup person according to the historical pickup behavior data; Acquiring context information related to time and space according to the pickup environment information; Based on the behavior feature vector, the time series feature and the context information, a prediction is made as to whether the user's pickup behavior is abnormal, so as to obtain the second verification result.

9. The method according to any one of claims 1 to 8, wherein: The sending of abnormal pickup warning information in response to the abnormal pickup verification result includes: Obtaining the predicted risk level of the pickup abnormality output by the large model, and generating the warning information according to the predicted risk level; Determine the contact information of the recipient associated with the target express to be picked up; The warning information is sent to the terminal device of the recipient based on the contact information.

10. The method according to claim 9, wherein: The contact information of the recipient associated with the target express to be picked up is determined, including: Determine the delivery information of the target courier based on the pickup code input by the user; or, Based on the cabinet identification of the express cabinet where the user currently picks up the package, determine the express information currently associated with the cabinet identification; Wherein, the express delivery information at least includes the contact information.

11. The method according to claim 8, wherein: After sending the alarm information to the terminal device of the recipient based on the contact information, the method further includes: Obtaining abnormal confirmation information fed back by the terminal device; In response to the abnormality confirmation information being no abnormality, marking the alarm information as a false alarm; In response to the abnormal confirmation information being abnormal, an alarm message is sent to the courier sender associated with the target courier.

12. The method according to claim 11, wherein: After marking the alarm information as a false alarm, the method further includes: Acquire multimodal data corresponding to the alarm information of the false alarm as sample data for model updating; The large model is optimized based on the sample data updated by the model.

13. The method according to any one of claims 1 to 7, wherein: After obtaining the pickup verification result corresponding to the user, the method further includes: Based on the multimodal data and the pickup verification result, a pickup log file is generated, and the pickup log file is stored in a first database, and the multimodal data is stored in a second database.

14. The method according to claim 13, wherein: The method further comprises: A data tracing request is received based on a query interface, and data is extracted from at least one of the first database and the second database according to the data tracing request to obtain a data tracing result.

15. A courier anti-theft device, wherein: The device comprises: An acquisition module, used to acquire multimodal data when a user picks up a package, wherein the multimodal data includes at least one type of biometric data of the user, pickup behavior data, and pickup environment data; A verification module, used to determine whether the pickup is abnormal based on the multimodal data based on a pre-trained large model, and obtain a pickup verification result corresponding to the user; A pickup module, configured to execute a pickup process in response to the pickup verification result being normal; The alarm module is used to issue an abnormal pickup alarm message in response to the abnormal pickup verification result.

16. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-14.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images