Express anti-theft method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an express anti-theft method and device, electronic equipment and storage medium, which relates to the technical field of artificial intelligence. The specific implementation scheme is: acquiring multi-modal data of a user when picking up a package, wherein the multi-modal data includes at least one type of biological feature data, picking-up behavior data and picking-up environment data of the user; determining whether the picking-up is abnormal based on a pre-trained large model on the multi-modal data to obtain a picking-up verification result corresponding to the user; in response to the picking-up verification result being normal, executing a picking-up process; and in response to the picking-up verification result being abnormal, issuing a picking-up abnormality alarm information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to an express delivery anti-theft method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the rapid development of express delivery business, the problems of lost express delivery and misdelivery of express delivery are increasingly common in express delivery stations and delivery points, which brings inconvenience to users' life and increases the difficulty of work for express delivery personnel. At present, most express delivery stations use mobile phone numbers and verification codes as the main means of identity verification, which leads to the fact that users can arbitrarily take other people's express delivery in unintentional or intentional cases, and further leads to the gradual increase in disputes caused by the lack of effective prevention and control measures. SUMMARY

[0003] The present disclosure provides an express delivery anti-theft method, device, electronic device, and storage medium.

[0004] According to an aspect of the present disclosure, an express delivery anti-theft method is provided, which includes: acquiring multi-modal data of a user when taking delivery, wherein the multi-modal data includes at least one type of biological feature data, taking delivery behavior data, and taking delivery environment data of the user; making a decision on whether the taking delivery is abnormal based on a pre-trained large model on the multi-modal data to acquire a taking delivery verification result corresponding to the user; executing a taking delivery process in response to the taking delivery verification result being normal; and issuing a taking delivery abnormality alarm information in response to the taking delivery verification result being abnormal.

[0005] According to another aspect of the present disclosure, an express delivery anti-theft device is provided, which includes: an acquisition module configured to acquire multi-modal data of a user when taking delivery, wherein the multi-modal data includes at least one type of biological feature data, taking delivery behavior data, and taking delivery environment data of the user; a verification module configured to make a decision on whether the taking delivery is abnormal based on a pre-trained large model on the multi-modal data to acquire a taking delivery verification result corresponding to the user; a taking delivery module configured to execute a taking delivery process in response to the taking delivery verification result being normal; and an alarm module configured to issue a taking delivery abnormality alarm information in response to the taking delivery verification result being abnormal.

[0006] According to another aspect of the present disclosure, an electronic device is provided, which includes: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the express delivery anti-theft method of the above-mentioned aspect embodiment.

[0007] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, and the computer instructions are stored on the computer readable storage medium and used to make the computer execute the express anti-theft method according to the embodiment of the above aspect.

[0008] According to another aspect of the present disclosure, a computer program product is provided, and the computer program product comprises computer instructions, and the computer instructions are executed by a processor to implement the express anti-theft method according to the embodiment of the above aspect.

[0009] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0011] Figure 1 A flowchart of an express anti-theft method provided by an embodiment of the present disclosure;

[0012] Figure 2 A flowchart of collecting multi-modal data provided by an embodiment of the present disclosure;

[0013] Figure 3 A flowchart of another express anti-theft method provided by an embodiment of the present disclosure;

[0014] Figure 4 A flowchart of another express anti-theft method provided by an embodiment of the present disclosure;

[0015] Figure 5 A flowchart of verifying the identity and behavior of a user provided by an embodiment of the present disclosure;

[0016] Figure 6 A flowchart of another express anti-theft method provided by an embodiment of the present disclosure;

[0017] Figure 7 A flowchart of storing and tracing data provided by an embodiment of the present disclosure;

[0018] Figure 8 A flowchart of performing abnormal alarm provided by an embodiment of the present disclosure;

[0019] Figure 9 A timing diagram of express anti-theft provided by an embodiment of the present disclosure;

[0020] Figure 10 A structural diagram of an express anti-theft device provided by an embodiment of the present disclosure;

[0021] Figure 11 A block diagram of an electronic device for implementing the express delivery theft prevention method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, which should be considered in a descriptive sense only. Thus, it will be apparent to one of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0023] Artificial Intelligence (AI) is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of human life, which includes both hardware technology and software technology. Artificial intelligence hardware technology generally includes computer vision technology, speech recognition technology, natural language processing technology, and learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0024] The express delivery theft prevention method, device, electronic device, and storage medium of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart of an express delivery theft prevention method provided by the embodiments of the present disclosure.

[0026] As shown in Figure 1 , the express delivery theft prevention method can include:

[0027] S101, acquiring multi-modal data at the time when the user takes the package, wherein the multi-modal data includes at least one type of biological feature data of the user, taking behavior data, and taking environment data.

[0028] It should be noted that the execution subject of the express delivery theft prevention method in the embodiments of the present disclosure can be a hardware device with data processing capability and / or necessary software required to drive the hardware device to work. Alternatively, the execution subject can include a server, a user terminal, and other intelligent devices. Alternatively, the user terminal includes but is not limited to a mobile phone, a computer, a smart voice interaction device, etc. Alternatively, the server includes but is not limited to a network server, an application server, and can also be a server of a distributed system, or a server combined with a blockchain, etc. The embodiments of the present disclosure are not limited specifically.

[0029] In some implementations, the multi-modal data of the user when picking up the package can be collected based on multiple sensors. Optionally, the multi-modal data of the user when picking up the package can be collected based on one or more image sensors, sound sensors. For example, when the user picks up the package in the delivery cabinet, the video, face image, voice data, picking-up behavior data and picking-up environment data of the user when picking up the package can be collected based on the camera, microphone and other devices of the delivery cabinet itself as multi-modal data. Among them, the face image and voice data can be used as biological feature data.

[0030] In S102, the pre-trained large model is used to determine whether the multi-modal data is abnormal, and the picking-up verification result of the user is obtained.

[0031] In some implementations, the identity of the user can be verified based on the biological feature data in the multi-modal data, and the behavior of the user can be verified based on the picking-up behavior data and the picking-up environment data in the multi-modal data, and the picking-up verification result of the user is determined according to the verification result.

[0032] In some implementations, the pre-trained large model can extract the identity feature vector and the behavior feature vector of the user from the multi-modal data, and verify the identity and behavior of the user based on the identity feature vector and the behavior feature vector to determine whether the user is abnormal.

[0033] Optionally, if the user fails the verification, it is determined that the picking-up verification result of the user is abnormal; if the user passes the verification, it is determined that the picking-up verification result of the user is normal.

[0034] Optionally, the identity and legitimacy of the user can be verified based on the identity feature vector, for example, the user can be verified by face recognition and the like. Further, based on the behavior feature vector, it is verified whether the behavior of the user matches the historical behavior, if matched, it is determined that the user passes the verification, and it is determined that the picking-up verification result of the user is normal.

[0035] In some implementations, the pre-trained large model can be a neural network model, and the large model can learn the multi-modal data and associate the relationship between the modalities through the cross-modal attention mechanism, to generate deeper feature vectors, that is, to obtain the identity feature vector and the behavior feature vector.

[0036] In S103, the picking-up process is performed in response to the normal picking-up verification result.

[0037] In some implementations, when the picking-up verification result is normal, it is determined that the identity and behavior of the user pass the verification, and the user can perform the picking-up process to take out the stored package. For example, if the package is stored in the delivery cabinet, the large model mounted on the delivery cabinet itself is used to verify the user, and it is determined that the picking-up verification result is normal, and the door of the delivery cabinet is opened to facilitate the user to pick up the package.

[0038] S104, in response to the pick-up verification result being abnormal, issuing a pick-up abnormality alarm information.

[0039] In some implementations, when the pick-up verification result is abnormal, an alarm process is triggered immediately. Alternatively, the pick-up abnormality alarm information can be sent to the user based on the degree of abnormality. If the degree of abnormality is high risk, the pick-up abnormality alarm information is sent to the user in the form of a short message; if the degree of abnormality is low risk, the pick-up abnormality alarm information is sent to the user in the form of a pop-up window.

[0040] Further, after receiving the pick-up abnormality alarm information, the user can confirm the abnormality. If the abnormality is confirmed, the user can be prompted to contact the express delivery company to confirm whether the express delivery is correctly delivered. If no abnormality is confirmed, it is marked as a false alarm, and the large model is updated based on the false alarm to reduce the occurrence of similar false alarms.

[0041] According to the express anti-theft method provided by the embodiments of the present disclosure, the multi-modal data of the user is obtained, and the multi-modal data is analyzed using a large model to verify the identity and behavior of the user to determine the pick-up verification result corresponding to the user. Further, whether the pick-up is abnormal can be determined according to the pick-up verification result to execute the corresponding process. By verifying the pick-up of the user, the express delivery can be effectively prevented from being mispicked or stolen, so as to ensure that the express delivery can be accurately and correctly delivered to the real recipient, and to reduce the delay or failure of delivery caused by information errors, and to improve the user satisfaction. The large model can automatically perform face recognition and behavior analysis to reduce the dependence on manual review, and improve the operation efficiency of the express delivery station.

[0042] As shown in the flowchart of collecting multi-modal data. Figure 2 When the user picks up, the express pick-up process is triggered, and multi-modal data is collected. Figure 2 In some implementations, the multi-modal data can be collected based on a camera, a microphone, and a sensor, such as a temperature sensor and an optical sensor. By starting the camera, the camera captures images and videos, and processes the images and videos to detect whether a face is photographed and the front of the user, and then stores the collected images and videos. By starting the microphone, the user's voice data is recorded, and the voice is processed for noise filtering, voiceprint recognition, etc., and the voice data is stored. By starting the sensor, the environmental data is collected, and the temperature, light, position, etc. data in the environmental data is obtained, and the environmental data is stored.

[0043] Further, the collected data is fused to obtain multi-modal data: biological feature data, pick-up behavior data and pick-up environment data, and the multi-modal data is analyzed using a large model to generate a pick-up verification result, and the corresponding process is executed according to the pick-up verification result.

[0044] Figure 3 A flowchart of an express anti-theft method provided by an embodiment of the present disclosure is shown.

[0045] As shown in Figure 3 , the express anti-theft method can include:

[0046] S301, acquiring multi-modal data of a user when picking up a package, wherein the multi-modal data includes at least one type of biological feature data of the user, pick-up behavior data, and pick-up environment data.

[0047] The related content of step S301 can be referred to the above-mentioned embodiments, which will not be repeated here.

[0048] S302, performing feature extraction on the multi-modal data based on a large model, and performing identity verification and pick-up behavior verification on the user based on the extracted feature information to obtain a pick-up verification result.

[0049] In some implementations, the large model can perform feature extraction on the biological feature data in the multi-modal data to obtain an identity feature vector corresponding to the user, and perform feature extraction on the pick-up behavior data and the pick-up environment data in the multi-modal data to obtain a behavior feature vector corresponding to the user.

[0050] Optionally, the identity feature vector can include face features and voiceprint features, and the behavior feature vector can include pick-up time features, pick-up frequency features, pick-up location features, behavior action features, and pick-up action features.

[0051] For example, the large model can perform feature extraction on the multi-modal data based on a convolutional neural network (CNN). For example, for an input face image, a 128-dimensional or higher-dimensional feature vector is extracted by the CNN to represent the face features. For example, the user action features in the video data are extracted based on the CNN.

[0052] Further, the identity and the pick-up behavior of the user are verified based on the identity feature vector and the behavior feature vector. If the identity verification and the pick-up behavior verification are passed, it is determined that the pick-up verification result is normal. If the identity verification and the pick-up behavior verification are not passed, it is determined that the pick-up verification result is abnormal.

[0053] S303, in response to the pick-up verification result being normal, performing a pick-up process.

[0054] S304, in response to the pick-up verification result being abnormal, issuing a pick-up abnormality alarm information.

[0055] The related content of steps S303-S304 can be referred to the above-mentioned embodiments, which will not be repeated here.

[0056] According to the express anti-theft method provided by the embodiments of the present disclosure, multi-modal data of a user is obtained, and a large model is used to extract features from the multi-modal data, and the extracted features are used to verify the identity and behavior of the user to determine the pick-up verification result corresponding to the user. Then, whether the pick-up is abnormal can be determined according to the pick-up verification result to perform a corresponding process. By verifying the pick-up of the user, it can effectively prevent the express from being mispicked or stolen, so as to ensure that the express can be accurately delivered to the real recipient, reduce the delay or failure of delivery caused by information errors, and improve the user satisfaction. The large model can automatically perform face recognition and behavior analysis to reduce the dependence on manual review and improve the operation efficiency of the express post station.

[0057] Figure 4 A flowchart of an express anti-theft method provided by the embodiments of the present disclosure is shown.

[0058] As shown in Figure 4 , the express anti-theft method can include:

[0059] S401, obtaining multi-modal data of a user when picking up, wherein the multi-modal data includes at least one type of biological feature data, pick-up behavior data and pick-up environment data of the user.

[0060] The related content of step S401 can be referred to the above-mentioned embodiments, which will not be repeated here.

[0061] S402, extracting features from the multi-modal data based on a large model to obtain an identity feature vector and a behavior feature vector.

[0062] In some implementations, the large model can extract the identity feature vector and the behavior feature vector from the multi-modal data. By extracting the identity feature vector from the multi-modal data, the comprehensiveness and accuracy of the identity feature can be improved, so that accurate verification can be performed based on the identity feature vector.

[0063] Optionally, by extracting features from the modal data in the multi-modal data, multi-dimensional modal features of the modal data are obtained, wherein the modal data can be any type of modal data in the multi-modal data, that is, features can be extracted from each type of modal data in the multi-modal data to obtain multi-dimensional modal features of each type of modal data. For example, the facial features of the user can be extracted from the face image as a type of modal feature, and the voiceprint features can be extracted from the voice data as a type of modal feature.

[0064] Further, the multi-dimensional modal features of the modal data can be fused to obtain fused modal features of the modal data, and cross-modal attention can be performed on the fused feature modal of the multi-modal data to obtain the identity feature vector. For example, the face features and the voiceprint features are fused and cross-modal attention is performed, and the identity feature vector can be obtained.

[0065] It can be understood that the behavior feature vector mainly describes the behavior habits and time sequence data of the user, and generally includes the action trajectory, time regularity, frequency and location of the user, and the behavior feature vector can further confirm the user, which helps to improve the response capability to potential security risks such as theft.

[0066] Optionally, the pick-up posture, action features and pick-up spatio-temporal information of the user are obtained by feature extraction on the multi-modal data, and the behavior feature vector is obtained according to the pick-up posture, action features and pick-up spatio-temporal information of the user. That is, the pick-up posture, action features and pick-up spatio-temporal information of the user can be fused to obtain the behavior feature vector.

[0067] S403, verifying the identity of the user according to the identity feature vector to obtain a first verification result.

[0068] In some implementations, the identity of the user can be verified based on the identity feature vector to determine whether the features match the stored reference features, so as to improve the accuracy and reliability of identity verification, effectively prevent theft, and enhance the security of the pick-up process. Optionally, the similarity of the feature vector can be calculated, and if the similarity is greater than a set threshold, it is determined that the user passes the identity verification, and the first verification result is normal.

[0069] That is, the identity reference feature vector corresponding to the biological feature data of the pick-up person can be obtained by obtaining the identification information of the pick-up person associated with the target express to be picked up, and according to the identification information. The first verification result is obtained by performing similarity matching on the identity feature vector and the identity reference feature vector.

[0070] Optionally, if the similarity is greater than a set similarity threshold, it is determined that the first verification result is normal; and if the similarity is less than or equal to the set similarity threshold, it is determined that the first verification result is abnormal.

[0071] S404, verifying the pick-up behavior of the user according to the behavior feature vector to obtain a second verification result.

[0072] In some implementations, the behavior of the user can be verified based on the behavior feature vector to determine whether the behavior matches the historical behavior data, so as to further enhance the security of the pick-up based on the identity verification. Optionally, the similarity between the behavior feature vector and the historical feature vector can be used to determine whether the second verification result is normal.

[0073] Optionally, the identifier information of the pickup person associated with the target express to be picked up is acquired, and the historical pickup behavior data of the pickup person is acquired according to the identifier information, and the historical behavior feature vector of the pickup person is acquired according to the historical pickup behavior data.

[0074] Further, the similarity between the behavior feature vector and the historical feature vector is calculated, and if the similarity is greater than a similarity threshold, it is determined that the behavior feature vector and the historical feature vector are matched. That is, in response to the behavior feature vector and the historical feature vector being matched, it is determined that the pickup behavior of the user is not abnormal, as the second verification result.

[0075] If the similarity is less than or equal to the similarity threshold, it is determined that the behavior feature vector and the historical feature vector are not matched. That is, in response to the behavior feature vector and the historical feature vector being not matched, it is determined that the pickup behavior of the user is abnormal, as the second verification result. For example, picking up at an unusual time period, moving too fast or too slow, or having an abnormal pickup posture, etc., the pickup behavior of the user is determined to be abnormal.

[0076] In some implementations, if the behavior feature vector and the historical feature vector are not matched, the behavior of the user can also be predicted, and the result of the prediction is used to determine whether the behavior of the user is abnormal, to enhance the accuracy and reliability of the behavior verification.

[0077] It can be understood that the behavior prediction requires the time-dependent relationship of the behavior, which can be analyzed and predicted based on time series. For example, the behavior can be predicted based on a Long Short-Term Memory (LSTM).

[0078] That is, in response to the behavior feature vector and the historical feature vector being not matched, the time sequence feature of the pickup person is acquired according to the historical pickup behavior data. The time sequence feature can be the pickup time of the pickup person. Further, the context information related to space and time is acquired according to the pickup environment data, such as weather, holidays, etc.

[0079] Further, whether the pickup behavior of the user is abnormal is predicted according to the behavior feature vector, the time sequence feature and the context information, to obtain the second verification result. For example, the behavior feature vector, the time sequence feature and the context information can be input into the LSTM to predict the behavior of the user, to determine whether the behavior of the user is affected by external factors. If the difference of the behavior of the user is related to the context information, it is determined that the pickup behavior of the user is not abnormal, as the second verification result, and if not, it is determined that the pickup behavior of the user is abnormal, as the second verification result.

[0080] S405, according to the first verification result and the second verification result, a pickup verification result is generated.

[0081] In some implementations, if both the first and second verification results are normal, the item retrieval verification result is determined to be normal; if either the first or second verification result is abnormal, the item retrieval verification result is determined to be abnormal.

[0082] S406, in response to the normal pickup verification result, the pickup process is executed.

[0083] S407, in response to an abnormal pickup verification result, issues a pickup abnormality alarm message.

[0084] The details of steps S406-S407 can be found in the above embodiments and will not be repeated here.

[0085] According to the express delivery anti-theft method provided in this disclosure, multimodal data of the user is acquired, and a large model is used to extract features from the multimodal data. The extracted features are then used to verify the user's identity and behavior to determine the corresponding package pickup verification result. Based on the pickup verification result, it can be determined whether the pickup is abnormal, and corresponding procedures are executed. By verifying the identity feature vector, it can be ensured that only verified users can pick up packages, effectively preventing accidental pickup or theft. By verifying the behavioral feature vector, abnormal pickup behavior can be accurately identified, and abnormal alarm information can be used to remind the user, enhancing security.

[0086] like Figure 5 The diagram illustrates the process of verifying a user's identity and behavior. First, multimodal data can be preprocessed, such as through standardization and noise reduction. This preprocessed data is then input into a large model, which extracts and fuses features to obtain identity feature vectors and behavior feature vectors. The identity feature vectors are then used to verify the user's identity, yielding a first verification result. The behavior feature vectors are used to verify the user's behavior, yielding a second verification result. If both the first and second verification results are normal, the pickup verification result is considered normal; otherwise, if either result is abnormal, the pickup verification result is considered abnormal.

[0087] If the pickup verification result is normal, the pickup process can be executed; if the pickup verification result is abnormal, a pickup abnormality alarm message will be issued.

[0088] Figure 6 This is a flowchart illustrating a method for preventing theft in express delivery, as provided in an embodiment of this disclosure.

[0089] like Figure 6 As shown, this anti-theft method for express delivery may include:

[0090] S601, acquire multi-modal data of the user when picking up the package, wherein the multi-modal data comprises at least one type of biological feature data, picking-up behavior data and picking-up environment data of the user.

[0091] S602, make a decision on whether the picking-up is abnormal based on the pre-trained large model, and acquire a picking-up verification result corresponding to the user.

[0092] The related content of steps S601-S602 can be referred to the above embodiments, which will not be repeated here.

[0093] In some implementations, in order to ensure the integrity and accuracy of the data, facilitate subsequent user verification, the acquired multi-modal data and the picking-up verification result can be stored. Optionally, a picking-up log file can be generated based on the multi-modal data and the picking-up verification result, and the picking-up log file is stored in the first database, and the multi-modal data is stored in the second database. The picking-up log file includes picking-up time, face image, video and other information. The first database can be a log database for storing log files and supporting efficient query and retrieval. The second database can be a data storage server for storing all collected data in a cloud database server, ensuring data security and persistence.

[0094] Further, after storing the data, the data can also be traced and retrieved based on the query interface, enhancing the trust and transparency of the picking-up process. The data traceability request can be received based on the query interface, and data extraction is performed in at least one of the first database and the second database according to the data traceability request, obtaining a data traceability result.

[0095] As shown in the flowchart of storing and tracing data. Figure 7 By collecting data, acquiring multi-modal data of the user, and performing picking-up verification based on the multi-modal data to obtain a picking-up verification result, and generating a picking-up log file based on the multi-modal data and the picking-up verification result, the picking-up log file can be stored in the log database, and the multi-modal data can be stored in the data storage server.

[0096] Further, by receiving a data traceability request and extracting data from the stored data based on the query interface, data traceability and retrieval are realized.

[0097] S603, in response to the picking-up verification result being normal, performing the picking-up process.

[0098] The related content of step S603 can be referred to the above embodiments, which will not be repeated here.

[0099] S604, in response to the pickup verification result being abnormal, obtaining a predicted risk level of the pickup abnormality output by the large model, and generating alarm information according to the predicted risk level.

[0100] In some implementations, when it is determined that the pickup verification result is abnormal, the large model predicts a risk level of the abnormality to obtain a predicted risk level of the pickup abnormality, wherein the predicted risk level includes high risk and low risk. Further, corresponding alarm information can be generated according to the predicted risk level.

[0101] For example, if the predicted risk level is high risk, the alarm information is generated based on the short message; if the predicted risk level is low risk, the alarm information is generated based on the pop-up window.

[0102] S605, determining the contact information of the pickup person associated with the target express to be picked up.

[0103] In some implementations, in order to alarm the user of the pickup abnormality to ensure the safety of the pickup process and avoid the occurrence of mis-pickup and theft of express, the pickup person information of the target express to be picked up can be determined, and alarm information can be sent to the pickup person.

[0104] Optionally, the express information of the target express can be determined based on the pickup code input by the user. Alternatively, the express information currently associated with the cabinet identifier of the express cabinet currently picked up by the user can be determined. The express information at least includes the contact information.

[0105] S606, sending the alarm information to the terminal device of the pickup person based on the contact information.

[0106] Optionally, after obtaining the contact information of the pickup person, the alarm information can be sent to the terminal device of the pickup person through the contact information. Optionally, the contact information can be a phone number, a terminal device identifier, or the like.

[0107] In some implementations, in order to determine whether there is really an abnormality to reduce the possibility of false alarm, after sending the alarm information, the user can confirm the alarm information to obtain abnormality confirmation information fed back by the terminal device.

[0108] Optionally, in response to the abnormality confirmation information being not abnormal, the alarm information is marked as false alarm, and the multi-modal data corresponding to the false alarm alarm information is obtained as sample data for model updating. Further, the large model is optimized based on the sample data for model updating to improve the accuracy of model detection.

[0109] Optionally, in response to the exception confirmation information being an exception, alarm information is sent to a delivery sender associated with the target express, so that the delivery sender can discover and correct errors in time, such as address errors, inconsistent recipient information, etc., thereby improving the accuracy of express service.

[0110] The express anti-theft method provided by the embodiments of the present disclosure acquires multi-modal data of a user, analyzes the multi-modal data using a large model, verifies the identity and behavior of the user, and determines a pick-up verification result corresponding to the user. Then, whether the pick-up is abnormal can be determined according to the pick-up verification result, and the user is alarmed in case of an exception, so as to ensure the safety of the pick-up process and avoid the occurrence of mis-pick-up or theft of express. By verifying the pick-up of the user, mis-pick-up or theft of express can be effectively prevented, so as to ensure that the express can be accurately and correctly delivered to the real recipient, reduce the delay or failure of delivery due to information errors, and improve user satisfaction. The large model can automatically perform face recognition and behavior analysis, reducing the dependence on manual review and improving the operation efficiency of the express station.

[0111] As shown in the flowchart of the abnormality alarm process in FIG. 1. Figure 8 As shown in the flowchart of the abnormality alarm process in FIG. 1.

[0112] As shown in the flowchart of the abnormality alarm process in FIG. 1. Figure 9 As shown in the flowchart of the abnormality alarm process in FIG. 1.

[0113] Corresponding to the express anti-theft method provided by the above several embodiments, an embodiment of the present disclosure also provides an express anti-theft device. Since the express anti-theft device provided by the embodiment of the present disclosure corresponds to the express anti-theft method provided by the above several embodiments, the implementation of the above express anti-theft method is also applicable to the express anti-theft device provided by the embodiment of the present disclosure, which will not be described in detail in the following embodiments.

[0114] Figure 10 A structural schematic diagram of an express anti-theft device provided by an embodiment of the present disclosure.

[0115] As shown in Figure 10 , the express anti-theft device 1000 of the embodiment of the present disclosure includes an acquisition module 1001, a verification module 1002, a pick-up module 1003, and an alarm module 1004.

[0116] The acquisition module 1001 is configured to acquire multi-modal data of a user when picking up an express, wherein the multi-modal data includes at least one type of biological feature data, pick-up behavior data, and pick-up environment data of the user.

[0117] The verification module 1002 is configured to make a decision on whether the pick-up is abnormal based on a pre-trained large model, and obtain a pick-up verification result corresponding to the user.

[0118] The pick-up module 1003 is configured to execute a pick-up process in response to the pick-up verification result being normal.

[0119] The alarm module 1004 is configured to issue a pick-up abnormality alarm information in response to the pick-up verification result being abnormal.

[0120] In an embodiment of the present disclosure, the verification module 1002 is further configured to perform feature extraction on the multi-modal data based on the large model, and perform identity verification and pick-up behavior verification on the user based on the extracted feature information to obtain the pick-up verification result.

[0121] In an embodiment of the present disclosure, the verification module 1002 is further configured to perform feature extraction on the multi-modal data based on the large model to obtain an identity feature vector and a behavior feature vector, perform identity verification on the user according to the identity feature vector to obtain a first verification result, perform pick-up behavior verification on the user according to the behavior feature vector to obtain a second verification result, and generate the pick-up verification result according to the first verification result and the second verification result.

[0122] In an embodiment of the present disclosure, the verification module 1002 is further configured to: perform feature extraction on the multi-modal data to obtain multi-dimensional modal features of the modal data; perform fusion on the multi-dimensional modal features of the modal data to obtain fused modal features of the modal data; and perform cross-modal attention on the fused modal features of the multi-modal data to obtain the identity feature vector.

[0123] In an embodiment of the present disclosure, the verification module 1002 is further configured to: obtain identity information of a pickup person associated with a target express to be picked up; obtain an identity reference feature vector corresponding to biological feature data of the pickup person according to the identity information; and perform similarity matching on the identity feature vector and the identity reference feature vector to obtain the first verification result.

[0124] In an embodiment of the present disclosure, the verification module 1002 is further configured to: perform feature extraction on the multi-modal data to obtain a pickup posture, action feature and pickup spatiotemporal information of the user; and obtain the behavior feature vector according to the pickup posture, action feature and pickup spatiotemporal information of the user.

[0125] In an embodiment of the present disclosure, the verification module 1002 is further configured to: obtain identity information of a pickup person associated with a target express to be picked up; obtain historical pickup behavior data of the pickup person according to the identity information, and obtain a historical behavior feature vector of the pickup person according to the historical pickup behavior data; in response to matching of the behavior feature vector and the historical feature vector, determine that the pickup behavior of the user is not abnormal as the second verification result; and in response to non-matching of the behavior feature vector and the historical feature vector, determine that the pickup behavior of the user is abnormal as the second verification result.

[0126] In an embodiment of the present disclosure, the verification module 1002 is further configured to: in response to non-matching of the behavior feature vector and the historical feature vector, obtain a timing feature of the pickup person according to the historical pickup behavior data; obtain context information related to space and time according to the pickup environment data; and predict whether the pickup behavior of the user is abnormal according to the behavior feature vector, the timing feature and the context information, to obtain the second verification result.

[0127] In an embodiment of the present disclosure, the alarm module 1004 is further configured to: obtain a predicted risk level of the pickup abnormality output by the large model, and generate the alarm information according to the predicted risk level; obtain contact information of a pickup person associated with a target express to be picked up; and send the alarm information to a terminal device of the pickup person based on the contact information.

[0128] In an embodiment of the present disclosure, the alarm module 1004 is further configured to: determine the express information of the target express based on the pick-up code input by the user; or determine the express information currently associated with the cabinet identifier of the express cabinet currently picked up by the user, wherein the express information at least includes the contact information.

[0129] In an embodiment of the present disclosure, the alarm module 1004 is further configured to: obtain the abnormality confirmation information fed back by the terminal device; in response to the abnormality confirmation information being non-abnormal, mark the alarm information as false alarm; and in response to the abnormality confirmation information being abnormal, send alarm information to the express sender associated with the target express.

[0130] In an embodiment of the present disclosure, the alarm module 1004 is further configured to: obtain the multi-modal data corresponding to the false alarm alarm information as sample data for model updating; and optimize the large model based on the sample data for model updating.

[0131] In an embodiment of the present disclosure, the verification module 1002 is further configured to: based on the multi-modal data and the pick-up verification result, generate a pick-up log file, and store the pick-up log file in a first database and store the multi-modal data in a second database.

[0132] In an embodiment of the present disclosure, the verification module 1002 is further configured to: receive a data traceability request based on a query interface, and extract data from at least one of the first database and the second database according to the data traceability request to obtain a data traceability result.

[0133] The express anti-theft device provided by the embodiments of the present disclosure can obtain the multi-modal data of the user, analyze the multi-modal data using a large model, verify the identity and behavior of the user, and determine the pick-up verification result corresponding to the user. Then, whether the pick-up is abnormal can be determined according to the pick-up verification result to execute corresponding processes. By verifying the pick-up of the user, the express can be effectively prevented from being mispicked or stolen, so as to ensure that the express can be accurately and correctly delivered to the real recipient, reduce the delivery delay or failure caused by information errors, and improve the user satisfaction. The large model can automatically perform face recognition and behavior analysis, reducing the dependence on manual review and improving the operation efficiency of the express post station.

[0134] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0135] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0136] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0137] As shown in Figure 11 The device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes in accordance with computer programs / instructions stored in a read-only memory (ROM) 1102 or loaded from a storage unit 1106 into a random access memory (RAM) 1103. Various programs and data required for operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other by a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0138] Various components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106 such as a keyboard, a mouse, etc.; an output unit 1107 such as various types of displays, speakers, etc.; a storage unit 1108 such as a magnetic disk, an optical disk, etc.; and a communication unit 1109 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0139] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs various methods and processes described above, such as the express theft prevention method. For example, in some embodiments, the express theft prevention method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1106. In some embodiments, part or all of the computer program / instructions can be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program / instructions are loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the express theft prevention method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the express theft prevention method by any other suitable means, such as by means of firmware.

[0140] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0141] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0142] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0144] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0145] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0146] It should be understood that the various forms of flow shown above can be re-ordered, added to, or have steps deleted, using the steps disclosed in the present disclosure. For example, the steps disclosed in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure are achieved, and the present disclosure is not limited herein.

[0147] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method of preventing theft of a courier, wherein, The method comprises: acquiring multi-modal data of the user when picking up the package, wherein the multi-modal data comprises pick-up behavior data, pick-up environment data and at least one type of biological feature data of the user; performing feature extraction on the multi-modal data based on a pre-trained large model to obtain an identity feature vector and a behavior feature vector, wherein the behavior feature vector is obtained according to pick-up posture, action features and pick-up space-time information of the user; verifying the identity of the user according to the identity feature vector to obtain a first verification result; the behavior feature vector is used to verify whether the behavior of the user matches historical behavior, and the behavior feature vector and a historical behavior feature vector associated with a pick-up person of a target express to be picked up are matched, so as to determine that the second verification result is that the pick-up behavior of the user is not abnormal; if the behavior feature vector and the historical behavior feature vector are not matched, the historical pick-up behavior data of the pick-up person is acquired to obtain a time sequence feature of the pick-up person, wherein the time sequence feature is pick-up time; context information related to space-time is acquired according to the pick-up environment data, wherein the context information comprises at least one of weather and holidays; the behavior feature vector, the time sequence feature and the context information are input into a long short-term memory network (LSTM) to predict whether the pick-up behavior of the user is abnormal, so as to obtain the second verification result; generating a pick-up verification result according to the first verification result and the second verification result; performing a pick-up process in response to the pick-up verification result being normal; issuing a pick-up abnormality alarm information in response to the pick-up verification result being abnormal.

2. The method of claim 1, wherein, The process of obtaining the identity feature vector comprises: performing feature extraction on the modal data in the multi-modal data to obtain multi-dimensional modal features of the modal data; fusing the multi-dimensional modal features of the modal data to obtain fused modal features of the modal data; performing cross-modal attention on the fused feature modal of the multi-modal data to obtain the identity feature vector.

3. The method of claim 1, wherein, The process of obtaining the identity feature vector comprises: acquiring identity information of a pick-up person associated with a target express to be picked up; acquiring an identity reference feature vector corresponding to biological feature data of the pick-up person according to the identity information; performing similarity matching on the identity feature vector and the identity reference feature vector to obtain the first verification result.

4. The method of claim 1, wherein, The process of obtaining the behavior feature vector comprises: performing feature extraction on the multi-modal data to obtain pick-up posture, action features and pick-up space-time information of the user; acquiring the behavior feature vector according to the pick-up posture, action features and pick-up space-time information of the user.

5. The method of claim 1, wherein, The method further comprises: acquiring identity information of a pick-up person associated with a target express to be picked up; acquiring historical pick-up behavior data of the pick-up person according to the identity information; acquiring a historical behavior feature vector of the pick-up person according to the historical pick-up behavior data.

6. The method of any one of claims 1-5, wherein, The process of issuing a pick-up abnormality alarm information in response to the pick-up verification result being abnormal comprises: obtain a predicted risk level of the pickup exception of the large model output, and generate the warning information according to the predicted risk level; determine contact information of a pickup person associated with a target express to be picked up; send the warning information to a terminal device of the pickup person based on the contact information.

7. The method of claim 6, wherein, The determination of the contact information of the pickup person associated with the target express to be picked up comprises: determining express information of the target express based on the pickup code input by the user; or determining the express information currently associated with the cabinet identifier based on the cabinet identifier of the express cabinet currently picked up by the user; wherein the express information at least includes the contact information.

8. The method of claim 6, wherein, After the sending of the warning information to the terminal device of the pickup person based on the contact information, the method further comprises: obtaining abnormality confirmation information fed back by the terminal device; in response to the abnormality confirmation information being non-abnormal, marking the warning information as a false alarm; in response to the abnormality confirmation information being abnormal, sending warning information to a sender associated with the target express.

9. The method of claim 8, wherein, After the false alarm marking of the warning information, the method further comprises: obtaining multi-modal data corresponding to the false alarm warning information as sample data for model updating; optimizing the large model based on the sample data for model updating.

10. The method of any one of claims 1-5, wherein, After the obtaining of the pickup verification result corresponding to the user, the method further comprises: based on the multi-modal data and the pickup verification result, generating a pickup log file, and storing the pickup log file in a first database and storing the multi-modal data in a second database.

11. The method of claim 10, wherein, The method further comprises: receiving a data traceability request based on a query interface, and extracting data in at least one of the first database and the second database according to the data traceability request to obtain a data traceability result.

12. A theft prevention device for express deliveries, wherein, The device comprises: an acquisition module configured to acquire multi-modal data when a user picks up, wherein the multi-modal data comprises pickup behavior data, pickup environment data and at least one type of biological feature data of the user; a verification module configured to perform feature extraction on the multi-modal data based on a pre-trained large model to obtain an identity feature vector and a behavior feature vector, the behavior feature vector being obtained based on a pickup posture, motion features and pickup space-time information of the user; verify the identity of the user based on the identity feature vector to obtain a first verification result; the behavior feature vector is used to verify whether the behavior of the user matches historical behavior, and the behavior feature vector and a historical behavior feature vector of a pickup person associated with a target express to be picked up are matched to determine a second verification result that the pickup behavior of the user is non-abnormal; the behavior feature vector and the historical behavior feature vector are not matched, and the time sequence feature of the pickup person is obtained based on historical pickup behavior data of the pickup person, the time sequence feature being pickup time; context information related to space-time is obtained based on the pickup environment data, the context information including at least one of weather and holidays. inputting the behavior feature vector, the timing feature and the context information into an LSTM, predicting whether the pickup behavior of the user is abnormal, to obtain a second verification result; generating a pickup verification result according to the first verification result and the second verification result; a pickup module configured to execute a pickup process in response to the pickup verification result being normal; an alarm module configured to issue a pickup abnormality alarm information in response to the pickup verification result being abnormal. 13.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method according to any one of claims 1-11.

15. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the method of any one of claims 1-11.

Images