Identity identification method, device, program product, medium and equipment

Abstract

One or more embodiments of the present specification provide an identity recognition method, apparatus, program product, medium, and device, wherein the identity recognition method comprises: for a completed delivery waybill, obtaining call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and a user's comment text on the delivery capacity to be identified; calling a trained multimodal model, wherein the multimodal model is used to extract frequency domain features of the call audio and text features of the comment text and, after fusing them, predict identity category information of the delivery capacity to be identified; and performing identity recognition on the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery capacity to be identified.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to identity recognition methods, devices, program products, media and equipment. Background Art

[0002] Delivery services are widely used in online shopping, food delivery, errand shopping and other scenarios. For example, in food delivery and other scenarios, after the user places an order using the client, the server will quickly allocate delivery capacity for the delivery order; the delivery capacity will arrive at the store, pick up the items to be delivered and deliver them to the delivery location specified by the user. Among them, the server's identity recognition of the delivery capacity can ensure the safety of the delivery capacity, the store, the user and the delivered items. Based on this, it is necessary to provide an identity recognition solution with a high accuracy rate. Summary of the invention

[0003] To overcome the problems existing in the related art, the embodiments of this specification provide identity recognition methods, devices, program products, media and equipment.

[0004] According to a first aspect of an embodiment of this specification, there is provided an identity recognition method, including:

[0005] For a completed delivery waybill, obtain the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill and the user's comment text on the delivery transport capacity to be identified;

[0006] Calling a trained multimodal model, the multimodal model is used to extract frequency domain features of the call audio and text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified;

[0007] The identity of the delivery transport capacity to be identified is identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified.

[0008] In some examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different regional information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the regional information of the geographical area to which the delivery capacity to be identified belongs to make predictions.

[0009] In some examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction.

[0010] In some examples, the training data for the multimodal model includes: historical commentary text containing personal names.

[0011] In some examples, the comment text includes one or more characters, and the text feature includes a pinyin feature of each character.

[0012] In some examples, the multimodal model is further used to: output the confidence of the predicted identity category information; and identify the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery capacity to be identified, including:

[0013] If the confidence level is greater than a preset threshold, the delivery transport capacity to be identified is identified according to the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified;

[0014] If the confidence level is less than a preset threshold, the prediction result is determined to be invalid and / or the prediction result is sent to a preset terminal.

[0015] In some examples, the identification of the delivery transport capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified includes:

[0016] If the identity category information predicted by the multimodal model matches the identity category information registered by the to-be-identified delivery transport capacity, it is determined that the to-be-identified delivery transport capacity has passed identity identification;

[0017] If the identity category information predicted by the multimodal model does not match the identity category information registered by the delivery transport capacity to be identified, it is determined that the delivery transport capacity to be identified has failed identity identification.

[0018] In some examples, after the step of determining that the to-be-identified delivery capacity has not passed identity identification, the method further includes:

[0019] A reminder message indicating that the delivery capacity to be identified has not passed identity identification is output to a preset terminal, and the reminder message carries at least one of the following information: identity category information registered for the delivery capacity, identity category information predicted by the multimodal model, confidence level of the identity category information predicted by the multimodal model, identification of the delivery capacity to be identified, identification of the delivery waybill, or delivery time of the delivery waybill.

[0020] According to a second aspect of an embodiment of this specification, there is provided an identity recognition method, including:

[0021] For a completed delivery waybill, obtaining the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery transport capacity belongs;

[0022] Calling a trained identity category prediction model, the identity category prediction model includes an attention module based on an attention mechanism, the identity category prediction model is used to utilize the influence of different regional information learned by the attention module in the training phase on the frequency domain characteristics of the delivery capacity of different identity categories, and predict the identity category information of the delivery capacity to be identified by extracting the frequency domain characteristics of the call audio and combining the regional information;

[0023] The identity of the delivery transport capacity to be identified is identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the delivery transport capacity to be identified.

[0024] In some examples, when predicting the identity category information of the delivery capacity to be identified, the identity category prediction model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction.

[0025] According to a third aspect of the embodiments of this specification, there is provided an identity recognition device, the device comprising:

[0026] An acquisition module is used to: for a completed delivery waybill, acquire the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill and the user's comment text on the delivery transport capacity to be identified;

[0027] A prediction module is used to: call a trained multimodal model, wherein the multimodal model is used to extract frequency domain features of the call audio and text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified;

[0028] The identification module is used to: identify the distribution capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the distribution capacity to be identified.

[0029] According to a fourth aspect of the embodiments of this specification, there is provided an identity recognition device, the device comprising:

[0030] An acquisition module is used to: for a completed delivery waybill, acquire the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery capacity belongs;

[0031] A prediction module is used to: call a trained identity category prediction model, wherein the identity category prediction model includes an attention module based on an attention mechanism, wherein the identity category prediction model is used to utilize the influence of different regional information learned by the attention module during the training phase on the frequency domain characteristics of the delivery capacity of different identity categories, and predict the identity category information of the delivery capacity to be identified by extracting the frequency domain characteristics of the call audio and combining the regional information;

[0032] The identification module is used to: identify the distribution capacity to be identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the distribution capacity to be identified.

[0033] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0034] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method embodiments described in the first or second aspect are implemented.

[0035] According to the seventh aspect of the embodiments of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method embodiments described in the aforementioned first aspect or the second aspect are implemented.

[0036] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0037] In the embodiment of this specification, it is designed to identify the identity of the delivery capacity from the dimension of identity category, and after extracting the frequency domain features of the call audio and the text features of the comment text using a multimodal model, the identity category information of the delivery capacity is predicted. Considering that the voice characteristics of people of different identity categories are quite different, this embodiment is designed to use frequency domain features to distinguish the identity category of the delivery capacity, and considering that the user's evaluation text of the delivery waybill contains the user's personal evaluation of the delivery capacity, the user's comment text information is integrated to predict the identity category of the delivery capacity; since the call audio and the comment text belong to information of different modes, a multimodal model is designed to integrate the call audio and the comment text to identify the identity category information, and a feature layer fusion method is designed to extract features from the two modal data of the call audio and the comment text respectively, and then fuse the features extracted from each modality, and predict the identity category of the delivery capacity based on the fusion result. Since the information of the call audio and the comment text is fused, the information of the two different modalities affects and complements each other, so that the multimodal model has better recognition accuracy and robustness.

[0038] The scheme of this embodiment is to identify the identity from the dimension of the delivery capacity identity category, which can provide highly reliable information for the delivery capacity identity fraud. In addition, compared with the aforementioned face recognition detection before the delivery capacity starts working and the face recognition random inspection during the order delivery process, these two methods are carried out at a certain time, and the delivery capacity may deliver multiple orders for a long time in a day, and there is a risk of missed detection. In this embodiment, considering that the delivery capacity often calls the user when delivering the waybill, such as communicating with the user about the waybill, such as informing the user that the item has been placed at a certain pickup point, or informing the user to pick up the meal when it is about to arrive, etc., therefore, using the call audio of the delivery capacity during the delivery process of the waybill and the user's comment text, it is possible to cover more delivery waybills and improve the coverage of identity recognition.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0041] Figure 1 It is a schematic diagram of a delivery scenario shown in this specification according to an exemplary embodiment.

[0042] Figure 2A It is a flow chart of an identity recognition method shown in this specification according to an exemplary embodiment.

[0043] Figure 2B It is a schematic diagram of a multimodal model shown in this specification according to an exemplary embodiment.

[0044] Figure 3 It is a flow chart of an identity recognition method shown in this specification according to an exemplary embodiment.

[0045] Figure 4 It is a block diagram of an identity recognition device shown in this specification according to an exemplary embodiment.

[0046] Figure 5 It is a block diagram of an identity recognition device shown in this specification according to an exemplary embodiment.

[0047] Figure 6 It is a block diagram of another identity recognition device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0049] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0051] like Figure 1Shown is a schematic diagram of a delivery scenario shown in this specification according to an exemplary embodiment, including a service provider, delivery capacity and users. Among them, the service provider has built a server and provided a client to users, and users can use the services provided by the service provider through the client; the service provider also provides a client for the merchant to use, and the merchant can use the services provided by the service provider through the client. Among them, each user can trade with the merchant through the client, and each transaction order corresponds to a delivery waybill. The service provider can allocate delivery capacity for the delivery waybill. The delivery capacity of this embodiment refers to the party with delivery capabilities. This embodiment includes delivery capacity, commonly known as riders. The delivery capacity can communicate with the server through the client used for the delivery capacity.

[0052] The service end may have tens of millions of delivery orders every day that require delivery capacity to deliver. The security of the delivery capacity's account is an important guarantee direction in offline security risk management. The inventor found that in actual applications, there may be a situation where the delivery capacity registers an account and then gives the account to other people for use, that is, the person who actually runs the delivery order in a delivery order is not the same person as the actual registrant of the account. This embodiment is called the problem of identity fraud of the delivery capacity. The problem of identity fraud of the delivery capacity will increase the management and operation costs of the service provider, and also increase the offline security probability or processing costs of the merchant side, the user side and the delivery capacity side, and will also bring about problems such as disrupting the fairness of the delivery capacity's order acceptance; or, if the delivery capacity encounters safety problems such as traffic accidents during the delivery process, the identity of the delivery capacity also needs to be accurately identified in some scenarios.

[0053] Some solutions are to have the service side conduct relevant security control throughout the entire life cycle of the delivery capacity, including registration, start of work, and delivery of orders. For example, there are background checks and other access controls when the delivery capacity is registered, and there are face scans to confirm the identity before starting work. During the delivery process, the delivery capacity will also be subject to face scan spot checks to confirm the authenticity of the rider's identity. However, the above solutions still have related risks of omission or skipping, such as when the delivery capacity passes the identity recognition at the start of work and then hands the terminal to other people to run the order, etc. Therefore, how to realize the identity recognition of the delivery capacity is a technical problem that needs to be solved urgently.

[0054] Next, the embodiments of this specification are described in detail.

[0055] like Figure 2A As shown, Figure 2A This is a flowchart of an identity recognition method according to an exemplary embodiment of the present specification, comprising the following steps:

[0056] In step 202, for the completed delivery waybill, the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill and the user's comment text on the delivery transport capacity to be identified are obtained;

[0057] In step 204, the trained multimodal model is called, and the multimodal model is used to extract the frequency domain features of the call audio and the text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified;

[0058] In step 206, the delivery transport capacity to be identified is identified based on the identity category information of the delivery transport capacity predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified.

[0059] This embodiment involves a trained multimodal model, which refers to a multimodal machine learning (MMML) model. This type of model can process and understand multi-source modal information through machine learning methods, where modality refers to the source or form of information, and multi-source modal information refers to multiple information with multiple sources or forms, that is, the multimodal model can process at least two types of information with different sources or forms.

[0060] The input of the multimodal model of this embodiment includes the call audio of the delivery capacity during the delivery process of the completed delivery waybill and the user's comment text on the delivery capacity to be identified. As an example, the call audio can be collected by the client of the delivery capacity with the authorization and consent of the delivery capacity, and the client can perform data protection locally through privacy computing and other methods. As an example, the user's comment text on the delivery capacity to be identified can be submitted to the server by the user using the client in an anonymous or public manner. Optionally, the server can also send the comment text to the client of the delivery capacity by encryption and other methods. In actual applications, any information related to the user / delivery capacity involved in this embodiment is obtained with the authorization and consent of the user / delivery capacity, and information desensitization and information protection can be performed through privacy computing and other technologies.

[0061] The trained multimodal model in this embodiment can be configured in the client or in the server. For example, taking the trained multimodal model configured in the client as an example, the scheme of this embodiment can be executed by the client of the distribution capacity, which collects the call audio and obtains the comment text, and then calls the multimodal model to obtain the identity category information of the distribution capacity predicted by the model; since the call audio collected locally by the client can be sent to the server without going through the network, the user's personal information security can be guaranteed.

[0062] In this embodiment, it is designed to identify the identity of the delivery capacity from the dimension of identity category. After extracting the voice features of the call audio and the text features of the comment text using a multimodal model, the identity category information of the delivery capacity is predicted. Considering that the voice characteristics of people of different identity categories are quite different, this embodiment is designed to use voice features to distinguish the identity category of the delivery capacity. At the same time, considering that the user's evaluation text of the delivery waybill contains the user's personal evaluation of the delivery capacity, the user's comment text information is integrated to predict the identity category of the delivery capacity; since the call audio and the comment text belong to information of different modes, a multimodal model is designed to integrate the call audio and the comment text to identify the identity category information, and a feature layer fusion method is designed to extract features from the two modal data of the call audio and the comment text respectively, and then fuse the features extracted from each modality, and predict the identity category of the delivery capacity based on the fusion result. Since the information of the call audio and the comment text is fused, the information of the two different modalities affects and complements each other, so that the multimodal model has better recognition accuracy and robustness.

[0063] The scheme of this embodiment is to identify the identity from the dimension of the delivery capacity identity category, which can provide highly reliable information for the delivery capacity identity fraud. In addition, compared with the aforementioned face recognition detection before the delivery capacity starts working and the face recognition random inspection during the order delivery process, these two methods are carried out at a certain time, and the delivery capacity may deliver multiple orders for a long time in a day, and there is a risk of missed detection. In this embodiment, considering that the delivery capacity often calls the user when delivering the waybill, such as communicating with the user about the waybill, such as informing the user that the item has been placed at a certain pickup point, or informing the user to pick up the meal when it is about to arrive, etc., therefore, using the call audio of the delivery capacity during the delivery process of the waybill and the user's comment text, it is possible to cover more delivery waybills and improve the coverage of identity recognition.

[0064] In practical applications, the above multimodal model can be implemented in a variety of ways as needed. For example, the implementation process of the multimodal model can be: first, a model is represented by modeling, then the model is evaluated by constructing an evaluation function, and finally, the evaluation function is optimized according to sample data and optimization methods to adjust the model to meet the set conditions.

[0065] Exemplarily, the multimodal model may be modeled using methods such as a multi-kernel learning method, a probabilistic graphical model, or a deep neural network model, and the neural network model may include a recursive neural network, an attention-based neural network, or a tensor-based neural network.

[0066] Exemplarily, the training data of the multimodal model can be a data set constructed from historical data that does not distinguish whether there has been identity fraud. Since identity fraud is a low-probability event, it accounts for a small proportion of historical data, and the model can tolerate a small amount of noise samples in the process of learning voiceprints, etc. In other examples, the training data of the multimodal model can also be obtained using historical data of historical delivery waybills where no identity fraud has occurred. For example, the historical call audio of the delivery capacity and the historical evaluation text of the user in the historical delivery waybills can be obtained as samples, and the label of the sample, that is, the identity category information of the delivery capacity, can be determined using the identity category information registered by the delivery capacity. Exemplarily, the identity category information includes category information of female or male, etc.

[0067] In some examples, feature processing is an important step in the model identity category recognition task, which directly affects the overall effect of the model identity category recognition. Therefore, effective and reasonable identity category-related features can be designed to ensure that the identity category-related information of each modality is fully utilized.

[0068] As an example, the audio features of the call audio may include statistical features such as Mel-frequency cepstral coefficients, zero-crossing rate, loudness, etc. extracted from the voice signal, or spectral features extracted from the spectrogram of the voice signal using an image algorithm. In some examples, the three elements of sound are considered to include pitch, timbre, and volume. Among them, timbre, also known as timbre, is determined by the harmonic spectrum and envelope of the sound waveform. It is an important factor in distinguishing the characteristics of sound. It is expressed in physical terms as the shape of the spectrum. Tone refers to the height of the sound, which is determined by the frequency. The higher the frequency, the higher the pitch. It is also an important factor in distinguishing the characteristics of sound. Volume is also known as sound pressure or loudness, which determines the size of the sound. The task of the multimodal model of this embodiment is to identify the identity category of the speaker through the call audio. Based on this, in some examples, the audio features of this embodiment may include frequency domain features, that is, the multimodal model can be designed to extract the voice features of the call audio from the spectrum information of the call audio.

[0069] Exemplarily, in this embodiment, the call audio can be processed by Linear Predictive Cepstral Coefficient Extraction Method (LPCC) or Mel Frequency Cepstral Coefficient Extraction Method (MFCC) and then input into the multimodal model, so that the multimodal model can extract frequency domain features. Among them, MFCC is mainly based on the nonlinear auditory mechanism of human beings, and imitates the function of the human ear to analyze the frequency of speech, so as to better extract the characteristics of speech signals. Mel is a unit of measurement for the perception of pitch or pitch frequency, and 1Mel is 1 / 1000 of the perceived degree of a pitch of 1000Hz. The Mel filter cepstral parameter feature occupies an important position in speech feature extraction, with simple calculation and outstanding distinguishing ability.

[0070] Exemplarily, taking MFCC processing as an example, the processing process may include pre-processing operations such as pre-emphasis, framing, and windowing on the call audio signal; then, discrete Fourier transform is performed on the single frame signal after framing; frequency domain data is obtained; then, the frequency domain data is filtered through w Mel frequency filters to extract the spectrum, Mel filter group and frequency envelope, and then the processed energy spectrum is logarithmized so that the amplitude multiplication in the Fourier transform is converted into addition to obtain logarithmic energy, and finally, the logarithmic energy is substituted into the discrete cosine transform DCT (Discrete Cosine Transform) to obtain the MFCC coefficient. Exemplarily, a call audio can be divided into n frames (the value of n is set according to the actual application), and the m-dimensional MFCC signal is extracted in each frame and stored as an n×m high-dimensional speech feature matrix.

[0071] Optionally, the MFCC processing process can also be set within the model, that is, the input of the multimodal model is the call audio, and the model performs MFCC processing on the input call audio to extract voice features.

[0072] The task of the model in this embodiment is to predict the identity category information of the delivery capacity. The voice characteristics of men and women are different. Therefore, based on this difference, the voice features of the call audio can be used to identify the identity category of the delivery capacity. The inventors found that the model that only uses voice features can already have a certain degree of recognition accuracy, but it still cannot reach a high level of accuracy, and there are still a small number of misidentifications. Further research on the subtle defects of the model is challenging and difficult, because in the field of machine learning, as mentioned above, there are many links involved from modeling to training, such as the selection and processing of sample data, the design of data features, the design of models, the design of loss functions or the design of optimization methods, etc. Differences in any link are factors that may lead to subtle defects in prediction accuracy.

[0073] After extensive research, the inventors found that dialect and age factors can affect the pitch. For example, my country has a vast territory and multiple dialects, that is, different places use different languages ​​to describe the same semantic content. Different dialects not only affect the language itself. The inventors found that due to the pronunciation habits of different dialects, people who are accustomed to different dialects will have differences in pitch height. For example, the dialect habits of people in certain areas make the pitch of men in this area higher than that of men in other areas (using other dialects), and the model only learns the pitch characteristics of most men. Therefore, a small number of samples with high pitches will be easily misidentified as women by the model. Similarly, the inventors found through research that different ages in people of the same identity category will also affect the pitch. Therefore, the voice feature includes information in multiple dimensions. This embodiment also innovatively designs the model to combine the influence of geographical regions and / or age factors on the voice features of different identity categories, and designs an attention mechanism to enable the model to accurately learn the influence.

[0074] As an example, a multimodal model may include an attention module based on an attention mechanism, and the principle of the attention module is to select information closely related to the current task from the global information. A common algorithm framework in a machine learning model is the Encoder-Decoder framework. Taking this framework as an example, the encoder Encoder encodes the input sequence of the model into a representation vector, and the decoder Decoder decodes the representation vector to learn the information hidden in the representation vector, etc., and finally restores it to an output sequence for the model to classify. Among them, in this embodiment, the call audio will last for a certain period of time, and the voice features include the timing information after the audio is framed, so the model can be designed to have long-term information memory dependency. Among them, audio data is time series data, and the time series data features need to be extracted through relevant algorithm structures, such as RNN (Recurrent Neural Network) or Transformer structure. For example, the LSTM model is a special RNN model that uses input gate, forget gate and output gate structure to solve the problem of long-term information memory dependency. This embodiment can also adopt the GRU structure. GRU combines the forget gate and the input gate into a single update gate, and also mixes the cell state and the hidden state. The model structure is simpler than LSTM. In this embodiment, the task of the model is to predict the identity category, that is, to pay more attention to the resonance peak and envelope information in the audio signal, rather than to the context information in the audio signal. The GRU structure is superior to the bidirectional structure and multi-head attention mechanism in the Transform, which is consistent with the task of the model in this embodiment. The GRU structure can pay more attention to the resonance peak and envelope information in the audio signal than the context information in the audio signal.

[0075] The addition of an attention module to the model can guide the model to focus on the information of certain dimensions in the representation vector or the correlation between certain dimensions when the decoder is decoding. That is, attention weights are introduced to the representation vector input to the decoder, and information of different dimensions in the representation vector has different weights.

[0076] In some examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different regional information learned by the attention module during the training phase on the voice characteristics of delivery capacity of different identity categories and the regional information of the geographical area to which the delivery capacity to be identified belongs to make predictions.

[0077] For example, the training data of the multimodal model includes call audio that carries the identity category information of the delivery capacity and the regional information of the geographical area to which the delivery capacity belongs. Optionally, the regional information of the geographical area to which the delivery capacity belongs can be obtained through the native place information registered by the delivery capacity. Optionally, a plurality of different geographical areas can be divided according to the area range that needs to be considered in the actual application, and the regional information of the geographical area to which the delivery capacity belongs can be determined according to the native place information registered by the delivery capacity; wherein, the division of geographical areas can be flexibly set according to the actual application as needed, and this embodiment does not limit this.

[0078] The multimodal model includes an attention module based on an attention mechanism. During the training process of the multimodal model, the attention module can learn the influence of the regional information on the speech features of the delivery capacity of different identity categories because the regional information of the delivery capacity is provided to the model, so that the weight of the extracted speech features can be set based on the regional information. The weight represents the influence of the regional information on the speech features of the delivery capacity of different identity categories. Therefore, when the trained multimodal model is applied, the regional information of the geographical area to which the delivery capacity to be identified belongs can be input into the multimodal model. The multimodal model can accurately predict the identity category information of the delivery capacity to be identified after weighting the extracted speech features based on the regional information and the extracted speech features using the above-mentioned weights learned by the attention model.

[0079] In other examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different age information on the voice characteristics of delivery capacity of different identity categories learned by the attention module during the training phase and the age information of the delivery capacity to be identified for prediction.

[0080] For example, the training data of the multimodal model includes call audio with the identity category information of the delivery transporter and the age information of the delivery transporter. Optionally, the age information of the delivery transporter can be obtained through the information provided when the delivery transporter is registered.

[0081] The multimodal model includes an attention module based on an attention mechanism. During the training process of the multimodal model, the age information of the delivery transport capacity is provided to the model, so that the attention module can learn the influence of the age information on the voice features of the delivery transport capacity of different identity categories, so that the weight of the extracted voice features can be set based on the age information. The weight represents the influence of the age information on the voice features of the delivery transport capacity of different identity categories. Therefore, when the trained multimodal model is applied, the age information of the delivery transport capacity to be identified can be input into the multimodal model. The multimodal model uses the age information and the extracted voice features to weight the extracted voice features using the above weights learned by the attention model, and predicts the identity category information of the delivery transport capacity to be identified.

[0082] The above embodiments respectively provide an attention mechanism for regional information and an attention mechanism for age information. In practical applications, the multimodal model may also include an attention module for both regional information and age information. The specific implementation method may refer to the above embodiments.

[0083] In actual applications, when multimodal models are applied, if identity impersonation occurs, the speaker in the call audio is someone else, and the regional information / age information used in model prediction is determined using the delivery capacity registration information. Therefore, it is possible that the actual speaker in the audio is inconsistent with the registration information.

[0084] Taking regional information as an example, the regional information used in model prediction is determined by the native place information registered by the delivery transport capacity, that is, it is possible that the actual speaker in the call audio is inconsistent with the native place information registered by the delivery transport capacity. For example, the native place of the delivery transport capacity assigned to the delivery waybill is A, and the waybill is handed over to other people by the delivery transport capacity to run the order. The native place of the other person is B, but the regional information used in model prediction is determined by the native place A of the delivery transport capacity. On the one hand, this phenomenon is less likely to occur in actual applications. When identity fraud occurs, the delivery transport capacity needs to hand over the electronic device to other people. The delivery transport capacity has a certain social relationship with other people, and there are fewer cases where their native places are very different; on the other hand, the area of ​​this embodiment covers a certain geographical range. Even if the native places are different, they may still belong to the same regional information. In addition, the native place information and age information of this embodiment play a role in the attention mechanism, that is, as auxiliary information to assist the model in prediction, so in the event of inconsistency, it will not have a significant impact on the prediction of the model.

[0085] In some examples, the call audio may be acquired by the client after detecting that the delivery transport capacity is talking to the user in the delivery waybill. In the call process, at least two speakers, the delivery transport capacity and the user, are involved. The task of the multimodal model of this embodiment is to identify the identity category of the delivery transport capacity. In some examples, the call audio input to the multimodal model may be processed call audio in which only the delivery transport capacity speaks. Optionally, call audio in which only the delivery transport capacity speaks and no user speaks may be acquired in a variety of ways.

[0086] In some examples, the call audio in which only the delivery capacity speaks can be extracted from the collected call audio containing the delivery capacity and the user. For example, the electronic device where the delivery capacity client is located includes a microphone module and a playback module. During the call, the operator will record the content of the call, and record the caller, the called party, the call signal of the caller, and the call signal of the called party. After the microphone module collects the sound wave signal and converts it into an analog signal, it is output by the processor of the electronic device through the antenna; the processor obtains the signal from the user end through the antenna, and outputs it by the playback module after processing. The operator fixedly places the caller's call analog signal in the left audio channel and the callee's call analog signal in the right channel. The client can use the function provided by the electronic device to record the audio of the call process. Combined with the calling and called behaviors recorded by the client, the signal of only the delivery capacity can be extracted by distinguishing the left channel signal and the right channel signal of the dual-channel signal of the call.

[0087] In other examples, the electronic device where the delivery capacity client is located includes a microphone module and a playback module. The call audio collected by the delivery capacity client in this embodiment may be the sound collected by the microphone module, without collecting the sound played by the playback module, thereby obtaining call audio with only the delivery capacity sound.

[0088] In this embodiment, it is considered that the user's comment text on the delivery capacity may involve information related to the identity category, such as "The rider is very punctual" or "Thank you for the delivery, the lady has worked hard", etc. Comment text, such as "Brother" or "Sister", etc., represents the user's name for the delivery capacity, and these name information is related to the identity category of the delivery capacity. Therefore, in this embodiment, a multimodal model is designed to comprehensively consider the user's comment text on the delivery capacity to identify the identity category of the delivery capacity.

[0089] Exemplarily, the training data of the multimodal model may include historical comment texts obtained from historical delivery waybills. During model training, the model will learn whether the comment texts carry address information, and determine whether the address information is a male or female address, thereby determining the identity category of the delivery capacity. Based on this, the training data of the multimodal model of this embodiment may include historical comment texts that carry personal addresses. Optionally, in actual applications, the user's comment texts on the delivery capacity may not include address information, and therefore, the training data of the multimodal model may also include historical comment texts that do not carry personal addresses.

[0090] There are many ways to extract text features of comment texts in practical applications. For example, the multimodal model includes a module for extracting text features of comment texts, which module can be textCNN, DCNN (Dynamic Convolutional Neural Network), RCNN (Recurrent Convolutional Neural Network) or HAN (Hierarchical Attention Networks), etc.

[0091] Exemplarily, in order to quickly and accurately obtain the feature representation of the comment text, the module for processing the comment text in the multimodal model of this embodiment can use the pre-trained BERT model (Bidirectional Encoder Representations from Transformer) to learn the feature representation of the comment text. The BERT model is used. On the one hand, the model has been pre-trained based on a large-scale data set to obtain a general language representation, so in practical applications, it can be fine-tuned according to the identity category recognition task of this embodiment; on the other hand, the BERT model uses a Transformer structure, and its ability to understand text is relatively strong.

[0092] For example, the BERT model includes three embedding layers, which are used to extract the word embeddings (token embeddings), paragraph embeddings (segment embeddings) and position embeddings (position embeddings) of the input comment text. Bert adopts the encoder structure of transformer, and the encoder encoder usually includes Multi-Head Attention (multi-head attention) and fully connected layers; the attention mechanism in transformer adopts the self-attention mechanism Self-Attention, and it is a multi-head attention mechanism. Taking the above three embeddings as an example, it is a three-head self-attention mechanism.

[0093] In some examples, for Chinese, users may use the pinyin input method to edit the comment text on the delivery capacity, so homophones may be entered incorrectly, such as "female rider" is mistakenly entered as "female chess player", while in the comment text in the scenario of this embodiment, "female chess player" and "female rider" have the same semantics, that is, the two should be synonyms, but the model will think that the two are different from the semantics of the words themselves. Therefore, due to the homophones caused by pinyin, the model cannot accurately recognize the semantics of the comment text. Based on this, in order to solve the problem that typos in the comment text may cause the model recognition rate to decrease, this embodiment designs a model to extract pinyin features for each character in the comment text, so that the model can combine the pinyin features to understand the semantics of the comment text, thereby more accurately obtaining the tendency of identity category information in the comment text.

[0094] Exemplarily, each character in the comment text can be identified, and the pinyin data corresponding to the comment text can be generated. The pinyin data can be a pinyin sequence composed of the pinyin of each character, so that the model can use the pinyin sequence to extract pinyin features. Optionally, the pinyin can be obtained before calling the model, so that the comment text carries the pinyin of each character and is input into the multimodal model; or the multimodal model can extract the pinyin features after obtaining the pinyin of each character based on the input comment text. Taking the aforementioned BERT as an example, an embedding layer for extracting pinyin information can be added to the existing BERT, and a self-attention mechanism can be designed to add an attention mechanism for pinyin information. In practical applications, the model can be used to extract pinyin features based on the actual network structure used, and this embodiment does not limit this.

[0095] In step 202, there are many ways to implement the multimodal model to extract the voice features of the call audio and the text features of the comment text and fuse them. As an example, joint representations (JointRepresentations) or coordinated representations (CoordinatedRepresentations) can be used to achieve the fusion of features from the two modalities.

[0096] Exemplarily, this embodiment may use joint representation to uniformly map information of multiple modalities into a multimodal vector space. For example, speech features and text features are concatenated and fused to form a multidimensional feature vector space, and the joint representation can retain the independent representation spaces of multiple modalities.

[0097] In other examples, collaborative representation can be used; collaborative representation focuses on capturing the complementarity of multiple modalities, which can be modeling the correlation between multiple modal data and mapping multiple modal information to collaborative space, expressed as f(x1)~f(x m), where “~” represents a collaborative relationship, and m represents the number of modes; correspondingly, the optimization goal of the network is to optimize the collaborative relationship. Therefore, the correlation between speech features and text features can be modeled and mapped to the collaborative space.

[0098] For example, in a delivery order, there may be one or more audio calls between the delivery carrier and the user during the delivery process; in the case of multiple calls, when applying the method of this embodiment, each audio call can be combined with the comment text input into the model for prediction, and the identity is identified based on the prediction result of each audio call by the model and the identity category information of the registered delivery carrier to be identified. Alternatively, multiple audio calls can be merged into one, and combined with the comment text input into the model for prediction, etc.

[0099] In step 206, based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified, there are multiple ways to identify the delivery transport capacity to be identified; illustratively, if the identity category information predicted by the multimodal model matches the identity category information registered by the delivery transport capacity to be identified, it can be considered that the identity of the delivery transport capacity to be identified in the delivery waybill is correct; if the identity category information predicted by the multimodal model does not match the identity category information registered by the delivery transport capacity to be identified, it can be considered that the identity of the delivery transport capacity to be identified in the delivery waybill is incorrect, and there may be a problem of identity fraud. Optionally, other factors can also be combined when performing identity identification.

[0100] For example, the multimodal model is also used to: output the confidence of the predicted identity category information; perform identity identification on the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered for the delivery capacity to be identified, including: if the confidence is greater than a preset threshold, perform identity identification on the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered for the delivery capacity to be identified; if the confidence is less than the preset threshold, determine that the prediction result is invalid and / or send the prediction result to a preset terminal.

[0101] In this embodiment, the output of the multimodal model may include predicted identity category information and the confidence level of the predicted identity category information. The reliability of the identity category information predicted by the model is determined based on the confidence level of the predicted identity category information. For example, if the confidence level of the predicted identity category information is high, it can be determined that the reliability of the identity category information predicted by the multimodal model is high, and the reliability of the identity identification of the distribution capacity to be identified based on this is high. If the confidence level of the predicted identity category information is not high, it can be determined that the reliability of the identity category information predicted by the multimodal model is not high, and the reliability of the identity identification of the distribution capacity to be identified based on this is not high. Based on this, it can be determined that the prediction result is invalid, and / or the prediction result is sent to a preset terminal, that is, the recognition result can be output as a prompt for manual review.

[0102] Exemplarily, the identity identification of the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery capacity to be identified includes: if the identity category information predicted by the multimodal model matches the identity category information registered by the delivery capacity to be identified, determining that the delivery capacity to be identified has passed the identity identification; if the identity category information predicted by the multimodal model does not match the identity category information registered by the delivery capacity to be identified, determining that the delivery capacity to be identified has not passed the identity identification. In this embodiment, by matching the identity category information predicted by the multimodal model with the identity category information registered by the delivery capacity to be identified, a highly reliable identity identification result can be obtained based on this.

[0103] Exemplarily, after the step of determining that the delivery capacity to be identified has not passed identity identification, the method further includes: outputting a reminder message to a preset terminal that the delivery capacity to be identified has not passed identity identification, and the reminder message carries at least one of the following information: identity category information registered for the delivery capacity, identity category information predicted by the multimodal model, confidence in the identity category information predicted by the multimodal model, identification of the delivery capacity to be identified, identification of the delivery waybill, or delivery time of the delivery waybill. In this embodiment, the preset terminal can be a client of the delivery capacity, or it can be a client used by relevant personnel on the service side. Through the above reminder message, a reminder can be output to the delivery capacity, and a message that the identity identification has not passed can also be output to relevant personnel on the service side to remind them of manual review.

[0104] Next, an example will be used for explanation.

[0105] like Figure 2B , which is a structural diagram of a multimodal model shown in the specification according to an exemplary embodiment.

[0106] The input of the multimodal model includes: comment text, MFCC obtained by processing the call audio, region information and age information.

[0107] Exemplarily, the multimodal model includes a BERT module, which is used to extract the following four features from the comment text: word feature, sentence feature, position feature, and pinyin feature.

[0108] The BERT module includes a multi-layer bidirectional Transformer encoder structure. There are multiple Transformer encoders. In the figure, “×N” is used to indicate that there are multiple series-connected Transformer encoders. Figure 2B The Transformer encoder shown in Figure 1 abandons the traditional network structure based on recurrent neural network (RNN) and CNN, and adopts the self-attention mechanism to associate each element in the sequence with all other elements to obtain the internal structure of the entire sequence.

[0109] Exemplarily, the Transformer encoder may include: a multi-head self-attention mechanism sublayer, a vector addition and normalization layer (Add+NOR), a feedforward neural network layer, and a vector addition and normalization layer (Add+NOR). Among them, this embodiment involves four features, and a four-head self-attention mechanism can be adopted. Self-attention means that each element in the input sequence is calculated with all elements in the sentence, helping the encoder to pay attention to all elements in the sentence when encoding each element. Through self-attention, the model can learn the dependency between each element in the sentence and capture the internal structure of the sentence.

[0110] The multimodal model may also include one or more RNN (Recurrent Neural Network) networks for encoding the input MFCC; wherein the RNN network structure may include GRU, BiGRU, LSTM, BiLSTM or BERT, etc. Exemplarily, the LSTM model is a special RNN model that uses input gates, forget gates, and output gates to solve the problem of long-term information memory dependency. This embodiment may use GRU, which combines the forget gate and the input gate into a single update gate, mixes the cell state and the hidden state, and the model structure is simpler than LSTM. After encoding, the pooling layer may be input, and an attention mechanism module is included after the pooling layer; this embodiment adds the attention mechanism of regional information and age information to the audio features, allowing the model to learn the different characteristics of different regions and ages in the frequency domain.

[0111] After the text features and weighted speech features of the comment text pass through the concatenation layer (concat), and then pass through the normalization layer of the residual network and the feedforward neural network layer in turn, the identity category score (sex score) can be output.

[0112] The performance of the multimodal model of this embodiment is also demonstrated through experiments. As mentioned above, based on the identity category recognition task of this embodiment, the speech feature uses frequency domain information, that is, the model pays more attention to the sound characteristics, and the attention to the sound characteristics is higher than the content of the speech itself; in the model, the representation vector of the speech feature is a matrix, the horizontal direction of the matrix is ​​the time series information of the speech, and the vertical direction is the frequency domain information; in the experiment, the attention mechanism can be used to perform attention on the feature matrix and matrix transposition after RNN extraction, and the frequency domain information and time series information are feature weighted respectively. The experimental results show that the weighted learning result of the frequency domain information is better than the weighted learning result of the time series information.

[0113] In addition, the results of GRU are better than other model structures through experimental comparison. As shown in the following table, taking the two performance indicators auc and f1 of the model as an example, the bold part refers to the model structure adopted in this embodiment. The test results show that the auc index and f1 index of the multimodal model mmml of this embodiment can obtain higher scores; in terms of the structure of the encoder, the GRU structure selected in this embodiment also has a relatively high score compared with other structures; the Spe-at-GRU structure of this embodiment also has a relatively high score compared with the Seq-at-GRU structure.

[0114] As shown in the following table:

[0115]

[0116] like Figure 3 FIG. 1 is another method for identifying the identity of a delivery transport capacity according to an exemplary embodiment of the present specification, and includes the following steps:

[0117] In step 302, for the completed delivery waybill, the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery capacity belongs are obtained;

[0118] In step 304, the trained identity category prediction model is called, the identity category prediction model includes an attention module based on an attention mechanism, the identity category prediction model is used to use the influence of different regional information learned by the attention module in the training stage on the frequency domain characteristics of the delivery capacity of different identity categories, and predict the identity category information of the delivery capacity to be identified by extracting the frequency domain characteristics of the call audio and combining the regional information;

[0119] In step 306, the identity of the delivery transport capacity to be identified is identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the delivery transport capacity to be identified.

[0120] The identity category prediction model of this embodiment can be a unimodal machine learning model or a multimodal model as described in the above embodiments.

[0121] Exemplarily, in this embodiment, the call audio may be processed by Linear Predictive Cepstral Coefficient (LPCC) or Mel Frequency Cepstral Coefficient (MFCC) and then input into the identity category prediction model, so that the identity category prediction model can extract frequency domain features. Alternatively, the MFCC processing process may also be set in the model, that is, the input of the identity category prediction model is the call audio, and the model performs MFCC processing on the input call audio and extracts frequency domain features.

[0122] The identity category prediction model of this embodiment includes an attention module, which can guide the model to focus on information of certain dimensions in the representation vector or the correlation between certain dimensions when the decoder is decoding, that is, introduce attention weights for the representation vector input to the decoder, and information of different dimensions in the representation vector has different weights.

[0123] For example, the training data of the identity category prediction model includes call audio that carries the identity category information of the delivery transport capacity and the regional information of the geographical area to which the delivery transport capacity belongs. Optionally, the regional information of the geographical area to which the delivery transport capacity belongs can be obtained through the native place information registered by the delivery transport capacity. Optionally, a plurality of different geographical areas can be divided according to the regional scope that needs to be considered in the actual application, and the regional information of the geographical area to which the delivery transport capacity belongs can be determined according to the native place information registered by the delivery transport capacity.

[0124] During the training process of the identity category prediction model, since the regional information of the delivery capacity is provided to the model, the attention module can learn the impact of the regional information on the frequency domain characteristics of the delivery capacity of different identity categories, so that the weight of the extracted frequency domain features can be set based on the regional information. The weight represents the impact of the regional information on the frequency domain characteristics of the delivery capacity of different identity categories. Therefore, when the trained identity category prediction model is applied, the regional information of the geographical area to which the delivery capacity to be identified belongs can be input into the identity category prediction model. The identity category prediction model uses the regional information and the extracted frequency domain features, and the extracted frequency domain features are weighted by the above-mentioned weights learned by the attention model to predict the identity category information of the delivery capacity to be identified.

[0125] In some examples, when predicting the identity category information of the delivery capacity to be identified, the identity category prediction model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction.

[0126] In this embodiment, in addition to paying attention to the area, an attention mechanism for age information is also added. For example, in the training data of the identity category prediction model, the call audio also carries the age information of the delivery capacity. Optionally, the age information of the delivery capacity can be obtained through the information provided when the delivery capacity is registered.

[0127] During the training process of the identity category prediction model, since the age information of the delivery capacity is provided to the model, the attention module can learn the impact of age information on the frequency domain characteristics of the delivery capacity of different identity categories, so that the weight of the extracted frequency domain features can be set based on the age information. The weight represents the impact of age information on the frequency domain characteristics of the delivery capacity of different identity categories. Therefore, when the trained identity category prediction model is applied, the age information of the delivery capacity to be identified can be input into the identity category prediction model. The identity category prediction model uses the age information and the extracted frequency domain features, and the extracted frequency domain features are weighted by the above weights learned by the attention model to predict the identity category information of the delivery capacity to be identified.

[0128] In the present embodiment, a method for identifying the identity of the delivery capacity is designed from the dimension of identity category. Taking into account that the voice characteristics of people of different identity categories vary greatly, the present embodiment is designed to use frequency domain features to distinguish the identity categories of the delivery capacity. Particularly, taking into account that the pronunciation habits of different dialects may cause differences in pitch among people accustomed to different dialects. Therefore, an attention mechanism for regional information is designed, which can accurately predict the identity category information of the delivery capacity to be identified.

[0129] The scheme of this embodiment is to identify the identity from the dimension of the delivery capacity identity category, which can provide highly reliable information for the delivery capacity identity fraud. In addition, compared with the aforementioned face recognition detection before the delivery capacity starts working and the face recognition random inspection during the order delivery process, these two methods are carried out at a certain time, and the delivery capacity may deliver multiple orders for a long time in a day, and there is a risk of missed detection. In this embodiment, considering that the delivery capacity often calls the user when delivering the waybill, such as communicating with the user about the waybill, such as informing the user that the item has been placed at a certain pickup point, or informing the user to pick up the meal when it is about to arrive, etc., therefore, using the call audio of the delivery capacity during the delivery process of the waybill and the user's comment text, it is possible to cover more delivery waybills and improve the coverage of identity recognition.

[0130] Corresponding to the embodiment of the aforementioned identity recognition method, this specification also provides embodiments of an identity recognition device and a computer used therein.

[0131] The embodiments of the identity recognition device of this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the file processing in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 4 The figure is a hardware structure diagram of the computer device where the identity recognition device of this manual is located, except Figure 4 In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the computer device where the identity recognition device 431 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0132] like Figure 5 As shown, Figure 5 is a block diagram of a device according to an exemplary embodiment of the present specification, wherein the device includes:

[0133] The acquisition module 51 is used to: for a completed delivery waybill, acquire the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and the user's comment text on the delivery capacity to be identified;

[0134] The prediction module 52 is used to: call a trained multimodal model, wherein the multimodal model is used to extract the frequency domain features of the call audio and the text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified;

[0135] The identification module 53 is used to: identify the delivery transport capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified.

[0136] In some examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different regional information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the regional information of the geographical area to which the delivery capacity to be identified belongs to make predictions.

[0137] In some examples, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction.

[0138] In some examples, the training data for the multimodal model includes: historical commentary text containing personal names.

[0139] In some examples, the comment text includes one or more characters, and the text feature includes a pinyin feature of each character.

[0140] In some examples, the multimodal model is further used to: output the confidence of the predicted identity category information; the recognition module is further used to:

[0141] If the confidence level is greater than a preset threshold, the delivery transport capacity to be identified is identified according to the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified;

[0142] If the confidence level is less than a preset threshold, the prediction result is determined to be invalid and / or the prediction result is sent to a preset terminal.

[0143] In some examples, the identification module is further used to:

[0144] If the identity category information predicted by the multimodal model matches the identity category information registered by the to-be-identified delivery transport capacity, it is determined that the to-be-identified delivery transport capacity has passed identity identification;

[0145] If the identity category information predicted by the multimodal model does not match the identity category information registered by the delivery transport capacity to be identified, it is determined that the delivery transport capacity to be identified has failed identity identification.

[0146] In some examples, the device also includes a reminder module, which is used to: after the step of determining that the delivery capacity to be identified has not passed the identity identification, output a reminder message to a preset terminal that the delivery capacity to be identified has not passed the identity identification, and the reminder message carries at least one of the following information: identity category information registered by the delivery capacity, identity category information predicted by the multimodal model, confidence in the identity category information predicted by the multimodal model, an identification of the delivery capacity to be identified, an identification of the delivery waybill, or a delivery time of the delivery waybill.

[0147] like Figure 6 As shown, Figure 6 is a block diagram of another identity recognition device according to an exemplary embodiment of the present specification, the device comprising:

[0148] The acquisition module 61 is used to: for a completed delivery waybill, acquire the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery capacity belongs;

[0149] Prediction module 62, used to: call the trained identity category prediction model, the identity category prediction model includes an attention module based on the attention mechanism, the identity category prediction model is used to use the influence of different regional information learned by the attention module in the training stage on the frequency domain characteristics of the delivery capacity of different identity categories, by extracting the frequency domain characteristics of the call audio and combining the regional information, to predict the identity category information of the delivery capacity to be identified;

[0150] The identification module 63 is used to: identify the delivery transport capacity to be identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the delivery transport capacity to be identified.

[0151] In some examples, when predicting the identity category information of the delivery capacity to be identified, the identity category prediction model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction.

[0152] The implementation process of the functions and effects of each module in the above-mentioned identity recognition device is specifically described in the implementation process of the corresponding steps in the above-mentioned identity recognition method, and will not be repeated here.

[0153] Accordingly, an embodiment of the present specification also provides a computer program product, including a computer program, which implements the steps of the aforementioned identity recognition method embodiment when executed by a processor.

[0154] Correspondingly, an embodiment of the present specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned identity recognition method embodiment when executing the program.

[0155] Accordingly, an embodiment of the present specification further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned identity recognition method embodiment are implemented.

[0156] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0157] The identity identification method of the distribution capacity of this embodiment can be applied to one or more electronic devices, where the electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the electronic device includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0158] The electronic device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0159] The electronic device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0160] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0161] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this application.

[0163] The description of "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0164] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0165] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0166] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. An identity recognition method, comprising: For a completed delivery waybill, obtain the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill, the regional information of the geographical area to which the delivery transport capacity to be identified belongs, and the comment text of the ordering user on the delivery transport capacity to be identified; Calling a trained multimodal model, the multimodal model is used to extract frequency domain features of the call audio and text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified; The training data of the multimodal model includes: historical call audios carrying information on the identity category of the delivery transport capacity and the regional information on the geographical area to which the delivery transport capacity belongs, and historical comment texts carrying personal names; The identity of the delivery transport capacity to be identified is identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified.

2. According to the method described in claim 1, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different regional information learned by the attention module in the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the regional information of the geographical area to which the delivery capacity to be identified belongs to make predictions.

3. According to the method described in claim 1, the multimodal model includes an attention module based on an attention mechanism. When predicting the identity category information of the delivery capacity to be identified, the multimodal model also combines the influence of different age information learned by the attention module in the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified for prediction. 4 . The method according to claim 1 , wherein the comment text comprises one or more characters, and the text feature comprises a pinyin feature of each of the characters.

5. According to the method of claim 1, the multimodal model is further used to: output the confidence of the predicted identity category information; the identity identification of the delivery capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery capacity to be identified comprises: If the confidence level is greater than a preset threshold, the delivery transport capacity to be identified is identified according to the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified; If the confidence level is less than a preset threshold, the prediction result is determined to be invalid and / or the prediction result is sent to a preset terminal.

6. The method according to claim 1 or 5, wherein the identity identification of the delivery transport capacity to be identified is performed based on the identity category information predicted by the multimodal model and the identity category information registered by the delivery transport capacity to be identified, comprising: If the identity category information predicted by the multimodal model matches the identity category information registered by the to-be-identified delivery transport capacity, it is determined that the to-be-identified delivery transport capacity has passed identity identification; If the identity category information predicted by the multimodal model does not match the identity category information registered by the delivery transport capacity to be identified, it is determined that the delivery transport capacity to be identified has failed identity identification.

7. The method according to claim 6, after the step of determining that the to-be-identified delivery capacity has not passed the identity identification, the method further comprises: A reminder message indicating that the delivery capacity to be identified has not passed identity identification is output to a preset terminal, and the reminder message carries at least one of the following information: identity category information registered for the delivery capacity, identity category information predicted by the multimodal model, confidence level of the identity category information predicted by the multimodal model, identification of the delivery capacity to be identified, identification of the delivery waybill, or delivery time of the delivery waybill.

8. A method for identifying a delivery capacity, comprising: For a completed delivery waybill, obtaining the call audio of the delivery transport capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery transport capacity belongs; Calling a trained identity category prediction model, the identity category prediction model includes an attention module based on an attention mechanism, the identity category prediction model is used to utilize the influence of different regional information learned by the attention module in the training phase on the frequency domain characteristics of the delivery capacity of different identity categories, and predict the identity category information of the delivery capacity to be identified by extracting the frequency domain characteristics of the call audio and combining the regional information; The identity of the delivery transport capacity to be identified is identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the delivery transport capacity to be identified.

9. According to the method described in claim 8, when predicting the identity category information of the delivery capacity to be identified, the identity category prediction model also combines the influence of different age information learned by the attention module during the training phase on the frequency domain characteristics of delivery capacity of different identity categories and the age information of the delivery capacity to be identified to make a prediction.

10. A distribution transport capacity identification device, the device comprising: The acquisition module is used to: for a completed delivery waybill, acquire the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill, the regional information of the geographical area to which the delivery capacity to be identified belongs, and the comment text of the ordering user on the delivery capacity to be identified; A prediction module is used to: call a trained multimodal model, wherein the multimodal model is used to extract frequency domain features of the call audio and text features of the comment text and fuse them to predict the identity category information of the delivery capacity to be identified; The training data of the multimodal model includes: historical call audios carrying information on the identity category of the delivery transport capacity and the regional information on the geographical area to which the delivery transport capacity belongs, and historical comment texts carrying personal names; The identification module is used to: identify the distribution capacity to be identified based on the identity category information predicted by the multimodal model and the identity category information registered by the distribution capacity to be identified.

11. A distribution transport capacity identification device, the device comprising: An acquisition module is used to: for a completed delivery waybill, acquire the call audio of the delivery capacity to be identified during the delivery process of the delivery waybill and the regional information of the geographical area to which the delivery capacity belongs; A prediction module is used to: call a trained identity category prediction model, wherein the identity category prediction model includes an attention module based on an attention mechanism, wherein the identity category prediction model is used to utilize the influence of different regional information learned by the attention module during the training phase on the frequency domain characteristics of the delivery capacity of different identity categories, and predict the identity category information of the delivery capacity to be identified by extracting the frequency domain characteristics of the call audio and combining the regional information; The identification module is used to: identify the distribution capacity to be identified based on the identity category information predicted by the identity category prediction model and the identity category information registered by the distribution capacity to be identified.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images