Data processing method, data processing device, storage medium and electronic device
By obtaining and analyzing video and audio data during the task execution process in the online ride-hailing platform, using the probability prediction model to predict the user's abnormal status and abnormal event probability, sending warnings to the task processing terminal, solving the risk of users riding in abnormal status The problem has achieved accurate and timely warning and reduction of risks.
Patent Information
- Application Number
- CN202011176910.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-11-15
AI Technical Summary
In online car-hailing platforms, abnormal events may occur when users ride in abnormal states, increasing the risk of personal or property losses for users, and it is difficult to accurately predict and warn in the existing technology.
By obtaining the video data and audio data collected during the execution of the target task, the probability of the user being in an abnormal state is determined based on the first probability prediction model, and the probability of abnormal events during the execution of the task is determined based on the second probability prediction model, when the probability reaches a predetermined When conditions are met, a warning message is sent to the task processing terminal.
It has achieved a relatively accurate and timely risk warning for tasks with a higher probability of abnormal events, reducing the possibility of user's personal or property being infringed.
Smart Images

Figure CN112329963B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, a data processing device, a storage medium and an electronic device. Background Art
[0002] With the continuous development of computer technology and Internet technology, more and more users choose to publish their car-hailing needs on online car-hailing platforms. Online car-hailing platforms have brought great convenience to users' travel, but sometimes if the user is in an abnormal state, abnormal events will occur during the execution of the car-hailing order, and the occurrence of abnormal events will cause losses to the user's life or property. Summary of the invention
[0003] In view of this, the purpose of an embodiment of the present invention is to provide a data processing method, a data processing device, a storage medium and an electronic device for more accurately determining the probability of abnormal events occurring during task execution, thereby subsequently issuing risk warnings for tasks with a higher probability of abnormal events occurring.
[0004] According to a first aspect of an embodiment of the present invention, a data processing method is provided, the method comprising:
[0005] Acquire task information of a target task, wherein the task information includes target video data and target audio data, wherein the target video data is video data collected during the execution of the target task, and the target audio data is audio data collected during the execution of the target task;
[0006] Based on a first probability prediction model, determining a first probability according to the target audio data and the target video data, wherein the first probability is used to characterize the probability that a target user is in an abnormal state, and the target user is a user corresponding to the target task;
[0007] Based on a second probability prediction model, determining a second probability according to the target video data and the first probability, wherein the second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task;
[0008] In response to the second probability satisfying a predetermined probability condition, a first message is sent to the task processing terminal to alert the task processing resources.
[0009] According to a second aspect of an embodiment of the present invention, there is provided a data processing device, the device comprising:
[0010] A task information acquisition unit, used to acquire task information of a target task, wherein the task information includes target video data and target audio data, wherein the target video data is video data collected during the execution of the target task, and the target audio data is audio data collected during the execution of the target task;
[0011] A first probability determination unit, configured to determine a first probability based on a first probability prediction model and according to the target audio data and the target video data, wherein the first probability is used to characterize a probability that a target user is in an abnormal state, and the target user is a user corresponding to the target task;
[0012] a second probability determination unit, configured to determine a second probability based on a second probability prediction model according to the target video data and the first probability, wherein the second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task;
[0013] The first message sending unit is used to send a first message to the task processing terminal in response to the second probability satisfying a predetermined probability condition, so as to alert the task processing resources.
[0014] According to a third aspect of an embodiment of the present invention, there is provided a computer-readable storage medium storing computer program instructions, wherein the computer program instructions implement the method as described in the first aspect when executed by a processor.
[0015] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, and wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.
[0016] The embodiment of the present invention obtains the video data and audio data collected during the execution of the target task as part of the task information, and then determines the probability of the target user being in an abnormal state based on the audio data and the video data based on the first probability prediction model, and determines the probability of an abnormal event occurring during the execution of the target task based on the video data and the probability of the target user being in an abnormal state based on the second probability prediction model, so that when the probability of an abnormal event occurring during the execution of the target task meets a predetermined probability condition, a first message is sent to the task processing terminal to alert the task processing resources. The embodiment of the present invention uses the probability of the target user being in an abnormal state as an important indicator for predicting the occurrence of abnormal events, and can more accurately and timely issue risk warnings for tasks with a high probability of abnormal events occurring. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0018] Figure 1 is a schematic diagram of the hardware system architecture of an embodiment of the present invention;
[0019] Figure 2 is a flow chart of a data processing method according to a first embodiment of the present invention;
[0020] Figure 3 is a flow chart of determining a first probability in an optional implementation manner of the first embodiment of the present invention;
[0021] Figure 4 is a data flow diagram for determining a second probability using the method of the first embodiment of the present invention;
[0022] Figure 5 is a schematic diagram of a data processing device according to a second embodiment of the present invention;
[0023] Figure 6 is a schematic diagram of an electronic device according to a third embodiment of the present invention. DETAILED DESCRIPTION
[0024] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, flows, components and circuits are not described in detail.
[0025] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0026] Unless the context clearly requires otherwise, words such as “include”, “including” and the like in the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as meaning “including but not limited to”.
[0027] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.
[0028] With the increasing popularity of online ride-hailing services, users can take a ride by posting a ride order on an online ride-hailing platform. Online ride-hailing platforms have brought great convenience to users' travel, but sometimes if the user is in an abnormal state (for example, drunk, unconscious, etc.), it will increase the risk of abnormal incidents (for example, robbery, sexual assault, etc.) during the ride, causing losses to the user's person or property.
[0029] Figure 1 It is a schematic diagram of the hardware system architecture of an embodiment of the present invention. Figure 1 The hardware system architecture shown may include at least one task issuing terminal, at least one server, at least one task processing terminal, at least one transportation tool, and at least one audio and video acquisition device. Figure 1 A task issuing terminal 11, a server 12, a task processing terminal 13, a transportation vehicle 14 and an audio and video acquisition device 15 are taken as an example for description. Figure 1 The task issuing terminal 11 and the task processing terminal 13 shown are terminals with positioning functions, the transportation vehicle 14 can be a vehicle, specifically a vehicle with a collision sensor, and the audio and video acquisition device 15 can be an image acquisition device that independently acquires video data and an audio acquisition device that independently acquires audio data, or an audio and video acquisition device that simultaneously acquires video data and video data, which is not specifically limited in this embodiment. The task issuing terminal 11, the server 12, the task processing terminal 13, and the audio and video acquisition device 15 can establish a communication connection with the server through a network.
[0030] The task release terminal 11 is also a user terminal. The user (that is, the passenger) can release a ride order on the online ride-hailing platform through the task release terminal 11. After the server 12 obtains the user's ride order through the online ride-hailing platform, it can assign the ride order to the task processing resource (that is, the driver) holding the task processing terminal 13, so that the task processing resource can provide the user with a ride service at an appropriate time according to the user's needs. During the execution of the target order (that is, after the passenger gets on the bus), the audio and video acquisition device 15 can collect the video data (that is, the target video data) and audio data (that is, the target audio data) inside the transportation vehicle 14, and send the target video data and the target audio data to the server 12. After obtaining the order information of the target order including the target video data and the target audio data, the server 12 can determine the first probability that the target user is in an abnormal state based on the first probability prediction model, the target audio data and the target video data, and then determine the second probability of an abnormal event occurring during the execution of the target task based on the second probability prediction model, the target video data and the first probability. If the second probability meets the predetermined probability condition, the server 12 sends a first message to the task processing terminal 13 to warn the task processing resource.
[0031] Figure 2 FIG. 1 is a flow chart of a data processing method according to a first embodiment of the present invention. Figure 2 As shown, the method of this embodiment includes the following steps:
[0032] Step S100, obtaining task information of the target task.
[0033] In this embodiment, during the execution of the target task, the audio and video acquisition device can collect video data and audio data of a predetermined time length, and send the collected target video data and target audio data to the server. After receiving the target video data and target audio data, the server can also obtain at least one of the starting position, the end position, the first interest range, the second interest range, the collision data, the first trajectory, and the second trajectory of the target task as the task information of the target task. In this embodiment, the predetermined time length can be a pre-set time length, or it can be the time length of the target task execution process.
[0034] Among them, the first interest range is the interest range to which the starting point position of the target task belongs, the second interest range is the interest range to which the end point position of the target task belongs, the collision data is used to characterize whether a collision occurs during the execution of the target task, the first trajectory is the movement trajectory of the target user, and the second trajectory is the movement trajectory of the task processing resource.
[0035] The interest range is used to reflect the category of the area to which the location belongs, such as a bar, subway station, restaurant, etc. Usually, the frequency of abnormal events occurring in the process of leaving an entertainment venue or going to a non-entertainment venue is higher than the frequency of abnormal events occurring in the process of leaving a non-entertainment venue or going to a non-entertainment venue. Therefore, optionally, the interest range of this embodiment can be divided into two categories, one for entertainment venues and the other for non-entertainment venues. Optionally, this embodiment can also take a point of interest as a category, such as taking a bar as a category, which is not specifically limited in this embodiment.
[0036] The collision data can be collected by a collision sensor installed on the vehicle. When the vehicle collides, the collision sensor can detect the intensity signal of the vehicle collision and send it to the server.
[0037] In this embodiment, the target user is the user corresponding to the target task, and may be the user who issues the ride order or the user who actually takes the ride, which is not specifically limited in this embodiment. The first track may be collected by the task issuing terminal at a predetermined period (e.g., 1 second) for the location information of the target user and sent to the server, and the second track may be collected by the task processing terminal at a predetermined period for the location information of the task processing resource and sent to the server.
[0038] Optionally, after obtaining the first trajectory and the second trajectory, the server may also determine the distance between the target user and the task processing resource at the same time, and when the distance satisfies the predetermined distance, it is determined that the target user and the task processing resource are not separated, which can be represented by 0, otherwise it is determined that the target user and the task processing resource are separated, which can be represented by 1. Then, the separation state sequence used to characterize whether the target user and the task processing resource are separated at each time is used as part of the task information of the target task.
[0039] For example, the separation state of the target user and the task processing resource at time t1 is 0, the separation state at time t2 is 0, ..., the separation state at time tk (k is a predetermined integer greater than or equal to 1) is 1. That is to say, before time t(k-1), the separation state of the target user and the task processing resource is 0, then the separation state sequence is (0, 0, ..., 1), where the number of 0s is (l-1).
[0040] Step S200: determining a first probability based on a first probability prediction model and according to target audio data and target video data.
[0041] In this embodiment, the first probability is used to characterize the probability that the target user is in an abnormal state. The occurrence of abnormal events is related to the abnormal state of the user to a certain extent. When the user is in an abnormal state such as drunkenness or unconsciousness, he is more likely to be robbed, sexually assaulted, etc. Therefore, this embodiment uses the first probability as an important indicator to determine whether an abnormal event will occur during the execution of the target task.
[0042] Figure 3 FIG. 4 is a flow chart of determining the first probability in an optional implementation of the first embodiment of the present invention. Figure 3 As shown, the method of this embodiment includes the following steps:
[0043] Step S210: Perform audio recognition on the target audio data to determine audio related information.
[0044] The audio-related information of this embodiment is information related to the intention of the target user and the task processing resources obtained based on the target audio data. In this step, the server can perform speech recognition on the target audio data in various existing ways, such as performing speech recognition on the target audio data by the method described in "Research on Speech Emotion Recognition Based on Separable Convolution and LSTM. Li Wenjie, Luo Wenjun, Li Yiwen, Su Chengyue, Chen Yuhuai, Cao Yue. Guangdong University of Technology. 2020" to obtain the speech recognition text corresponding to the target audio data, which is not specifically limited in this embodiment.
[0045] After obtaining the speech recognition text, the server can determine the word embedding vector (i.e., word vector) of each word in the speech recognition text based on the word vector model. Specifically, the server can segment the speech recognition text through various existing methods, such as stuttering word segmentation, to obtain multiple words, and then obtain the word vector corresponding to each word based on the pre-trained word vector model.
[0046] The word vector model is a model obtained by training based on the distribution hypothesis. The distribution hypothesis assumes that words with similar contexts have similar semantics, that is, similar words have similar word vectors. Context is also called context. For any word in a text, the n (n is an integer greater than or equal to 1) words in front of the word and the m (m is an integer greater than or equal to 1) words after the word in the text are all the context of the word. In this embodiment, the word vector model is also a word2vec model, which can be specifically skip-gram, CBOW (Continuous Bag-of-Word, continuous bag of words model), etc. The word vector can be obtained based on multiple training texts. Taking skip-gram as an example, in the training process of the word vector model, the server can represent each word through a one-hot vector, and then use the one-hot vector of each word as input and the one-hot vector of each word in the context of the word as the training target to train skip-gram.
[0047] After obtaining multiple words corresponding to the speech recognition text, the server can obtain the one-hot vector of each word based on the full vocabulary obtained from multiple training texts, and then use the one-hot vector of each word as the input of skip-gram to obtain the word vector of each word. When obtaining the one-hot vector of a word based on the full vocabulary, the server can determine the element of the position corresponding to the word as 1 according to the position of the word in the full vocabulary, and determine the elements of other positions as 0. For example, the full vocabulary includes 10,000 words such as: I, you, don’t, how much, etc., then for the word "how much", the server can determine the fourth element in the one-hot vector (that is, the position corresponding to "how much") as 1, and determine the elements of other positions as 0, and obtain the one-hot vector corresponding to "how much" (0,0,0,1,...,0), where the ellipsis includes 9995 zeros.
[0048] After obtaining the word vectors corresponding to each word, the server can determine the similarity corresponding to each word based on the word vectors of each word and the predetermined word list. In this embodiment, the predetermined word list is a word list composed of sensitive words related to at least one of the abnormal state and abnormal events in the full word list. Therefore, the server can effectively identify the abnormal intentions of users and task processing resources during the ride based on the predetermined word list, thereby playing a positive protective role in the occurrence of abnormal events. Similar to the way the word vectors of each word in the speech recognition text are obtained, the word vectors of each sensitive word in the predetermined word list are also obtained based on the full word list and the word vector model.
[0049] When determining the similarity of each word, the server may calculate the similarity of the word vector of each word with the word vector of each sensitive word, and then use the maximum value of the similarities between each word and each sensitive word as the similarity of each word. For example, sensitive words include words w1, word 2, and word w3. When determining the similarity corresponding to the word "how much", the server may calculate the similarity between "how much" and the word vector of word w1, the similarity between "how much" and the word vector of word w2, and the similarity between "how much" and the word vector of word w3, and then determine the similarity between "how much" and the word vector of word w1 (that is, the maximum value of the similarities) as the similarity corresponding to "how much".
[0050] In this embodiment, the server can calculate the similarity between the words in the text and the sensitive words by various existing similarity calculation methods, such as cosine value, Euclidean distance, Pearson correlation, etc., which are not specifically limited in this embodiment.
[0051] At the same time, the server can also obtain the text vector of the speech recognition text according to the word vector of each word. Specifically, the server can use the mean vector of the word vector of each word in the speech recognition text (that is, the vector composed of the average value of the elements at the same position in the word vector of each word) as the text vector of the speech recognition text.
[0052] After obtaining the similarity of each word in the speech recognition text and the text vector of the speech recognition text, the server can determine the audio-related information corresponding to the target audio data according to the similarities and the text vectors.
[0053] Step S220: performing image recognition on the target video data to determine video related information.
[0054] The video-related information of this embodiment is information related to the number and location of the target users, etc., obtained based on the target video image. In this step, the server can perform image recognition on the target video data based on various existing methods, for example, by performing image recognition on the target video data to obtain video-related information using the method described in "A New Image Recognition Method Based on Dual Network Model. Ma Linmei, Tang Xiao, Wang Yinhe. Guangdong University of Technology, Guangzhou Junshi Electric Co., Ltd. R&D Center. 2020", which is not specifically limited in this embodiment.
[0055] The video-related information of this embodiment includes at least the number of target users and the relative positions of the target users and the task processing resources. When the vehicle of the task processing resource is a vehicle with a left cab, the relative positions may include on the right side of the task processing resource, behind the task processing resource, behind the right of the task processing resource, etc.
[0056] Step S230: determining a first probability based on the first probability prediction model and according to the audio-related information and the video-related information.
[0057] After determining the audio-related information and the video-related information, in this step, the server may use the audio-related information and the video-related information as inputs to a first probability prediction model to obtain a first probability corresponding to the target task.
[0058] In this embodiment, the first probability prediction model can be a decision tree, a Bayesian classifier, a neural network, etc., and is obtained by training based on the first sample set. The selection of the first probability prediction model can be determined based on the characteristics of each parameter in the historical data and / or actual needs. The first sample set used to train the first probability prediction model includes the first historical task information of multiple historical tasks, wherein the first historical task information includes the abnormal state identifier of the historical task. The abnormal state identifier is the training target of the second probability prediction model. If the historical user corresponding to the historical task is in an abnormal state during the execution of the historical task, the abnormal state identifier can be 1, otherwise it can be 0. Similarly, the first historical task information also includes historical video-related information and historical audio-related information corresponding to each historical task, and may also include other historical information.
[0059] Take neural network as an example. The full name of neural network is artificial neural network (ANN), which is an information processing model formed by the interconnection of a large number of processing units. Common artificial neural networks include convolutional neural network (CNN), recurrent neural network (RNN), etc. ANN has the characteristics of nonlinearity (suitable for processing nonlinear information), non-locality (that is, the overall state of a system depends on the interaction between processing units), non-qualitativeness (that is, it has adaptive, self-organizing, self-learning capabilities, and can continuously self-learn in the process of processing information) and non-convexity (the activation function of the model has multiple extreme values, which makes the model have multiple relatively stable equilibrium states, so that the changes of the model are diverse). Therefore, it can be widely used in various fields for more accurate data prediction.
[0060] During the training process of the first probability prediction model, the input of the model may include historical information such as historical video related information and historical audio related information of each historical task, and the training target is the abnormal state identification corresponding to each historical task. After obtaining the trained first probability prediction model based on the above training method, the server can use the video related information, audio related information and other task information of the target task as input to obtain the first probability corresponding to the target task.
[0061] Step S300, based on the second probability prediction model, determine the second probability according to the target video data and the first probability.
[0062] In this embodiment, the second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task. In this step, the server can also obtain video-related information based on the target video data, and then obtain the second probability corresponding to the target task based on the second probability prediction model according to the video-related information and the first probability.
[0063] Similar to the first probability prediction model, in this embodiment, the second probability prediction model can be a decision tree, a Bayesian classifier, a neural network, etc., and is obtained by training according to the second sample set. The selection of the second probability prediction model can be determined according to the characteristics of each parameter in the historical data and / or actual needs. The second sample set used to train the second probability prediction model includes second historical task information of multiple historical tasks, wherein the second historical task information includes abnormal event identifiers of historical tasks. The abnormal event identifier is the training target of the second probability prediction model. If an abnormal event has occurred during the execution of the historical task (that is, a real complaint about an abnormal event has been received from a historical user), the abnormal event identifier can be 1, otherwise it can be 0. Similarly, the second historical task information also includes historical video related information corresponding to each historical task, and may also include other historical information.
[0064] During the training process of the first probability prediction model, the input of the model may include historical information such as historical video related information of each historical task, and the training target is the abnormal event identification corresponding to each historical task. After obtaining the trained second probability prediction model based on the above training method, the server can use the video related information of the target task, the first probability and other task information as input to obtain the second probability corresponding to the target task.
[0065] Step S400: In response to the second probability satisfying a predetermined probability condition, sending a first message to a task processing terminal.
[0066] The predetermined probability condition is used to characterize that the probability of an abnormal event occurring during the execution of the target task is greater than the first probability value, and there is a greater possibility of an abnormal event occurring. Therefore, when the second probability meets the predetermined probability condition, the server can send a first message to the task processing terminal to alert the task processing resources, thereby curbing the occurrence of abnormal events to a certain extent from the perspective of the parties involved, and reducing the possibility of personal or property infringement of users.
[0067] In some optional implementations, this embodiment may further include the following steps:
[0068] Step S500: Send a second message to a predetermined terminal.
[0069] Similar to step S400, in this step, the server can send a second message to the predetermined terminal when the second probability meets the predetermined probability condition, so as to prompt the risk of abnormal events in the execution of the target task, thereby curbing the occurrence of abnormal events to a certain extent from the perspective of a third party, and reducing the possibility of personal injury or property damage to the user. In this embodiment, the predetermined terminal can be a terminal held by the manager of the online ride-hailing platform, a terminal held by a security guard, or another third-party terminal.
[0070] Optionally, before this step, in order to make the second message more targeted, this embodiment may also determine the second message through the following steps:
[0071] Step S500A: in response to the second probability satisfying a predetermined probability condition, determining at least one label corresponding to the target task according to the task information.
[0072] In this step, the server can determine at least one label corresponding to the target task based on at least part of the information in the task information. For example, if the first track and the second track are not separated within a predetermined time period, the server can determine a label of the target task as "tracks are not separated"; if the first probability is greater than the second probability value, it can be considered that the target user is in an abnormal state, so the server can determine a label of the target task as "abnormal state"; if the target task execution period is evening or night, the server can determine a label of the target task as "late night".
[0073] Step S500B: determining a second message according to at least one tag.
[0074] After determining at least one tag corresponding to the target task, the server may determine a second message based on the at least one tag, so that the predetermined terminal can display the abnormal points of the target task, so that the person holding the predetermined terminal can intervene in the target task more timely and effectively. Specifically, the server may use at least one tag of the target task as part of the second message.
[0075] Figure 4 is a data flow diagram for determining the second probability using the method of the first embodiment of the present invention. Figure 4As shown, the task information of the target task includes target audio data 41, target video data 42 and other information 43. The server can perform speech recognition on the target audio data 41 to obtain audio-related information 44, and perform image recognition on the target video data 42 to obtain video-related information 45, and then use the audio-related information 44, video-related information 45 and other information 43 as inputs of a first probability prediction model to obtain a first probability 46, and then use the first probability 46, video-related information 45 and other information 43 as inputs of a second probability prediction model to obtain a second probability.
[0076] This embodiment obtains the video data and audio data collected during the execution of the target task as part of the task information, and then determines the probability of the target user being in an abnormal state based on the audio data and the video data based on the first probability prediction model, and determines the probability of an abnormal event occurring during the execution of the target task based on the video data and the probability of the target user being in an abnormal state based on the second probability prediction model, so that when the probability of an abnormal event occurring during the execution of the target task meets a predetermined probability condition, a first message is sent to the task processing terminal to warn the task processing resources. This embodiment uses the probability of the target user being in an abnormal state as an important indicator for predicting the occurrence of abnormal events, and can more accurately and timely issue risk warnings for tasks with a high probability of abnormal events occurring.
[0077] Figure 5 Schematic diagram of a data processing device according to a second embodiment of the present invention. Figure 5 As shown, the apparatus of this embodiment includes a task information acquiring unit 51 , a first probability determining unit 52 , a second probability determining unit 53 and a first message sending unit 54 .
[0078] Among them, the task information acquisition unit 51 is used to obtain the task information of the target task, and the task information includes target video data and target audio data. The target video data is the video data collected during the execution of the target task, and the target audio data is the audio data collected during the execution of the target task. The first probability determination unit 52 is used to determine a first probability based on a first probability prediction model according to the target audio data and the target video data. The first probability is used to characterize the probability that the target user is in an abnormal state. The target user is the user corresponding to the target task. The second probability determination unit 53 is used to determine a second probability based on a second probability prediction model according to the target video data and the first probability. The second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task. The first message sending unit 54 is used to send a first message to the task processing terminal in response to the second probability satisfying a predetermined probability condition to alert the task processing resources.
[0079] This embodiment obtains the video data and audio data collected during the execution of the target task as part of the task information, and then determines the probability of the target user being in an abnormal state based on the audio data and the video data based on the first probability prediction model, and determines the probability of an abnormal event occurring during the execution of the target task based on the video data and the probability of the target user being in an abnormal state based on the second probability prediction model, so that when the probability of an abnormal event occurring during the execution of the target task meets a predetermined probability condition, a first message is sent to the task processing terminal to warn the task processing resources. This embodiment uses the probability of the target user being in an abnormal state as an important indicator for predicting the occurrence of abnormal events, and can more accurately and timely issue risk warnings for tasks with a high probability of abnormal events occurring.
[0080] Figure 6 is a schematic diagram of an electronic device according to a third embodiment of the present invention. Figure 6 The electronic device shown is a general data processing device, which includes a general computer hardware structure, which at least includes a processor 61 and a memory 62. The processor 61 and the memory 62 are connected via a bus 63. The memory 62 is suitable for storing instructions or programs executable by the processor 61. The processor 61 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 61 executes the commands stored in the memory 62, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 63 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 64 and the display device and the input / output (I / O) device 65. The input / output (I / O) device 65 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output (I / O) device 65 is connected to the system via an input / output (I / O) controller 66.
[0081] The memory 62 may store software components such as an operating system, a communication module, an interaction module, and an application program. Each of the modules and applications described above corresponds to a set of executable program instructions that implement one or more functions and methods described in the embodiments of the invention.
[0082] The flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention described above describe various aspects of the present invention. It should be understood that each block of the flowchart and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that (via the processor of a computer or other programmable data processing device) instructions are created to implement the device for the function / action specified in the flowchart and / or block diagram block or block.
[0083] At the same time, as will be appreciated by those skilled in the art, various aspects of the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, various aspects of the embodiments of the present invention may take the following forms: a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.) or an implementation that combines software aspects with hardware aspects, which may generally be referred to herein as a "circuit", "module" or "system". In addition, aspects of the present invention may take the following forms: a computer program product implemented in one or more computer-readable media, the computer-readable media having a computer-readable program code implemented thereon.
[0084] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, (but not limited to) electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or apparatus or a program used in conjunction with an instruction execution system, device or apparatus.
[0085] A computer readable signal medium may include a propagated data signal having a computer readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any of the following computer readable media: not a computer readable storage medium, and may communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.
[0086] Computer program code for performing operations for various aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, PHP, Python, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server as a stand-alone software package. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A data processing method, It is characterized in that The method comprises: Acquire task information of a target task, wherein the task information includes target video data and target audio data, wherein the target video data is video data collected during the execution of the target task, and the target audio data is audio data collected during the execution of the target task; Based on a first probability prediction model, determining a first probability according to the target audio data and the target video data, wherein the first probability is used to characterize the probability that a target user is in an abnormal state, and the target user is a user corresponding to the target task; Based on a second probability prediction model, determining a second probability according to the target video data and the first probability, wherein the second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task; In response to the second probability satisfying a predetermined probability condition, sending a first message to the task processing terminal to alert the task processing resource; Wherein, determining the second probability based on the second probability prediction model according to the target video data and the first probability includes: Determine the second probability based on the second probability prediction model according to video related information and the first probability, wherein the video related information is determined by performing image recognition on the target video data; The determining the second probability based on the second probability prediction model according to the video related information and the first probability includes: The first probability and the video related information are used as inputs of the second probability prediction model to obtain the second probability.
2. The method according to claim 1, It is characterized in that The method further comprises: A second message is sent to a predetermined terminal to prompt a risk of an abnormal event occurring during the execution of the target task.
3. The method according to claim 1 or 2, It is characterized in that The method further comprises: In response to the second probability satisfying the predetermined probability condition, determining at least one label corresponding to the target task according to the task information; A second message is determined based on the at least one tag.
4. The method according to claim 1, It is characterized in that The determining the first probability according to the target audio data and the target video data based on the first probability prediction model comprises: Performing audio recognition on the target audio data to determine audio related information; Based on the first probability prediction model, the first probability is determined according to the audio-related information and the video-related information.
5. The method according to claim 4, It is characterized in that The performing audio recognition on the target audio data to determine audio related information includes: Performing speech recognition on the target audio data to obtain speech recognition text; Determine the word vector of each word in the speech recognition text based on the word vector model; Determine the similarity corresponding to each of the words according to each of the word vectors and a predetermined word list; Determine the text vector of the speech recognition text according to each of the word vectors; The audio related information is determined according to the similarities and the text vector.
6. The method according to claim 5, It is characterized in that Determining the text vector of the speech recognition text according to each of the word vectors includes: The mean vector of each of the word vectors is calculated as the text vector.
7. The method according to claim 1 or 4, It is characterized in that The video related information includes the user number of the target user and the relative position to the task processing resource.
8. The method according to claim 1, It is characterized in that The target task is a ride-hailing order for an online ride-hailing service; The task information also includes at least one of the starting position, the ending position, the first interest range, the second interest range, the collision data, the first trajectory and the second trajectory of the target task, wherein the first interest range is the interest range to which the starting position belongs, and the second interest range is the interest range to which the ending position belongs. The collision data is used to characterize whether a collision occurs during the execution of the target task, the first trajectory is the movement trajectory of the target user, and the second trajectory is the movement trajectory of the task processing resource.
9. The method according to claim 8, It is characterized in that The first probability prediction model is obtained by training according to a first sample set, the first sample set includes first historical task information of a plurality of historical tasks, the first historical task information includes an abnormal state identifier of the historical task, and the abnormal state identifier is used to indicate whether the corresponding historical user is in an abnormal state; The second probability prediction model is obtained by training based on a second sample set, wherein the second sample set includes second historical task information of multiple historical tasks, and the second historical task information includes abnormal event identifiers of the historical tasks, and the abnormal event identifiers are used to characterize whether abnormal events occur during the execution of the historical tasks.
10. A data processing device, It is characterized in that The device comprises: A task information acquisition unit, used to acquire task information of a target task, wherein the task information includes target video data and target audio data, wherein the target video data is video data collected during the execution of the target task, and the target audio data is audio data collected during the execution of the target task; A first probability determination unit, configured to determine a first probability based on a first probability prediction model and according to the target audio data and the target video data, wherein the first probability is used to characterize a probability that a target user is in an abnormal state, and the target user is a user corresponding to the target task; a second probability determination unit, configured to determine a second probability based on a second probability prediction model, according to the target video data and the first probability, wherein the second probability is used to characterize the probability of an abnormal event occurring during the execution of the target task; a first message sending unit, configured to send a first message to the task processing terminal in response to the second probability satisfying a predetermined probability condition, so as to alert the task processing resource; Wherein, the second probability determination unit includes: a second probability determination subunit, configured to determine the second probability based on the second probability prediction model, according to video related information and the first probability, wherein the video related information is determined by performing image recognition on the target video data; The second probability determination subunit is further used for: The first probability and the video related information are used as inputs of the second probability prediction model to obtain the second probability.
11. A computer readable storage medium having computer program instructions stored thereon, It is characterized in that The computer program instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.
12. An electronic device comprising a memory and a processor, It is characterized in that The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 9.
