Data processing method, related equipment and computer readable medium

Abstract

The embodiment of the invention discloses a data processing method, a data processing device, electronic equipment, a computer readable medium and a computer program product. The method comprises the following steps: constructing a corresponding hypergraph based on perception result data of a plurality of perception objects on an environment observation event; determining a hyperedge cardinal number and hyperedge strength of each hyperedge in the hypergraph, and node strength of each node in the hypergraph; estimating an initial parameter value of a preset parameter according to the hyperedge cardinal number and the hyperedge strength of each hyperedge and the node strength of each node; based on the sensing result data, expectation maximization processing is carried out on the initial parameter value of the preset parameter, an event confirmation result of the environment observation event is obtained, and the event confirmation result represents whether the corresponding environment observation event occurs truly or not. According to the technical scheme provided by the embodiment of the invention, the truth value discovery accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular, to a data processing method, a data processing device, an electronic device, a computer-readable medium, and a computer program product. Background Art

[0002] With the popularity of various smart mobile devices and wearable smart devices, ordinary individuals can collect a wide variety of environmental perception data anytime and anywhere through the electronic devices they carry, so that these data can be collected and applied in various technical fields such as the Internet of Things, Internet of Vehicles, big data, and artificial intelligence. Since the task of collecting environmental perception data is generally open to the public, the reliability of the participants in the collection task is generally unknown, and there are even cases where some participants deliberately upload erroneous perception result data. Therefore, it is necessary to mine authentic and reliable information from the perception result data set reported by all participants without knowing the reliability of the participants, that is, to solve the truth discovery of the perception result data (Truth Discovery is a data mining paradigm generated to solve the conflicts in multi-source noise data. It can mine accurate results, that is, true values, from multiple conflicting data) problem.

[0003] In related technologies, voting or expectation maximization (EM) algorithms are usually used to mine the true value in the data set, but neither of the above two solutions can accurately predict and discover the true value in the perception result data. Therefore, how to improve the accuracy of true value discovery is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, a data processing device, an electronic device, a computer-readable medium, and a computer program product, which can improve the accuracy of truth value discovery.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0007] Constructing a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events, wherein the environmental observation events are used to constitute the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object;

[0008] Determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; wherein the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation event;

[0009] According to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node, the initial parameter value of the preset parameter is estimated; wherein the preset parameter includes at least one of the event proportion of the environmental observation event that actually occurred, the first probability that each perception object perceives the environmental observation event that actually occurred, and the second probability that each perception object perceives the environmental observation event that did not actually occur;

[0010] Based on the perception result data, the initial parameter value of the preset parameter is subjected to expectation maximization processing to obtain an event confirmation result of the environmental observation event, and the event confirmation result represents whether the corresponding environmental observation event actually occurs.

[0011] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising a construction unit, a determination unit and a processing unit, wherein:

[0012] The construction unit is used to construct a corresponding hypergraph based on the perception result data of the environmental observation events by the multiple perception objects, wherein the environmental observation events are used to constitute the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object;

[0013] The determination unit is used to determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; wherein the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation event;

[0014] The determination unit is further used to estimate the initial parameter value of the preset parameter according to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node; wherein the preset parameter includes at least one of the event proportion of the environmental observation event that actually occurred, the first probability that each perception object perceives the environmental observation event that actually occurred, and the second probability that each perception object perceives the environmental observation event that did not actually occur;

[0015] The processing unit is used to perform expectation maximization processing on the initial parameter value of the preset parameter based on the perception result data to obtain an event confirmation result of the environmental observation event, and the event confirmation result represents whether the corresponding environmental observation event actually occurs.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors; a storage device for storing one or more computer programs, so that when the one or more computer programs are executed by the one or more processors, the electronic device implements the data processing method as described above.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable medium on which a computer program is stored. When the computer program is executed by a processor of an electronic device, the electronic device executes the data processing method as described above.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device performs the data processing method as described above.

[0019] In the technical solution provided in the embodiment of the present application, the corresponding hypergraph is first constructed through the perception result data, so that the initial parameter values ​​of the preset parameters can be accurately and reasonably estimated through the hypergraph topology indicators such as the hyperedge cardinality of each hyperedge, the hyperedge strength and the node strength of each node, which are associated with the preset parameters required by the expectation maximization algorithm, thereby obtaining better and more reasonable initial parameter values. It can be seen that the embodiment of the present application can provide accurate and reasonable initial parameter values ​​for the preset parameters required by the subsequent expectation maximization algorithm by constructing a hypergraph, which is conducive to improving the algorithm accuracy of the expectation maximization algorithm, thereby making the final event confirmation result of the environmental observation event more accurate, thereby greatly improving the accuracy of truth value discovery, especially conducive to improving the accuracy of truth value discovery for sparse data such as perception result data.

[0020] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 It is a comparative schematic diagram of a traditional graph and a hypergraph provided in an embodiment of the present application;

[0023] Figure 2 It is a structural diagram of a data processing system provided in an embodiment of the present application;

[0024] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a confirmed observation matrix provided in an embodiment of the present application;

[0026] Figure 5 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0027] Figure 6 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0028] Figure 7 This is a schematic diagram of a process of truth value discovery in a smart transportation scenario provided by an embodiment of the present application;

[0029] Figure 8a It is a comparative schematic diagram of the accuracy of true value judgment of an environmental observation event provided in an embodiment of the present application;

[0030] Figure 8b is a comparative schematic diagram of the accuracy of true value judgment of another environmental observation event provided in an embodiment of the present application;

[0031] Figure 9a is a comparative schematic diagram of an average error of the reliability of a perceived object provided in an embodiment of the present application;

[0032] Figure 9b is a comparative schematic diagram of another average error of perceived object reliability provided by an embodiment of the present application;

[0033] Fig.10 is a structural block diagram of a data processing device provided in an embodiment of the present application;

[0034] Fig.11 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all contents and operations, nor must they be executed in the order described. For example, some operations may be decomposed, while some operations may be combined or partially combined, so the actual execution order may change according to actual conditions.

[0038] It should also be noted that the "multiple" mentioned in this application refers to two or more than two. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0039] With the continuous development and accelerated integration of various technologies such as the Internet of Things, Internet of Vehicles, Big Data, and Artificial Intelligence, crowd sensing, as an emerging data perception model, has become an international frontier research hotspot in recent years. Crowd sensing includes mobile crowd sensing (MCS) and sparse crowd sensing (SCS).

[0040] Among them, mobile crowd intelligence perception refers to a new data acquisition model that combines crowdsourcing ideas with the perception capabilities of mobile devices. With the popularization of various smart mobile devices and wearable smart devices, ordinary individuals can collect a wide variety of environmental perception data anytime and anywhere through the smart devices they carry and their built-in sensors (such as accelerometers, gyroscopes, cameras, compasses, thermometers, microphones, car phones, etc.), and upload them to the corresponding servers. The environmental perception data obtained through mobile crowd intelligence perception can be applied in various technical fields such as the Internet of Things, Internet of Vehicles, big data, and artificial intelligence; specifically, the environmental perception data obtained through mobile crowd intelligence perception has been applied in business scenarios such as machine learning model training, environmental pollution monitoring, environmental noise maps, urban traffic conditions, social networks, and health care. In the foreseeable future, it will be applied to more business scenarios.

[0041] The so-called artificial intelligence technology refers to the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science; it mainly produces a new intelligent machine that can respond in a similar way to human intelligence by understanding the essence of intelligence, so that the intelligent machine has multiple functions such as perception, reasoning and decision-making. Accordingly, AI technology is a comprehensive discipline, which mainly includes several major directions such as computer vision technology (Computer Vision, CV), speech technology, natural language processing technology and machine learning / deep learning.

[0042] The so-called machine learning is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of AI and the fundamental way to make computer devices intelligent; the so-called machine learning is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines; it specializes in studying how computer devices simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Deep learning is a technology that uses deep neural network systems to perform machine learning; machine learning / deep learning can usually include artificial neural networks, reinforcement learning (RL), supervised learning, unsupervised learning and other technologies; the so-called supervised learning refers to the processing method of using training samples with known categories (with labeled categories) for model training, and unsupervised learning refers to the processing method of using training samples with unknown categories (not labeled) for model training.

[0043] Since the environmental perception data collection task is generally open to the public, the reliability of the participants in the collection task is generally unknown, and there are even cases where some participants deliberately upload wrong perception result data, thereby disrupting the collection process of environmental perception data. At the same time, even if it is believed that most of the participants are reliable, their cognitive abilities can guarantee the quality of the collected environmental perception data to a certain extent; however, in the process of participating in data collection, the reliability of the participants may also be damaged due to lack of attention to the corresponding collection task, obstructed observation process, improper operation process, etc.

[0044] Therefore, it is necessary to mine authentic and reliable information from the perception result data set reported by all participating objects without knowing the reliability of the participating objects, that is, to solve the truth discovery problem of the perception result data.

[0045] Truth value discovery is a data mining paradigm that was created to resolve conflicts in multi-source noise data. It can mine accurate results, namely the true value, from multiple conflicting data. Compared with traditional voting or averaging methods, an important feature of truth value discovery is that it assumes that the reliability of the data source of each data is different, and can estimate the reliability of the data source. Based on the reliability of the data source, truth value discovery can perform weighted aggregation on the data from each data source to infer the true value. Since the reliability and true value of the data source are unknown in advance, truth value discovery is essentially a type of unsupervised data mining.

[0046] In related technologies, majority voting or truth value discovery methods based on expectation maximization (EM) are used to mine the truth values ​​in a data set.

[0047] The majority voting method is to trust the judgment results of the majority of participants, that is, "the minority obeys the majority". However, the main problem with this method is that it does not consider and infer the reliability of the participants. This may lead to a situation where the perception results of some unreliable sources may be more convincing than the perception results of several reliable sources. Therefore, the truth value discovery accuracy of the majority voting method is not high.

[0048] Expectation Maximization (EM) is a general algorithm for finding the maximum likelihood estimate of the target parameter in a statistical model with missing data. Simply put, after assigning appropriate parameters to the latent variables and estimated variables, EM uses the value of the latent variable to make guesses, and uses the guessed value to iteratively supplement the missing parts of the data set to re-predict the estimated variable. Specifically, the EM algorithm estimates the value of the latent variable and regards the estimated value as the true value to re-estimate the unknown parameters. The EM algorithm is an iterative algorithm and each iteration consists of two steps: the E step, finding the expectation; the M step, finding the maximum. Among them, the EM algorithm uses the likelihood function to find the maximum likelihood estimate of the unknown parameters.

[0049] However, not every participant will perceive all environmental observation events included in the collection task and upload the corresponding perception result data. In real scenarios, after participating in the collection task, the participants will only perceive a small number of environmental observation events included in the collection task due to various factors such as region, distance, route planning, human energy and time. Therefore, there is not much perception result data for each environmental observation event, so the collected perception result data is a sparse data set.

[0050] Since several parameters required for expectation maximization processing in the truth value discovery method based on the expectation maximization algorithm are not reasonably selected, they simply randomly select initial values ​​for subsequent iterative calculations; therefore, for sparse data sets, this method cannot accurately predict and discover the true value.

[0051] In addition, hypergraph, as a generalized variant of traditional graph, can more accurately describe the relationship between objects with multiple associations. Figure 1 , which shows a schematic diagram comparing a traditional graph and a hypergraph. Figure 1Figure 101 in the figure is a hypergraph. In a hypergraph, an edge is called a hyperedge. A hyperedge can connect any number of nodes. This is the biggest difference between a hypergraph and a traditional graph in terms of graph structure. When a hyperedge can only connect two nodes at most, the hypergraph composed of such hyperedges will degenerate into a traditional graph that we are familiar with, such as Figure 1 As shown in Figure 102. In other words, the traditional graph is a form of hypergraph.

[0052] Based on this, an embodiment of the present application provides a data processing scheme, which introduces hypergraph theory into the truth discovery algorithm. The corresponding hypergraph will be first constructed through the collected perception result data, so as to determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, as well as the node strength of each node in the hypergraph. Then, based on the hyperedge cardinality and hyperedge strength of each hyperedge, and the node strength of each node, the initial parameter values ​​of the preset parameters required by the expected maximum algorithm are estimated. Finally, by maximizing the expectation of the initial parameter values ​​of the preset parameters, the event confirmation result of the environmental observation event is obtained.

[0053] Among them, the event confirmation result represents whether the corresponding environmental observation event actually occurred. The collected perception result data includes the perception result data of each perception object of the environmental observation event among multiple perception objects. The perception object refers to a person or electronic device participating in the perception task of the environmental observation event, which is not limited here.

[0054] An environmental observation event can be any event that can be observed or needs to be observed in the environment. The number of environmental observation events can include one or more. For example, in a traffic scenario, an environmental observation event can be whether a traffic light at a certain intersection is operating normally, whether a car accident has occurred, etc. In a medical scenario, an environmental observation event can be whether the medical equipment is clean, whether there is a nurse on duty at the reception desk, etc.

[0055] In addition, the nodes in the hypergraph are composed of environmental observation events, and the hyperedges in the hypergraph are determined by the set of environmental observation events perceived by the same perception object. There is a corresponding relationship between each node and each environmental observation event, and there is a corresponding relationship between each hyperedge and each perception object.

[0056] At the same time, the hyperedge cardinality of each hyperedge in the hypergraph represents the number of environmental observation events perceived by the corresponding perception object; the hyperedge strength of each hyperedge is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects; the node strength of each node represents the number of perception objects that perceive the corresponding environmental observation event.

[0057] In addition, the preset parameters may include at least one of the event proportion of actual environmental observation events, the first probability that each perception object perceives the actual environmental observation event, and the second probability that each perception object perceives the environmental observation event that did not actually occur.

[0058] The environmental observation events perceived by each sensing object are the environmental observation events that each sensing object determines to occur after observation. The rest are called environmental observation events that are not perceived by each sensing object.

[0059] The hyperedge strength can comprehensively reflect the degree of association between hyperedges, that is, the hyperedge strength of a hyperedge can reflect the degree of similarity between the perception object corresponding to the hyperedge and the remaining other perception objects in multiple perception objects. Generally speaking, it is rare for multiple perception objects to upload erroneous perception result data for the same environmental observation event. Therefore, if a perception object is very similar to other perception objects, it can be said to a certain extent that the reliability of the perception object is very high. At the same time, the hyperedge cardinality can reflect the workload of each perception object, and the amount of workload can also reflect the reliability of the corresponding perception object to a certain extent. Therefore, a more accurate and reasonable preliminary estimate of the first probability and second probability of each perception object can be made based on the hyperedge strength and the hyperedge cardinality.

[0060] The node strength of a node can be understood as the number of perceptual objects that determine the occurrence of the corresponding environmental observation event; if most people are sure that an environmental observation event has occurred, then the possibility of the environmental observation event actually occurring is higher. Therefore, the node strength of each node can be used to make a relatively accurate and reasonable preliminary estimate of the event proportion.

[0061] It is not difficult to see that this solution first constructs a hypergraph through the collected perception result data, so that the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, as well as the node strength of each node, can be used to accurately and reasonably estimate the parameter values ​​of the preset parameters required by the expectation maximization algorithm, thereby obtaining a better and more reasonable initial parameter value for the preset parameter. It can be seen that this solution can provide accurate and reasonable initial parameter values ​​for the preset parameters required by the subsequent expectation maximization algorithm by constructing a hypergraph, thereby improving the algorithm accuracy of the expectation maximization algorithm, making the final event confirmation result of the environmental observation event more accurate, and achieving the purpose of improving the accuracy of true value discovery.

[0062] Based on the above data processing solution, the present application embodiment provides a data processing system, which can be seen in Figure 2 , Figure 2The data processing system shown may include multiple terminal devices 201 and multiple servers 202. Among them, a communication connection is established between any terminal device and any server. For example, the terminal device 201 may include any one or more of a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted terminal, and a smart wearable device. Multimedia playback clients, social clients, browser clients, information stream clients, educational clients, etc. can be run in the terminal device 201. The server 202 can be a server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Any terminal device 201 and any server 202 can be directly or indirectly connected in communication via wired or wireless communication, and this application is not limited here.

[0063] In some embodiments, the above data processing method can be performed only by Figure 2 The server 202 in the data processing system shown is executed, and the specific execution process is: the server 202 can build a corresponding hypergraph through the collected perception result data. Then, the server 202 can determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, as well as the node strength of each node in the hypergraph. Afterwards, the server 202 can estimate the initial parameter values ​​of the preset parameters based on the hyperedge cardinality and hyperedge strength of each hyperedge, as well as the node strength of each node. Finally, the server 202 can obtain the event confirmation result of the environmental observation event by maximizing the expectation of the initial parameter values ​​of the preset parameters.

[0064] Optionally, the server 202 may also filter out data that is consistent with the event confirmation result of the environmental observation event from the collected environmental perception data as reliable perception data, so that the reliable perception data can be subsequently applied to various technical fields such as training machine learning models and big data processing. Figure 2 The terminal device 201 in the data processing system shown is executed. The specific execution process can refer to the specific execution process of the server 202, which will not be repeated here.

[0065] In other embodiments, the above data processing method may be run in a data processing system, which may include a terminal device and a server. Figure 2The terminal device 201 and the server 202 included in the data processing system shown are jointly completed. The terminal device 201 can be a sensing object or an electronic device carried by the sensing object. The number of terminal devices 201 can include multiple. The specific execution process is: each terminal device 201 can collect the sensing result data of the environmental observation event and upload it to the server 202; after receiving the sensing result data of the environmental observation event collected by multiple terminal devices 201, the server 202 can build a corresponding hypergraph through the received sensing result data; then, the server 202 can determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; then, the server 202 can estimate the initial parameter value of the preset parameter according to the hyperedge cardinality and hyperedge strength of each hyperedge, and the node strength of each node. Finally, the server 202 can obtain the event confirmation result of the environmental observation event by maximizing the expectation of the initial parameter value of the preset parameter.

[0066] It should be noted that the embodiments of the present application can be used to discover true values ​​in various data sets, especially sparse data sets, and can be applied to various scenarios, including but not limited to cloud technology, AI (Artificial Intelligence), smart home, smart transportation and other scenarios, and can also be used for various applications such as navigation applications, financial applications, social applications, shopping applications, etc., without limitation.

[0067] Specifically, the technical solution of the embodiment of the present application can be applied to intelligent traffic scenarios, such as the terminal device can be set to be one or more of various devices such as mobile terminal devices carried by pedestrians in the intelligent traffic scenario, vehicle-mounted terminals installed in various vehicles, and intelligent garbage bins at various intersections. The server can be a traffic data server that provides data processing services for terminal devices in the intelligent traffic scenario.

[0068] The traffic data server can send data collection tasks about a batch of traffic observation events to each terminal device in the intelligent traffic scene, and each terminal device collects the perception result data of the traffic observation event and uploads it to the traffic data server. Then, the traffic data server can construct the corresponding traffic hypergraph based on the received perception result data, and determine the hyperedge cardinality and hyperedge strength of each hyperedge in the traffic hypergraph, as well as the node strength of each node in the traffic hypergraph; after that, the traffic data server can estimate the initial parameter value of the preset parameter based on the hyperedge cardinality and hyperedge strength of each hyperedge in the traffic hypergraph, as well as the node strength of each node. Finally, the traffic data server can obtain the event confirmation result of each traffic observation event by maximizing the expectation of the initial parameter value of the preset parameter, so as to determine whether each traffic observation event actually occurred.

[0069] For another example, the technical solution of the embodiment of the present application is applied to a shopping application, the terminal device may be a terminal device running a shopping application, and the server may be an application server providing services for the shopping application. The application server may send data collection tasks about commodity observation events for commodities to multiple terminal devices running the shopping application. For example, commodity observation events may include whether the purchased commodity is damaged, whether the purchased commodity is stained, etc.

[0070] Each terminal device can collect the perception result data of the commodity observation event and upload it to the application server. Then, the application server can construct the corresponding commodity hypergraph through the received perception result data, and determine the hyperedge cardinality and hyperedge strength of each hyperedge in the commodity hypergraph, as well as the node strength of each node in the commodity hypergraph; after that, the application server can estimate the initial parameter value of the preset parameter according to the hyperedge cardinality and hyperedge strength of each hyperedge in the commodity hypergraph, as well as the node strength of each node. Finally, the application server can obtain the event confirmation result of each commodity observation event by maximizing the expectation of the initial parameter value of the preset parameter, so as to determine whether each commodity observation event actually occurred. Optionally, the application server can also evaluate the commodity quality of the corresponding commodity through the event confirmation result of the commodity observation event.

[0071] It should be noted that in the specific implementation of this application, if the perception result data and other related data are related to the object, when the embodiment of this application is applied to a specific product or technology, it is necessary to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0072] The following is a detailed description of various implementation details of the technical solution of the embodiment of the present application:

[0073] like Figure 3 As shown, Figure 3 is a flow chart of a data processing method according to an embodiment of the present application, which can be applied to Figure 2 The data processing system shown in the figure, the method can be executed by the terminal device or the server, or can be executed by the terminal device and the server together. In the embodiment of the present application, the method is executed by the server as an example for explanation, wherein the data processing method may include S301 to S304, which are described in detail as follows:

[0074] S301. Construct a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events.

[0075] In the embodiments of the present application, the perception object refers to a person or electronic device that participates in the perception task of the environmental observation event, which is not limited here. An environmental observation event can be any event that can be observed and needs to be observed in the environment. The number of environmental observation events can include one or more. For example, in a traffic scenario, an environmental observation event can be whether the traffic lights at a certain intersection are operating normally, whether a car accident has occurred, etc. In a medical scenario, an environmental observation event can be whether the medical equipment is clean, whether there is a nurse on duty at the guide desk, etc.

[0076] The perception result data of multiple perception objects on environmental observation events can be composed of the perception result data of each perception object on each environmental observation event. The perception result data of a perception object on an environmental observation event can specifically be a value, character, field, etc. used to indicate whether the corresponding environmental observation event occurs after the corresponding perception object observes it, which is not limited here.

[0077] For example, "1" may be set to indicate that the corresponding perception object determines that the corresponding environmental observation event occurs, and "0" may be used to indicate other situations. Among them, other situations include that the corresponding perception object determines that the corresponding environmental observation event does not occur, the corresponding perception object does not observe the corresponding environmental observation event, and so on.

[0078] In one embodiment, the environmental observation events are used to form the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object. In other words, there is a corresponding relationship between each node and each environmental observation event, and there is a corresponding relationship between each hyperedge and each perception object.

[0079] In one embodiment, since the perception result data of each perception object to the environmental observation event mentioned above can be a binary conclusion like "1" or "0", the specific method of constructing a corresponding hypergraph based on the perception result data of multiple perception objects to the environmental observation event can be: constructing a binary observation matrix based on the perception result data of multiple perception objects to the environmental observation event; processing the observation matrix according to the preset hypergraph topology indicators to obtain the corresponding hypergraph.

[0080] The hypergraph can be specifically characterized by an observation matrix having hypergraph characteristics. The preset hypergraph topology index can include one or more of hyperedge cardinality, hyperedge strength, and node strength.

[0081] In a specific implementation, the data processing method proposed in the embodiment of the present application can be applied in a smart traffic scenario to discover the true value in the traffic perception result data. The traffic perception result data may include traffic perception result data of multiple traffic observation events uploaded by multiple traffic perception objects after the data collection tasks for multiple traffic observation events are sent to multiple traffic perception objects (specifically, taxi drivers, pedestrians, etc.).

[0082] In addition, any traffic perception result data is used to indicate whether the corresponding traffic perception object confirms whether the corresponding traffic observation event has occurred. Traffic observation events can specifically be observation events related to traffic conditions, such as whether the traffic lights at a certain intersection are operating normally, whether a traffic accident has occurred, whether there is congestion, whether the traffic flow or pedestrian flow is too large. After the traffic perception object observes a traffic observation event at a certain intersection, it can upload a binary conclusion about the traffic observation event. For example, if a traffic observation event exists or a traffic observation event occurs, the binary conclusion can be "1", otherwise it can be "0". At the same time, relevant photos or text can also be uploaded.

[0083] The server can firstly M ,P i represents the i-th traffic sensing object, there are M traffic sensing objects in total) for a set of traffic observation events E(E1,...,E N , E j Represents the jth traffic observation event, there are N total) binary conclusions (i.e., traffic perception data) to create a confirmed observation matrix PE. See Figure 4 , Figure 4 A schematic diagram of a confirmed observation matrix is ​​shown in FIG. Figure 4 As shown, any element in the confirmed observation matrix 401 is the corresponding traffic perception object P i For the corresponding traffic observation event E j The binary conclusion is 1 and 0. Among them, P i E j =1 indicates traffic sensing object P i Traffic observation event E j After the observation confirms its occurrence, the binary conclusion reported is "1", and the rest of the cases are reported as P i E j =0 indicates.

[0084] After creating the confirmed observation matrix PE, the server can process the confirmed observation matrix PE according to preset hypergraph topology indicators such as hyperedge cardinality, hyperedge strength, node strength, etc. to obtain the corresponding traffic hypergraph.

[0085] S302: Determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph.

[0086] In an embodiment of the present application, the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects among multiple perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation events.

[0087] The environmental observation events perceived by each sensing object are the environmental observation events that each sensing object determines to occur after observation. The rest are called environmental observation events that are not perceived by each sensing object.

[0088] In one embodiment, the hyperedge cardinality in hypergraph theory is defined as i The number of nodes included. In this embodiment, the set of perception result data of each perception object for the environmental observation event is a hyperedge, and an environmental observation event can be regarded as a node. Therefore, the hyperedge cardinality of each hyperedge is the number of environmental observation events perceived by the corresponding perception object. The hyperedge cardinality of a hyperedge can intuitively reflect the workload of the corresponding perception object to a certain extent, and thus can reflect the reliability of the corresponding perception object to a certain extent. At the same time, the hyperedge cardinality of the overall hyperedge in the hypergraph can reflect the sparsity of the perception result data to a certain extent.

[0089] In the specific implementation, referring to the definition of the hyperedge cardinality, in the confirmed observation matrix PE mentioned in the example of step S301, each traffic perception object P i The hyperedge cardinality of the traffic sensing object P i The number of traffic perception result data uploaded with a binary conclusion of "1".

[0090] In one embodiment, the node overdegree in the hypergraph theory is defined as the number of hyperedges containing the node. In this embodiment, the node strength of a node can be specifically understood as the number of perceptual objects that determine the occurrence of the corresponding environmental observation event. Therefore, the node overdegree of each node can reflect the probability of each environmental observation event tending to the benchmark true value to a certain extent, and the node overdegree of the entire node in the hypergraph can reflect the probability of the environmental observation event actually occurring in the environmental observation event to a certain extent.

[0091] In the specific implementation, referring to the definition of node strength, in the confirmed observation matrix PE mentioned in the example of step S301, each traffic observation event E j The node over-degree is the traffic observation event E j The number of traffic sensing objects that make observations and report a binary conclusion of "1".

[0092] In one embodiment, the hyperedge strength in the hypergraph theory can be summarized as the number of nodes contained in a hyperedge and all its adjacent hyperedges. The adjacent hyperedges of a hyperedge refer to the hyperedges that contain the same nodes as the hyperedge. The hyperedge strength of each hyperedge can reflect the similarity between the corresponding perceived object and other perceived objects to a certain extent.

[0093] Generally speaking, it is rare for multiple perception objects to upload erroneous perception result data for the same environmental observation event. Therefore, the overall data of other perception objects can be comprehensively considered as a reference for evaluating the reliability of the corresponding perception object. Especially for sparse data sets, hyperedge strength can play a role in comprehensively using the overall data as much as possible and amplifying the similarity between perception objects.

[0094] In a specific implementation, in the confirmed observation matrix PE mentioned in the example of step S301, the traffic perception object P i The corresponding hyperedge H i With traffic sensing object P i' The corresponding hyperedge H i The weight of It is defined as being able to be simultaneously contained in the hyperedge H i and hyperedge H i' The number of nodes in, further, the hyperedge H i Super edge strength The calculation formula (1-1) is as follows:

[0095]

[0096] in Represents the hyperedge H i The set of adjacent hyperedges.

[0097] S303: Estimate initial parameter values ​​of preset parameters according to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node.

[0098] In an embodiment of the present application, the preset parameters include at least one of the event proportion of actual environmental observation events, the first probability that each perception object perceives an actual environmental observation event, and the second probability that each perception object perceives an environmental observation event that has not actually occurred.

[0099] In one embodiment, when there are multiple environmental observation events, the event ratio specifically refers to the ratio of the number of environmental observation events that actually occurred to the total number of environmental observation events.

[0100] In one embodiment, referring to the explanation of the environmental observation events perceived and not perceived by each perception object in step S302, the first probability of each perception object can be specifically understood as the probability of each perception object determining that any environmental observation event occurs when any environmental observation event is a real event; and the second probability of each perception object can be specifically understood as the probability of each perception object determining that any environmental observation event occurs when any environmental observation event is an unreal event. It can be seen that the first probability and the second probability can respectively reflect the perception accuracy of the corresponding perception object for real events and unreal events, thereby representing the reliability of the corresponding perception object.

[0101] In one embodiment, since the hyper-edge strength and hyper-edge cardinality mentioned in step S302 can both reflect the reliability of the corresponding perceived object, the specific method for estimating the initial parameter value of the first probability or the second probability can be: based on the hyper-edge cardinality and hyper-edge strength of each hyper-edge, estimate the initial parameter value of the first probability or the second probability.

[0102] In one embodiment, since the node over-degree of each node mentioned in step S302 can reflect the probability of each environmental observation event tending to the benchmark true value to a certain extent, the node over-degree of each node can reflect the probability of the corresponding environmental observation event actually occurring to a certain extent. Therefore, the event proportion of the environmental observation event that actually occurred can be accurately and reasonably estimated with the help of the node over-degree of each node.

[0103] Therefore, the specific method of estimating the initial parameter value of the event ratio can be: based on the node over-measurement of each node, estimate the probability of the corresponding environmental observation event actually occurring; based on the probability of each environmental observation event actually occurring, estimate the number of environmental observation events that actually occur; finally, based on the number of environmental observation events that actually occur and the total number of environmental observation events, determine the initial parameter value of the event ratio.

[0104] S304: Based on the perception result data, an expectation maximization process is performed on the initial parameter values ​​of the preset parameters to obtain an event confirmation result of the environmental observation event.

[0105] In the embodiment of the present application, the event confirmation result of the environmental observation event represents whether the corresponding environmental observation event actually occurred. Specifically, the event confirmation result of the environmental observation event can be a number, field, or character indicating that the corresponding environmental observation event actually occurred or did not actually occur, and can also include the probability of the corresponding environmental observation event actually occurring or not actually occurring, etc., which is not limited here.

[0106] Optionally, the specific process of the expectation maximization processing may be: calling the expectation maximization algorithm, iteratively processing the initial parameter value of the preset parameter based on the perception result data, and obtaining the event confirmation result of the environmental observation event. In the iterative processing, the parameter value of the preset parameter is iterated in the direction of maximizing the parameter value of the preset parameter.

[0107] Optionally, after obtaining the event confirmation result of the environmental observation event, the server can also filter out data that is consistent with the event confirmation result of the environmental observation event from the perception result data as reliable perception data, so that the reliable perception data can be subsequently applied in various technical fields such as training machine learning models and big data processing.

[0108] Optionally, the server may also determine the perception reliability of each perception object based on the perception result data of each perception object for the environmental observation event and the event confirmation result; then, the perception object whose perception reliability is greater than the preset reliability threshold is determined as a reliable object. The preset reliability threshold may be manually set or may be set by the server or terminal device in the above data processing system, which is not limited here. For example, the mode or average value of multiple perception reliabilities may be selected as the preset reliability threshold, which is not limited here.

[0109] Furthermore, when the server obtains the data collection task for a new environmental observation event, it can publish the new data collection task to the reliable object; the server can directly use the perception result data uploaded by the received reliable object as reliable perception data, so that the reliable perception data can be subsequently applied in various technical fields such as training machine learning models and big data processing.

[0110] In an embodiment of the present application, a hypergraph is first constructed using the perception result data, so that the initial parameter values ​​of the preset parameters can be accurately and reasonably estimated through the hypergraph topology indicators associated with the preset parameters required by the expectation maximization algorithm, such as the hyperedge cardinality of each hyperedge, the hyperedge strength, and the node strength of each node, thereby obtaining better and more reasonable initial parameter values. It can be seen that the embodiment of the present application can provide accurate and reasonable initial parameter values ​​for the preset parameters required by the subsequent expectation maximization algorithm by constructing a hypergraph, which is conducive to improving the algorithm accuracy of the expectation maximization algorithm, thereby making the final event confirmation results of the environmental observation events more accurate, thereby greatly improving the accuracy of truth value discovery, especially for sparse data such as perception result data.

[0111] In one embodiment of the present application, another data processing method is provided, which can be applied to Figure 2The data processing system shown in the figure, the method can be executed by the terminal device or the server, or by the terminal device and the server together. In the embodiment of the present application, the method is described by taking the server as an example. Figure 5 FIG. 1 is a flow chart showing another data processing method. Figure 3 The method shown in is expanded on the basis of that in the embodiment of the present application, the preset parameters include event proportion, and the first probability and the second probability of each perceived object.

[0112] Among them, S501 to S504 are described in detail as follows:

[0113] S501: Construct a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events.

[0114] In the embodiment of the present application, the specific implementation of step S501 can refer to the specific implementation of step S301 in the above embodiment, which will not be repeated here.

[0115] S502: Determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph.

[0116] In the embodiment of the present application, the specific implementation of step S502 can refer to the specific implementation of step S302 in the above embodiment, which will not be repeated here.

[0117] S503. Estimate the initial parameter value of the event proportion, the initial parameter value of the first probability of each perceived object, and the initial parameter value of the second probability of each perceived object based on the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node.

[0118] In an embodiment of the present application, a specific method for estimating the initial parameter value of the first probability of each perceived object may be: determining the maximum hyper-edge cardinality among multiple hyper-edge cardinalities; taking the maximum hyper-edge strength among multiple hyper-edge strengths as the target hyper-edge strength; and determining the initial parameter value of the first probability of each perceived object based on the hyper-edge strength of each hyper-edge and the target hyper-edge strength.

[0119] Optionally, in order to avoid the situation where the initial parameter value estimated based on the maximum super-edge strength is inaccurate due to the large gap between the maximum super-edge strength and the minimum super-edge strength; the specific method of estimating the initial parameter value of the first probability of each perceived object can also be: determining the maximum hyper-edge cardinality among multiple hyper-edge cardinalities; determining the target hyper-edge strength based on the maximum hyper-edge cardinality and the total number of perceived objects; determining the initial parameter value of the first probability of each perceived object according to the hyper-edge strength of each hyper-edge and the target hyper-edge strength.

[0120] In a specific implementation, the target super-edge strength can be calculated based on the calculation formula (1-1). Formula (2-1) is as follows:

[0121] HInmax=HInumax×(M-1)×ω (2-1)

[0122] Among them, HInmax represents the target hyperedge strength; HInumax represents the maximum hyperedge cardinality; M represents the total number of perceived objects; ω is a coefficient ranging from 0 to 1, which can be set manually or by the server or terminal device in the above data processing system, and there is no limitation on this.

[0123] In actual observation, the perception result data of the same environmental observation event and the perceived environmental observation event of different perception objects are generally not exactly the same, that is, the number and type of nodes contained in two hyperedges are generally not exactly the same. In order to avoid the large gap between the maximum hyperedge strength and the minimum hyperedge strength caused by the hyperedge strength corresponding to some perception objects being too large and the hyperedge strength corresponding to some perception objects being too small, it is necessary to find a value similar to the mode in the hyperedge strength as the target hyperedge strength; therefore, when calculating the target hyperedge strength, a coefficient ω between 0 and 1 is required for numerical correction.

[0124] In one embodiment, the specific method for determining the initial parameter value of the first probability of each perceived object based on the hyper-edge strength of each hyper-edge and the target hyper-edge strength can be: estimating the original parameter value of the first probability of each perceived object based on the hyper-edge strength of each hyper-edge and the target hyper-edge strength; adding the original parameter values ​​of the first probability of each perceived object to obtain the sum of probabilities; and determining the initial parameter value of the first probability of each perceived object based on the sum of probabilities and the total number of perceived objects.

[0125] Specifically, since the hyperedge strength of each hyperedge can reflect the similarity between the corresponding perceived object and other perceived objects to a certain extent, and the more similar a perceived object is to the majority of other perceived objects, the higher the perception accuracy of the perceived object is. At the same time, by comparing the perception accuracy of a perceived object with the perception accuracy of the overall perceived object, the perception accuracy of the perceived object on the environmental observation event can be evaluated more objectively and accurately.

[0126] Therefore, a specific method for estimating the original parameter value of the first probability of each perceived object may be: taking the ratio of the hyper-edge strength of each hyper-edge and the target hyper-edge strength as the original parameter value of the first probability of each perceived object.

[0127] In addition, when the first probabilities of each perceived object are the same, the expected maximum algorithm has better algorithm effect and more accurate prediction of event confirmation results. Therefore, the initial parameter value of the first probability of each perceived object can also be determined based on the sum of probabilities and the total number of perceived objects.

[0128] In the specific implementation, referring to the example in step S301, each traffic sensing object P i The first probability a i The calculation formulas (2-2) and (2-3) for the original parameter values ​​and initial parameter values ​​are as follows:

[0129] a[i]=HIn[i] / HInmax (2-2)

[0130] a i =sum(a[i]) / M (2-3)

[0131] Among them, a[i] refers to each traffic perception object P i The first probability a i The original parameter value; sum(a[i]) represents the sum of a[1] to a[M], where M is the total number of traffic sensing objects.

[0132] Optionally, the estimated original parameter value of the first probability of each perceived object may be directly used as the initial parameter value of the first probability of each perceived object.

[0133] In one embodiment, the situations where the perception result data is erroneous due to intentional uploading of erroneous perception result data, lack of attention, obstructed observation and other factors are rare; therefore, the probability that a perception object perceives an environmental observation event but the environmental observation does not actually occur (i.e., the second probability) is usually smaller than the probability that a perception object perceives an environmental observation event but the environmental observation actually occurs (i.e., the first probability).

[0134] Therefore, the second probability of each perceived object can be accurately estimated based on the first probability of each perceived object. Then, the specific method of estimating the initial parameter value of the second probability of each perceived object can be: the initial parameter value of the first probability of each perceived object is subjected to probability reduction processing to obtain the initial parameter value of the second probability of each perceived object. Among them, the initial parameter value of the second probability of each perceived object is smaller than the initial parameter value of the first probability of each perceived object.

[0135] In the specific implementation, refer to the calculation formulas (2-2) and (2-3) in the above example, each traffic sensing object P i The second probability b i The calculation formula (2-4) for the initial parameter value is as follows:

[0136] b i =a i / 2 (2-4)

[0137] In one embodiment, since the event ratio is the ratio of the number of actual environmental observation events to the total number of environmental observation events, intentional uploading of erroneous perception result data, and the erroneous perception result data caused by lack of attention, obstructed observation, etc. are rare; therefore, in the initial estimation, the perception result data of each perception object can be regarded as reliable.

[0138] Therefore, the original parameter value of the event ratio can be determined based on the total number of environmental observation events perceived by each perception object and the total number of environmental observation events not perceived by each perception object.

[0139] Optionally, since the node excess of each node mentioned in step S302 can reflect the probability of each environmental observation event tending to the benchmark true value to a certain extent, and the expectation-maximization algorithm iterates in the direction of maximizing the parameter value of the preset parameter; therefore, if the event proportion is set too large in advance, the accuracy of the event confirmation result obtained based on the expectation-maximization algorithm will be reduced.

[0140] Therefore, the original parameter value of the event proportion can also be updated based on the total number of perceived objects and the node strength of each node to obtain the initial parameter value of the event proportion; wherein the product of the initial parameter value of the event proportion and the total number of perceived objects is less than the node strength of any node.

[0141] In the specific implementation, referring to the example in step S301, the proportion dxp of the binary conclusion "1" in the confirmed observation matrix PE can be calculated as the original parameter value of the event proportion; then, the original parameter value of the event proportion is updated by calculating formula (2-5) to obtain the initial parameter value of the event proportion. The calculation formula (2-5) is as follows:

[0142] NHn[j]≥dxp×M (2-5)

[0143] Among them, NHn[j] represents the traffic observation event E j The node strength of the corresponding node; the initial dxp represents the proportion of "1" in the confirmation observation matrix. After re-defining the value range of dxp according to the calculation formula (3-5), a value is selected from the value range of dxp as the initial parameter value of d. Optionally, the initial parameter value of d can be the maximum value of dxp.

[0144] Optionally, the original parameter value of the event proportion may be directly used as the initial parameter value of the event proportion.

[0145] S504: Based on the perception result data, the initial parameter value of the event proportion, the initial parameter value of the first probability of each perception object, and the initial parameter value of the second probability of each perception object are subjected to expectation maximization processing to obtain the event confirmation result of the environmental observation event.

[0146] In the embodiment of the present application, the specific implementation of step S504 can refer to the specific implementation of step S304 in the above embodiment, which will not be repeated here.

[0147] In an embodiment of the present application, the initial parameter values ​​of the first probability and the second probability of each perceived object are estimated by the corresponding hyperedge cardinality and hyperedge strength that can also reflect the reliability of the perceived object; the initial parameter value of the event proportion is estimated by the node strength that can reflect the trend of the environmental observation event to the benchmark true value. Therefore, the initial parameter values ​​of the preset parameters estimated by the hypergraph topology indicators associated with the preset parameters are accurate, reasonable, and better parameter values. It can be seen that the embodiment of the present application can provide accurate and reasonable initial parameter values ​​for the preset parameters required for the subsequent expectation-maximization algorithm by constructing a hypergraph, which is conducive to improving the algorithm accuracy of the expectation-maximization algorithm, thereby making the final event confirmation results of the environmental observation events more accurate, thereby greatly improving the accuracy of truth discovery.

[0148] At the same time, hypergraph topology indicators such as hyperedge cardinality and node strength can reflect the sparsity of the perception result data. Therefore, the embodiment of the present application can flexibly adjust the initial parameter values ​​of the preset parameters according to the sparsity of the perception result data, so that the initial parameter values ​​of the preset parameters used by the expectation-maximization algorithm are accurate parameter values ​​that match the sparsity of the perception result data, which is beneficial to improving the algorithm accuracy of the expectation-maximization algorithm, making the final event confirmation results of the environmental observation events more accurate, thereby improving the accuracy of true value discovery in data sets with different sparsity levels.

[0149] In one embodiment of the present application, another data processing method is provided, which can be applied to Figure 2 The data processing system shown in the figure, the method can be executed by the terminal device or the server, or can be executed by the terminal device and the server together. In the embodiment of the present application, the method is executed by the server as an example for explanation. Figure 6 FIG. 1 is a flow chart of another data processing method. Figure 3 and Figure 5 The expectation maximization process is extended based on the method shown in .

[0150] Among them, S601 to S606 are described in detail as follows:

[0151] S601: Construct a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events.

[0152] In the embodiment of the present application, the specific implementation of step S601 can refer to the specific implementation of step S301 in the above embodiment, which will not be repeated here.

[0153] S602: Determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph.

[0154] In the embodiment of the present application, the specific implementation of step S602 can refer to the specific implementation of step S302 in the above embodiment, which will not be repeated here.

[0155] S603: Estimate initial parameter values ​​of preset parameters according to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node.

[0156] In the embodiment of the present application, the specific implementation of step S503 can refer to the specific implementation of step S303 and step S403 in the above embodiment, which will not be repeated here.

[0157] S604: In the process of iteratively processing the preset parameters based on the perception result data, the initial parameter values ​​of the preset parameters are used as the first-round processing parameter values ​​for the first-round processing to obtain the predicted occurrence probability of the environmental observation event.

[0158] In an embodiment of the present application, the specific process of using the initial parameter values ​​of the preset parameters as the first-round processing parameter values ​​may include: using the initial parameter values ​​of the event proportions as the first-round processing parameter values ​​of the event proportions, using the initial parameter values ​​of the first probability of each perceived object as the first-round processing parameter values ​​of the first probability of each perceived object, and using the initial parameter values ​​of the second probability of each perceived object as the first-round processing parameter values ​​of the second probability of each perceived object.

[0159] In one embodiment, during the first round of processing, the first round of processing parameter values ​​are also the current round of processing parameter values ​​for the first round of processing, and each round of processing will generate a predicted probability of occurrence of an environmental observation event.

[0160] Specifically, the process of performing the current round of processing based on the current round of processing parameter values ​​may include:

[0161] 1) Based on the current round processing parameter value of the event proportion, estimate the first reference probability that the environmental observation event actually occurs and the second reference probability that the environmental observation event does not actually occur;

[0162] Specifically, since the event proportion represents the proportion of environmental observation events that actually occurred, the current round processing parameter value of the event proportion can be directly used as the first reference probability that the environmental observation event actually occurred; at the same time, since the environmental observation events that actually occurred are removed, what remains are the environmental observation events that did not actually occur. Therefore, the second reference probability that the environmental observation event did not actually occur can be directly calculated through the first reference probability that the environmental observation event actually occurred (that is, the current round processing parameter value of the event proportion), which is the actual occurrence of the environmental observation event.

[0163] 2) determining first perception data of the multiple perception objects on the environmental observation event when the environmental observation event actually occurs according to the current round processing parameter value of the first probability of each perception object and the perception result data of the multiple perception objects on the environmental observation event;

[0164] Specifically, since the perception result data of multiple perception objects to environmental observation events are composed of the perception result data of each perception object to the environmental observation events; therefore, the current round processing parameter value of the first probability of each perception object can be multiplied by the perception result data of each perception object to the environmental observation event to obtain the predicted perception data of each perception object to the environmental observation event; then the predicted perception data of each perception object to the environmental observation event are added to obtain the first perception data.

[0165] 3) determining second perception data of multiple perception objects on the environmental observation event when the environmental observation event has not actually occurred according to the current round processing parameter value of the second probability of each perception object and the perception result data;

[0166] Specifically, referring to the process of acquiring the first perception data, the current round processing parameter value of the second probability of each perception object can be multiplied by the perception result data of each perception object on the environmental observation event to obtain the estimated perception data of each perception object on the environmental observation event; then the estimated perception data of each perception object on the environmental observation event are added to obtain the second perception data.

[0167] 4) Based on the first reference probability, the second reference probability, the first perception data and the second perception data, determine the predicted occurrence probability corresponding to the current round of processing.

[0168] Specifically, the specific process of determining the predicted occurrence probability corresponding to the current round of processing can be: for the environmental observation event, multiply the first perception data of the environmental observation event by the first reference probability to obtain a first product; multiply the second perception data of the environmental observation event by the second reference probability to obtain a second product; divide the first product by the sum of the first product and the second product to obtain the predicted occurrence probability of the environmental observation event corresponding to the current round of processing.

[0169] It should be noted that when there are multiple environmental observation events, the first perception data should be the first perception data of multiple perception objects on each environmental observation event when each environmental observation event actually occurs, that is, each environmental observation event has corresponding first perception data. Similarly, each environmental observation event has corresponding second perception data. Therefore, the predicted probability of occurrence corresponding to the current round of processing is also the predicted probability of occurrence of each environmental observation event corresponding to the current round of processing.

[0170] S605: Determine the next round of processing parameter values ​​of preset parameters according to the predicted occurrence probability obtained from the current round of processing, and perform the next round of processing based on the next round of processing parameter values ​​to obtain the corresponding predicted occurrence probability.

[0171] In an embodiment of the present application, the number of environment perception observation events may include multiple ones.

[0172] In one embodiment, when the preset parameters include the first probability of each perceived object, the specific process of determining the next round of processing parameter values ​​of the preset parameters based on the predicted occurrence probability obtained from the current round of processing includes: for any perceived object, adding the predicted occurrence probabilities of each target environmental observation event perceived by any perceived object to obtain the target probability; adding the predicted occurrence probabilities of each environmental observation event to obtain the total probability; and predicting the next round of processing parameter values ​​of the first probability of any perceived object based on the target probability and the total probability.

[0173] Optionally, based on the target probability and the total probability, a specific method for predicting the next round of processing parameter values ​​of the first probability of any perceived object may be: using the ratio between the target probability and the total probability as the next round of processing parameter value of the first probability of any perceived object.

[0174] It should be noted that the predicted occurrence probability of each target environmental observation event and the predicted occurrence probability of each environmental observation event are the predicted occurrence probabilities obtained by the current round of processing, and will not be repeated later.

[0175] For example, for environmental observation events 1 to 5, the predicted occurrence probabilities obtained by the current round of processing are 0.5, 0.3, 0.1, 0.6, and 0.2, respectively. After observation, perception object a confirms that environmental observation events 1 and 3 occur, that is, perception object a only perceives environmental observation events 1 and 3, then the target environmental observation events are environmental observation events 1 and 3. The target probability is the sum of 0.5 and 0.1, that is, 0.6. The total probability is the sum of the predicted occurrence probabilities of each of the five environmental observation events, that is, 1.7. Then, the next round of processing parameter value of the first probability of perception object a is approximately 0.353.

[0176] In one embodiment, when the preset parameters include the first probability of each perceived object, the specific process of determining the parameter value of the next round of processing of the preset parameters based on the predicted occurrence probability obtained from the current round of processing includes: for any perceived object, based on the predicted occurrence probability of each target environmental observation event perceived by any perceived object, determining the probability that each target environmental observation event has not actually occurred; based on the predicted occurrence probability of each environmental observation event, determining the probability that each environmental observation event has not actually occurred; based on the predicted occurrence probability of each environmental observation event, predicting the next round of processing parameter value of the second probability of any perceived object.

[0177] Optionally, based on the probability that each target environmental observation event did not actually occur and the probability that each environmental observation event did not actually occur, the specific method of predicting the next round of processing parameter values ​​of the second probability of any perceived object can be: adding the probabilities that each target environmental observation event did not actually occur to obtain a reference probability; adding the probabilities that each environmental observation event did not actually occur to obtain a total probability value; and using the ratio between the reference probability and the total probability value as the next round of processing parameter values ​​for the second probability of any perceived object.

[0178] Referring to the above example, the probability that the perception object a perceives that environmental observation event 1 does not actually occur is 1-0.5, that is, 0.5; similarly, the probability that the perception object a perceives that environmental observation events 2 to 5 do not actually occur is 0.7, 0.9, 0.4, and 0.8, respectively. Therefore, the reference probability is 1.4, and the total probability value is 4.3. Then, the next round of processing parameter value of the second probability of the perception object a is approximately 0.326.

[0179] In one embodiment, when the preset parameters include event proportions, the specific process of determining the parameter values ​​for the next round of processing of the preset parameters based on the predicted probability of occurrence obtained from the current round of processing includes: obtaining the parameter values ​​for the next round of processing of the event proportions based on the predicted probability of occurrence of each environmental observation event obtained from the current round of processing and the total number of environmental observation events.

[0180] S606. If the difference between the next round processing parameter value and the current round processing parameter value is less than or equal to the preset parameter difference, stop the iterative processing and generate the event confirmation result of the environmental observation event based on the predicted occurrence probability obtained in the last round of processing.

[0181] In the embodiment of the present application, the preset parameter difference may be a reasonable difference set artificially based on experience, or may be set by a server or terminal device in the above-mentioned data processing system, and there is no limitation on this.

[0182] Since there can be multiple preset parameters, the difference between the next round processing parameter value and the current round processing parameter value can be: the difference between the next round processing parameter value of any preset parameter and the current round processing parameter value of any preset parameter; the difference between the next round processing parameter value and the current round processing parameter value can also be: the sum of the differences obtained by adding the differences between the next round processing parameter values ​​of each preset parameter and the current round processing parameter values ​​of each preset parameter, which is not limited here.

[0183] In a specific implementation, referring to the examples in step S301 and step S403, after obtaining the initial parameter values ​​of each preset parameter, the expectation maximization algorithm can be used to calculate the target parameter values ​​of each preset parameter and the event confirmation results of each traffic observation event. The algorithm process of the expectation maximization algorithm is as follows:

[0184] 1) According to the first probability a of each traffic perception object i The initial parameter value, the second probability b i The initial parameter value of and the initial parameter value of event proportion d constitute the processing parameter set θ for the first round of processing (1) (i.e. the first round of processing parameter values ​​of the preset parameters), θ (1) represents the processing parameter set θ during the first iteration.

[0185] 2) Before θ reaches the convergence condition, the traffic observation events E obtained during the nth round of processing can be calculated based on the calculation formula (3-1): j The predicted probability of occurrence

[0186]

[0187] Among them, X j represents the jth traffic observation event E j ,θ (n) represents the processing parameter set θ during the nth round of processing; p(X j ,θ (n) |z j =1) and p(X j ,θ (n) |z j = 0) respectively indicates that the jth traffic observation event is actually true (i.e., z j =1) or false (i.e. z j =0), the conditional probability of estimating the parameter set θ for the jth traffic observation event and the nth round of processing.

[0188] a i (n) Represents the traffic perception object P i The first probability ai The nth round of processing parameter value; b i (n) Represents the traffic perception object P i The second probability b i The nth round of processing parameter value; and, d (n) Indicates the processing parameter value of the nth round with event proportion d.

[0189] P i E j It is to confirm the binary conclusion of the i-th traffic perception object in the observation matrix PE on the j-th traffic observation event; This is equivalent to calculating the first perception data mentioned in step S605. In, when P i E j =1 (i.e. the binary conclusion is "1"), then retain a i (n) Do the product, when P i E j =0 (i.e. the binary conclusion is "1"), then retain (1-a i (n) ) to do the product. Similarly, I will not go into details here. is equivalent to calculating the second perception data mentioned in step S604. (n) This is equivalent to calculating the first reference probability mentioned in step S604, and (1-d (n) ) is equivalent to calculating the second reference probability mentioned in step S604.

[0190] 3) Before θ reaches the convergence condition, calculate the traffic perception object P based on calculation formulas (3-2), (3-3) and (3-4) i The n+1th round processing parameter values ​​of the first probability and the second probability, and the n+1th round processing parameter value of the event proportion.

[0191]

[0192]

[0193]

[0194] Among them, PEt i Refers to the traffic perception object P in the confirmed observation matrix i The event set consisting of the traffic observation events with the binary conclusion of "1" uploaded. In other words, The meaning is to PEt i Each traffic observation event E obtained when all traffic observation events in the nth round are processedj The probability of occurrence is indeed added. i is the traffic sensing object P i The total number of traffic observation events with a binary conclusion of "1" uploaded, i.e., the traffic perception object P i The hyperedge cardinality of .

[0195] In addition, for ease of understanding, formula (3-3) is expanded as shown in formula (3-5):

[0196]

[0197] Among them, j1,j2,...,jBH i ∈PEt i .

[0198] 4) Repeat steps 2) and 3) in step 4 until convergence is reached. (n) and θ (n+1) If the difference between is less than or equal to the preset parameter difference, it is determined that the convergence condition is met; if θ (n) and θ (n+1) If the difference between them is greater than the preset parameter difference, it is determined that the convergence condition has not been met.

[0199] 5) When the processing parameter set θ reaches the convergence condition, the last iteration As traffic observation event E j The predicted probability of occurrence, and the last round of processing and d (n) The current round processing parameter value is taken as a i , b i , the target parameter value of d.

[0200] At the same time, according to the last iteration Determine traffic observation event E j The true value of the traffic observation event E j In fact, it actually happened), that is, the traffic observation event E j The event confirmation result. Because the traffic perception result uploaded by the traffic perception object is a binary conclusion, When the traffic observation event E j It is true and it really happened; otherwise it is false and it did not really happen.

[0201] In practical applications, please refer to the attached Figure 7 , which shows a schematic diagram of the process of truth value discovery in a smart transportation scenario. Figure 7The smart traffic scenario in is related to the example in the above step S301 and its related examples. The formulas and characters are the same and can be understood in combination.

[0202] like Figure 7 As shown, the server can receive the traffic network information in the actual traffic scene 701, and receive binary conclusions (i.e., the binary conclusions of a group of traffic observation events E in the actual traffic scene 701) uploaded by a group of taxi drivers (i.e., a group of traffic perception objects P). Figure 7 Traffic perception result data in ).

[0203] The first step, such as Figure 7 As shown, the server can build a model based on the received data and information, that is, establish a more general mathematical model, and define the actual meaning of the parameters used in the model analysis. Among them, the mathematical model in this solution refers to the use and representation of traffic perception data in the smart traffic scenario, and the process of establishing the corresponding mapping relationship.

[0204] Specifically, we can firstly M ,P i represents the i-th traffic sensing object, there are M traffic sensing objects in total) for a set of traffic observation events E(E1,...,E N , E j Represents the jth traffic observation event, there are N binary conclusions in total to create a confirmed observation matrix PE.

[0205] Then, the three parameters used in the model analysis can be defined as follows:

[0206] 1)t i : represents the traffic perception object P i The probability that the uploaded binary conclusion is consistent with the benchmark truth value of the traffic observation event can also be called the perception reliability of the traffic perception object; the benchmark truth value of the traffic observation event refers to the true value of whether the traffic observation event occurs. i The calculation formula (4-1) is as follows:

[0207] t i =P(E tj |P i E j ) (4-1)

[0208] Among them, E tj Indicates traffic observation event E j is actually true, that is, its base truth value is 1; similarly, we can define E fj Indicates traffic observation event E j It is actually false, that is, its benchmark truth value is 0, and will not be further explained later.

[0209] 2)a i Represents the traffic perception object P i The uploaded binary conclusion confirms that a traffic observation event actually occurred. The probability that this traffic observation event actually occurred is the first probability; b i Represents the traffic perception object P i The binary conclusion of represents the probability that a traffic observation event did not actually occur, but the traffic observation event actually occurred, that is, the second probability. i and b i are two parameters that can be used to measure the perception reliability of traffic perception objects. i and b i The calculation formulas (4-2) and (4-3) are as follows:

[0210] a i =P(P i E j |E tj ) (4-2)

[0211] b i =P(P i E j |E fj ) (4-3)

[0212] 3) d represents the overall probability that a traffic observation event randomly selected from multiple traffic observation events is true, that is, for all traffic observation events, E j =1, that is, the event proportion; where the calculation formula (4-4) of d is as follows:

[0213] d=P(E tj |E j ) (4-4)

[0214] Step 2: If Figure 7 As shown in block diagram 702, the collected traffic perception data can be further abstracted, and the mathematical model established in the first step can be improved by introducing hypergraph theory and related topological indicators to obtain the corresponding hypergraph (i.e., the confirmed observation matrix with hypergraph characteristics in block diagram 702).

[0215] Specifically, if Figure 7 As shown in block diagram 703 in FIG. 7 , three hypergraph topology indicators in hypergraph theory can be introduced: hyperedge cardinality, node hyperdegree, and hyperedge intensity.

[0216] In the smart traffic scenario, the traffic perception result of each traffic perception object for multiple traffic observation events is a hyperedge, and a traffic observation event can be regarded as a node.

[0217] Therefore, in the confirmed observation matrix PE, each traffic sensing object P i The hyperedge cardinality represents the traffic sensing object P i The number of binary conclusions uploaded as "1", the hyperedge cardinality can intuitively reflect the workload of each traffic perception object to a certain extent, and the overall hyperedge cardinality can reflect the sparsity of the confirmation observation matrix PE to a certain extent. The hyperedge cardinality can be used for subsequent optimization calculation of each traffic perception object P i Reliability i .

[0218] In the confirmed observation matrix PE, each traffic observation event E j The node excess degree of represents the number of traffic sensing objects that observe it and report the binary conclusion "1". The node excess degree can reflect the probability of each traffic observation event tending to the benchmark truth value to a certain extent, and the overall node excess degree can reflect the value range of d in the confirmation observation matrix PE to a certain extent. The node excess degree is used to calculate the better initial parameter value of d in the subsequent calculation.

[0219] The hyperedge strength can be summarized as the number of nodes contained in a hyperedge and all its adjacent hyperedges. In the confirmed observation matrix PE, the hyperedge strength of a traffic perception object can reflect its similarity with other traffic perception objects to a certain extent, that is, comprehensively consider the overall data of the remaining traffic perception objects as a reference for evaluating the reliability of traffic perception objects. Especially for sparse matrices, the hyperedge strength can play a role in making comprehensive use of the overall data as much as possible and amplifying the similarity between traffic perception objects. The hyperedge strength can be used for subsequent calculations with each traffic perception object P i Related i and b i Better initial parameter values.

[0220] Step 3: If Figure 7 As shown in block diagram 703 in FIG. 1 , the confirmed observation matrix with hypergraph features can be processed by HGPI Alogorithm (i.e., parameter optimization algorithm) to determine the preset parameter a i , b i and d better initial parameter values.

[0221] Specifically, the processing of HGPI Alogorithm is as follows:

[0222] 1) For each traffic perception object P i, each traffic sensing object P can be calculated i The calculation formula of the super-edge strength can be referred to formula (2-1) and will not be repeated here.

[0223] 2) Calculate each traffic perception object P i The hyperedge cardinality HInum[i].

[0224] 3) Calculate the proportion dxp of binary conclusions “1” in the confirmed observation matrix PE.

[0225] 4) Find the maximum hyperedge cardinality from the hyperedge cardinality calculated in step 2), and calculate the target hyperedge strength HInmax based on the calculation formula (2-1) in step S503

[0226] 5) Based on the calculation results of the previous steps 1) to 4), a better first probability a of each traffic sensing object is calculated by the calculation formulas (2-2), (2-3) and (2-4) in step S503. i and the second probability b i The initial parameter value of .

[0227] 6) Calculate the node excess NHn[j] of each traffic observation event.

[0228] 7) Calculate the initial parameter value of the event proportion d based on dxp and the calculation formula (2-5) mentioned in step S503.

[0229] Step 4: If Figure 7 As shown in the block diagram 703 in the figure, based on the more optimal initial estimated values ​​of each preset parameter obtained in the third step, the expectation maximization algorithm (i.e., EM Alogorithm) can be used to calculate the target parameter values ​​of each preset parameter and the predicted occurrence probability of each traffic observation event. The specific algorithm process of the expectation maximization algorithm can refer to the specific description of the algorithm process of the expectation maximization algorithm in step S606, which will not be repeated here.

[0230] At the same time, according to the last iteration Determine traffic observation event E j The true value of the traffic observation event E j In fact, it actually happened), that is, the traffic observation event E j The event confirmation result. Because the traffic perception result uploaded by the traffic perception object is a binary conclusion, When the traffic observation event E j It is true and it really happened; otherwise it is false and it did not really happen.

[0231] Step 5: If Figure 7As shown in block diagram 703 in FIG. 1 , reliable data can be screened out to form a reliable data set according to the algorithm result of EM Alogorithm, and reliable objects can be screened out from traffic perception objects.

[0232] Specifically, according to the above t i From the definition of , we can know that we can directly confirm the traffic perception object P in the observation matrix i Based on the binary conclusions of each traffic observation event and the event confirmation results of each traffic observation event obtained in the fourth step, the traffic perception object P is calculated. i Perceived reliability of i .

[0233] Optionally, the first probability a can be i and the second probability b i Substitute the target parameter value into the Bayesian formula to obtain each traffic perception object P i Perceived reliability of i .

[0234] Then, the traffic perception objects whose perception reliability is greater than the preset reliability threshold can be determined as reliable objects. Furthermore, when the server obtains the data collection task for a new traffic observation event, it can publish the new data collection task to the reliable object; then, the server can directly use the perception result data uploaded by the reliable object as reliable data, so that the reliable perception data can be applied to various aspects such as machine learning model training and big data processing in the fields of traffic and navigation.

[0235] At the same time, reliable traffic perception data can be screened out from the traffic perception data through the event confirmation results of each traffic observation event, so as to carry out corresponding machine learning, big data and other processing through the reliable traffic perception data.

[0236] In practical applications, in order to verify the effectiveness of the algorithm of this scheme, two benchmark algorithms, the majority voting algorithm and the conventional truth value discovery algorithm based on the EM algorithm, can be used as references to conduct comparative experiments with the truth value discovery algorithm of the fusion hypergraph and EM proposed in this scheme.

[0237] Among them, the two benchmark algorithms and the truth value discovery algorithm of the fusion hypergraph and EM proposed in the embodiment of the present application can be evaluated and compared on different data sets for multiple indicators. The indicators used for evaluation and comparison can include the accuracy of the truth value judgment of the environmental perception event and the average error of the reliability of the perception object.

[0238] The accuracy of true value judgment of environmental perception events indicates the proportion of correct judgments of the benchmark true value of environmental perception events by the corresponding algorithm. The average error of the reliability of the perception object indicates the mean of the reliability ratios of all perception objects. The reliability ratio of any perception object refers to the ratio of the difference between the reliability of any perception object predicted by the corresponding algorithm and the true reliability of any perception object to the true reliability of any perception object. In addition, the greater the accuracy of true value judgment, the higher the accuracy of true value discovery of the corresponding algorithm; the smaller the average error of the reliability of the perception object, the higher the accuracy of true value discovery of the corresponding algorithm.

[0239] For details, please see the attached Figure 8a , shows a schematic diagram of the comparison of the accuracy of the true value judgment of an environmental observation event. Figure 8a The horizontal axis of the statistical graph in is the data set (i.e., data), and the vertical axis is the accuracy of true value judgment. The light gray column corresponds to the majority voting algorithm, the dark gray column corresponds to the truth value discovery algorithm based on the EM algorithm, and the black column corresponds to the truth value discovery algorithm of the fusion hypergraph and EM proposed in the embodiment of the present application, which will not be repeated later. Figure 8a The dxp (i.e., the proportion of binary conclusions "1" in the confirmed observation matrix, which can be used to measure the sparsity of the confirmed observation matrix; the smaller the dxp, the sparser the confirmed observation matrix) of data11 to data15 is fixed, and d (i.e., the overall probability that a randomly selected environmental observation event is true) varies from 0.1 to 0.5, with a step size of 0.1.

[0240] Depend on Figure 8a As can be seen from the statistical chart in, as d increases, the truth value judgment accuracy of all algorithms shows a downward trend; but no matter how d changes, the truth value judgment accuracy of the algorithm in this application is always much higher than the truth value judgment accuracy of the other two benchmark algorithms.

[0241] Please see attached Figure 8b , which shows a comparative diagram of the accuracy of true value judgment of another environmental observation event. Figure 8b In the example, d from data16 to data20 is fixed, and dxp varies from 0.10 to 0.30 with a step size of 0.05. Figure 8b From the statistical chart in , we can see that as dxp increases, the accuracy of the true value judgment of the two benchmark algorithms remains unchanged at first, and then increases; while the accuracy of the true value judgment of the algorithm of this solution increases first and then decreases, indicating that the algorithm of this application is more effective for sparse matrices. At the same time, no matter how dxp changes, the accuracy of the true value judgment of the algorithm of this application is always much higher than that of the other two benchmark algorithms.

[0242] Please see attached Figure 9a , which shows a comparative diagram of the average error of the perceived object reliability. Figure 9a The horizontal axis of the statistical graph in is the data set (i.e., data), and the vertical axis is the average error of the reliability of the perceived object. The light gray column corresponds to the majority voting algorithm, the dark gray column corresponds to the truth value discovery algorithm based on the EM algorithm, and the black column corresponds to the truth value discovery algorithm of the fusion hypergraph and EM proposed in the embodiment of the present application, which will not be repeated later. Figure 9a In the data11 to data15, dxp is fixed, d varies from 0.1 to 0.5, and the step size is 0.1.

[0243] Depend on Figure 9a From the statistical graph in , we can see that, for the average error of perceived object reliability, as d increases, the majority voting algorithm shows a trend of first decreasing and then increasing, the truth value discovery algorithm based on the EM algorithm shows a trend of first increasing and then decreasing, and the algorithm of this scheme shows a trend of first decreasing and then increasing. However, no matter how d changes, the average error of perceived object reliability of the algorithm of this scheme has always been much lower than the average error of perceived object reliability of the other two benchmark algorithms.

[0244] Please see attached Figure 9b , showing another comparative schematic diagram of the average error of perceived object reliability. Figure 9b In the example, d from data16 to data20 is fixed, and dxp varies from 0.10 to 0.30 with a step size of 0.05. Figure 9b From the statistical chart in , we can see that as dxp increases, the average error of the perceived object reliability of the two benchmark algorithms shows a slight downward trend overall; while the average error of the perceived object reliability of the algorithm of the present application shows a trend of first decreasing and then slightly increasing overall. However, no matter how dxp changes, the average error of the perceived object reliability of the algorithm of the present application has always been much lower than the average error of the perceived object reliability of the other two benchmark algorithms.

[0245] It can be seen that compared with the other two benchmark algorithms, the algorithm of this application can greatly improve the accuracy of true value discovery and is effective.

[0246] In an embodiment of the present application, a hypergraph is first constructed through the perception result data, so that the initial parameter values ​​of the preset parameters can be accurately and reasonably estimated through the hypergraph topology indicators such as the hyperedge cardinality of each hyperedge, the hyperedge strength, and the node strength of each node, which are associated with the preset parameters required by the expectation-maximization algorithm, thereby obtaining better and more reasonable initial parameter values. In an embodiment of the present application, the better initial parameter values ​​of the preset parameters provide a good start for the iterative processing of the expectation-maximization algorithm, so that the predicted probability of occurrence obtained in each iterative processing will not deviate too much from the benchmark true value, which is conducive to improving the algorithm accuracy of the expectation-maximization algorithm, thereby making the final event confirmation result of the environmental observation event more accurate, thereby greatly improving the accuracy of truth discovery.

[0247] Here, the device embodiment of the present application is introduced, which can be used to execute the data processing method in the above embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the above embodiment of the data processing method of the present application.

[0248] The present application embodiment provides a data processing device, such as Fig.10 As shown, the device includes a construction unit 1001, a determination unit 1002 and a processing unit 1003, wherein:

[0249] A construction unit 1001 is used to construct a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events, wherein the environmental observation events are used to constitute the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object;

[0250] The determination unit 1002 is used to determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; wherein the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation event;

[0251] The determination unit 1002 is further configured to estimate the initial parameter value of the preset parameter according to the hyperedge cardinality and the hyperedge strength of each hyperedge and the node strength of each node; wherein the preset parameter includes at least one of the event proportion of the environmental observation event that actually occurred, the first probability that each sensing object senses the environmental observation event that actually occurred, and the second probability that each sensing object senses the environmental observation event that did not actually occur;

[0252] The processing unit 1003 is used to perform expectation maximization processing on the initial parameter values ​​of the preset parameters based on the perception result data to obtain an event confirmation result of the environmental observation event, and the event confirmation result represents whether the corresponding environmental observation event actually occurs.

[0253] In one embodiment of the present application, based on the aforementioned scheme, when the processing unit 1003 performs expectation maximization processing on the initial parameter value of the preset parameter based on the perception result data to obtain the event confirmation result of the environmental observation event, it can be specifically used for: in the process of iterative processing of the preset parameter based on the perception result data, the initial parameter value of the preset parameter is used as the first-round processing parameter value for the first round of processing to obtain the predicted occurrence probability of the environmental observation event; the next round of processing parameter value of the preset parameter is determined according to the predicted occurrence probability obtained by the current round of processing, and the next round of processing is performed based on the next round of processing parameter value to obtain the corresponding predicted occurrence probability; if the difference between the next round of processing parameter value and the current round of processing parameter value is less than or equal to the preset parameter difference, the iterative processing is stopped, and the event confirmation result of the environmental observation event is generated based on the predicted occurrence probability obtained by the last round of processing.

[0254] In one embodiment of the present application, based on the aforementioned scheme, when the processing unit 1003 performs the current round of processing based on the current round of processing parameter values, it can be specifically used to: estimate a first reference probability that an environmental observation event actually occurs and a second reference probability that the environmental observation event does not actually occur based on the current round of processing parameter values ​​of the event proportion; determine first perception data of multiple perception objects on the environmental observation event when the environmental observation event actually occurs according to the current round of processing parameter values ​​of the first probability of each perception object and the perception result data; determine second perception data of multiple perception objects on the environmental observation event when the environmental observation event does not actually occur according to the current round of processing parameter values ​​of the second probability of each perception object and the perception result data; determine the predicted occurrence probability corresponding to the current round of processing based on the first reference probability, the second reference probability, the first perception data and the second perception data.

[0255] In one embodiment of the present application, the number of environmental perception observation events includes multiple; based on the aforementioned scheme, when the processing unit 1003 determines the next round of processing parameter values ​​of the preset parameters according to the predicted occurrence probability obtained by the current round of processing, it can be specifically used to: for any perception object, add the predicted occurrence probabilities of each target environmental observation event perceived by any perception object to obtain the target probability; add the predicted occurrence probabilities of each environmental observation event to obtain the total probability; based on the target probability and the total probability, predict the next round of processing parameter values ​​of the first probability of any perception object.

[0256] In one embodiment of the present application, based on the aforementioned scheme, when the processing unit 1003 determines the parameter value of the next round of processing of the preset parameters according to the predicted occurrence probability obtained from the current round of processing, it can be specifically used to: for any perceived object, determine the probability that each target environmental observation event does not actually occur according to the predicted occurrence probability of each target environmental observation event perceived by any perceived object; determine the probability that each environmental observation event does not actually occur according to the predicted occurrence probability of each environmental observation event; predict the next round of processing parameter value of the second probability for any perceived object based on the probability that each target environmental observation event does not actually occur and the probability that each environmental observation event does not actually occur.

[0257] In one embodiment of the present application, based on the aforementioned scheme, when the determination unit 1002 estimates the initial parameter value of the preset parameter according to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node, it can be specifically used to: determine the maximum hyperedge cardinality among multiple hyperedge cardinalities; determine the target hyperedge strength based on the maximum hyperedge cardinality and the total number of perceived objects; determine the initial parameter value of the first probability of each perceived object according to the hyperedge strength of each hyperedge and the target hyperedge strength.

[0258] In one embodiment of the present application, based on the aforementioned scheme, when the determination unit 1002 determines the initial parameter value of the first probability of each perceived object according to the hyper-edge strength of each hyper-edge and the target hyper-edge strength, it can be specifically used to: estimate the original parameter value of the first probability of each perceived object according to the hyper-edge strength of each hyper-edge and the target hyper-edge strength; add the original parameter values ​​of the first probability of each perceived object to obtain the sum of probabilities; and determine the initial parameter value of the first probability of each perceived object based on the sum of probabilities and the total number of perceived objects.

[0259] In one embodiment of the present application, based on the above-mentioned scheme, the determination unit 1002 can also be specifically used to: perform probability reduction processing on the initial parameter value of the first probability of each perceived object to obtain the initial parameter value of the second probability of each perceived object.

[0260] In one embodiment of the present application, based on the aforementioned scheme, when the determination unit 1002 determines the initial parameter value of the first probability of each perceived object according to the hyper-edge strength of each hyper-edge and the target hyper-edge strength, it can be specifically used to: determine the original parameter value of the event share based on the total number of environmental observation events perceived by each perceived object and the total number of environmental observation events not perceived by each perceived object; update the original parameter value of the event share based on the total number of perceived objects and the node strength of each node to obtain the initial parameter value of the event share; wherein the product of the initial parameter value of the event share and the total number of perceived objects is less than the node strength of any node.

[0261] Based on the above-mentioned scheme, the processing unit 1003 can also be specifically used to: determine the perception reliability of each perception object based on the perception result data and event confirmation results of each perception object on the environmental observation event; determine the perception object whose perception reliability is greater than the preset reliability threshold as a reliable object.

[0262] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0263] The device provided by the above-mentioned embodiment can be arranged in a terminal device or in a server. The device provided by the embodiment of the present application can first construct a hypergraph through the perception result data, so that the initial parameter values ​​of the preset parameters can be accurately and reasonably estimated through the hypergraph topology indicators such as the hyperedge cardinality of each hyperedge, the hyperedge strength and the node strength of each node, which are associated with the preset parameters required by the expectation maximization algorithm, thereby obtaining better and more reasonable initial parameter values. It can be seen that the embodiment of the present application can provide accurate and reasonable initial parameter values ​​for the preset parameters required by the subsequent expectation maximization algorithm by constructing a hypergraph, which is conducive to improving the algorithm accuracy of the expectation maximization algorithm, so that the event confirmation results of the environmental observation events finally obtained are more accurate, thereby greatly improving the accuracy of truth value discovery, especially the truth value discovery of sparse data such as perception result data.

[0264] An embodiment of the present application also provides an electronic device, comprising one or more processors and a storage device, wherein the storage device is used to store one or more computer programs, and when the one or more computer programs are executed by one or more processors, the electronic device implements the above data processing method.

[0265] Fig.11 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0266] It should be noted that Fig.11 The computer system 1100 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0267] like Fig.11As shown, the computer system 1100 includes a processor (Central Processing Unit, CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (Read-Only Memory, ROM) 1102 or the program loaded from the storage part 1108 to the random access memory (Random Access Memory, RAM) 1103, such as executing the method in the above embodiment. In the RAM 1103, various programs and data required for system operation are also stored. The CPU 1101, ROM 1102 and RAM 1103 are connected to each other through a bus 1104. An input / output (Input / Output, I / O) interface 1105 is also connected to the bus 1104.

[0268] In some embodiments, the following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0269] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1109, and / or installed from a removable medium 1111. When the computer program is executed by a processor (CPU) 1101, various functions defined in the system of the present application are executed.

[0270] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (Erasable Programmable Read Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0271] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and a computer program.

[0272] The units or modules involved in the embodiments described in this application may be implemented by software or hardware, and the units or modules described may also be set in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0273] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the data processing method as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0274] Another aspect of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the data processing method as described above in the above embodiments.

[0275] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0276] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0277] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person of ordinary skill in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A data processing method, characterized in that: include: Constructing a corresponding hypergraph based on the perception result data of multiple perception objects on environmental observation events, wherein the environmental observation events are used to constitute the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object; Determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; wherein the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation event; According to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node, the initial parameter value of the preset parameter is estimated; wherein the preset parameter includes at least one of the event proportion of the environmental observation event that actually occurred, the first probability that each perception object perceives the environmental observation event that actually occurred, and the second probability that each perception object perceives the environmental observation event that did not actually occur; Based on the perception result data, the initial parameter value of the preset parameter is subjected to expectation maximization processing to obtain an event confirmation result of the environmental observation event, and the event confirmation result represents whether the corresponding environmental observation event actually occurs.

2. The method according to claim 1, characterized in that The method of performing expectation maximization processing on the initial parameter value of the preset parameter based on the perception result data to obtain the event confirmation result of the environmental observation event includes: In the process of iteratively processing the preset parameters based on the perception result data, the initial parameter values ​​of the preset parameters are used as the first-round processing parameter values ​​for the first-round processing to obtain the predicted occurrence probability of the environmental observation event; Determine the next round of processing parameter values ​​of the preset parameters according to the predicted occurrence probability obtained by the current round of processing, and perform the next round of processing based on the next round of processing parameter values ​​to obtain the corresponding predicted occurrence probability; If the difference between the next round processing parameter value and the current round processing parameter value is less than or equal to the preset parameter difference, the iterative processing is stopped, and the event confirmation result of the environmental observation event is generated based on the predicted occurrence probability obtained in the last round of processing.

3. The method according to claim 2, characterized in that The process of performing the current round of processing based on the current round of processing parameter values ​​includes: Based on the current round processing parameter value of the event proportion, estimate a first reference probability that the environmental observation event actually occurs and a second reference probability that the environmental observation event does not actually occur; Determine, according to the current round processing parameter value of the first probability of each perception object and the perception result data, the first perception data of the multiple perception objects on the environmental observation event when the environmental observation event actually occurs; Determining, according to the current round processing parameter value of the second probability of each perception object and the perception result data, the second perception data of the multiple perception objects on the environmental observation event when the environmental observation event does not actually occur; Based on the first reference probability, the second reference probability, the first perception data and the second perception data, the predicted occurrence probability corresponding to the current round of processing is determined.

4. The method according to claim 2, characterized in that: The number of the environmental perception observation events includes a plurality of events; and the next round of processing parameter values ​​of the preset parameters are determined according to the predicted probability of occurrence obtained from the current round of processing, including: For any perception object, the predicted occurrence probabilities of each target environment observation event perceived by the perception object are added together to obtain the target probability; Add the predicted probability of occurrence of each environmental observation event to obtain the total probability; Based on the target probability and the total probability, predict the next round of processing parameter values ​​of the first probability of any perceived object.

5. The method according to claim 2, characterized in that: The step of determining the next round of processing parameter values ​​of the preset parameters according to the predicted occurrence probability obtained from the current round of processing includes: For any perception object, according to the predicted probability of occurrence of each target environment observation event perceived by the any perception object, determine the probability that each target environment observation event has not actually occurred; Determining the probability that each environmental observation event has not actually occurred based on the predicted probability of occurrence of each environmental observation event; Based on the probability that each target environmental observation event does not actually occur and the probability that each environmental observation event does not actually occur, the next round of processing parameter values ​​of the second probability of any perceived object are predicted.

6. The method according to any one of claims 1 to 5, characterized in that: The estimating the initial parameter value of the preset parameter according to the hyperedge cardinality and the hyperedge strength of each hyperedge and the node strength of each node includes: determining a maximum hyperedge cardinality among a plurality of hyperedge cardins; Determining a target hyperedge strength based on the maximum hyperedge cardinality and the total number of perceived objects; According to the hyper-edge strength of each hyper-edge and the target hyper-edge strength, an initial parameter value of the first probability of each perceived object is determined.

7. The method according to claim 6, characterized in that The determining, according to the hyper-edge strength of each hyper-edge and the target hyper-edge strength, an initial parameter value of the first probability of each perceived object comprises: estimating original parameter values ​​of the first probabilities of the respective perceived objects according to the hyper-edge strengths of the respective hyper-edges and the target hyper-edge strength; Adding the original parameter values ​​of the first probabilities of the respective perceived objects to obtain a probability sum; Based on the probability sum and the total number of the perceived objects, an initial parameter value of the first probability of each perceived object is determined.

8. The method according to claim 6, characterized in that The method further comprises: The initial parameter values ​​of the first probabilities of the respective perceived objects are subjected to probability reduction processing to obtain the initial parameter values ​​of the second probabilities of the respective perceived objects.

9. The method according to any one of claims 1 to 5, characterized in that: The estimating the initial parameter value of the preset parameter according to the hyperedge cardinality and the hyperedge strength of each hyperedge and the node strength of each node includes: Determine the original parameter value of the event proportion based on the total number of environmental observation events perceived by each of the perception objects and the total number of environmental observation events not perceived by each of the perception objects; Based on the total number of perceived objects and the node strength of each node, the original parameter value of the event proportion is updated to obtain the initial parameter value of the event proportion; wherein the product of the initial parameter value of the event proportion and the total number of perceived objects is less than the node strength of any node.

10. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Determining the perception reliability of each perception object based on the perception result data of each perception object on the environmental observation event and the event confirmation result; A perceived object whose perceived reliability is greater than a preset reliability threshold is determined as a reliable object.

11. A data processing device, characterized in that: It includes a construction unit, a determination unit and a processing unit, wherein: The construction unit is used to construct a corresponding hypergraph based on the perception result data of the environmental observation events by the multiple perception objects, wherein the environmental observation events are used to constitute the nodes included in the hypergraph, and the hyperedges included in the hypergraph are determined by the set of environmental observation events perceived by the same perception object; The determination unit is used to determine the hyperedge cardinality and hyperedge strength of each hyperedge in the hypergraph, and the node strength of each node in the hypergraph; wherein the hyperedge cardinality represents the number of environmental observation events perceived by the corresponding perception object, the hyperedge strength is obtained by the number of environmental observation events perceived by the corresponding perception object and other perception objects, and the node strength represents the number of perception objects that perceive the corresponding environmental observation event; The determination unit is further used to estimate the initial parameter value of the preset parameter according to the hyperedge cardinality and hyperedge strength of each hyperedge and the node strength of each node; wherein the preset parameter includes at least one of the event proportion of the environmental observation event that actually occurred, the first probability that each perception object perceives the environmental observation event that actually occurred, and the second probability that each perception object perceives the environmental observation event that did not actually occur; The processing unit is used to perform expectation maximization processing on the initial parameter value of the preset parameter based on the perception result data to obtain an event confirmation result of the environmental observation event, and the event confirmation result represents whether the corresponding environmental observation event actually occurs.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the data processing method as described in any one of claims 1 to 10.

Images