Data management method and device based on semantic hypergraph, electronic equipment and medium
By extracting and matching vehicle features in vehicle event data and building a semantic hypergraph of vehicle event, the problem of low efficiency in traditional vehicle event management is solved, and more efficient and accurate data management is achieved.
Patent Information
- Application Number
- CN202510212055.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-25
- Publication Date
- 2025-05-27
AI Technical Summary
Traditional vehicle event management and data construction mainly rely on manual labor, with limitations such as low operational efficiency and incomplete information coverage, making it difficult to adapt to the rapid development needs of modern urban transportation.
By obtaining historical vehicle information and vehicle event data, the target image corresponding to the target event data is determined for image extraction, the vehicle picture is obtained, and the vehicle picture is extracted to obtain vehicle features, and the vehicle features are obtained. Based on the matching results of vehicle pictures and historical vehicle information or the matching degree between vehicle characteristics and historical vehicle information, the target vehicle corresponding to the target event data is determined, and a vehicle event semantic hypergraph is constructed for data management.
It realizes the more accurate determination of the vehicle corresponding to the event data, and then determines the relationship between the event and the vehicle. It provides a more intuitive and compact way to represent the relationship between the data through the vehicle event semantic hypergraph, which improves the efficiency and accuracy of data management.
Smart Images
Figure CN120045743A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of intelligent transportation, and particularly relates to a data management method, device, electronic device and medium based on a semantic hypergraph. Background Art
[0002] With the continuous progress of intelligent transportation systems, the demand for data management such as vehicle event management and safety management is increasing day by day. Under the framework of a smart city, intelligent transportation has become an important part of promoting the sustainable development of the city, aiming to improve traffic efficiency, reduce traffic accidents, and optimize traffic resource allocation through advanced information technology. However, traditional data management such as vehicle event management and data construction mainly relies on manual work, with limitations such as low operation efficiency and incomplete information coverage, and it is difficult to meet the rapid development needs of modern urban transportation.
[0003] The relevant vehicle data sources are extensive and complex, and effective data collection, cleaning and integration have become a major problem. There may be problems such as inconsistent formats and uneven quality between different data sources, which affect the usability of the data and cannot intuitively display the relevant data information, resulting in problems such as low efficiency. Obviously, there is an urgent need for a data management method based on a semantic hypergraph to solve at least one of the above problems.
[0004] It should be noted that the above content only provides background technical information related to the present application and does not necessarily constitute prior art. Summary of the Invention
[0005] In view of the above-mentioned disadvantages of the prior art, the present application provides a data management method, device, electronic device and medium based on a semantic hypergraph, so as to provide a more intuitive and compact way to represent the relationship between data through a vehicle event semantic hypergraph, thereby improving the efficiency and accuracy of data management.
[0006] Other features and advantages of the present application will become apparent through the following detailed description, or be learned in part through the practice of the present application.
[0007] According to one aspect of the embodiments of the present application, a data management method based on a semantic hypergraph is provided, including: obtaining historical vehicle information and vehicle event data; determining target event data from the vehicle event data, extracting a target picture corresponding to the target event data to obtain a vehicle picture, and extracting features of the vehicle picture to obtain vehicle features; determining a target vehicle corresponding to the target event data based on a first matching result between the vehicle picture and the historical vehicle information, or a second matching result between the vehicle features and the historical vehicle information; constructing a vehicle event semantic hypergraph according to the target vehicles corresponding to each target event data, so as to perform data management according to the vehicle event semantic hypergraph.
[0008] In an embodiment of the present application, based on the foregoing solution, the target picture corresponding to the target event data is extracted to obtain a vehicle picture, and the vehicle picture is subjected to feature extraction to obtain vehicle features, including: cropping the target picture based on the vehicle position data in the target event data to obtain the vehicle picture; performing feature extraction on the vehicle picture according to preset extraction features to obtain the vehicle features.
[0009] In an embodiment of the present application, based on the foregoing solution, based on the first matching result between the vehicle picture and the historical vehicle information, or the second matching result between the vehicle features and the historical vehicle information, the target vehicle corresponding to the target event data is determined, including: performing license plate detection and recognition on the vehicle picture, if the target license plate of the to-be-determined vehicle is detected and recognized, then matching the target license plate with the historical license plates in the historical vehicle information, if the matching is successful, then the vehicle corresponding to the successfully matched historical license plate is determined as the target vehicle corresponding to the target event data; if the matching fails, then a new vehicle information is added according to the target license plate, and the historical vehicle information is updated; if the target license plate is not detected or not recognized, then calculating the first matching degree between the vehicle features and the historical features in the historical vehicle information, if there is a first matching degree greater than the first preset threshold, then the vehicle corresponding to the maximum first matching degree is determined as the target vehicle corresponding to the target event data; if there is no first matching degree greater than the first preset threshold, then the to-be-determined vehicle is marked as an unmatched vehicle.
[0010] In an embodiment of the present application, based on the foregoing solution, after marking the to-be-determined vehicle as an unmatched vehicle, the method further includes: calculating the second matching degree between the vehicle features corresponding to the unmatched vehicle and the historical features in the historical vehicle information; if there is a second matching degree greater than the second preset threshold, then the vehicle corresponding to the maximum second matching degree is determined as the target vehicle corresponding to the target event data.
[0011] In an embodiment of the present application, based on the foregoing solution, a vehicle event semantic hypergraph is constructed according to the target vehicles corresponding to the respective target event data, including: defining an entity set and a relationship set in the vehicle event semantic hypergraph, where the entity set includes vehicles and event types, and the relationship set includes the corresponding relationship between the vehicle and the vehicle event; corresponding the historical vehicles and historical events to the entities in the entity set to obtain a hypergraph entity set; establishing the entity relationship in the vehicle event semantic hypergraph according to the target vehicles corresponding to the respective target event data; constructing the vehicle event semantic hypergraph based on the hypergraph entity set and the entity relationship.
[0012] In one embodiment of the present application, based on the foregoing solution, after constructing a vehicle event semantic hypergraph according to the target vehicles corresponding to each of the target event data, the method further includes: performing knowledge hypergraph completion on the vehicle event semantic hypergraph through a preset semantic hypergraph completion model to update the vehicle event semantic hypergraph; determining a target entity from the vehicle event semantic hypergraph, and generating an induced subgraph based on the target entity and the vehicle event semantic hypergraph; determining an entity to be analyzed from the induced subgraph, extracting features of the entity to be analyzed to obtain entity features of the entity to be analyzed, and analyzing the entity to be analyzed based on the entity features and a preset data analysis model to perform data management according to the analysis result.
[0013] In one embodiment of the present application, based on the foregoing solution, after obtaining historical vehicle information and vehicle event data, the method further includes: determining an event time difference based on the event occurrence time of the vehicle event data and the current time; performing target feature detection on the event pictures in the vehicle event data to obtain a first detection result; performing repeatability detection on the vehicle event data to obtain a second detection result; and performing data cleaning on the vehicle event data based on the event time difference, the first detection result, and the second detection result.
[0014] According to one aspect of the embodiments of the present application, there is provided a data management device based on a semantic hypergraph, including: an acquisition module, configured to acquire historical vehicle information and vehicle event data; an extraction module, configured to determine target event data from the vehicle event data, perform picture extraction on a target picture corresponding to the target event data to obtain a vehicle picture, and perform feature extraction on the vehicle picture to obtain vehicle features; a matching module, configured to determine a target vehicle corresponding to the target event data based on a first matching result between the vehicle picture and the historical vehicle information or a second matching result between the vehicle features and the historical vehicle information; and a construction module, configured to construct a vehicle event semantic hypergraph according to the target vehicles corresponding to each of the target event data to perform data management according to the vehicle event semantic hypergraph.
[0015] According to one aspect of the embodiments of the present application, there is provided an electronic device, including: one or more processors; a storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enabling the electronic device to implement the data management method based on a semantic hypergraph according to any one of the foregoing embodiments.
[0016] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor of a computer, enabling the computer to execute the data management method based on a semantic hypergraph according to any one of the foregoing embodiments.
[0017] Advantages of the present application: By obtaining historical vehicle information and vehicle event data, determining target event data from the vehicle event data, extracting target pictures corresponding to the target event data to obtain vehicle pictures, extracting features from the vehicle pictures to obtain vehicle features, and determining the target vehicle corresponding to the target event data based on the first matching result between the vehicle pictures and the historical vehicle information or the second matching result between the vehicle features and the historical vehicle information, it is possible to more accurately determine the vehicle corresponding to the event data, and then determine the relationship between the event and the vehicle. A vehicle event semantic hypergraph is constructed based on the target vehicles corresponding to each target event data for data management. The vehicle event semantic hypergraph provides a more intuitive and compact way to represent the relationship between data, thereby improving the efficiency and accuracy of data analysis and management.
[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. Brief Description of the Drawings
[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:
[0020] Figure 1 is a schematic diagram of an exemplary system architecture shown in an exemplary embodiment of the present application;
[0021] Figure 2 is a schematic flowchart of a data management method based on a semantic hypergraph shown in an exemplary embodiment of the present application;
[0022] Figure 3 is a schematic flowchart of data fusion of a data management method based on a semantic hypergraph shown in an exemplary embodiment of the present application;
[0023] Figure 4 is a schematic diagram of a vehicle event semantic hypergraph of a data management method based on a semantic hypergraph shown in an exemplary embodiment of the present application;
[0024] Figure 5 is a schematic flowchart of a data management method based on a semantic hypergraph shown in another exemplary embodiment of the present application;
[0025] Figure 6 is a block diagram of a data management device based on a semantic hypergraph shown in an exemplary embodiment of the present application;
[0026] Figure 7 The structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown. Specific embodiments
[0027] The embodiments of the present application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, rather than for limiting the protection scope of the present application.
[0028] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.
[0029] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.
[0030] First of all, it should be noted that the PPOCR network model, namely PaddleOCR, is a practical ultra-lightweight optical character recognition (OCR) system, mainly composed of three parts: text detection, detection box correction, and text recognition. It can accurately recognize the text in pictures in various scenarios, and has the characteristics of high efficiency, accuracy, and light weight. It is widely used in fields such as document digitization, identity authentication, and license plate recognition.
[0031] The HyConvE method is a knowledge hypergraph link prediction method based on an embedding model and a convolutional neural network. It uses three-dimensional convolution to capture the deep interaction between entities and relationships, and captures the semantic and position information in each multi-relation through two-dimensional convolution operations, so as to achieve effective link prediction.
[0032] The CLIP-ReID model is an innovative method for image re-identification. It makes full use of the capabilities of vision-language models, especially for image re-identification without specific text labels. The CLIP-ReID model is improved based on the CLIP (Contrastive Language–Image Pre-training) architecture and is specifically designed for the image re-identification (ReID) task. In the CLIP model, images and texts are mapped into a shared latent space, so that similar images and the texts describing them are close to each other in the space. This cross-modal alignment ability makes the CLIP model perform excellently in image understanding and recognition.
[0033] Figure 1 It is a schematic diagram of an exemplary system architecture shown in an exemplary embodiment of the present application.
[0034] Refer to Figure 1 As shown, the system architecture may include a data acquisition device 101 and a computer device 102. Among them, the computer device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The data acquisition device 101 is used to collect historical vehicle information and vehicle event data. In this embodiment, after the data acquisition device 101 obtains the above data, it provides them to the computer device 102 for processing. Relevant technicians can use the computer device 102 to determine target event data from the vehicle event data, extract target pictures corresponding to the target event data to obtain vehicle pictures, extract features from the vehicle pictures to obtain vehicle features, and determine the target vehicle corresponding to the target event data based on the first matching result between the vehicle pictures and the historical vehicle information or the second matching result between the vehicle features and the historical vehicle information. A vehicle event semantic hypergraph is constructed according to the target vehicles corresponding to each target event data, so as to perform data management according to the vehicle event semantic hypergraph. It should be noted that the data acquisition device 101 and the computer device 102 provided in this embodiment are only examples and should not bring any limitations to the functions and usage scopes of the embodiments of the present application.
[0035] It should be noted that the data management method based on the semantic hypergraph provided in the embodiments of the present application is generally executed by the computer device 102. Correspondingly, the data management device based on the semantic hypergraph is generally set in the computer device 102.
[0036] Figure 2It is a schematic flowchart of a data management method based on a semantic hypergraph shown in an exemplary embodiment of the present application. The data management method based on the semantic hypergraph can be executed by a computing processing device, and the computing processing device can be Figure 1 the computer device 102 shown in Figure 2 As shown, the data management method based on the semantic hypergraph at least includes steps S210 to S240, which are introduced in detail as follows:
[0037] In step S210, historical vehicle information and vehicle event data are obtained.
[0038] In an embodiment of the present application, historical data stored in edge devices, i.e., edge computing units, and cloud devices, and newly collected vehicle event data are obtained. Among them, the historical data includes, but is not limited to, historical vehicle information and historical event data. The newly collected vehicle event data is generated by event triggers such as vehicle parking violation detectors, vehicle reverse driving detectors, and vehicle speeding detectors, and these triggers record the position data of the vehicle in the event pictures and other relevant event characteristic data.
[0039] In this embodiment, the data can be divided into structured data and unstructured data according to whether it is structured. Among the structured data, it mainly includes data stored in the cloud vehicle event library, vehicle entity library, etc., and descriptive text data, etc.; among the unstructured data, it mainly includes signal data, picture data, video data, etc. related to vehicles and vehicle events. In the embodiment, the data in the cloud device can be empty, and the data in the cloud device is automatically constructed from bottom to top through the data in the edge device, or the data in the edge device is controlled to be fused with the data in the cloud device. Through in-depth fusion and matching at the data level of the structured and unstructured cloud data and edge data, automated bottom-up vehicle data construction is realized, providing a solid data foundation for the comprehensive management and intelligent analysis of vehicle events and improving the data utilization efficiency.
[0040] In an embodiment of the present application, the process after obtaining historical vehicle information and vehicle event data includes the following steps: determining the event time difference based on the event occurrence time and the current time of the vehicle event data; performing target feature detection on the event pictures in the vehicle event data to obtain a first detection result; performing repeatability detection on the vehicle event data to obtain a second detection result; and performing data cleaning on the vehicle event data based on the event time difference, the first detection result, and the second detection result.
[0041] In this embodiment, if the event time difference is greater than the preset time difference, the first detection result does not include the target feature, or the second detection result indicates that the vehicle event data is duplicate data, that is, the vehicle event data is duplicate with other vehicle event data, then the vehicle event data is cleaned, that is, the vehicle event data is filtered out. The repeatability detection can be performed in the following way: compare the similarity between the target event data and any vehicle event data except the target event vehicle. If the similarity is greater than the first preset similarity, it is determined that the target event data is duplicate with the vehicle event data.
[0042] In some embodiments, a vehicle event set D is generated based on vehicle event data ie , and the vehicle event set D ie is cleaned and filtered. After receiving an event data, the validity of the event is judged. The event data is effectively cleaned and filtered by judging whether the event data contains target features such as vehicles and vehicle positions, whether it belongs to a duplicate event, and whether it has timeliness. Among them, the timeliness of the vehicle event data can be judged by at least one of the following methods: determine the event time difference based on the event occurrence time of the vehicle event data and the current time. If the event time difference is greater than the preset time difference, it is determined that it does not have timeliness; compare the similarity between the vehicle event data and other vehicle event data, and calculate the time interval between the event occurrence time of the vehicle event data and the event occurrence time of other vehicle event data. If the similarity is greater than the second preset similarity and the time interval is less than the preset duration threshold, it is determined that it does not have timeliness.
[0043] In this embodiment, for the vehicle event set D ie ={e 1 ,e 2 ,…,e n}, the filtering and cleaning are performed, where D ie represents the newly added vehicle event data set, and e i is a certain vehicle event. By methods such as detecting the existence of the target in the event, detecting the timeliness of the event, and filtering duplicate events, the event is cleaned to ensure the data quality, consistency, and availability of the data.
[0044] In this embodiment, the collected data is cleaned and preprocessed, mainly involving cleaning and preprocessing the collected source data. In the cleaning process, the structured data and unstructured data in the cloud device and edge device are mainly filtered to filter out invalid data or noise data, especially filtering the false alarms and misreports in the edge event trigger; and then better extracting the features of the involved picture data to facilitate subsequent feature comparison and building effective links.
[0045] In step S220, determine the target event data from the vehicle event data, extract the target picture corresponding to the target event data to obtain a vehicle picture, and extract features from the vehicle picture to obtain vehicle features.
[0046] In an embodiment of the present application, the process of extracting the target picture corresponding to the target event data to obtain a vehicle picture and extracting features from the vehicle picture to obtain vehicle features includes the following steps: crop the target picture based on the vehicle position data in the target event data to obtain a vehicle picture; extract features from the vehicle picture according to the preset extraction features to obtain vehicle features.
[0047] In this embodiment, determine the vehicle to be processed from the target event data, extract the target picture corresponding to the vehicle to be processed to obtain the vehicle picture corresponding to the vehicle to be processed, and extract features from the vehicle picture corresponding to the vehicle to be processed to obtain the vehicle features corresponding to the vehicle to be processed.
[0048] In some embodiments, the target event data can be preprocessed through the following steps to obtain a vehicle picture and vehicle features:
[0049] In step S2-1, obtain the vehicle event e i vehicle information in For the determined valid vehicle event data, take an unprocessed vehicle in a vehicle event as the vehicle to be processed, and obtain vehicle information such as the vehicle position data of the vehicle from the corresponding event picture Perform pixel cropping based on the vehicle position data to obtain the vehicle picture corresponding to the vehicle to be processed
[0050] In step S2-2, extract vehicle features For the obtained vehicle picture Perform a vehicle feature extraction operation to obtain vehicle features
[0051] In step S2-3, save information such as vehicle features to the event vehicle information D icar as the basis for subsequent analysis.
[0052] In step S2-4, repeat steps S2-1 to S2-3 until all unprocessed vehicles in the target event data have completed feature extraction and information saving.
[0053] In step S2-5, when there are no unprocessed vehicles in the target event data, mark the end of the preprocessing stage of the target event data.
[0054] In this embodiment, vehicle information is included in the vehicle event data represents event e i the vehicle car included j ; through event e i perform pixel clipping on the vehicle position data included in it, and extract the vehicle picture Use the trained feature extraction model (such as the CLIP-ReID model) to perform feature extraction operations on the vehicle picture to obtain vehicle features Save uniformly as The vehicle data in all events constitutes an event vehicle library Thus, the preprocessing of the event data is completed.
[0055] In step S230, based on the first matching result between the vehicle picture and the historical vehicle information, or the second matching result between the vehicle feature and the historical vehicle information, determine the target vehicle corresponding to the target event data.
[0056] In an embodiment of the present application, the process of determining the target vehicle corresponding to the target event data based on the first matching result between the vehicle picture and the historical vehicle information, or the second matching result between the vehicle feature and the historical vehicle information, includes the following steps: perform license plate detection and recognition on the vehicle picture. If the target license plate of the pending vehicle is detected and recognized, then match the target license plate with the historical license plates in the historical vehicle information to obtain the first matching result. If the first matching result is a successful match, then determine the vehicle corresponding to the successfully matched historical license plate as the target vehicle corresponding to the target event data; if the first matching result is a failed match, then add a new vehicle information according to the target license plate and update the historical vehicle information; if the target license plate is not detected, or the target license plate is not recognized, then calculate the first matching degree between the vehicle feature and the historical feature in the historical vehicle information according to the second matching result between the vehicle feature and the historical vehicle information. If there is a first matching degree greater than the first preset threshold, then determine the vehicle corresponding to the maximum first matching degree as the target vehicle corresponding to the target event data; if there is no first matching degree greater than the first preset threshold, then mark the pending vehicle as an unmatched vehicle. The matching degree can also be understood as the similarity. In this embodiment, the cosine similarity between the vehicle feature and the historical vehicle information is calculated to obtain the similarity between the vehicle feature and the historical vehicle information, and the matching degree is determined according to the similarity.
[0057] In some embodiments Figure 3 is a schematic diagram of the data fusion process of the data management method based on the semantic hypergraph shown in an exemplary embodiment of the present application. Refer to Figure 3 shown, the data in the edge device and the data in the cloud device are fused through the following steps:
[0058] In step S3-1, receive a target event data e that has completed preprocessing i , and randomly determine a vehicle car from the target event data e i . j .
[0059] In step S3-2, perform license plate detection and recognition on the vehicles car in the target event data e that have not been updated i ; if the license plate can be detected and recognized, enter S3-4, otherwise enter step S3-3; j
[0060] In step S3-3, match the vehicle features of the current vehicle with the features of the existing vehicles in the vehicle entity library ; if there are paired vehicles with a matching degree exceeding the matching degree threshold threshold match , select the vehicle car with the highest matching score as the matching vehicle and enter step S3-5; otherwise, mark the vehicle as an unmatched vehicle and enter step S3-6; k
[0061] In step S3-4, check if there is matching data corresponding to the license plate in the vehicle entity library. If the matching can be completed, enter step S3-5; otherwise, create a new vehicle entity and add it to the vehicle entity library D car and enter step S3-5.
[0062] In step S3-5, update the link relationship between the vehicles in the matching vehicle entity library and the vehicles in the vehicle event library;
[0063]
[0063] In step S3-6, repeat steps S3-2 to S3-3 until all the vehicles in the target event data have been processed
[0064] When there are no longer unupdated vehicles in the target event data, mark the update process of the target event data to the cloud as completed until all vehicle event data have been updated to the cloud
[0065] In this embodiment, the vehicle matching work is completed through the vehicle unique identifier - license plate. By performing license plate detection through a trained target detection model (such as the YOLOv7 network model) to obtain the license plate position corresponding to the vehicle Furthermore, the license plate is recognized by the PPOCR network model to obtain the final license plate. The matching between the event vehicle and the vehicle entity library is completed by comparing the license plate text. If the license plate does not exist, a new vehicle entity is added. However, in actual scenarios, problems such as blurred camera or blocked license plate often occur, resulting in the inability to detect or recognize the license plate. Therefore, the vehicle images that cannot detect or recognize the license plate are subjected to feature extraction through a trained feature extraction model (such as the CLIP-ReID model) to obtain vehicle features And the vehicle features and the vehicle entity feature library The feature data in are compared one by one, where Get the comparison score Select those exceeding the threshold threshold match And having the highest score The vehicle entity car k As the matching result of the event vehicle to construct a link. If the matching cannot be completed by the above two methods, the event vehicle is marked as an unmatched vehicle
[0066] In an embodiment of the present application, the process after marking the undetermined vehicle as an unmatched vehicle further includes the following steps: calculating the second matching degree between the vehicle features corresponding to the unmatched vehicle and the historical features in the historical vehicle information; if there is a second matching degree greater than the second preset threshold, the vehicle corresponding to the maximum second matching degree is determined as the target vehicle corresponding to the target event data. Both the second preset threshold and the first preset threshold are matching degree thresholds, and the specific values of the second preset threshold and the first preset threshold can be determined according to actual needs. The second preset threshold and the first preset threshold can be the same or different
[0067] In this embodiment, the unmatched vehicle is matched through the following steps:
[0068] In step S4-1, take a vehicle information that has not been matched and enter step S4-2
[0069] In step S4-2, the vehicle features are matched with the existing vehicle features in the vehicle entity library, that is, the historical features. If there is a pairing vehicle exceeding the matching degree threshold t h res h o ld match Then select the vehicle with the highest matching score as the matching vehicle and enter step S4-3; otherwise, no additional operation is performed, skip this vehicle, and enter step S4-4
[0070] In step S4-3, update the link relationship between the vehicles in the matching vehicle entity library and the vehicles in the vehicle event library with each other.
[0071] In step S4-4, repeat steps S4-1 to S4-3 until there are no vehicles that can be matched.
[0072] In this embodiment, to solve the problem of unmatched vehicles in the event library, compare the vehicle with the vehicle entity feature library again, and clean and update the vehicle information in the vehicle event library.
[0073] In step S240, construct a vehicle event semantic hypergraph based on the target vehicles corresponding to each target event data, so as to manage the data according to the vehicle event semantic hypergraph.
[0074] In an embodiment of the present application, the process of constructing a vehicle event semantic hypergraph based on the target vehicles corresponding to each target event data includes the following steps: Define the entity set and relationship set in the vehicle event semantic hypergraph, where the entity set includes vehicles and event types, and the relationship set includes the corresponding relationship between vehicles and vehicle events; Corresponding the historical vehicles and historical events with the entities in the entity set to obtain a hypergraph entity set; Establish the entity relationship in the vehicle event semantic hypergraph according to the target vehicles corresponding to each target event data; Construct a vehicle event semantic hypergraph based on the hypergraph entity set and the entity relationship.
[0075] In this embodiment, the vehicle event semantic hypergraph is constructed through the following steps:
[0076] In step S5-1, define the entity set E and relationship set R included in the hypergraph. For example, define the vehicle event semantic hypergraph KHG car There are entities with abstract concepts such as vehicle E car , event type E et , vehicle attribute E va etc. in the concept layer. In the hypergraph KHG car In the instance layer, it mainly includes vehicle event instance entity E ie , vehicle instance entity E icar , vehicle attribute instance entity E iva ; At the same time, define the relationships existing in the hypergraph KHG car : In the KHG car instance layer, there is a subordinate relationship R iva between the vehicle attribute entity E icar and the vehicle E Affiliation , and there is an inclusion relationship R icar between the vehicle E ie and the event E include etc.; In the hypergraph, there is a matching / mapping relationship R match, there is a subordinate relationship R between the vehicle event types at the conceptual level and the events at the instance level Affiliation and so on. In one embodiment, the above inclusion relationship is specifically that event E ie includes vehicle E icar .
[0077] In step S5-2, a hypergraph entity set is constructed from the cloud database. After being processed through the foregoing steps, the cloud database has become structured data, and the hypergraph entity set is obtained by corresponding the content of the structured data with the defined hypergraph entities.
[0078] In step S5-4, the entities are divided into hypergraph levels. The generated entities, vehicle event types, etc. in the vehicle entity library D car are regarded as the conceptual level, and the vehicle entities generated in the vehicle event library D ie etc. are regarded as the entities in the instance level.
[0079] In step S5-3, the relationships between the entities in the hypergraph are established from the cloud database. After the foregoing steps in the cloud database, the vehicle matching between the instance level and the conceptual level, as well as data induction, etc. are completed, and thus the relationships between the entities in the hypergraph are generated according to the relationship definitions in the hypergraph. The vehicle event semantic hypergraph KHG car as Figure 4 shown, Figure 4 is a schematic diagram of the vehicle event semantic hypergraph of the data management method based on the semantic hypergraph shown in an exemplary embodiment of the present application.
[0080] By analyzing and designing a multi-level vehicle semantic hypergraph for the vehicle safety field in the smart city scenario, and implementing the construction of the vehicle semantic hypergraph for structured and unstructured data based on the cloud data, while providing a structured and visual data foundation for vehicle safety, it provides graph data content for vehicle event analysis and vehicle safety analysis to improve vehicle management efficiency and vehicle safety.
[0081] In one embodiment of the present application, the process after constructing the vehicle event semantic hypergraph according to the target vehicle corresponding to each target event data includes the following steps: updating the vehicle event semantic hypergraph by complementing the knowledge hypergraph of the vehicle event semantic hypergraph through a preset semantic hypergraph completion model; determining target entities from the vehicle event semantic hypergraph, generating an induced subgraph based on the target entities and the vehicle event semantic hypergraph; determining entities to be analyzed from the induced subgraph, extracting features of the entities to be analyzed to obtain the entity features of the entities to be analyzed, and analyzing the entities to be analyzed based on the entity features and a preset data analysis model to perform data management according to the analysis results.
[0082] In this embodiment, the process of analyzing and managing vehicles and events according to the vehicle event semantic hypergraph may include the following steps:
[0083] In step S6-1, for the generated vehicle event semantic hypergraph KHG car perform knowledge hypergraph completion and enter step S6-2. Among them, the semantic hypergraph completion can be completed by the HyConvE method.
[0084] In step S6-2, for a certain entity car in the completed vehicle event semantic hypergraph KHG generate an induced subgraph and enter step S6-3.
[0085] In step S6-3, for the nodes (entities) to be analyzed in this induced subgraph perform node feature operations and enter step S6-4.
[0086] In step S6-4, perform CLS classification operations on the node features to obtain the final analysis result, so as to perform risk assessment, vehicle behavior pattern recognition, trend prediction, etc. according to the final analysis result.
[0087] In this embodiment, by generating an induced subgraph for the entity to be analyzed (which can be a vehicle entity in the concept layer or an event vehicle entity in the instance layer ), an entity-induced subgraph to be analyzed is obtained In the generation of the induced subgraph, multiple methods such as N-hop (N-hop Neighbors), DFS (Depth First Search), and BFS (Breadth First Search) can be used for generation. Taking vehicle risk assessment as an example, by putting the induced subgraph and vehicle entities into a trained GNN (Graph Neural Network) for feature extraction, the feature representation of the entity nodes is obtained, and then through a trained risk classifier CLS risk perform risk assessment to obtain a prediction result so as to determine the risk level of the entity according to the prediction result.
[0088] In some embodiments, based on the vehicle event semantic hypergraph, the cause of the event can be deeply analyzed for events and vehicles, and the traffic flow at the target time or target location can be predicted, etc. Furthermore, based on the above results, forward-looking maintenance suggestions and safety warnings can be provided. This not only improves the utilization rate and processing efficiency of data, but also can provide a more efficient and accurate solution for vehicle event management, significantly improving the safety and management efficiency of vehicles, and contributing new strength to the construction and development of smart cities.
[0089] By applying the knowledge hypergraph analysis method to vehicle event management, we can achieve multi-level analysis of vehicles, events, etc., and build a multi-faceted analysis system for vehicle safety and vehicle event management. This provides strong technical support for vehicle safety management in smart city scenarios such as smart highways and smart parks, significantly improves vehicle safety performance and management efficiency, and improves user experience and safety.
[0090] Figure 5 FIG. 1 is a flow chart of a data management method based on a semantic hypergraph according to another exemplary embodiment of the present application. Figure 5 As shown, the data management method based on semantic hypergraph includes the following steps:
[0091] In step S1, multi-source data of "cloud-edge-end" is collected. Multi-source data includes but is not limited to historical data and new event data, where new event data is generated by event triggers such as vehicle illegal parking detectors, vehicle reverse driving detectors, and vehicle speeding detectors, which record the location data of the vehicle in the event picture and other related event characteristic data.
[0092] In step S2, the collected data is cleaned and preprocessed, including data cleaning, vehicle target feature extraction and preservation, to ensure data quality, consistency and availability;
[0093] In step S3, the pre-processed edge multi-source data is integrated with the historical data stored in the cloud. The new event data is matched and linked with the vehicle information in the vehicle entity data through methods such as license plate matching and feature matching. The matched vehicle information will update the vehicle entity library;
[0094] In step S4, the data in the cloud library is self-cleaned and updated. Implement a self-cleaning and dynamic update mechanism for the cloud library data, regularly clean up expired or invalid data, retry to match data items that were previously unsuccessfully associated, and maintain the timeliness and integrity of the cloud library;
[0095] In step S5, construct the vehicle event semantic hypergraph KHG car By defining the core entities in vehicle events and their relationships, a semantic hypergraph network reflecting the overall picture of vehicle events is established;
[0096] In step S6, according to the semantic hypergraph KHG car Analyze and manage vehicles or events. Use the constructed semantic hypergraph for in-depth analysis and refined management, including but not limited to vehicle behavior pattern recognition, risk assessment, trend prediction and other functions, and formulate effective vehicle safety management strategies based on this.
[0097] The specific manners of the above steps have been described in detail in the method embodiments, and will not be elaborated here.
[0098] This application integrates structured and unstructured data to automatically construct a vehicle semantic hypergraph containing rich semantic information, and uses a deep learning graph analysis method based on the hypergraph to achieve comprehensive management of vehicle events. This method not only improves the utilization rate of data, but also provides a more efficient and accurate way for vehicle event management, thereby enhancing vehicle safety and management efficiency, and contributing new strength to the construction and development of smart cities. In addition, by using the generated vehicle event semantic hypergraph as the data basis, data can be correlated, thereby optimizing the data processing process.
[0099] Figure 6 is a block diagram of a data management device based on a semantic hypergraph shown in an exemplary embodiment of this application. This device can be applied to Figure 1 the shown implementation environment and is specifically configured in the computer device 102. This device can also be applicable to other exemplary implementation environments and be specifically configured in other devices. This embodiment does not limit the implementation environment applicable to this device.
[0100] As Figure 6 shown, this exemplary data management device based on a semantic hypergraph includes: an acquisition module 610, an extraction module 620, a matching module 630, and a construction module 640.
[0101] Among them, the acquisition module 610 is used to obtain historical vehicle information and vehicle event data; the extraction module 620 is used to determine target event data from the vehicle event data, extract pictures of the target pictures corresponding to the target event data to obtain vehicle pictures, and extract features of the vehicle pictures to obtain vehicle features; the matching module 630 is used to determine the target vehicle corresponding to the target event data based on the first matching result between the vehicle pictures and the historical vehicle information or the second matching result between the vehicle features and the historical vehicle information; the construction module 640 is used to construct a vehicle event semantic hypergraph according to the target vehicles corresponding to each target event data, so as to perform data management according to the vehicle event semantic hypergraph.
[0102] It should be noted that the data management device based on a semantic hypergraph provided in the above embodiment and the data management method based on a semantic hypergraph provided in the above embodiment belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiments, and will not be elaborated here. In practical applications, the data management device based on a semantic hypergraph provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here either.
[0103] An embodiment of the present application further provides an electronic device, including: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the data management method based on semantic hypergraph provided in each of the above embodiments.
[0104] Figure 7 The structure diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that, Figure 7 The computer system 700 of the electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.
[0105] As Figure 7 shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage section 708 into the random access memory (RAM) 703, such as executing the methods provided in each of the above embodiments. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0106] The following components are connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read from it can be installed into the storage portion 708 as needed.
[0107] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, various functions defined in the system of the present application are executed.
[0108] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The computer program included on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0110] The units involved in the embodiments described in the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.
[0111] Another aspect of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute the data management method based on a semantic hypergraph provided in each of the above embodiments. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist separately without being assembled into the electronic device.
[0112] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0113] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the data management method based on a semantic hypergraph provided in each of the above embodiments.
[0114] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.
[0115] After considering the specification and practicing the embodiments disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application.
[0116] The above embodiments are only used to exemplarily illustrate the principles and effects of the present application, rather than to limit the present application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present application should still be covered by the claims of the present application.
Claims
1. A data management method based on semantic hypergraph, characterized in that: include: Obtain historical vehicle information and vehicle event data; Determine target event data from the vehicle event data, perform image extraction on a target image corresponding to the target event data to obtain a vehicle image, and perform feature extraction on the vehicle image to obtain a vehicle feature; Determine the target vehicle corresponding to the target event data based on a first matching result between the vehicle image and the historical vehicle information, or a second matching result between the vehicle feature and the historical vehicle information; A vehicle event semantic hypergraph is constructed according to the target vehicles corresponding to each of the target event data, so as to perform data management according to the vehicle event semantic hypergraph.
2. The data management method based on semantic hypergraph according to claim 1, characterized in that: Performing image extraction on the target image corresponding to the target event data to obtain a vehicle image, and performing feature extraction on the vehicle image to obtain a vehicle feature, including: Cropping the target image based on the vehicle position data in the target event data to obtain the vehicle image; Feature extraction is performed on the vehicle image according to preset extraction features to obtain the vehicle features.
3. The data management method based on semantic hypergraph according to claim 1, characterized in that: Determining a target vehicle corresponding to the target event data based on a first matching result between the vehicle image and the historical vehicle information, or a second matching result between the vehicle feature and the historical vehicle information, includes: Perform license plate detection and recognition on the vehicle image. If the target license plate of the pending vehicle is detected and recognized, match the target license plate with the historical license plates in the historical vehicle information. If the match is successful, determine the vehicle corresponding to the successfully matched historical license plate as the target vehicle corresponding to the target event data. If the match fails, add a new vehicle information according to the target license plate and update the historical vehicle information. If the target license plate is not detected or identified, the first matching degree between the vehicle features and the historical features in the historical vehicle information is calculated; if there is a first matching degree greater than a first preset threshold, the vehicle corresponding to the largest first matching degree is determined as the target vehicle corresponding to the target event data; if there is no first matching degree greater than the first preset threshold, the pending vehicle is marked as an unmatched vehicle.
4. The data management method based on semantic hypergraph according to claim 3 is characterized in that: After marking the pending vehicle as an unmatched vehicle, the method further includes: Calculating a second matching degree between the vehicle feature corresponding to the unmatched vehicle and the historical feature in the historical vehicle information; If there is a second matching degree greater than the second preset threshold, the vehicle corresponding to the largest second matching degree is determined as the target vehicle corresponding to the target event data.
5. The data management method based on semantic hypergraph according to any one of claims 1 to 4, characterized in that: Constructing a vehicle event semantic hypergraph according to the target vehicles corresponding to each of the target event data, including: Defining an entity set and a relationship set in the vehicle event semantic hypergraph, wherein the entity set includes vehicles and event types, and the relationship set includes a corresponding relationship between the vehicles and vehicle events; Matching historical vehicles and historical events with entities in the entity set to obtain a hypergraph entity set; Establishing entity relationships in the vehicle event semantic hypergraph according to the target vehicles corresponding to each target event data; The vehicle event semantic hypergraph is constructed based on the hypergraph entity set and the entity relationship.
6. The data management method based on semantic hypergraph according to any one of claims 1 to 4, characterized in that: After constructing a vehicle event semantic hypergraph according to the target vehicles corresponding to each of the target event data, the method further includes: Performing knowledge hypergraph completion on the vehicle event semantic hypergraph through a preset semantic hypergraph completion model to update the vehicle event semantic hypergraph; Determine a target entity from the vehicle event semantic hypergraph, and generate an induced subgraph based on the target entity and the vehicle event semantic hypergraph; An entity to be analyzed is determined from the induced subgraph, features are extracted from the entity to be analyzed to obtain entity features of the entity to be analyzed, and the entity to be analyzed is analyzed based on the entity features and a preset data analysis model to perform data management according to the analysis results.
7. The data management method based on semantic hypergraph according to any one of claims 1 to 4, characterized in that: After acquiring historical vehicle information and vehicle event data, the method further includes: determining an event time difference based on the event occurrence time and the current time of the vehicle event data; Performing target feature detection on the event picture in the vehicle event data to obtain a first detection result; Performing a repeatability test on the vehicle event data to obtain a second test result; The vehicle event data is cleansed based on the event time difference, the first detection result, and the second detection result.
8. A data management device based on semantic hypergraph, characterized in that: include: The acquisition module is used to obtain historical vehicle information and vehicle event data; An extraction module, used to determine target event data from the vehicle event data, perform image extraction on a target image corresponding to the target event data to obtain a vehicle image, and perform feature extraction on the vehicle image to obtain a vehicle feature; A matching module, configured to determine a target vehicle corresponding to the target event data based on a first matching result between the vehicle image and the historical vehicle information, or a second matching result between the vehicle feature and the historical vehicle information; A construction module is used to construct a vehicle event semantic hypergraph according to the target vehicles corresponding to each target event data, so as to perform data management according to the vehicle event semantic hypergraph.
9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the data management method based on the semantic hypergraph as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the data management method based on a semantic hypergraph as described in any one of claims 1 to 7.