Data processing method and related device

Abstract

One or more embodiments of the present disclosure provide a data processing method and related equipment, the method comprising: acquiring first data of a plurality of types of data sources; performing data fusion based on the first data to generate second data; the data fusion includes feature cascade fusion and decision confidence fusion, wherein the feature cascade fusion aims to generate a feature vector of the first data of a single data source, cross-semantic features of the first data of a plurality of data sources, and decision features of the first data, and the decision confidence fusion aims to generate a decision confidence; based on the first data and the second data, constructing a data model; and performing operation compliance detection based on the data model.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a data processing method and related equipment. Background Technology

[0002] As large data centers move towards high-density deployment and industry security and compliance standards continue to tighten, comprehensive monitoring of personnel operations, equipment operation, and environmental status in data centers, as well as full-process behavior traceability, has become a core business requirement.

[0003] Currently, the industry mostly uses various separate monitoring systems. Each subsystem is independent of the others, data cannot be shared, and the spatiotemporal benchmarks are difficult to unify, making it impossible to achieve comprehensive, blind-spot-free monitoring or support real-time and intelligent management and control requirements. Summary of the Invention

[0004] This disclosure proposes a data processing method and related equipment to solve or partially solve the above-mentioned problems to a certain extent.

[0005] In a first aspect, this disclosure provides a data processing method, comprising:

[0006] Retrieve first data from multiple types of data sources; Based on the first data, data fusion is performed to generate the second data; the data fusion includes feature concatenation fusion and decision confidence fusion, wherein the purpose of feature concatenation fusion is to generate the feature vector of the first data from a single data source, the cross-semantic features of the first data from multiple data sources, and the decision features of the first data, and the purpose of decision confidence fusion is to generate decision confidence. Based on the first data and the second data, a data model is constructed; Operational compliance is checked based on the data model.

[0007] A second aspect of this disclosure provides a data processing apparatus, comprising: The acquisition module is configured to acquire first data from multiple types of data sources; The fusion module is configured to perform data fusion based on the first data to generate the second data; the data fusion includes feature concatenation fusion and decision confidence fusion, wherein the purpose of feature concatenation fusion is to generate feature vectors of the first data from a single data source, cross-semantic features of the first data from multiple data sources, and decision features of the first data, and the purpose of decision confidence fusion is to generate decision confidence. The building module is configured to build a data model based on the first data and the second data; The detection module is configured to perform operational compliance detection based on the data model.

[0008] A third aspect of this disclosure provides a computer device including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the one or more programs include instructions for performing the method of the first aspect.

[0009] A fourth aspect of this disclosure provides a non-volatile computer-readable storage medium comprising a computer program that, when executed by one or more processors, causes the one or more processors to perform the method described in the first aspect.

[0010] A fifth aspect of this disclosure provides a computer program product comprising one or more computer programs that, when executed by one or more processors, implement the method described in the first aspect.

[0011] As can be seen from the above, the data processing method, apparatus, electronic device, and storage medium provided in this disclosure process first data from multiple types of data sources to achieve operational compliance detection, thereby improving the reliability of the compliance detection results. Based on the first data, multi-level fusion such as feature cascade fusion and decision confidence fusion is performed, and effective information is extracted layer by layer based on the correlation between features. This effectively improves the scenario adaptability of the extracted features to a certain extent, providing a more effective basis for operational compliance detection, thereby improving the accuracy of operational compliance detection, reducing the probability of false judgment, and providing a basis for more reliable and rigorous security decisions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in one or more embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only one or more embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating an exemplary data processing method provided in an embodiment of this disclosure is shown.

[0014] Figure 2 A schematic diagram of the structure of an exemplary data processing method apparatus provided in an embodiment of this disclosure is shown.

[0015] Figure 3 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0017] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0018] As described in the background section, security monitoring systems in related technologies typically consist of multiple subsystems, each of which performs security monitoring based on its own security specifications.

[0019] However, due to the constraints of data source (such as sensors, surveillance cameras, etc.) settings, the accuracy of security monitoring results will be significantly reduced when the data source malfunctions (such as surveillance cameras being obstructed). Furthermore, because the data sources and security standards of each subsystem differ, different security monitoring results may be generated for the same area, which also affects the accuracy of the security monitoring results and leads to poor performance of the security monitoring system.

[0020] In view of this, a first aspect of the present disclosure provides a data processing method that can solve or partially solve the above-mentioned problems to a certain extent.

[0021] Figure 1 A schematic flowchart of an exemplary data processing method provided in an embodiment of this disclosure is shown. This data processing method can be used to process data for operational compliance detection, thereby achieving security monitoring.

[0022] like Figure 1 As shown, the data processing method may further include the following steps.

[0023] In step S101, the first data from multiple types of data sources is obtained.

[0024] In this step, multiple types of data sources can refer to data sources with different source types. For example, it can refer to data obtained by different subsystems in a security monitoring system, or data obtained by different types of data collectors.

[0025] For example, the subsystem may include a monitoring subsystem, an access control subsystem, an intelligent robot subsystem, and a manual maintenance subsystem, etc., and this disclosure does not limit it.

[0026] For example, different types of data acquisition devices can be surveillance cameras (such as fixed video surveillance cameras), access control systems, smart cabinet locks, smart robots, law enforcement recorders, and infrared thermal imagers, etc., and this disclosure does not limit them.

[0027] Among them, fixed video surveillance cameras are set in fixed locations, with a fixed coverage area and an unchangeable viewing angle; access control systems have the authority to control personnel entry and exit, allowing only authorized personnel to pass through, and can also record personnel entry and exit; smart cabinet locks have the authority to control the opening and closing of individual cabinets, and different smart cabinet locks operate independently, and can also record the opening and closing records of individual cabinets; smart robots can perform inspections of specific areas according to instructions and record the inspection process; law enforcement recorders can record the detection process during personnel inspections, requiring law enforcement personnel to activate them, and can automatically upload the data to the manual maintenance subsystem.

[0028] In this step, the first data can refer to the raw data obtained from the above-mentioned multiple types of data sources, or it can be data after preprocessing the raw data.

[0029] Optionally, preprocessing may include data cleaning, normalization, standardization, etc., which are not limited in this disclosure.

[0030] For example, the first data from multiple types of data sources can be synchronized in terms of time and space. For instance, the PTP protocol can be used to achieve nanosecond-level time synchronization and unify all data into the same spatial coordinate system to facilitate subsequent data analysis.

[0031] This step allows us to obtain primary data from multiple sources, providing a data foundation for subsequent data processing and operational compliance checks.

[0032] In step S102, data fusion is performed based on the first data to generate the second data.

[0033] In related technologies, each of the above-mentioned multiple types of data sources is used for single-point perception and independent operation. This approach lacks the ability to fuse multi-source data and make intelligent decisions.

[0034] For example, the relevant technology cannot respond or handle the situation in a timely manner when the following situations occur: fixed video surveillance cameras have physical blind spots, such as the back of the cabinet or cold aisles, which are not covered; fixed video surveillance cameras have physical blind spots, such as being partially or completely blocked by moving objects; fixed video surveillance cameras cannot identify personnel operations in special environments such as low temperature or low light; the access control system time is deviated, making it impossible to effectively correlate access control events with other events, and making it difficult to reconstruct personnel operation trajectories; and intelligent cabinets have a disconnect between authorization and actual operation behavior, which may lead to the risk of unauthorized operation after authorization, such as authorized personnel performing unauthorized operations after unlocking the door.

[0035] In the event of the above situations, manual post-event verification is required. This approach suffers from problems such as slow response, high false positive rate, and incomplete traceability.

[0036] Therefore, this disclosure proposes to perform data fusion on the first data to obtain the fused second data.

[0037] In this step, data fusion may include feature cascade fusion and / or decision confidence fusion.

[0038] Optionally, the purpose of feature concatenation fusion is to generate feature vectors of first data from a single data source, cross-semantic features of first data from multiple data sources, and decision features of the first data, while the purpose of decision confidence fusion is to generate decision confidence.

[0039] In other words, feature cascade fusion not only focuses on the features of the first data from a single data source, but also on the cross-semantic features between the first data from multiple data sources, and provides decision features of the first data for operational compliance detection.

[0040] Decision confidence fusion can generate decision features based on primary data, providing direct decision-making basis for operational compliance detection.

[0041] The multi-level feature cascade fusion and decision confidence fusion in this step can provide a more reliable data foundation for operational compliance testing, thereby improving the accuracy of operational compliance testing.

[0042] In step S103, a data model is constructed based on the first data and the second data.

[0043] In this step, the first and second data are further fused to provide a unified data foundation for the final operational compliance test.

[0044] Optionally, the data model may include multiple data dimensions, such as five or six dimensions, and this disclosure does not limit this.

[0045] Optionally, the dimensions of the data model can be set manually based on requirements and data formats.

[0046] In step S104, operational compliance is checked based on the data model.

[0047] In this step, operational compliance is checked based on the data model obtained in the previous steps.

[0048] Operational compliance testing refers to judging whether the operations performed by workers conform to standard operating procedures.

[0049] Optionally, when an operator's operation does not conform to the standard operating procedure, it can be determined that the operator has failed the operation compliance test or is unqualified, and an early warning can be issued accordingly.

[0050] The data processing method provided in this disclosure can effectively improve the accuracy of operational compliance testing results to a certain extent, and enhance the testing performance and effectiveness of compliance testing.

[0051] In some embodiments, the first data may include personnel identity data, operation work order data, historical behavior pattern data, and equipment status data, etc., as the basis for data fusion of the first data. The specific data types included in the first data are not limited in this disclosure.

[0052] Optionally, personnel identity data and operation work order data can be obtained based on the pre-entry data of the operators, historical behavior pattern data can be obtained from a database storing historical data, and equipment status data can be obtained through sensors installed on the equipment.

[0053] In some embodiments, step S102, which involves performing feature concatenation fusion based on the first data, may include: Step S1021: Generate the feature vector of the single data source based on the first data from the single data source.

[0054] In this step, the first data from a single data source can refer to data such as video frames captured by a fixed video surveillance camera, the locking status of a smart cabinet, or infrared images captured by an infrared thermal imager; this disclosure does not limit the data.

[0055] Optionally, the first data from a single data source can be encoded independently to obtain its corresponding feature vector.

[0056] It is understandable that the feature vectors obtained through this step, while retaining the independent information of the first data, provide a data foundation with unified dimensions for subsequent data processing.

[0057] Step S1022: Based on feature vectors, personnel identity data, and operation work order data, use a cross-semantic attention mechanism algorithm to generate cross-semantic features of the first data from multiple data sources.

[0058] In this step, based on the feature vectors obtained in the previous step, as well as personnel identity data and operation work order data, the second-level cross-semantic features are obtained.

[0059] In this step, a cross-semantic attention mechanism algorithm can be used to extract the second-level cross-semantic features.

[0060] Optionally, a feature extraction model based on a cross-semantic attention mechanism algorithm can be constructed; the feature vectors, personnel identity data, and operation work order data obtained in the previous step are input into the feature extraction model to output the cross-semantic features of the first data.

[0061] Among them, the query vector Q can be generated based on operation work order data and personnel identity data, indicating the task to be solved. The key vector K and value vector V can be generated based on the feature vector. The key vector indicates the problems that multiple types of data sources are good at solving, and the value vector indicates the feature information of multiple types of data sources.

[0062] Feature extraction models can calculate the similarity between key vectors and query vectors to obtain an importance score for each data source to the current task. The softmax function can then be used to convert these importance scores into weights ranging from 0 to 1. Finally, a weighted sum of the value vectors based on these weights is performed to obtain the final cross-semantic features. This process can be represented as follows: ,in, This represents the dimension (vector length) of the key vector of the k-th data source.

[0063] For example, when the work order data is "open the smart cabinet door", the query vector is the feature vector generated based on the scenario. Through the feature extraction model based on the cross-semantic attention mechanism algorithm, it can be determined that the locking status data of the smart cabinet and the video data collected by the smart robot in this area have higher weights, while the video data of the fixed camera with obstruction has lower weights. Based on the above first data and the above weights, the corresponding cross-semantic features can be obtained.

[0064] Alternatively, the cross-semantic attention mechanism algorithm can be replaced by an adaptive weighted convolutional network algorithm, which is not limited in this disclosure.

[0065] Cross-semantic features break down information barriers between various types of data sources, enabling semantic-level association and fusion of primary data. Furthermore, by assigning dynamically adaptable weights to data from different data sources, they mitigate data interference from invalid data to some extent.

[0066] On the other hand, by combining operation work order data and personnel identity data, a semantic context is provided for the first data, thereby providing a judgment background for the subsequent calculation of safety decision confidence and operation compliance detection, improving the efficiency and accuracy of decision-making.

[0067] Step S1023: Based on cross-semantic features, historical behavior pattern data, and device status data, a graph neural network algorithm is used to generate decision features.

[0068] In this step, decision features for operational compliance are further generated based on the cross-semantic features obtained in the previous step.

[0069] Optionally, the cross-semantic features, historical behavior pattern data, and equipment status data from the previous step can be used as input data. A graph neural network algorithm is used to construct a relationship graph based on the above input data. The nodes of this relationship graph can represent operators, equipment, and operational behaviors, and the edges can represent the relationships between nodes. The compliance of the current operation is inferred based on this relationship graph to generate decision features.

[0070] This step provides a direct basis for operational compliance testing. By extracting decision features, abstract perceptual and semantic data can be transformed into quantitative decision-making data that can be directly used for security management.

[0071] In some embodiments, step S102, which involves performing decision confidence fusion based on the first data, may further include: Step S1024: Based on the first data, use the trust function theory to generate a confidence level that there is a malfunction in the predetermined security decision.

[0072] In this step, safety decisions can refer to decisions made in response to failure events.

[0073] Compared to the decision features described above, this confidence level further considers the conflict between the first data from each data source, thus providing a more reliable data foundation for subsequent operational compliance checks.

[0074] Optionally, the confidence level can be obtained based on the conflict between the first data and the first data from multiple types of data sources on the predetermined proposition (the erroneous operation to be verified).

[0075] Alternatively, the multi-source evidence fusion (DS evidence theory) method can be used to fuse first data from multiple data sources into a more reliable confidence level.

[0076] For example, the expression can be: ,in, Indicates data source and The conflict coefficient between them This indicates a proposition to be verified (an error to be verified). Indicates data source Regarding the confidence level of a verified proposition, Indicates data source The confidence level of a proposition to be verified.

[0077] Optionally, when the data source and When the conflict coefficient between the algorithms exceeds a preset threshold, a manual reconciliation process can be triggered to avoid algorithm misjudgment.

[0078] For example, in the data source and When the conflict coefficient between them is greater than 0.8, a manual reprocessing procedure can be triggered.

[0079] Alternatively, the DS evidence theory method can be replaced by Bayesian decision fusion algorithms, etc., and this disclosure does not limit it.

[0080] It is understandable that, unlike decision features which judge confidence levels based on cross-semantic features, decision confidence fusion generates confidence levels based on the independent confidence levels of multiple data sources and the conflict coefficients between data sources.

[0081] In other words, feature cascade fusion and decision confidence fusion generate the confidence level of the operation from different perspectives, providing more comprehensive data assurance for subsequent operation compliance testing.

[0082] In some embodiments, the first data may further include fixed camera data.

[0083] In some embodiments, the method further includes the following steps.

[0084] Step S105: In response to the visibility of the fixed camera data acquired by the fixed camera being less than a preset threshold, the inspection robot and / or the operator are mobilized to the area where the fixed camera is located to collect the inspection robot data and the law enforcement recorder data.

[0085] As mentioned above, when the first data from a single data source is used as the basis for judging operational compliance in related technologies, any problems with the first data from that data source will greatly affect the accuracy of the operational compliance judgment.

[0086] For example, when a fixed camera is obstructed, the visibility of the data it collects will be significantly reduced, which may lead to errors in judging operational compliance.

[0087] Therefore, in some embodiments of this disclosure, when the visibility of the fixed camera data is less than a preset threshold, further data is obtained by mobilizing inspection robots and / or operators to arrive at the area where the fixed camera is located to collect inspection robot data and / or law enforcement recorder data, so as to provide a data basis for field-of-view fusion of fixed camera data, inspection robot data and / or law enforcement recorder data.

[0088] Step S106: Based on data from fixed cameras, patrol robots, and / or law enforcement recorders, a multi-view stereo vision algorithm is used to perform field-of-view fusion to generate second data.

[0089] In this step, the data from the fixed camera, the patrol robot, and / or the law enforcement recorder are all video data. The three video data can be spatiotemporally aligned and pixel-level fused using a multi-view stereo vision algorithm to reconstruct the operation scene of the occluded area.

[0090] Alternatively, its fusion algorithm can be expressed as: ,in, This represents the first data from the k-th data source. Indicates the number of data sources. This represents the valid field-of-view mask (excluding occluded areas) for the k-th data source. and This represents the coordinates of a pixel in the first data set. Indicates the timestamp of the first data. Indicates time-varying weights, , This indicates the priority of the k-th data source (e.g., the inspection robot has the highest priority, and the fixed camera has the lowest priority). This indicates the priority of the j-th data source. This represents the confidence level of the image quality of the k-th data source at time t. Let represent the image quality confidence of the j-th data source at time t.

[0091] It is understood that this disclosure may also utilize first data from other data sources to reconstruct fixed camera data, and this disclosure does not limit this.

[0092] For example, when the operator's body obscures their hands, field-of-view fusion can be performed based on infrared thermal data generated by an infrared thermal imaging instrument and data from the operator's law enforcement recorder to estimate hand posture and reconstruct operating gestures.

[0093] For example, when the smart cabinet door obstructs the internal equipment after it is opened, the opening angle data obtained by the angle sensor on the smart cabinet door and the internal view obtained by the inspection robot moving to the side of the smart cabinet can be obtained to perform field fusion to determine the operation performed by the current operator, and based on the operation, determine whether the compliance test can be passed.

[0094] This step allows for the reconstruction of fixed camera data, providing a more reliable data foundation for subsequent compliance checks.

[0095] In some embodiments, step S103 may further include the following steps.

[0096] Step S1031: Perform time and spatial data alignment on the first data from multiple data sources to determine the location coordinates, orientation angle, and timestamp corresponding to the first data.

[0097] In this step, data alignment in the time dimension can represent nanosecond-level time synchronization based on the PTP protocol, while data alignment in the spatial dimension can represent unifying the first data from all types of data sources into a unified spatial coordinate system.

[0098] Step S1031: Based on the first data, determine the sensing range and resolution of the first data.

[0099] In this step, the perception range can represent the area range corresponding to the first data, and the resolution can indicate the clarity of the first data.

[0100] Step S1031: Based on the location coordinates, orientation angle, timestamp, and the feature vector, cross-semantic features, and decision features of the first data, as well as the perception range and resolution of the first data, construct a data model.

[0101] Based on the above steps, the data model constructed in this step can be represented as follows: in, Indicates position coordinates and orientation angle. Represents a timestamp. This represents the feature vector, cross-semantic features, and decision features of the first data. Indicates the confidence level of the decision. This indicates the sensing range and resolution.

[0102] In some embodiments, step S104 may include the following steps.

[0103] Step S1041: Establish a finite state machine based on the predefined operation process.

[0104] Optionally, the finite state machine in this step can be represented as: ,in, It can represent a set of operation states. It can represent a set of triggering events. Represents the state transition function. Indicates the initial state. This represents the set of legal termination states.

[0105] Step S1042: Extract the sequence of operation events based on the data model.

[0106] Based on this finite state machine, the sequence of operation events (i.e., at least part of the standard operation procedure) can be extracted.

[0107] Step S1043: Match the operation event sequence with the finite state machine to generate the deviation of the operation events.

[0108] Alternatively, this deviation can be calculated using the following formula. ,in, This indicates the number of successful transitions. This indicates the total number of transfers.

[0109] Step S1044: Perform operational compliance checks based on deviation.

[0110] After obtaining the deviation through the above steps, it is possible to determine whether an event has occurred that violates compliance requirements, and an early warning can be issued.

[0111] For example, when the deviation exceeds a preset threshold, it can be determined that a violation has occurred and an early warning can be issued.

[0112] Finite state machines can be used to achieve complete traceability of the "personnel-time-space-operation-object" chain.

[0113] Although this disclosure is based on the foregoing preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make some changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of protection of the invention is determined by the claims.

[0114] It should be noted that the data processing method in this embodiment is not based on... Figure 1 The specific order is a limitation. It should be understood that in other embodiments, the order of some steps in the data processing method of this disclosure can be interchanged according to actual needs, or some steps can be omitted or deleted.

[0115] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0116] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a data processing apparatus. For example... Figure 2 As shown, the data processing device includes: Module 11 is configured to acquire first data from multiple types of data sources; The fusion module 12 is configured to perform data fusion based on the first data to generate the second data; the data fusion includes feature concatenation fusion and decision confidence fusion, wherein the purpose of feature concatenation fusion is to generate feature vectors of the first data from a single data source, cross-semantic features of the first data from multiple data sources, and decision features of the first data, and the purpose of decision confidence fusion is to generate decision confidence. Module 13 is configured to build a data model based on the first data and the second data; The detection module 14 is configured to perform operational compliance detection based on the data model.

[0118] Optionally, the first data includes personnel identification data, operation work order data, historical behavior pattern data, and equipment status data; Fusion module 12 is specifically configured as follows: Generate the feature vector of a single data source based on the first data from that single data source. Based on the feature vector, the personnel identity data, and the operation work order data, cross-semantic features of the first data from multiple data sources are generated using a cross-semantic attention mechanism algorithm. Based on the cross-semantic features, the historical behavior pattern data, and the device status data, a graph neural network algorithm is used to generate decision features.

[0119] Optionally, the fusion module 12 is specifically configured as follows: Based on the first data, the confidence level of the predetermined security decision is generated using the trust function theory. Among them, when the system of conflicting data between the first data from multiple data sources exceeds a preset threshold, a manual reconciliation process is triggered.

[0120] Optionally, the first data may further include fixed camera data; the data fusion may further include field-of-view fusion. The device is also configured to: In response to the visibility of the fixed camera data acquired by the fixed camera being less than a preset threshold, an inspection robot and / or personnel are mobilized to the area where the fixed camera is located to collect inspection robot data and / or law enforcement recorder data. Based on the data from the fixed camera, the data from the patrol robot, and / or the data from the law enforcement recorder, a second set of data is generated by fusing the field of view using a multi-view stereo vision algorithm.

[0121] Optionally, module 13 is specifically configured as follows: Data alignment is performed on the first data from multiple types of data sources in terms of time and space to determine the location coordinates, orientation angle, and timestamp of the first data. Based on the first data, determine the sensing range and resolution of the first data; The data model is constructed based on the location coordinates, the orientation angle, the timestamp, and the feature vector, cross-semantic features, and decision features of the first data, as well as the perception range and resolution of the first data.

[0122] Optionally, the detection module 14 is specifically configured as follows: A finite state machine is built based on a predefined operation process; Extract the sequence of operation events based on the data model; The sequence of operation events is matched with the finite state machine to generate the deviation of the operation events; Operational compliance is checked based on the deviation.

[0123] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing the embodiments of this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0124] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0125] like Figure 3As shown, the computer device 500 may include: a processor 502, a memory 504, a network interface 506, a peripheral interface 508, and a bus 510. The processor 502, memory 504, network interface 506, and peripheral interface 508 are interconnected within the computer device 500 via the bus 510.

[0126] Processor 502 may be a central processing unit (CPU), image processor, neural network processor (NPU), microcontroller (MCU), programmable logic device, digital signal processor (DSP), application-specific integrated circuit (ASIC), or one or more integrated circuits. Processor 502 can be used to perform functions related to the techniques described in this disclosure. In some embodiments, processor 502 may also include multiple processors integrated as a single logic component. For example, such as... Figure 3 As shown, processor 502 may include multiple processors 502a, 502b and 502c.

[0127] Memory 504 can be configured to store data (e.g., instructions, computer code, etc.). Figure 3 As shown, the data stored in memory 504 may include program instructions (e.g., one or more programs for implementing the data detection method of this disclosure embodiment) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). Processor 502 may also access the program instructions and data stored in memory 504 and execute the program instructions to operate on the data to be processed. Memory 504 may include volatile or non-volatile storage devices. In some embodiments, memory 504 may include random access memory (RAM), read-only memory (ROM), optical disk, magnetic disk, hard disk, solid-state drive (SSD), flash memory, memory stick, etc.

[0128] Network interface 506 can be configured to provide communication with other external devices to computer device 500 via a network. This network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the specific examples described above.

[0129] The peripheral interface 508 can be configured to connect the computer device 500 to one or more peripheral devices to enable information input and output. For example, peripheral devices may include input devices such as keyboards, mice, touchpads, touch screens, microphones, and various sensors, as well as output devices such as displays, speakers, vibrators, and indicator lights.

[0130] Bus 510 can be configured to transfer information between various components of computer device 500 (such as processor 502, memory 504, network interface 506, and peripheral interface 508), such as internal buses (e.g., processor-memory bus), external buses (USB port, PCI-E bus), etc.

[0131] It should be noted that although the architecture of the computer device 500 described above only shows the processor 502, memory 504, network interface 506, peripheral interface 508, and bus 510, in specific implementations, the architecture of the computer device 500 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the architecture of the computer device 500 described above may only include the components necessary for implementing the embodiments of this disclosure, and does not necessarily include all the components shown in the figures.

[0132] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the one or more processors to perform the data detection method.

[0133] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0134] The computer program stored in the storage medium of the above embodiments is used to cause the one or more processors to execute the data detection method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0135] Based on the same inventive concept, corresponding to the data detection method of any of the above embodiments, this disclosure also provides a computer program product, which includes one or more computer programs. In some embodiments, the one or more computer programs are executable by one or more processors to cause the one or more processors to perform the data detection method. Corresponding to the execution entity for each step in each embodiment of the data detection method, the processor executing the corresponding step may belong to the corresponding execution entity.

[0136] The computer program product of the above embodiments is used to cause the processor to execute the data detection method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0137] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0138] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0139] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0140] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, characterized in that, include: Retrieve first data from multiple types of data sources; Based on the first data, data fusion is performed to generate the second data; The data fusion includes feature concatenation fusion and decision confidence fusion. The purpose of feature concatenation fusion is to generate feature vectors of first data from a single data source, cross-semantic features of first data from multiple data sources, and decision features of the first data. The purpose of decision confidence fusion is to generate decision confidence. Based on the first data and the second data, a data model is constructed; Operational compliance is checked based on the data model.

2. The method according to claim 1, characterized in that, The first data includes personnel identity data, operation work order data, historical behavior pattern data, and equipment status data; Based on the first data, feature concatenation and fusion are performed, including: Generate the feature vector of a single data source based on the first data from that single data source. Based on the feature vector, the personnel identity data, and the operation work order data, cross-semantic features of the first data from multiple data sources are generated using a cross-semantic attention mechanism algorithm. Based on the cross-semantic features, the historical behavior pattern data, and the device status data, a graph neural network algorithm is used to generate decision features.

3. The method according to claim 1, characterized in that, Decision confidence fusion based on the first data includes: Based on the first data, the confidence level of the predetermined security decision is generated using the trust function theory. Among them, when the system of conflicting data between the first data from multiple data sources exceeds a preset threshold, a manual reconciliation process is triggered.

4. The method according to claim 1, characterized in that, The first data also includes fixed camera data; the data fusion also includes field-of-view fusion. The method further includes: In response to the visibility of the fixed camera data acquired by the fixed camera being less than a preset threshold, an inspection robot and / or personnel are mobilized to the area where the fixed camera is located to collect inspection robot data and / or law enforcement recorder data. Based on the data from the fixed camera, the data from the patrol robot, and / or the data from the law enforcement recorder, a second set of data is generated by fusing the field of view using a multi-view stereo vision algorithm.

5. The method according to claim 1, characterized in that, Based on the first data and the second data, a data model is constructed, including: Data alignment is performed on the first data from multiple types of data sources in terms of time and space to determine the location coordinates, orientation angle, and timestamp of the first data. Based on the first data, determine the sensing range and resolution of the first data; The data model is constructed based on the location coordinates, the orientation angle, the timestamp, and the feature vector, cross-semantic features, and decision features of the first data, as well as the perception range and resolution of the first data.

6. The method according to claim 1, characterized in that, Operational compliance checks are performed based on the aforementioned data model, including: A finite state machine is built based on a predefined operation process; Extract the sequence of operation events based on the data model; The operation event sequence is matched with the finite state machine to generate the deviation of the operation events; Operational compliance is checked based on the deviation.

7. A data processing apparatus, characterized in that, include: The acquisition module is configured to acquire first data from multiple types of data sources; The fusion module is configured to perform data fusion based on the first data to generate the second data; the data fusion includes feature concatenation fusion and decision confidence fusion, wherein the purpose of feature concatenation fusion is to generate feature vectors of the first data from a single data source, cross-semantic features of the first data from multiple data sources, and decision features of the first data, and the purpose of decision confidence fusion is to generate decision confidence. The building module is configured to build a data model based on the first data and the second data; The detection module is configured to perform operational compliance detection based on the data model.

8. A computer device comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the one or more programs comprising instructions for performing the method of any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium comprising a computer program, which, when executed by one or more processors, causes the one or more processors to perform the method of any one of claims 1 to 6.

10. A computer program product comprising one or more computer programs that, when executed by one or more processors, implement the method as described in any one of claims 1 to 6.

Images