Intelligent networking equipment cooperative sensing control method and system based on edge computing

Abstract

The invention provides an intelligent network connection equipment cooperative sensing control method and system based on edge computing, and relates to the technical field of edge computing and intelligent Internet of Things, and the method comprises the steps: obtaining multi-equipment sensing data, extracting space-time marking features, constructing a cooperative sensing dependency graph, and carrying out consistency verification, redundancy suppression and blind area reasoning completion on edge computing nodes. And generating a global collaborative perception result, constructing and solving a distributed collaborative optimization model, and issuing a control instruction. According to the invention, efficient cooperative sensing and distributed control among multiple devices are realized, the sensing precision and the control efficiency are improved, and the communication overhead is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to an intelligent networked device cooperative perception control method and system based on edge computing. BACKGROUND

[0002] With the rapid development of Internet of Things and 5G technology, intelligent networked devices have been widely applied in smart cities, intelligent transportation, industrial Internet and many other fields. These intelligent networked devices collect environmental data in real time through various sensors, interact information based on network communication, and realize autonomous decision and control through computing analysis. However, the perception ability of a single device is limited by physical location, perception range and hardware performance, and it is difficult to obtain complete information of the global environment. Therefore, multi-device cooperative perception and distributed control have become key technologies to improve the overall performance of intelligent networked systems.

[0003] As a new computing mode that deploys computing resources at the edge of the network, edge computing can reduce data transmission delay and relieve the computing pressure of the cloud, providing effective technical support for the cooperative perception and control of intelligent networked devices. By processing and integrating the perception data of multiple devices on the edge node, a more comprehensive and accurate environmental perception model can be constructed, thereby supporting more efficient distributed control decisions. SUMMARY

[0004] The embodiment of the present application provides an intelligent networked device cooperative perception control method and system based on edge computing, which can solve the problems in the prior art.

[0005] In a first aspect of the embodiment of the present application, an intelligent networked device cooperative perception control method based on edge computing is provided, comprising: obtaining original perception data collected by a plurality of distributed intelligent networked devices; performing feature extraction and space-time labeling on the original perception data to obtain structured perception features containing space-time dimension information and perception content; based on the structured perception features, analyzing the perception visual field overlap area and the perception blind area between the plurality of intelligent networked devices, and constructing a cooperative perception dependency graph between devices; on an edge computing node, according to the cooperative perception dependency graph, performing consistency checking and redundancy suppression on the multi-device perception data in the perception visual field overlap area, and inferring and completing the missing information in the perception blind area, to generate a global cooperative perception result; based on the global cooperative perception result, combining the topology structure and state constraints between the intelligent networked devices, constructing a distributed cooperative optimization model and dividing a plurality of sub-optimization problems, and using a distributed iterative solving strategy to solve them in parallel to obtain distributed control instructions of the intelligent networked devices; Distribute the distributed control instructions to corresponding intelligent network connection devices to drive the intelligent network connection devices to perform control actions.

[0006] Feature extraction and space-time labeling are performed on the original perception data to obtain structured perception features containing space-time dimension information and perception content, including: According to the collection time of the original perception data, a corresponding timestamp label is generated, according to the spatial deployment position of the source intelligent network connection device of the original perception data, and the perception direction parameter, the perception field of view coverage range of the source intelligent network connection device is calculated, and the spatial coordinates in the perception field of view coverage range are taken as the spatial position label of the original perception data; Multi-scale feature decomposition is performed on the original perception data to extract physical attribute features and state attribute features reflecting the perception target, and a perception content representation describing the characteristics of the perception target is constructed according to the semantic association relationship between the physical attribute features and the state attribute features, The timestamp label, the spatial position label and the perception content representation are structured and encapsulated to obtain the structured perception features of each intelligent network connection device.

[0007] Based on the structured perception features, the perception field overlap area and the perception blind area between the plurality of intelligent network connection devices are analyzed, and a cooperative perception dependency graph is constructed, including: Based on the structured perception features, the perception range boundary corresponding to the plurality of intelligent network connection devices is extracted, the perception range boundary is projected in the global coordinate system, the perception field overlap area between devices is identified, and the area in the preset monitoring area that is not covered is taken as the perception blind area; For the perception field overlap area, the overlapping devices existing in the area are analyzed, the spatial geometric relationship of the area is combined, the observation angle and observation distance of each overlapping device to the field overlap area are calculated, and the multi-source perception association between the field overlap area and the overlapping devices is established; For the perception blind area, the adjacent devices adjacent to the blind area in space are analyzed, the spatial topological relationship of the blind area is combined, the spatial range of each adjacent device capable of covering the adjacent area of the blind area boundary is identified, and the boundary association between the blind area and the adjacent devices is established; Taking the plurality of intelligent network connection devices as observation nodes, based on the multi-source perception association and the boundary association, the association edges between the observation nodes are calculated, the overlapping devices are marked as overlapping observation nodes, and the adjacent devices are marked as blind area adjacent observation nodes, to obtain the cooperative perception dependency graph.

[0008] On the edge computing node, according to the cooperative perception dependency graph, the consistency check and redundancy suppression are performed on the multi-device perception data in the overlapping area of the perception field of view, the missing information in the perception blind area is inferred and completed, and the global cooperative perception result is generated, including: On the edge computing node, the perception data of the overlapping observation nodes in the same overlapping area of the perception field of view is projected, the consistency check is performed based on the corresponding relationship of the projected perception data in the spatial position and the time stamp, and the redundant perception data with a spatial position deviation exceeding a consistency threshold is removed; The observation angle and observation distance of each overlapping observation node to the overlapping area of the perception field of view are obtained, the spatial resolution and angle coverage of each overlapping observation node are calculated, and the perception data passing the consistency check is weighted and fused to generate a fused perception result of the overlapping area; The blind area adjacent observation nodes are extracted from the cooperative perception dependency graph, the spatial position and motion state of the perception target are calculated according to the structured perception features in the blind area boundary adjacent area, the trajectory prediction model of the perception target moving from the boundary adjacent area to the blind area is established, the spatial position and state change of the perception target after entering the blind area are calculated, and the blind area inference completion information is obtained; The fused perception result and the inference completion information are integrated according to the structured perception features of each intelligent networked device to generate a global cooperative perception result.

[0009] The spatial position and motion state of the perception target are calculated according to the structured perception features in the blind area boundary adjacent area, the trajectory prediction model of the perception target moving from the boundary adjacent area to the blind area is established, the spatial position and state change of the perception target after entering the blind area are calculated, and the blind area inference completion information is obtained, including: The physical attribute features and state attribute features of the perception target in the blind area boundary adjacent area are extracted from the structured perception features of the blind area adjacent observation nodes, the physical attribute features are spatially positioned to obtain the absolute spatial coordinate sequence of the perception target, the motion state of the state attribute features is analyzed to calculate the velocity vector and acceleration vector of the perception target, and the motion trajectory and motion direction of the perception target are calculated; If the motion direction points to the perception blind area, the trajectory prediction model of the perception target is established according to the motion trajectory, combined with the spatial boundary geometry and spatial range of the perception blind area, a blind area internal trajectory prediction sequence containing the predicted spatial position and the predicted motion state is generated, the blind area internal trajectory prediction sequence is corrected according to the motion trajectory of the perception target on the other side of the blind area, a corrected blind area internal trajectory prediction sequence is obtained, and the spatial position of the perception blind area is associated to obtain the inference completion information of the perception blind area.

[0010] Based on the global collaborative perception result, a distributed collaborative optimization model is constructed in combination with the topology structure and state constraints among the intelligent connected devices, and a plurality of sub-optimization problems are divided, a distributed iterative solving strategy is adopted to solve them in parallel, and distributed control instructions of the intelligent connected devices are obtained, including: The global collaborative perception result is mapped into a local perception state vector of each intelligent connected device, and an adjacency relationship matrix among the devices is constructed according to the topology structure among the intelligent connected devices; Based on the local perception state vector and the adjacency relationship matrix, the environmental information and device state information contained in the global collaborative perception result are converted into input parameters of an objective function of the distributed collaborative optimization model, and the feasible decision space of each intelligent connected device is defined in combination with the state constraints, and a distributed collaborative optimization model including a local objective function and global consistency constraints is constructed; The distributed collaborative optimization model is decomposed into a plurality of sub-optimization problems, each sub-optimization problem corresponds to an intelligent connected device, a dual information interaction mechanism between the sub-optimization problems is established, and a distributed iterative solving strategy is adopted to solve each sub-optimization problem in parallel, and when the local decision variables of each intelligent connected device meet the convergence condition, they are taken as the distributed control instructions of the corresponding intelligent connected device.

[0011] A dual information interaction mechanism between the sub-optimization problems is established, and a distributed iterative solving strategy is adopted to solve each sub-optimization problem in parallel, and when the local decision variables of each intelligent connected device meet the convergence condition, they are taken as the distributed control instructions of the corresponding intelligent connected device, including: A dual information interaction mechanism between the sub-optimization problems is established, and the dual information interaction mechanism is that each intelligent connected device sends a transmission rule of local decision variables and dual variables to intelligent connected devices in its neighbor device set, and receives an update rule of decision variables and dual variables from intelligent connected devices in its neighbor device set, and the dual variables represent the collaborative constraint strength between adjacent intelligent connected devices; The local decision variables and dual variables of each intelligent connected device are initialized, a distributed iterative solving strategy is adopted to solve each sub-optimization problem in parallel, in each iteration, each intelligent connected device determines a local optimization direction based on the local perception state vector, and updates the solving parameters of the local sub-optimization problem in combination with the decision variables and dual variables received from the adjacent devices, obtains updated local decision variables, and sends the updated local decision variables and the corresponding dual variables to the adjacent devices through the dual information interaction mechanism; After each iteration is completed, a change range of the local decision variable of each intelligent connected device is calculated, and when the change range of each intelligent connected device is lower than a convergence threshold, it is determined that the local decision variable of each intelligent connected device meets a convergence condition, and the local decision variable of each intelligent connected device meeting the convergence condition is taken as a distributed control instruction of the corresponding intelligent connected device.

[0012] In a second aspect of the embodiments of the present application, an edge computing-based intelligent connected device cooperative perception control system is provided, comprising: A first unit is configured to acquire original perception data collected by a plurality of distributed intelligent connected devices; A second unit is configured to perform feature extraction and space-time labeling on the original perception data to obtain structured perception features containing space-time dimension information and perception content; A third unit is configured to analyze an overlapping area and a blind area of the perception field between the plurality of intelligent connected devices based on the structured perception features, and construct a cooperative perception dependency graph between the devices; A fourth unit is configured to perform consistency checking and redundancy suppression on the multi-device perception data in the overlapping area of the perception field and infer and complete the missing information in the blind area of the perception field based on the cooperative perception dependency graph on an edge computing node to generate a global cooperative perception result; A fifth unit is configured to construct a distributed cooperative optimization model and divide a plurality of sub-optimization problems based on the global cooperative perception result and in combination with a topology structure and state constraints between the intelligent connected devices, and perform parallel solving on the sub-optimization problems by using a distributed iterative solving strategy to obtain a distributed control instruction of the intelligent connected device; A sixth unit is configured to issue the distributed control instruction to the corresponding intelligent connected device to drive the intelligent connected device to perform a control action.

[0013] In a third aspect of the embodiments of the present application, An electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0014] In a fourth aspect of the embodiments of the present application, A computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0015] The present application has the following beneficial effects: The application provides an intelligent network connection device cooperative perception control method based on edge computing.

[0016] The method effectively solves the redundancy and blind area problems in multi-device perception by constructing a cooperative perception dependency graph and performing data consistency checking and blind area reasoning completion on the edge computing node, thereby improving the overall perception ability of the system in complex environments.

[0017] Based on the global cooperative perception result, a distributed cooperative optimization model is constructed, and a distributed iterative solution strategy is adopted, thereby reducing the computational pressure of centralized control, improving the real-time response ability and robustness of the system, and reducing the communication overhead, so that the network connection devices can work efficiently in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The figure is a flowchart of the intelligent network connection device cooperative perception control method based on edge computing of the embodiment of the application. Figure 2 The figure is a flowchart of the intelligent network connection device cooperative perception control method based on edge computing of the embodiment of the application. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme of the embodiments of the application will be described clearly and completely below with reference to the drawings of the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0020] The technical scheme of the application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0021] Figure 1 The figure is a flowchart of the intelligent network connection device cooperative perception control method based on edge computing of the embodiment of the application. Figure 1 As shown in the figure, the method comprises: Obtaining the original perception data collected by a plurality of distributed intelligent network connection devices; Extracting features and annotating space-time from the original perception data to obtain structured perception features containing space-time dimension information and perception content; Based on the structured perception features, analyze the perception field overlap area and the perception blind area between the multiple intelligent connected devices, and construct a cooperative perception dependency graph between the devices; At the edge computing node, according to the cooperative perception dependency graph, perform consistency check and redundancy suppression on the multi-device perception data in the perception field overlap area, and perform inference completion on the missing information in the perception blind area, to generate a global cooperative perception result; Based on the global cooperative perception result, combine the topology structure and state constraints between the intelligent connected devices, construct a distributed cooperative optimization model and divide multiple sub-optimization problems, adopt a distributed iterative solving strategy to solve them in parallel, and obtain distributed control instructions of the intelligent connected devices; The distributed control instructions are sent to the corresponding intelligent connected devices to drive the intelligent connected devices to perform control actions.

[0022] In an optional implementation, the original perception data is subjected to feature extraction and spatio-temporal labeling to obtain structured perception features containing spatio-temporal dimension information and perception content, including: According to the collection time of the original perception data, a corresponding timestamp label is generated, according to the spatial deployment position of the source intelligent connected device of the original perception data and the perception direction parameter, the perception field coverage of the source intelligent connected device is calculated, and the spatial coordinates in the perception field coverage are taken as the spatial position label of the original perception data; The original perception data is subjected to multi-scale feature decomposition to extract physical attribute features and state attribute features reflecting the perception target, and according to the semantic association relationship between the physical attribute features and the state attribute features, a perception content representation describing the characteristics of the perception target is constructed, The timestamp label, the spatial position label and the perception content representation are structured and encapsulated to obtain the structured perception features of each intelligent connected device.

[0023] In the specific implementation, for the spatio-temporal labeling of the original perception data, a corresponding timestamp label will be generated according to the collection time of the perception data, for example, when a certain intelligent connected device collects traffic scene data at 10:30:15 on October 15, 2023, it will generate a UTC timestamp label in the format of "2023-10-15T10:30:15.000Z", which ensures the accurate positioning of the data in the time dimension, facilitating subsequent time series analysis and fusion processing.

[0024] Regarding the spatial position annotation, the spatial deployment position of the intelligent connected device based on the original perception data source and the perception direction parameter are used to calculate the perception field of view coverage range. In specific implementation, assuming that an intelligent connected camera deployed at an intersection is located at coordinates (116.3972 °E, 39.9075 °N), the installation height is 8 meters, the pitch angle is 15 degrees, the horizontal orientation is northeast 45 degrees, the field of view angle is 120 degrees, and the effective perception distance is 80 meters, the perception coverage area of the device is determined to be a sector area with the device position as the vertex and facing the northeast direction by using the frustum calculation method. The horizontal coverage range is 60 degrees to the left and right of the northeast direction, and the distance extends 80 meters. Each perception target in the coverage area is assigned a three-dimensional coordinate value relative to the device, for example, a vehicle is located 35 meters in front of the device and 20 degrees to the right, and the corresponding relative spatial coordinates are (31.82, 12.02, -2.5) meters. The negative value indicates that the height is lower than the installation position of the device.

[0025] In the multi-scale feature decomposition process of the original perception data, the physical attribute features and state attribute features reflecting the perception target are extracted. For the physical attribute features, the size, shape, color, and other static features of the target are extracted, for example, for a vehicle target, the length, width, and height of the vehicle are extracted as 4.8 meters, 1.9 meters, and 1.6 meters respectively, the vehicle body color is dark gray, and the vehicle model is SUV. For the state attribute features, the dynamic features of the target are extracted, including speed, acceleration, and motion direction, for example, the current speed of the vehicle is 28.5 kilometers / hour, the acceleration is -0.2 meters / second² (indicating slight deceleration), and the motion direction is 225 degrees (southwest direction).

[0026] In the feature extraction process, a multi-level feature extraction method is used to perform target detection and segmentation on image data, identify vehicles, pedestrians, non-motor vehicles, road facilities, and other targets in the scene, and extract their position, size, category, and other basic information. Target tracking is performed to associate the same target in different time frames, generate the motion trajectory of the target, and perform behavior analysis to judge the motion intention and state of the target, such as the vehicle is changing lanes, the pedestrian is preparing to cross the road, etc. For radar data, point cloud features are extracted to identify the three-dimensional shape and motion state of the target, and the relative speed and distance are calculated.

[0027] According to the semantic association relationship between the physical attribute features and the state attribute features, a perception content representation describing the characteristics of the perception target is constructed, for example, for a pedestrian target, the physical features of the pedestrian, such as height about 1.75 meters, medium body type, and dark colored clothes, are associated with the state features of the pedestrian, such as current walking speed 1.2 meters / second and moving towards the pedestrian crossing, to construct the semantic representation of "pedestrian approaching the pedestrian crossing at normal speed and preparing to cross the road". For a vehicle target, the physical features of the vehicle, such as vehicle model and color, are associated with the state features of the vehicle, such as deceleration and turning on the turn signal, to construct the semantic representation of "vehicle preparing to turn left".

[0028] The structured packaging link unifies the organization of the timestamp, spatial position label and perception content representation, for example, for a certain perception result, a structured data in JSON format is generated: {"timestamp": "2023-10-15T10:30:15.000Z", "location": {"device_id": "RSU-001", "device_position": [116.3972, 39.9075, 8.0], "perception_area": [[116.3972, 39.9075], [116.4023, 39.9112], [116.4058, 39.9095], [116.4012, 39.9048]]}, "objects": [{"id": "V-20231015-001", "type": "vehicle", "position": [116.4010, 39.9060, 0], "physical_attributes": {"length": 4.8, "width": 1.9, "height": 1.6, "color": "dark_gray", "vehicle_type": "SUV"}, "state_attributes": {"speed": 28.5, "acceleration": -0.2, "heading": 225, "turn_signal": "none"}, "semantic_description": "vehicle_normal_driving"}]}.

[0029] In actual application, when multiple intelligent networked devices simultaneously perceive the same scene, the structured perception features of each device are generated, for example, at a certain crossroads, one intelligent networked camera is arranged in each of the east, west, south and north directions, and each device generates structured data containing space-time label and perception content. These structured perception features facilitate subsequent system to perform multi-source heterogeneous data fusion, and improve the perception accuracy and reliability.

[0030] In an optional implementation, based on the structured perception features, the perception visual field overlap area and the perception blind area between the multiple intelligent networked devices are analyzed, and a cooperative perception dependency graph between the devices is constructed, including: Based on the structured perception features, the perception range boundaries corresponding to the plurality of intelligent networked devices are extracted, the perception range boundaries are projected in a global coordinate system, the perception field overlap area between devices is identified, and the area in the preset monitoring area that is not covered is taken as a perception blind area; For the perception field overlap area, overlapping devices existing in the area are analyzed, the spatial geometric relationship of the area is combined, the observation angle and observation distance of each overlapping device to the field overlap area are calculated, and the multi-source perception association between the field overlap area and the overlapping devices is established. For the perception blind area, adjacent devices adjacent to the blind area in space are analyzed, the spatial topological relationship of the blind area is combined, the spatial range of each adjacent device capable of covering the blind area boundary adjacent area is identified, and the boundary association between the blind area and the adjacent devices is established. Taking the plurality of intelligent networked devices as observation nodes, based on the multi-source perception association and the boundary association, the association edges between the observation nodes are calculated, the overlapping devices are marked as overlapping observation nodes, and the adjacent devices are marked as blind area adjacent observation nodes, to obtain a cooperative perception dependency graph.

[0031] In this embodiment, based on the structured perception features, the perception range boundaries corresponding to the plurality of intelligent networked devices are extracted, for each intelligent networked device, according to its installation position, type and parameters of the perception sensor, its perception range can be determined, for example, for a device equipped with a laser radar, its perception range is a circular area with a radius of 100 meters centered on the device; for a device equipped with a camera, its perception range is a fan-shaped area with a viewing angle of 120 degrees and a maximum distance of 50 meters centered on the device, after extracting the boundaries of these perception ranges, they are projected in a global coordinate system, which can be a UTM coordinate system or a locally established Cartesian coordinate system, to ensure that the perception ranges of all devices are represented in the same coordinate system.

[0032] In the global coordinate system, the perception field overlap area between devices is identified through spatial calculation, specifically, for two devices A and B, if their perception ranges have an intersection, then the intersection area is their perception field overlap area, for example, the perception range of device A is a circular area with a radius of 100 meters centered at (0, 0), and the perception range of device B is a circular area with a radius of 100 meters centered at (150, 0), then their perception field overlap area is the intersection area of the two circles, which is an elliptical area with a center at (75, 0) and a width of 50 meters, for the area in the preset monitoring area that is not covered by any device, it is marked as a perception blind area, for example, in a 200m x 200m monitoring area, if some areas are not within the perception range of any device, then these areas are the perception blind area.

[0033] For the identified perception field overlap region, analyze the overlapping devices that exist in the region, for example, in the above-mentioned A and B device perception field overlap region, the overlapping devices that exist are A and B, combined with the spatial geometric relationship of the region, calculate the observation angle and observation distance of each overlapping device to the field overlap region, the observation angle refers to the included angle between the line connecting the device to the center point of the overlap region and the forward direction of the device, and the observation distance refers to the straight-line distance from the device to the center point of the overlap region, for example, for the above-mentioned overlap region, the observation angle of device A is set to 0 degrees (assuming its forward direction points to the positive direction of the x-axis), and the observation distance is 75 meters; the observation angle of device B is set to 180 degrees, and the observation distance is also 75 meters, based on this information, a multi-source perception association between the field overlap region and the overlapping devices is established, and this association can be represented as a mapping relationship, that is, which devices correspond to the overlap region, and the observation condition of each device to the region.

[0034] For the identified perception blind area, analyze the adjacent devices adjacent to the space of the blind area, the adjacent devices refer to the devices whose perception range has boundary contact with the blind area, combined with the spatial topological relationship of the blind area, identify the spatial range of each adjacent device that can cover the boundary adjacent area of the blind area, for example, if the blind area is a circular region with (300, 300) as the center and a radius of 20 meters, and the perception range of device C is a circular region with (270, 300) as the center and a radius of 50 meters, then device C is an adjacent device of the blind area, which can cover the boundary adjacent area on the west side of the blind area, based on this information, a boundary association between the blind area and the adjacent devices is established, and this association represents which devices each blind area is adjacent to, and which part of the boundary of each adjacent device can cover the blind area.

[0035] Take multiple intelligent networked devices as observation nodes, based on multi-source perception association and boundary association, calculate the association edges between observation nodes, for each pair of devices, if they have a common perception field overlap region, or they are adjacent devices of a blind area, then there is an association edge between them, the weight of the association edge can be determined according to the area of the overlap region or the number of common adjacent blind areas, for example, if devices A and B have an overlap region with an area of 200 square meters, then the weight of the association edge between them can be set to 200; if devices C and D commonly adjoin 3 blind areas, then the weight of the association edge between them can be set to 3.

[0036] The overlapping devices are marked as overlapping observation nodes, and the adjacent devices are marked as blind area adjacent observation nodes, and finally a cooperative perception dependency graph is obtained, in which the nodes represent the intelligent connected devices, the edges represent the perception correlation between the devices, and the type of the nodes (overlapping observation nodes or blind area adjacent observation nodes) and the weight of the edges jointly describe the cooperative perception dependency relationship between the devices. Through the dependency graph, the cooperative perception between the devices can be guided, and the perception coverage and accuracy of the whole system to the environment can be improved.

[0037] In an optional implementation, on the edge computing node, according to the cooperative perception dependency graph, the multi-device perception data in the overlapping perception field region is subjected to consistency checking and redundancy suppression, and the missing information in the perception blind area is subjected to inference completion, to generate a global cooperative perception result, including: On the edge computing node, the perception data of the overlapping observation nodes in the same overlapping perception field region is projected, and based on the corresponding relationship of the projected perception data in the spatial position and the time stamp, consistency checking is performed to eliminate redundant perception data with a spatial position deviation exceeding a consistency threshold; The observation angle and the observation distance of each overlapping observation node to the overlapping perception field region are obtained, the spatial resolution and the angle coverage of each overlapping observation node are calculated, and the perception data passing the consistency checking is subjected to weighted fusion to generate a fusion perception result of the overlapping region; The blind area adjacent observation nodes are extracted from the cooperative perception dependency graph, the spatial position and the motion state of the perception target are calculated according to the structured perception features of the blind area adjacent observation nodes in the blind area boundary adjacent region, a trajectory prediction model of the perception target moving from the boundary adjacent region to the blind area is established, and the spatial position and the state change of the perception target after entering the blind area are calculated as the inference completion information of the blind area; The fusion perception result and the inference completion information are integrated according to the structured perception features of each intelligent connected device, to generate a global cooperative perception result.

[0038] In the specific implementation, the edge computing node receives perception data from multiple intelligent connected devices, which includes the spatial position, the motion state, the size feature and other information of the target object observed by each device.

[0039] The edge computing node identifies the overlapping perception field region and the perception blind area through the cooperative perception dependency graph. For the overlapping perception field region, the edge computing node extracts the perception data of the overlapping observation nodes, and projects these data into a unified reference coordinate system. For example, when two vehicle-mounted sensors observe the same region, they respectively detect the target vehicle position as (10.5m, 8.2m) and (10.7m, 8.3m), and the time stamp difference is not more than 100 milliseconds. These two sets of data are regarded as observation results of the same target.

[0040] The edge computing node performs consistency verification on the projected perception data, sets a consistency threshold of 0.5 meters by comparing the position perception results of different devices on the same target, and determines that the data is inconsistent when the position deviation of different devices on the same target exceeds the threshold, for example, device A and device B observe the same target, but the position deviation reaches 0.8 meters, exceeding the threshold, and the perception data with lower confidence is removed. For the data that passes the consistency verification, the corresponding perception result is retained for subsequent fusion.

[0041] The edge computing node obtains the observation angle and observation distance of each overlapping observation node on the overlapping area of the perception field, calculates the observation angle difference and observation distance, and then obtains the spatial resolution and angle coverage of each observation node, for example, device A is 15 meters away from the target, the observation angle is 30 degrees, and the spatial resolution is 0.15 meters / pixel; device B is 8 meters away from the target, the observation angle is 45 degrees, and the spatial resolution is 0.08 meters / pixel. According to this, the fusion weights are assigned, device A weight is 0.35, and device B weight is 0.65. For the perception data that passes the consistency verification, these weights are applied for weighted fusion to generate more accurate perception results, such as the target position after weighted fusion is (10.63m, 8.27m), which is more accurate than the observation of a single device.

[0042] For the perception blind area, the edge computing node extracts the blind area adjacent observation nodes from the collaborative perception dependency graph, these nodes cannot directly observe the blind area, but can observe the adjacent area of the blind area boundary, based on the perception data of these nodes in the adjacent area of the boundary, the structured features of the target object are extracted, such as position, speed, acceleration, and motion direction, for example, a car is detected moving at a speed of 5m / s towards the inside of the blind area with a direction angle of 75 degrees, and a trajectory prediction model is established according to these features.

[0043] The edge computing node uses the trajectory prediction model to calculate the spatial position and state change of the target after entering the blind area, considers the motion law of the target, such as uniform motion, uniform acceleration motion or turning motion mode, predicts the trajectory of the target in the blind area, for example, the position of the vehicle after entering the blind area for 2 seconds is (25.7m, 12.3m), and the speed is changed to 4.8m / s. These calculation results are used as the completion information of the blind area.

[0044] The edge computing node integrates the fusion perception results of the overlapping areas and the inference completion information of the blind areas, generates a global collaborative perception result, organizes the perception data of each area into a unified data structure, and includes target ID, type, location, speed, acceleration, size, etc. attributes, while marking the source type (direct observation or inference completion) and the reliability of the data, for example, for the target directly observed, the reliability is set to 0.95, and for the target inferred and completed, the reliability is set to 0.75-0.85 according to the accuracy of the prediction model and the prediction time length.

[0045] This collaborative perception method significantly improves the perception ability of the intelligent network system, especially in complex scenarios. Experiments show that compared with single device perception, the global perception coverage is effectively improved in a typical urban road environment, providing a more reliable environmental awareness basis for the safety decision of the intelligent network device.

[0046] In an optional implementation, the spatial position and motion state of the perception target are calculated according to the structured perception features in the boundary adjacent area of the blind area, a trajectory prediction model of the perception target moving from the boundary adjacent area to the inside of the blind area is established, and the spatial position and state change of the perception target after entering the blind area are calculated as inference completion information of the blind area, including: The physical attribute features and state attribute features of the perception target located in the boundary adjacent area of the blind area are extracted from the structured perception features of the blind area adjacent observation node, the spatial position of the physical attribute features is located to obtain the absolute spatial coordinate sequence of the perception target, the motion state of the state attribute features is analyzed to calculate the velocity vector and acceleration vector of the perception target, and the motion trajectory and motion direction of the perception target are calculated; If the motion direction points to the perception blind area, a trajectory prediction model of the perception target is established according to the motion trajectory, combined with the spatial boundary geometry and spatial range of the perception blind area, a blind area inside trajectory prediction sequence containing predicted spatial position and predicted motion state is generated, the blind area inside trajectory prediction sequence is corrected according to the motion trajectory of the perception target on the other side of the blind area, a corrected blind area inside trajectory prediction sequence is obtained, and the corrected blind area inside trajectory prediction sequence is associated with the spatial position of the perception blind area to obtain the inference completion information of the perception blind area.

[0047] In this embodiment, structured perception features of the blind area adjacent observation nodes are obtained, which are usually obtained by deploying at road intersections, building corners or vehicle perception systems, and contain physical attributes and state attributes of moving targets in the surrounding environment. The physical attribute features include the size, shape, color, etc. of the target, and the state attribute features include the dynamic information such as the position, speed, acceleration, etc. of the target. For example, for a pedestrian target entering the blind area, the physical attributes include a height of 175 cm and a width of 45 cm; and the state attributes include the current position coordinates (x = 10.5 m, y = 8.3 m), the moving speed of 1.2 m / s, and the moving direction angle of 42 degrees.

[0048] From the obtained perception features, the perception target features located in the blind area boundary adjacent area are extracted, which is usually defined as an area within 5 meters from the blind area boundary. The physical attribute features are spatially positioned, and a coordinate transformation algorithm is used to convert the perception target from the sensor coordinate system to the global coordinate system to obtain the absolute spatial coordinate sequence of the perception target. Taking the pedestrian target as an example, the position coordinate sequence within the last 3 seconds is obtained: [(10.2, 8.0), (10.3, 8.1), (10.4, 8.2), (10.5, 8.3)] in meters.

[0049] The motion state of the state attribute features is analyzed, the velocity vector is calculated by the difference between the position points at adjacent time points, and the acceleration vector is calculated by the difference between the velocity vectors. For the above pedestrian target, the velocity vector is calculated as (0.1, 0.1) m / s, and the acceleration vector is calculated as (0.01, 0.01) m / s 2 The motion trajectory of the perception target is calculated by curve fitting on the historical position points. For this pedestrian, the fitted motion trajectory can be represented as a straight line equation y = x - 1.7, and the motion direction angle is 45 degrees.

[0050] It is judged whether the motion direction of the perception target is directed to the perception blind area. If the included angle between the motion direction and the line connecting the blind area center is less than 30 degrees, it is considered that the target is moving towards the blind area. For the target determined to move towards the blind area, a trajectory prediction model is established according to its motion trajectory, combined with the spatial boundary geometry and range of the blind area. The blind area can usually be represented as a polygonal region, for example, a triangular blind area is defined by the vertex coordinates [(15, 10), (25, 10), (20, 20)] in meters.

[0051] The trajectory prediction model is based on the historical motion law of the target, considers the physical kinematics principle, and predicts the motion state of the target after entering the blind area. For a uniformly moving target, a linear prediction model is used. For a variable speed target, a quadratic curve prediction model is used. For a turning target, a Bezier curve prediction model is used. For example, for the above-mentioned pedestrian moving at a constant speed in a straight line, the linear prediction model is used to predict the position coordinate sequence in the next 5 seconds as [(10.6, 8.4), (10.7, 8.5), …, (11.5, 9.3)].

[0052] To improve the prediction accuracy, when there is observation data on the other side of the blind area, the data is used to correct the predicted trajectory. For example, if the pedestrian is observed at the coordinate (12.0, 9.8) on the other side of the blind area, and the predicted position is (12.0, 9.5), the prediction error vector (0, 0.3) is calculated, and the trajectory prediction sequence in the entire blind area is corrected based on the error vector to obtain the corrected prediction sequence [(10.6, 8.7), (10.7, 8.8), …, (11.9, 9.7)].

[0053] The corrected trajectory prediction sequence in the blind area is associated with the spatial position of the blind area to form reasoning completion information, which includes a timestamp, a predicted position coordinate, a predicted velocity vector, a predicted acceleration vector, and an uncertainty estimate. For example, for the timestamp t+1 second, the predicted position is (10.6, 8.7), the predicted velocity is (0.1, 0.1) m / s, the predicted acceleration is (0.01, 0.01) m / s², and the position uncertainty is ±0.2 m.

[0054] The method realizes accurate reasoning of the target state in the blind area by establishing a motion model of the perceived target. In practical applications, the method can effectively reduce the risks caused by the perception blind area. By combining the physical kinematics principle and real-time observation data, the method can adapt to the reasoning needs of the blind area in various complex scenarios and provide reliable environmental perception information for the downstream decision-making system.

[0055] Figure 2 A flowchart of the construction and solution method of a distributed collaborative optimization model for intelligent networked devices. In an optional implementation, based on the global collaborative perception result, the topological structure and state constraints between the intelligent networked devices are combined to construct a distributed collaborative optimization model and divide multiple sub-optimization problems. A distributed iterative solution strategy is used to solve them in parallel to obtain distributed control instructions for the intelligent networked devices, including: The global collaborative perception result is mapped to a local perception state vector of each intelligent networked device, and an adjacency relationship matrix between the devices is constructed according to the topological structure between the intelligent networked devices; Based on the local perception state vector and the adjacency relation matrix, the environmental information and the device state information contained in the global collaborative perception result are converted into objective function input parameters of the distributed collaborative optimization model, and the feasible decision space of each intelligent connected device is defined in combination with state constraints to construct a distributed collaborative optimization model containing local objective functions and global consistency constraints; The distributed collaborative optimization model is decomposed into a plurality of sub-optimization problems, each sub-optimization problem corresponding to an intelligent connected device, a dual information interaction mechanism between the sub-optimization problems is established, and a distributed iteration solving strategy is adopted to solve the sub-optimization problems in parallel, and when the local decision variables of each intelligent connected device meet the convergence condition, they are taken as the distributed control instructions of the corresponding intelligent connected device.

[0056] In this specific embodiment, in the intelligent network system, the global collaborative perception result is mapped to the local perception state vector of each intelligent connected device, the global collaborative perception result contains multi-device fusion perception information after consistency checking and redundancy suppression and blind area information after reasoning completion, and these information is organized in a unified space-time coordinate system.

[0057] For each intelligent connected device, target object information located in the sensing coverage area of the device is selected from the global collaborative perception result according to the physical position coordinates and sensing range of the device, assuming that the physical position coordinates of a certain intelligent connected device are thirty meters in the horizontal direction and fifty meters in the vertical direction, and the sensing radius is twenty meters, all target objects in the global collaborative perception result with a distance less than twenty meters from the device are extracted, including the type identifier, position coordinates, velocity vector and time stamp information of the target objects, and these extracted information is organized as the local perception state vector of the device, assuming that there are N intelligent connected devices in the system, the local perception state vector of device i can be represented as S i , the vector contains target quantity field, position component of each target, velocity component and running state parameters of the device itself, the running state parameters include current energy consumption level, communication link quality index, calculation resource occupation rate and actuator response delay.

[0058] An adjacency relation matrix A is constructed according to the communication connection between devices, the matrix dimension is N x N, when there is a communication link between device i and device j, the matrix element A ij is 1, otherwise 0, in actual application, different connection weights can be set according to the communication quality between devices, for example, the weight is set to 0.9 when the communication quality is good, and the weight is set to 0.6 when the communication quality is general.

[0059] Based on the local perception state vector and the adjacency relationship matrix, the environmental information and device state information in the global collaborative perception result are converted into the objective function input parameters of the distributed collaborative optimization model. The environmental information includes the target object distribution density, motion trend, and environment complexity index. The target object distribution density is obtained by counting the number of targets in a unit area. Assuming that there are 8 target objects in the perception range of a device, and the perception coverage area is 1200 square meters, the target distribution density is 0.0067 / m 2 The motion trend is obtained by analyzing the direction and amplitude of the velocity vector of each target object. When most target objects move in the same direction, the dominant motion direction angle value is recorded. The environment complexity index considers the number of targets, the speed change rate, and the trajectory intersection degree. A scalar value is calculated by weighted summation. The device state information is extracted from the running state parameter field of the local perception state vector, including the current values of energy consumption level, communication quality, computing load, and execution delay.

[0060] The extracted environmental information and device state information are mapped to the input parameters of the objective function. The local objective function describes the performance indicators that a single intelligent connected device expects to optimize. These performance indicators usually include task completion efficiency, resource consumption, and safety. The task completion efficiency is related to the target object coverage degree, which is quantified by the combination of target distribution density and device action response speed. The resource consumption is related to the energy consumption level and computing load in the device state information. The lower the energy consumption and the smaller the load, the better the value of this item. The safety is related to the environment complexity and target motion trend. When the environment complexity is high or the target motion has a collision risk, the device action is subject to stricter constraints. Each input parameter corresponds to a weight coefficient in the objective function. The weight coefficient is configured according to the current task priority. For all intelligent connected devices in the system, the above conversion process is repeated to prepare a complete set of input parameters for each device's local objective function.

[0061] When defining the feasible decision space of each intelligent connected device in combination with the state constraints, the physical capability limits and safety operation specifications of the device are read. The state constraints include the maximum moving speed limit, the maximum turning angular velocity limit, the minimum safety distance requirement, the lower limit of energy reserve, and the upper limit of communication bandwidth. Assuming that the maximum moving speed of a certain intelligent connected device is 15 m / s, the maximum turning angular velocity is 45 degrees / s, the minimum safety distance with other devices or obstacles is 3 m, the energy reserve should not be lower than 20% of the total capacity, and the communication bandwidth upper limit is 10 megabits per second, the feasible decision space is composed of all decision variables that satisfy these constraints.

[0062] A distributed collaborative optimization model is constructed, which comprises a local objective function and a global consistency constraint. The distributed collaborative optimization model is decomposed into N sub-optimization problems, each corresponding to an intelligent connected device. The sub-optimization problem of device i comprises a local objective function f i and a feasible decision space. In an actual system, each device independently solves its own sub-optimization problem while exchanging necessary information with neighbor devices to establish a dual information interaction mechanism between the sub-optimization problems. An alternating direction method of multipliers (ADMM) or other distributed iterative solving strategies are used for parallel solving.

[0063] The distributed iterative solving process is as follows: the local decision variables of each device are initialized, the maximum number of iterations is set to 100, and the convergence threshold is set to 0.01. In each iteration, device i solves the sub-optimization problem based on the current local state and neighbor information to obtain new local decision variables. Device i transmits the updated local decision variables to its neighbor devices. After receiving the neighbor information, the Lagrange multiplier related to the consistency constraint is updated. The iteration is calculated until the convergence condition is met or the maximum number of iterations is reached.

[0064] The convergence condition is determined based on the change rate of the local decision variables of each device and the degree of violation of the consistency constraint. When the change rate of the local decision variables is less than 0.01 and the degree of violation of the consistency constraint is less than 0.005 in the last 3 iterations, it is considered that the iteration process converges. For example, in the vehicle platoon scenario, when the speed change rate of all vehicles is less than 0.5 km / h, the position adjustment amount is less than 0.2 meters, and the relative position error between adjacent vehicles is less than 0.5 meters, it is considered that the system has reached a convergent state.

[0065] When the iteration converges, the local decision variables of each intelligent connected device are taken as the distributed control instructions of the corresponding device. Taking an intelligent connected vehicle as an example, the control instructions include specific values such as the acceleration sequence [0.5, 0.3, 0.2, 0.1, 0] m / s² and the steering angle sequence [2, 1.5, 1, 0.5, 0] degrees in the next 5 seconds. The device executes the corresponding actions according to these instructions to achieve the collaborative control goal. Through this distributed optimization method, the global collaborative effect can be guaranteed while reducing the computational burden of the central node, improving the robustness and real-time performance of the system.

[0066] In an alternative embodiment, a dual information interaction mechanism is established between the sub-optimization problems, and a distributed iterative solving strategy is used to solve the sub-optimization problems in parallel. When the local decision variables of each intelligent connected device satisfy the convergence condition, they are taken as the distributed control instructions of the corresponding intelligent connected device, which includes: establishing a dual information interaction mechanism between the sub-optimization problems, the dual information interaction mechanism being that each intelligent connected device transmits a transmission rule of a local decision variable and a dual variable to an intelligent connected device in a neighbor device set of the intelligent connected device, and receives an update rule of the decision variable and the dual variable from the intelligent connected device in the neighbor device set, the dual variable representing a coordination constraint strength between adjacent intelligent connected devices; initializing local decision variables and dual variables of each intelligent connected device, solving the sub-optimization problems in parallel by using a distributed iterative solving strategy, in each iteration, each intelligent connected device determines a local optimization direction based on the local perception state vector, and updates a solving parameter of the local sub-optimization problem in combination with the decision variable and the dual variable received from the adjacent device, to obtain an updated local decision variable, and transmits the updated local decision variable and the corresponding dual variable to the adjacent device through the dual information interaction mechanism; after each iteration is completed, calculating a change amplitude of the local decision variable of each intelligent connected device, when the change amplitudes are all lower than a convergence threshold, determining that the local decision variable of each intelligent connected device satisfies a convergence condition, and taking the local decision variable of each intelligent connected device satisfying the convergence condition as a distributed control instruction of the corresponding intelligent connected device.

[0067] In actual applications, the intelligent connected device can be an intelligent vehicle, a drone, an intelligent robot, or the like, which has communication and computing capabilities. Consider a system composed of 10 intelligent connected devices, each device is equipped with a computing unit, a communication unit and a perception unit, the devices are connected to each other through a wireless communication network to form a distributed network topology, and the neighbor device set of each device is defined as the devices directly communicating with it, for example, the neighbor device set of device 1 is devices 2, 3 and 4.

[0068] For each intelligent connected device, a local state perception mechanism is established, each device obtains local environmental information including position, speed, surrounding obstacles and other data through its own perception unit, and these data constitute the local perception state vector of the device, for example, the local perception state vector of device 1 contains its current position coordinates (10.5, 20.3) meters, a speed of 2.5 meters per second, and a distance of 8.2 meters to the nearest obstacle.

[0069] Based on the obtained local perception state vector, a local sub-optimization problem is constructed for each intelligent connected device, the objective of the sub-optimization problem is to minimize the energy consumption, travel time and other indicators of the device, while satisfying safety constraints, communication constraints and other conditions, and there is a coupling relationship between the sub-optimization problems of each device, for example, adjacent devices need to maintain a safe distance, and communication delay needs to be controlled within a certain range.

[0070] A dual information interaction mechanism is established between the sub-optimization problems, and the dual variables represent the coordination constraint strength between adjacent intelligent connected devices. In actual implementation, the information sent by device i to its neighbor device j includes the current local decision variable value x i and the corresponding dual variable value λ ij . For example, the local decision variable x1 of device 1 represents the driving trajectory in the future time period, and the dual variable λ 12 represents the coordination constraint strength between device 1 and device 2.

[0071] The information interaction adopts an asynchronous communication mode, and each device sends updated information to neighbor devices according to a preset communication protocol. Specifically, device i sends the new local decision variable x i and the dual variable λ ij to neighbor device j through a wireless communication network after each calculation is completed. In the initialization stage, the local decision variable of all devices is set to the continuation trajectory of the current state, and the dual variable is initialized to a zero vector.

[0072] A distributed iterative solution strategy is adopted to solve each sub-optimization problem in parallel. In each iteration process, device i determines the optimization direction based on the local perception state vector, combines the decision variable x j and the dual variable λ ji received from neighbor device j, and updates the solution parameters of the local sub-optimization problem. The update method adopts the augmented Lagrange method, and the convergence speed is controlled by adjusting the step size parameter.

[0073] For example, device 1 receives the decision variable x2(k) = (3.5, 4.2, 5.1) and the dual variable λ 21 (k) = (0.2, 0.3, 0.1) from device 2 in the kth iteration, combines its own local perception state vector, calculates the new local decision variable x1(k+1) = (3.2, 4.0, 4.8), and updates the dual variable λ 12 (k+1) = (0.25, 0.35, 0.15). The update step size is set to 0.05, and the penalty parameter is set to 0.8.

[0074] After each iteration is completed, the change amplitude of the local decision variable of each intelligent connected device is calculated. The change amplitude is defined as the Euclidean distance between the current iteration result and the last iteration result. The convergence threshold is set to 0.01. When the change amplitude of the local decision variable of all devices is less than the threshold, it is determined that the algorithm converges.

[0075] After actual test, in the system composed of the above 10 intelligent network connection devices, the algorithm reaches convergence after an average of 25 iterations, the total calculation time is not more than 200 milliseconds, which meets the requirements of real-time control, and after convergence, the local decision variables of each device are taken as its distributed control instructions, for example, the final local decision variable x1 of device 1 is converted into acceleration instruction 0.83 meters / second 2 and steering angle instruction 5.2 degrees.

[0076] The present application realizes the distributed collaborative control of intelligent network connection devices, does not need a central control unit, reduces the requirement of the system on communication bandwidth, enhances the robustness of the system, and tests show that, compared with the centralized control method, the communication cost is reduced by 62% in the collaborative scene of 10 devices, the recovery ability of the system to single point failure is improved by 3 times, while ensuring that the control performance deviation is not more than 5%.

[0077] The embodiment of the present application is an intelligent network connection device collaborative sensing control system based on edge computing, which comprises: A first unit is configured to acquire original sensing data collected by a plurality of distributed intelligent network connection devices; A second unit is configured to perform feature extraction and space-time labeling on the original sensing data to obtain structured sensing features containing space-time dimension information and sensing content; A third unit is configured to analyze the sensing visual field overlap area and sensing blind area between the plurality of intelligent network connection devices based on the structured sensing features, and construct a collaborative sensing dependency graph between the devices; A fourth unit is configured to perform consistency checking and redundancy suppression on the multi-device sensing data in the sensing visual field overlap area and infer and complete the missing information in the sensing blind area according to the collaborative sensing dependency graph on the edge computing node, and generate a global collaborative sensing result; A fifth unit is configured to construct a distributed collaborative optimization model and divide a plurality of sub-optimization problems based on the global collaborative sensing result, the topology structure between the intelligent network connection devices and the state constraints, and adopt a distributed iterative solving strategy to solve the problems in parallel to obtain distributed control instructions of the intelligent network connection devices; A sixth unit is configured to issue the distributed control instructions to the corresponding intelligent network connection devices to drive the intelligent network connection devices to perform control actions.

[0078] In a third aspect, the embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0079] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon computer program instructions, which when executed by a processor, implement the method described above.

[0080] The present application can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for performing various aspects of the present application.

[0081] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still modify the technical solutions recorded in the above-mentioned embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A collaborative sensing and control method for intelligent connected devices based on edge computing, characterized in that, include: Acquire raw sensing data collected by multiple intelligent connected devices deployed in a distributed manner; Feature extraction and spatiotemporal annotation are performed on the raw sensory data to obtain structured sensory features containing spatiotemporal dimension information and sensory content; Based on the structured perception features, the overlapping areas and blind spots of perception fields among the multiple intelligent connected devices are analyzed, and a collaborative perception dependency graph among the devices is constructed. On the edge computing node, based on the collaborative perception dependency graph, consistency verification and redundancy suppression are performed on the multi-device perception data in the overlapping area of ​​the perception field, and the missing information in the perception blind zone is inferred and completed to generate global collaborative perception results. Based on the global collaborative perception results, combined with the topology and state constraints among the intelligent connected devices, a distributed collaborative optimization model is constructed and divided into multiple sub-optimization problems. A distributed iterative solution strategy is used to solve these problems in parallel to obtain the distributed control commands for the intelligent connected devices. The distributed control commands are sent to the corresponding intelligent connected devices to drive the intelligent connected devices to perform control actions.

2. The method according to claim 1, characterized in that, Feature extraction and spatiotemporal annotation are performed on the raw sensory data to obtain structured sensory features containing spatiotemporal dimension information and sensory content, including: A corresponding timestamp is generated based on the acquisition time of the original sensing data. The sensing field coverage of the source intelligent connected device is calculated based on the spatial deployment location of the source intelligent connected device and the sensing direction parameters. The spatial coordinates within the sensing field coverage are used as the spatial location marker of the original sensing data. The raw sensing data is subjected to multi-scale feature decomposition to extract physical attribute features and state attribute features reflecting the sensing target. Based on the semantic association between the physical attribute features and the state attribute features, a sensing content representation describing the characteristics of the sensing target is constructed. The timestamp annotation, the spatial location annotation, and the perceived content representation are encapsulated in a structured manner to obtain the structured perception features of each intelligent connected device.

3. The method according to claim 1, characterized in that, Based on the structured perception features, the overlapping areas and blind spots of perception fields among the multiple intelligent connected devices are analyzed, and a collaborative perception dependency graph among the devices is constructed, including: Based on the structured perception features, the perception range boundaries corresponding to the multiple intelligent connected devices are extracted, and the perception range boundaries are spatially projected in the global coordinate system to identify the overlapping areas of perception fields between devices, and the uncovered areas in the preset monitoring area are taken as perception blind spots. For the overlapping area of ​​the sensing field of view, analyze the overlapping devices with sensing coverage in the area, and calculate the observation angle and observation distance of each overlapping device in the overlapping area of ​​the sensing field of view based on the spatial geometric relationship of the area, and establish a multi-source sensing association between the overlapping area of ​​the sensing field of view and the overlapping devices. For the aforementioned perception blind zone, the adjacent devices spatially adjacent to the blind zone are analyzed. Based on the spatial topology of the blind zone, the spatial range that each adjacent device can cover in the vicinity of the boundary of the blind zone is identified, and the boundary association between the blind zone and the adjacent devices is established. Using the multiple intelligent connected devices as observation nodes, based on the multi-source perception association and the boundary association, the association edges between the observation nodes are calculated, the overlapping devices are marked as overlapping observation nodes, and the adjacent devices are marked as blind zone adjacent observation nodes, thus obtaining a cooperative perception dependency graph.

4. The method according to claim 1, characterized in that, On edge computing nodes, based on the collaborative sensing dependency graph, consistency verification and redundancy suppression are performed on multi-device sensing data in overlapping sensing fields, and missing information in sensing blind spots is inferred and completed to generate global collaborative sensing results, including: On the edge computing node, the sensing data of the overlapping observation nodes in the same sensing field of view are projected. Based on the correspondence between the spatial location and the timestamp of the projected sensing data, consistency verification is performed, and redundant sensing data with spatial location deviations exceeding the consistency threshold are eliminated. Obtain each overlapping observation node, calculate the spatial resolution and angular coverage of each overlapping observation node for the observation angle and observation distance of the overlapping area of ​​the sensing field, and perform weighted fusion on the sensing data that has passed the consistency check to generate the fused sensing result of the overlapping area. The blind zone adjacent observation nodes are extracted from the collaborative perception dependency graph. Based on their structured perception features in the area near the boundary of the blind zone, the spatial position and motion state of the perceived target are calculated. A trajectory prediction model for the perceived target moving from the area near the boundary to the inside of the blind zone is established. The spatial position and state changes of the perceived target after entering the blind zone are calculated and used as the reasoning completion information for the blind zone. Based on the structured perception characteristics of each intelligent connected device, the fused perception results and the inference completion information are integrated to generate a global collaborative perception result.

5. The method according to claim 4, characterized in that, Based on the structured perception characteristics of the target within the region adjacent to the blind zone boundary, the spatial position and motion state of the target are calculated. A trajectory prediction model for the target's movement from the region adjacent to the boundary into the blind zone is established. The changes in the target's spatial position and state after entering the blind zone are calculated and used as inference completion information for that blind zone, including: From the structured perception features of the observation nodes adjacent to the blind zone, the physical attribute features and state attribute features of the perceived target located in the vicinity of the blind zone boundary are extracted. The physical attribute features are spatially located to obtain the absolute spatial coordinate sequence of the perceived target. The state attribute features are analyzed for motion state to calculate the velocity vector and acceleration vector of the perceived target, and the motion trajectory and direction of the perceived target are calculated. If the direction of movement points to the perception blind zone, a trajectory prediction model for the perceived target is established based on the movement trajectory and the spatial boundary geometry and spatial range of the perception blind zone. A trajectory prediction sequence within the blind zone, including the predicted spatial position and predicted movement state, is generated. Based on the movement trajectory of the perceived target on the other side of the blind zone, the trajectory prediction sequence within the blind zone is corrected to obtain a corrected trajectory prediction sequence within the blind zone. This corrected sequence is then associated with the spatial position of the perception blind zone to obtain the reasoning completion information for the perception blind zone.

6. The method according to claim 1, characterized in that, Based on the global collaborative perception results, and combined with the topology and state constraints among the intelligent connected devices, a distributed collaborative optimization model is constructed and divided into multiple sub-optimization problems. A distributed iterative solution strategy is used to solve these problems in parallel, resulting in distributed control commands for the intelligent connected devices, including: The global collaborative perception results are mapped to the local perception state vectors of each intelligent connected device, and an adjacency relationship matrix between the devices is constructed based on the topology between the intelligent connected devices. Based on the local perception state vector and the adjacency matrix, the environmental information and device state information contained in the global collaborative perception result are transformed into the objective function input parameters of the distributed collaborative optimization model. Combined with state constraints, the feasible decision space of each intelligent connected device is defined, and a distributed collaborative optimization model containing local objective functions and global consistency constraints is constructed. The distributed collaborative optimization model is decomposed into multiple sub-optimization problems, each corresponding to an intelligent connected device. A dual information interaction mechanism is established between the sub-optimization problems, and a distributed iterative solution strategy is adopted to solve each sub-optimization problem in parallel. When the local decision variables of each intelligent connected device meet the convergence condition, they are used as the distributed control commands of the corresponding intelligent connected device.

7. The method according to claim 6, characterized in that, A dual information interaction mechanism is established among the sub-optimization problems. A distributed iterative solution strategy is adopted to solve each sub-optimization problem in parallel. When the local decision variables of each intelligent connected device satisfy the convergence condition, they are used as distributed control commands for the corresponding intelligent connected device, including: A dual information interaction mechanism is established between sub-optimization problems. The dual information interaction mechanism is that each intelligent connected device sends the transmission rules of local decision variables and dual variables to the intelligent connected devices in its neighboring device set, and receives the update rules of decision variables and dual variables from the intelligent connected devices in its neighboring device set. The dual variables represent the cooperative constraint strength between adjacent intelligent connected devices. The local decision variables and dual variables of each intelligent connected device are initialized, and the distributed iterative solution strategy is used to solve each sub-optimization problem in parallel. In each iteration, each intelligent connected device determines the local optimization direction based on the local perception state vector, and updates the solution parameters of the local sub-optimization problem by combining the decision variables and dual variables received from neighboring devices to obtain the updated local decision variables. The updated local decision variables and the corresponding dual variables are sent to the neighboring devices through the dual information interaction mechanism. After each iteration, the change magnitude of the local decision variables of each intelligent connected device is calculated. When the change magnitude is lower than the convergence threshold, it is determined that the local decision variables of each intelligent connected device meet the convergence condition. The local decision variables of each intelligent connected device that meet the convergence condition are used as the distributed control commands of the corresponding intelligent connected device.

8. A collaborative sensing and control system for intelligent connected devices based on edge computing, used to implement the method as described in any one of claims 1-7, characterized in that, include: The first unit is used to acquire raw sensing data collected by multiple intelligent connected devices deployed in a distributed manner; The second unit is used to extract features and perform spatiotemporal annotation on the original sensing data to obtain structured sensing features containing spatiotemporal dimension information and sensing content. The third unit is used to analyze the overlapping areas and blind spots of the perception fields among the multiple intelligent connected devices based on the structured perception features, and to construct a collaborative perception dependency graph among the devices. The fourth unit is used to perform consistency verification and redundancy suppression on the multi-device perception data in the overlapping area of ​​the perception field of view on the edge computing node, based on the collaborative perception dependency graph, and to perform reasoning to complete the missing information in the perception blind zone, so as to generate a global collaborative perception result. The fifth unit is used to construct a distributed collaborative optimization model based on the global collaborative perception results, combined with the topology and state constraints between the intelligent connected devices, and divide it into multiple sub-optimization problems. A distributed iterative solution strategy is used to solve the sub-optimization problems in parallel to obtain the distributed control commands of the intelligent connected devices. The sixth unit is used to send the distributed control commands to the corresponding intelligent connected devices to drive the intelligent connected devices to perform control actions.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images