Sensing fusion method, communication device, and storage medium

Abstract

Provided are a sensing fusion method, a communication device, and a storage medium. The sensing fusion method is applied to a first sensing node, and comprises: acquiring first sensing information and second sensing information, wherein the first sensing information is sensing information obtained by sensing a sensing target by the first sensing node, and the second sensing information is sensing information obtained by sensing the sensing target by a second sensing node; fusing the first sensing information and the second sensing information to obtain fused sensing information of the sensing target; and sending the fused sensing information and global identity information of the sensing target to the second sensing node, wherein the global identity information is used for identifying the sensing target in a sensing network.

Description

Perception fusion method, communication device and storage medium

[0001] This disclosure claims priority to Chinese patent application No. 202410210750.5, filed on February 26, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of communication technology, and in particular to a perception fusion method, a communication device, and a storage medium. Background Art

[0003] Synaesthesia integration technology is gaining increasing attention from academia, industry, and standardization organizations. This technology will enable the integration of mobile communication networks, perception networks, and computing networks. Summary of the Invention

[0004] In a first aspect, an embodiment of the present disclosure provides a perception fusion method, applied to a first perception node. The perception fusion method includes:

[0005] Acquire first perception information and second perception information, where the first perception information is perception information obtained by the first perception node perceiving the perception target, and the second perception information is perception information obtained by the second perception node perceiving the perception target;

[0006] Fusing the first perception information and the second perception information to obtain fused perception information of the perception target;

[0007] The fused perception information and the global identity information of the perception target are sent to the second perception node, where the global identity information is used to identify the perception target in the perception network.

[0008] In a second aspect, an embodiment of the present disclosure provides a perception fusion method, applied to a second perception node. The perception fusion method includes:

[0009] Sending second perception information to the first perception node, where the second perception information is perception information obtained by the second perception node perceiving the perception target;

[0010] Receive the fused perception information sent by the first perception node and the global identity information of the perception target; the fused perception information is obtained by fusing the first perception information and the second perception information; the first perception information is the perception information obtained by the first perception node perceiving the perception target; the global identity information is used to identify the perception target in the perception network.

[0011] In a third aspect, embodiments of the present disclosure provide a communication device comprising: a memory and a processor; the memory and the processor are coupled; the memory is configured to store a computer program; and the processor implements any of the aforementioned perception fusion methods when executing the computer program.

[0012] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the perception fusion method of any of the above aspects.

[0013] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the perception fusion method of any of the above aspects is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present disclosure, the following briefly introduces the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] FIG1 is a schematic diagram of the architecture of a perception fusion system according to some embodiments.

[0016] FIG2 is a flowchart of a perception fusion method according to some embodiments.

[0017] FIG3 is a flowchart of another perception fusion method according to some embodiments.

[0018] FIG4 is a flowchart of another perception fusion method according to some embodiments.

[0019] FIG5 is a flowchart of another perception fusion method according to some embodiments.

[0020] FIG6 is a schematic diagram of the distribution of sensing nodes according to some embodiments.

[0021] FIG7 is a schematic diagram of another distribution of sensing nodes according to some embodiments.

[0022] FIG8 is a schematic diagram of another distribution of sensing nodes according to some embodiments.

[0023] FIG9 is a flowchart of yet another perception fusion method according to some embodiments.

[0024] FIG10 is a flowchart of another perception fusion method according to some embodiments.

[0025] FIG11 is a schematic structural diagram of a perception fusion device according to some embodiments.

[0026] FIG12 is a schematic structural diagram of another perception fusion device according to some embodiments.

[0027] FIG13 is a schematic structural diagram of a communication device according to some embodiments. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions of this disclosure in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this disclosure, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0029] It should be noted that in this disclosure, expressions such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this disclosure as "exemplarily" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of expressions such as "exemplarily" or "for example" is intended to present the relevant concepts in a detailed manner.

[0030] In the following, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0031] In the description of this disclosure, unless otherwise specified, " / " means "or." For example, A / B can mean A or B. "And / or" herein is simply a way to describe an association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: only A, A and B, and only B. Furthermore, "at least one" means one or more, and "a plurality" means two or more.

[0032] Synaesthesia integration is based on the existing perception network architecture and communication waveform, and uses existing communication equipment to realize perception applications, promoting the development of innovative application services.

[0033] In the currently used perception functions, due to the increase in the scale of the perception network and the number of perception targets, the corresponding computing power and data interaction volume of the perception nodes and perception centers need to be increased accordingly. How to reduce the amount of information interaction between perception nodes and realize the fusion of perception information between each perception node is still less studied in the industry.

[0034] In response to the above technical problems, an embodiment of the present disclosure provides a perception fusion method, which fuses the first perception information and the second perception information through a first perception node to obtain fused perception information, and assigns global identity information to the perception target through the first perception node, and sends the fused perception information and the global identity information to the second perception node, thereby realizing information interaction between each perception node and reducing the information processing cardinality between each perception node, and also making the identity of the perception target in the perception fusion system easier to identify.

[0035] Figure 1 is a schematic diagram of the architecture of a perception fusion system according to some embodiments. As shown in Figure 1 , the system includes: perception nodes and a perception center.

[0036] In some embodiments, a user can perform a sensing demand operation on the sensing center (for example, the sensing demand operation is used to sense the speed and trajectory of a drone (a sensing target) within a preset range of a sensing node). Accordingly, the sensing center can respond to the sensing demand operation by sending a sensing signal to each sensing node, which then processes the sensing signal to sense the sensing target and obtain sensing information.

[0037] In some embodiments, the sensing node is also used to transmit sensing information to a sensing center or other sensing nodes.

[0038] It should be noted that the number of sensing nodes in the above-mentioned sensing fusion system can be one or more. The sensing node can be a generation node B (gNB) base station or a user equipment (UE), and the sensing target can be a drone, a vehicle, or a pedestrian, etc., which moves in a sensing network composed of distributed sensing nodes. This disclosure does not limit the number of sensing nodes and the type of sensing targets.

[0039] For example, a sensing node comprising a first sensing node and a second sensing node may be located within a wireless access network. Each sensing node forms a distributed sensing network. The sensing node senses the target and processes the sensing signal. It also has information connections with other adjacent sensing nodes, enabling inter-node information exchange based on the Sn interface. The sensing information is then transmitted to the sensing center based on the S1 interface.

[0040] In some embodiments, the number of the second perception nodes may be one or more, and the present disclosure does not limit the number of the second perception nodes.

[0041] It should be noted that the Sn interface is a communication interface for transmitting information between a sensing node and its adjacent sensing nodes, and the S1 interface is a communication interface for transmitting information between a sensing node and a sensing center.

[0042] In some embodiments, the sensing nodes may also be connected indirectly via a wired optical fiber connection, a wireless link connection, or a sensing center. This disclosure does not limit the connection method between the sensing nodes.

[0043] In some embodiments, the perception center performs perception process control, perception resource configuration, and perception information processing on each perception node based on the S1 interface according to detailed perception requirements. The perception center can be set up as a perception network element in the core network, or it can be located in the central computer room of a certain area. The perception center is the perception control center and computing power server center of all perception nodes in this area.

[0044] It should be noted that the system architecture and application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0045] For example, the above-mentioned perception fusion system can also be a device-to-device (D2D) communication system, a vehicle-to-everything (V2X) communication system, or a system integrating multiple systems, etc., and the embodiments of the present disclosure are not limited to this. The V2X communication system includes sub-fields such as vehicle-to-vehicle (V2V), vehicle-to-pedestrian (V2P), and vehicle-to-infrastructure (V2I).

[0046] In some embodiments, each device in the above-mentioned communication system may be a UE (such as a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook computer, and cellular phone), augmented reality (AR) or virtual reality (VR) device, etc., with SL communication capabilities. The embodiments of the present disclosure do not impose any special restrictions on the form of each device in the above-mentioned communication system.

[0047] Each device in the above-mentioned systems is a sensing node in the embodiments of the present disclosure.

[0048] The following describes a perception fusion method provided by an embodiment of the present disclosure in conjunction with the accompanying drawings.

[0049] Referring to Figure 2 , which is a flow chart of a perception fusion method according to some embodiments, the method is applied to a first perception node. As shown in Figure 2 , the method includes: S101 to S103 .

[0050] S101. The first perception node obtains first perception information and second perception information.

[0051] The first perception information is the perception information obtained by the first perception node when perceiving the perception target, and the second perception information is the perception information obtained by the second perception node when perceiving the perception target.

[0052] In some embodiments, the sensing target may be a drone, a vehicle, or a pedestrian. The sensing target may move in a sensing network composed of distributed sensing nodes. Depending on the sensing target, the sensing information perceived by the sensing node may also be different.

[0053] In some embodiments, the first sensing node performs a sensing task in response to the sensing signal, senses the sensing target, and obtains first sensing information. The second sensing node performs a sensing task in response to the sensing information, senses the sensing target, and obtains second sensing information.

[0054] In some embodiments, a user can perform a sensing request operation on the sensing center (for example, the sensing request operation is used to sense the speed and trajectory of a drone (a sensing target) within a preset range of a sensing node). Accordingly, the sensing center can respond to the sensing request operation by sending a sensing signal to each sensing node, which then processes the sensing signal to sense the sensing target and obtain sensing information.

[0055] Furthermore, after the second perception node obtains the second perception information, the second perception node can actively send the second perception information to the first perception node, or it can send the second perception information to the first perception node after receiving the reporting instruction sent by the first perception node. The present disclosure does not limit the reporting method of the second perception information.

[0056] As an example, the first perception information can be used to characterize the running trajectory of the perception target within the perception range of the first perception node, and the second perception information can be used to characterize the running trajectory of the perception target within the perception range of the second perception node.

[0057] The running trajectory of the sensing target may include at least one position and speed of the sensing target, and the at least one position and speed of the sensing target may be represented by a trajectory point sequence, such as [(Lo1, La1, H1, V1), (Lo2, La2, H2, V2), ..., (Lo t , La t , H t , V t)..., (Lo n , La n , H n , V n )].

[0058] Lo t La represents the longitude of the target trajectory point at time t, t represents the latitude of the target trajectory point perceived at time t, H t represents the height of the target trajectory point perceived at time t, V t Indicates that the target is perceived at the trajectory point (Lo t , La t , H t ), t = 1, 2, ..., n.

[0059] It should be noted that the above-mentioned contents of the first perception information and the second perception information are merely exemplary. In a detailed implementation, the first perception information and the second perception information may also contain more or less contents, and this disclosure does not limit this.

[0060] In some embodiments, the sensing area of ​​the second sensing node overlaps with the sensing area of ​​the first sensing node, or the distance between the second sensing node and the first sensing node is less than a preset distance threshold. Therefore, the second sensing node is also called an adjoint-sensing node of the first sensing node. The number of second sensing nodes can be one or more.

[0061] For example, taking the preset distance threshold of 200 meters as an example, if the distance between any perception node and the first perception node is less than 200 meters, the perception node is determined as the second perception node, that is, the perception node is the companion perception node of the first perception node.

[0062] It should be noted that the perception area of ​​the perception node can be a preset perception area, or a circular area within a preset range centered on the current perception node. The perception area of ​​the perception node can also be an irregular area. This disclosure does not limit the range and type of the perception area.

[0063] In some embodiments, since the number of second sensing nodes can be multiple, the second sensing information can be multiple sensing information perceived by multiple second sensing nodes.

[0064] Exemplarily, if the second sensing nodes include sensing node A and sensing node B, the second sensing information includes sensing information sensed by sensing node A and sensing information sensed by sensing node B. If the second sensing nodes include sensing node A, sensing node B, and sensing node C, the second sensing information includes sensing information sensed by sensing node A, sensing information sensed by sensing node B, and sensing information sensed by sensing node C.

[0065] S102: The first perception node fuses the first perception information and the second perception information to obtain fused perception information of the perception target.

[0066] In some embodiments, after obtaining the first perception information and the second perception information, in order to reduce the repeated perception information in the first perception information and the second perception information, thereby reducing the calculation cardinality of the perception center, the first perception information and the second perception information can be fused through the first perception node to obtain fused perception information of the perception target.

[0067] The first perception node may fuse the first perception information and the second perception information by deduplicating and merging the first perception information and the second perception information.

[0068] For example, taking the example that both the first perception information and the second perception information contain the trajectory points (Lo1, La1, H1, V1) of the perception target, the first perception node can deduplicate the trajectory points and only retain one of the trajectory points.

[0069] Alternatively, the first sensing node may deduplicate the sensing information with duplicate sensing areas in the first sensing information and the second sensing information, and retain only one of the two identical sensing areas.

[0070] For example, if the first perception information and the second perception information both include perception information in perception area 1, the first perception node only retains the perception information of one perception area 1 and removes the perception information of the other perception area 1.

[0071] Alternatively, the first perception node retains the perception information with the highest perception confidence among all the perception information, and removes other perception information.

[0072] For example, taking the first perception confidence of 0.7 and the second perception confidence including 0.5, 0.3 and 0.6 as an example, the first perception node only retains the first perception information with a perception confidence of 0.7, and removes the perception information with perception confidences of 0.5, 0.3 and 0.6.

[0073] Alternatively, the first sensing node adds the three-dimensional coordinates of the trajectory points of the sensed target and calculates the average value of each dimension, retains the average trajectory point, and removes the original trajectory point.

[0074] For example, if the first perception information contains the trajectory points (Lo11, La11, H11, V11) of the perception target, and the second perception information contains the trajectory points (Lo21, La21, H21, V21), (Lo31, La31, H31, V31) and (Lo41, La41, H41, V41) of the perception target, the first perception node can merge the above trajectory points and retain the trajectory points of the perception target. And remove the original trajectory points (Lo21, La21, H21, V21), (Lo31, La31, H31, V31) and (Lo41, La41, H41, V41).

[0075] It should be noted that the above are only exemplary ways of fusing perception information. In a detailed implementation, the way of fusing perception information may also be other ways. This disclosure does not limit the way of fusing perception information.

[0076] S103. The first sensing node sends the fused sensing information and the global identity information of the sensing target to the second sensing node.

[0077] Global identity information is used to identify sensing targets in the sensing network.

[0078] In some embodiments, after the first perception node fuses the first perception information and the second perception information to obtain the fused perception information of the perception target, the first perception node assigns a global identity (global-identity document, G-ID) to the perception target, and sends the fused perception information and the global identity information of the perception target to the second perception node.

[0079] Exemplarily, the first sensing node may record a unique identifier (such as a graphic, a label, text, etc.) for the sensing target and use the unique identifier as the global identity information of the sensing target. The global identity information enables the sensing target to be recognized by all sensing nodes in the sensing network.

[0080] Based on the perception fusion method provided by the embodiments of the present disclosure, the first perception node fuses the first perception information with the second perception information, effectively deduplicating the perception information. This not only reduces the computational base of the perception network but also improves the fusion accuracy of the perception information. Simultaneously, the first perception node transmits the fused perception information to the second perception node, enabling intercommunication of perception information between the nodes. The global identity information of the perception target is transmitted to the second perception node, making the identity of the perception target easier to identify within the perception fusion system. This avoids repeated processing of the same perception information by the first perception node, reduces the perception workload, and enables convenient interaction and efficient transmission of information between the perception nodes.

[0081] Referring to Figure 3 , which is a flow chart illustrating another perception fusion method according to some embodiments, the perception fusion method is applied to a first perception node. As shown in Figure 3 , before S101 above, the method includes: S201 to S203 .

[0082] S201. The first perception node sends a first message to the second perception node.

[0083] The first message is used to request the first perception node to be the fusion perception node corresponding to the perception target (request to combine, RTC). The first message includes the first perception information, the first perception confidence and the first local identity information of the perception target. The fusion perception node is the node responsible for fusing the perception information of the perception target.

[0084] In some embodiments, before the first perception node obtains the first perception information and the second perception information, the first perception node needs to send a first message to the second perception node requesting the first perception node to be the fused perception node corresponding to the perception target based on whether the first perception node meets the conditions of the fused perception node.

[0085] It should be noted that the establishment of the fusion perception node determines the anchor node of the perception target in the perception network, that is, there is a one-to-one correspondence between the perception target and the fusion perception node.

[0086] It should be noted that perception confidence is used to characterize the credibility of the perceptual information obtained by a perception node when processing a perception signal. It can be a real number in the range [0, 1], with 0 representing the least trustworthy and 1 representing the most trustworthy. Perception confidence is related to the signal-to-noise ratio (SNR) of the perception signal obtained by the perception node and the correlation between various perception parameters. Generally, a higher SNR indicates a greater correlation between the perception parameters, and thus a higher perception confidence.

[0087] In some embodiments, the first perception confidence is the confidence of the first perception information, and the first local identity information is used to locally identify the perception target at the first perception node.

[0088] Exemplarily, the first perception node can record a proprietary identifier (such as a graphic, a label, text, etc.) for the perception target and use the proprietary identifier as the first local identity document (L-ID) information of the perception target. The first local identity information enables the perception target to be identified by the first perception node.

[0089] It should be noted that the global identity information of the sensing target can be recognized by all sensing nodes in the sensing network, and the first local identity information of the sensing target can only be recognized by the first sensing node.

[0090] As an example, the first perception node assigns a local identity to the perceived perception target and calculates a first perception confidence of the first perception information. Then, the first perception node sends the local identity information and the first perception confidence to the second perception node.

[0091] In some embodiments, since the perception information may be affected by the signal-to-noise ratio of the perception signal, spatial interference signals, device insertion loss, etc., there may be a gap between the perception information perceived by the perception node and the actual information of the perception target.

[0092] In some embodiments, the signal-to-noise ratio (SNR) of a perception signal can, to a certain extent, represent perception confidence. Specifically, a higher SNR indicates less noise interference. For actual perception targets, the distance and speed of the perceived target are physically related. Therefore, signal processing can be performed by calculating the square of the difference between the cumulative distance and speed of the perceived target over time. A smaller value indicates more accurate perception information, and therefore, a higher perception confidence.

[0093] In some embodiments, the perception confidence is calculated as shown in formula (1):

[0094] N is the number of perception coordinate points contained in a certain target trajectory of the perception node, γ n is the signal-to-noise ratio of the perception signal at the nth perception coordinate point, ΔS n is the spatial flight distance between the perceived target from the nth perceived coordinate point to the previous perceived coordinate point, v n is the flight speed of the perceived target at the nth perception coordinate point, and Δt is the time between two adjacent perception coordinate points.

[0095] As shown in formula (1), the higher the signal-to-noise ratio of the perception signal, the more correlated the perception parameters are, and the higher the perception confidence.

[0096] Furthermore, the first perception node obtains the first perception information perceived by the first perception node and the second perception information perceived by the second perception node, and matches and associates the first perception information with the second perception information. After the first perception node calculates the difference between the first perception confidence and the second perception confidence of the second perception information, it sends a first message to the second perception node whose difference is less than a preset confidence threshold to request the first perception node to be the fusion perception node corresponding to the perception target.

[0097] It should be noted that the preset confidence threshold is set by the perception fusion system manufacturer and stored in the memory. The preset confidence threshold set by different perception fusion system manufacturers may be different, and this disclosure does not limit this.

[0098] As an example, the preset confidence threshold may be -0.2.

[0099] As an example, the first perception node sends a first message to the second perception node, whose difference between the first perception confidence and the second perception confidence of the second perception information is less than -0.2, requesting the first perception node to be used as the fusion perception node corresponding to the perception target.

[0100] In some embodiments, the first sensing node sends a message (reject, REJ) for rejecting the first sensing node as the fusion sensing node corresponding to the sensing target to the sensing node whose difference between the first sensing confidence and the second sensing confidence is greater than a preset confidence threshold.

[0101] As an example, the first sensing node sends a message for rejecting the fusion sensing node corresponding to the first sensing node as a sensing target to the sensing node whose difference between the first sensing confidence and the second sensing confidence is greater than -0.2.

[0102] S202. The first perception node receives a second message sent by the second perception node.

[0103] The second message is a response message to the first message. The response message may be an indication of agreeing to use the first sensing node as the fused sensing node corresponding to the sensing target, or an indication of rejecting to use the first sensing node as the fused sensing node corresponding to the sensing target.

[0104] In some embodiments, after a first sensing node sends a first message to a second sensing node, the second sensing node may determine, based on the first message, whether the first sensing node meets the requirements for being a fused sensing node. After determining that the first sensing node meets the requirements for being a fused sensing node, the second sensing node may send a second message to the first sensing node. Accordingly, the first sensing node receives the second message sent by the second sensing node.

[0105] S203. The first perception node sends a third message to the second perception node.

[0106] In some embodiments, after receiving the second message sent by the second sensory node, the first sensory node sends a third message to the second sensory node based on the fact that all the second messages include an agreement indication.

[0107] The third message is used to indicate that the first sensing node is a fusion sensing node.

[0108] In some embodiments, the third message includes global identity information of the sensing target and second local identity information of the sensing target, where the second local identity information is used to locally identify the sensing target at the second sensing node.

[0109] Exemplarily, the second perception node can record a proprietary identifier (such as a graphic, a label, text, etc.) for the perception target and use the proprietary identifier as the second local identity document (L-ID) information of the perception target. The second local identity information enables the perception target to be recognized by the second perception node.

[0110] It should be noted that the global identity information of the perception target can be recognized by all perception nodes in the perception network (i.e., the first perception node and the second perception node), the first local identity information of the perception target can only be recognized by the first perception node, and the second local identity information of the perception target can only be recognized by the second perception node.

[0111] As an example, when the first sensing node receives the second message, if all messages in the second message include an agreement indication (ACP), the first sensing node is used as a combine-sensing node (C-SN).

[0112] For example, taking the second perception node including perception node A, perception node B and perception node C as an example, the second message includes a message sent by perception node A, a message sent by perception node B and a message sent by perception node C. If the second message includes the message sent by perception node A, the message sent by perception node B and the message sent by perception node C, all of which include an agreement indication (ACP), indicating that the first perception node meets the conditions of being a fused perception node, then the first perception node will be used as a fused perception node.

[0113] It should be noted that the fused perception node sends the fused perception information and the global identity information of the perception target to the perception center, so that the perception center can control the perception process of each perception node by processing the fused perception information and the user's perception task requirements, and reconfigure the perception resources of each perception node to complete the perception task.

[0114] Furthermore, the first perception node fuses the perception information perceived by all the perception nodes to obtain fused perception information, and assigns the perception target a global identity. The first perception node then sends the fused perception information and the global identity information to the perception center.

[0115] In some embodiments, the first sensing node further sends the fused sensing information, the global identity information, and the local identity information to the second sensing node to enable information interaction between the sensing nodes in the sensing network.

[0116] In some embodiments, when the second perception node receives the fused perception information, global identity information, and local identity information, it sends an acknowledgement instruction (ACK) to the first perception node. All perception nodes associate the global identity information and local identity information of the perception target, and continue to perceive the perception target and record the perception information of the perception target.

[0117] In some embodiments, due to the overlap of the perception areas of the various perception nodes, there may be situations where other non-second perception nodes perceive the target. When the difference between the first perception confidence of the first perception node and the second perception confidence of the second perception node is greater than a preset confidence threshold, the fusion perception node needs to be switched. At this time, the second perception node performs corresponding processing to ensure the uniqueness of the fusion perception node during mobility management. If the fusion perception node switches, the switching decision and instruction must be made by the current fusion perception node (that is, the first perception node), and the new fusion perception node must also be selected from the current second perception node.

[0118] Referring to Figure 4 , which is a flowchart illustrating another perception fusion method according to some embodiments, the perception fusion method is applied to a first perception node and is used to switch a fused perception node. As shown in Figure 4 , the method includes steps S301 to S303 .

[0119] S301. The first perception node receives a fourth message sent by the second perception node.

[0120] In some embodiments, when the first sensing node is a fusion sensing node and the difference between the first sensing confidence of the first sensing node and the second sensing confidence of the second sensing node is greater than a preset confidence threshold, the second sensing node sends a fourth message to the first sensing node.

[0121] The fourth message is used to request the second perception node to serve as the fusion perception node of the perception target. The fourth message includes the second perception information, the second perception confidence and the second local identity information of the perception target. The fusion perception node is the node responsible for fusing the perception information of the perception target.

[0122] In some embodiments, the second perception confidence is the confidence of the second perception information, and the second local identity information is used to locally identify the perception target at the second perception node.

[0123] Furthermore, the first perception node determines whether the second perception node has perceived the perception target based on the global identity information of the perception target. For any second perception node, if the second perception node has not perceived the perception target, the second perception confidence of the second perception node is 0.

[0124] Alternatively, for any second perception node, if the perception target perceived by the second perception node is matched and associated with the perception target perceived by the second perception node adjacent to it, and it is found that a fused perception node already exists, the second perception confidence of the second perception node adjacent to it is determined to be 0.

[0125] In some embodiments, the first sensing node receives a fourth message sent by a sensing node whose difference between the second sensing confidence and the first sensing confidence is less than a preset confidence threshold.

[0126] In some embodiments, the preset confidence threshold may be -0.2.

[0127] As an example, the first perception node sends a fourth message including second perception information, second perception confidence, and second local identity information of the perception target to a perception node whose difference between the second perception confidence and the first perception confidence is less than 0.2, requesting the second perception node to serve as the fusion perception node of the perception target.

[0128] S302. The first perception node sends a fifth message to the second perception node.

[0129] The fifth message is a response message to the fourth message. The response message may indicate that the second sensing node is accepted as the new fused sensing node corresponding to the sensing target, or may indicate that the second sensing node is rejected as the new fused sensing node corresponding to the sensing target.

[0130] In some embodiments, after the first sensing node receives the fourth message sent by the second sensing node, the first sensing node may determine, based on the fourth message, whether the second sensing node meets the conditions for becoming a new fused sensing node. After determining that the second sensing node meets the conditions for becoming a fused sensing node, the first sensing node may send a fifth message to the second sensing node. In response, the second sensing node receives the fifth message sent by the first sensing node.

[0131] In some embodiments, the first sensing node sends a fifth message to the second sensing node based on the first sensing confidence and the second sensing confidence.

[0132] As an example, the first perception node can first determine whether the second perception node meets the conditions of being a fused perception node based on the first perception confidence and the second perception confidence, and send the fifth message to the second perception node based on whether the second perception node meets the conditions of being a fused perception node.

[0133] The condition for the second perception node to serve as a fusion perception node includes: a difference between the first perception confidence and the second perception confidence is less than a preset confidence threshold.

[0134] As an example, when the difference between the first perception confidence and the second perception confidence is less than -0.2, it is determined that the second perception node meets the condition of being a fusion perception node.

[0135] In some embodiments, if the second sensing node meets the conditions for being a fused sensing node, the first sensing node sends a fifth message including a consent indication to the second sensing node.

[0136] In some embodiments, if the second sensing node does not meet the conditions for being a fused sensing node, the first sensing node sends a fifth message including a rejection indication to the second sensing node.

[0137] In some embodiments, when the first perception node is already a fused perception node, the condition for the second perception node to be a fused perception node also includes: the second perception confidence of the second perception node is the maximum value among the second perception confidences of all second perception nodes.

[0138] In some embodiments, the first sensing node may determine from the second sensing nodes that a target second sensing node exists.

[0139] The target second perception node is the perception node with the highest perception confidence of the perception target among all the second perception nodes, and the difference between the perception confidence of the first perception node to the perception target and the perception confidence of the target second perception node to the perception target is less than a preset confidence threshold.

[0140] As an example, the perception confidence of the target second perception node is the maximum value among the second perception confidences, and the difference between the perception confidence of the first perception node to the perception target and the perception confidence of the target second perception node to the perception target is less than -0.2.

[0141] For example, taking the second perception node including perception node A, perception node B and perception node C as an example, assuming that the perception confidence of the perception information of perception node A is 0.5, the perception confidence of the perception information of perception node B is 0.6 and the perception confidence of the perception information of perception node C is 0.7, if the first perception confidence is 0.8, the perception node C in the second perception node is determined as the target second perception node, that is, the perception node C is determined as the new fusion perception node. In this way, the frequent switching of the fusion perception node of the perception target is avoided, and the uniqueness of the fusion perception node is guaranteed.

[0142] S303. The first perception node sends a sixth message to the second perception node.

[0143] In some embodiments, after the target sensing node is determined to be the new fused sensing node, the first sensing node sends a sixth message to all second sensing nodes. The sixth message is used to instruct the fused sensing node to switch from the first sensing node to the target second sensing node. The sixth message includes the node identifier of the target second sensing node and the global identity information of the sensing target.

[0144] Furthermore, the new fusion perception node (ie, the target perception node) fuses the perception information of all perception nodes to obtain fused perception information, and sends the fused perception information and global identity information to the perception center.

[0145] In some embodiments, the new fused perception node (i.e., the target perception node) sends the fused perception information, global identity information of the original fused perception node (i.e., the first perception node) and the fused perception information and global identity information of the new fused perception node (i.e., the target perception node) to all second perception nodes, and the second perception node sends the global identity information to the perception nodes adjacent to the second perception node, so that the adjacent perception nodes can subsequently interact with each other to perceive information.

[0146] In some embodiments, when the second perception node receives the fused perception information, fused perception node switching information, global identity information and local identity information, it sends a confirmation instruction ACK to the fused perception node. All perception nodes associate the global identity information and local identity information of the perception target, and continue to perceive the perception target and record the perception information of the perception target.

[0147] It should be noted that if a fused sensing node switches, the new fused sensing node sends its identity to its neighboring sensing nodes and identifies the neighboring sensing node as the second sensing node. This avoids frequent switching of the fused sensing node perceiving the target and ensures the stability of the perception information fusion process.

[0148] 5 , which is a flow chart of another perception fusion method according to some embodiments. The perception fusion method is applied to a second perception node. As shown in FIG5 , the method includes: S401 to S402 .

[0149] S401. The second perception node sends second perception information to the first perception node.

[0150] The second perception information is the perception information obtained by the second perception node when perceiving the perception target.

[0151] In some embodiments, before the second sensing node sends the second sensing information to the first sensing node, the second sensing node receives a first message sent by the first sensing node. The first message is used to request that the first sensing node be used as a fusion sensing node corresponding to a sensing target, where the fusion sensing node is a node responsible for fusing the sensing information of the sensing target.

[0152] In some embodiments, the first message includes first perception information, first perception confidence, and first local identity information of the perception target, where the first perception confidence is the confidence of the first perception information; the first local identity information is used to locally identify the perception target at the first perception node.

[0153] Furthermore, after receiving the first message sent by the first sensing node, the second sensing node sends a second message to the first sensing node, where the second message is a response message to the first message.

[0154] As an example, the second sensing node sends the second message to the first sensing node based on the first sensing confidence and the second sensing confidence, where the second sensing confidence is the confidence of the second sensing information.

[0155] As an example, the second sensing node may first determine whether the first sensing node meets the conditions for serving as a fused sensing node based on the first sensing confidence and the second sensing confidence. The conditions for the first sensing node serving as a fused sensing node include: a difference between the first sensing confidence and the second sensing confidence is less than a preset confidence threshold.

[0156] As an example, the preset confidence threshold may be -0.2.

[0157] Exemplarily, when the difference between the first perception confidence and the second perception confidence is less than -0.2, the second perception node determines that the first perception node meets the condition of being a fusion perception node.

[0158] Further, when the first perception node meets the conditions for being a fused perception node, the second perception node sends a second message containing an approval indication to the first perception node; or, when the first perception node does not meet the conditions for being a fused perception node, the second perception node sends a second message containing a rejection indication to the first perception node.

[0159] As an example, when the first perception node satisfies that the difference between the first perception confidence and the second perception confidence is less than -0.2, the second perception node sends a second message containing an approval indication to the first perception node; or, when the first perception node does not satisfy that the difference between the first perception confidence and the second perception confidence is less than -0.2 (such as the difference between the first perception confidence and the second perception confidence is greater than -0.2), the second perception node sends a second message containing a rejection indication to the first perception node.

[0160] In some embodiments, if the first sensing node meets the conditions of the fused sensing node and the second sensing node, the second sensing node receives the third message sent by the first sensing node.

[0161] The third message is used to indicate that the first perception node is a fusion perception node. The third message includes global identity information and second local identity information of the perception target. The second local identity information is used to locally identify the perception target at the second perception node.

[0162] Furthermore, the second perception node sends the perceived second perception information to the first perception node, and the first perception node fuses the second perception information with the first perception information to obtain fused perception information.

[0163] Furthermore, the first sensing node sends the fused sensing information to the second sensing node.

[0164] S402. The second perception node receives the fused perception information and the global identity information of the perception target sent by the first perception node.

[0165] The fused perception information is obtained by fusing the first perception information and the second perception information. The first perception information is the perception information obtained by the first perception node perceiving the perception target. The global identity information is used to identify the perception target in the perception network.

[0166] In some embodiments, after the second sensing node receives the global identity information sent by the first sensing node, an association relationship is established between the global identity information and the second local identity information.

[0167] In some embodiments, if the fused sensing node needs to switch to a sensing node among the second sensing nodes, the second sensing node receives a sixth message sent by the first sensing node. The sixth message is used to instruct the fused sensing node that senses the target to switch from the first sensing node to the target second sensing node. The sixth message includes the node identifier of the target second sensing node and the global identity information of the sensing target. The fused sensing node is the node responsible for fusing the sensing information of the sensing target.

[0168] In some embodiments, after determining the first sensing node as the fusion sensing node of the sensing target, the second sensing node receives the seventh message sent by the third sensing node.

[0169] The third perception node is a perception node in the second perception node that does not meet the conditions of being a fusion perception node. The seventh message is used to request the third perception node to be used as the fusion perception node. The fusion perception node is a node responsible for the perception information of the fusion perception target.

[0170] In some embodiments, the second sensing node sends an eighth message including a rejection indication to the third sensing node. The eighth message is a response message to the seventh message.

[0171] As an example, since the third perception node is a perception node among the second perception nodes that does not meet the conditions of being a fused perception node, the second perception node needs to reject the third perception node's request to become a fused perception node, that is, the second perception node sends an eighth message containing a rejection indication to the third perception node.

[0172] Based on the perception fusion method provided by the embodiment of the present disclosure, the second perception information is sent to the first perception node through the second perception node, so that the first perception node can fuse the first perception information and the second perception information to obtain fused perception information, so that the second perception node receives the fused perception information sent by the first perception node, reducing the computing base of the perception fusion system and enhancing the information interaction between the various perception nodes. At the same time, according to the global identity information sent by the first perception node, the second perception node can be accurately matched to the perception target, thereby realizing the precise positioning of the perception target in the perception fusion system.

[0173] See Figure 6, which is a schematic diagram of the distribution of sensing nodes according to some embodiments. As shown in Figure 6, taking the case where a sensing network of a sensing fusion system has six sensing nodes deployed (sensing node 1, sensing node 2, sensing node 3, sensing node 4, sensing node 5, and sensing node 6), and the sensing target is an unmanned aerial vehicle (UAV) at position 1, the sensing nodes are connected via Sn interfaces.

[0174] In some embodiments, the various perception nodes in the perception network realize the fusion of the perception information of the drone and the construction of the global identity through distributed collaboration. In the initial perception stage of the perception network for the drone, it is necessary to establish a fusion perception node.

[0175] For example, if sensing node 1, sensing node 2, and sensing node 3 can all sense the drone, the drone is assigned a local identity L-1-1 by sensing node 1, and the perception confidence δ of the perception information sensed by sensing node 1 is calculated. 1-1 =0.6.

[0176] As an example, the sensing node 2 assigns the drone a local identity L-2-1 and obtains the perception confidence s of the perception information perceived by the sensing node 2. 2-1 =0.85, the sensing node 3 assigns the drone a local identity L-3-1, and at the same time obtains the perception confidence δ of the perception information perceived by the sensing node 3 3-1 =0.5, the sensing node 4 assigns the drone a local identity L-4-1.

[0177] Furthermore, perception node 1 sends the perception information of the drone to the adjacent perception node 2 and sends a fusion request RTC. Perception node 2 sends the perception information of the drone to the adjacent perception node 1, perception node 3 and perception node 4 and sends a fusion request RTC. Perception node 3 sends the perception information of the drone to the adjacent perception node 2 and perception node 4 and sends a fusion request RTC.

[0178] In some embodiments, the sensing node 1 matches and associates the sensing information of the drone sent by the sensing node 2 with the drone of the sensing node 1, and finds that they are the same sensing target, so the difference in the sensing confidence is calculated as Δ δ =δ 1-1 -δ 2-1 =-0.25.

[0179] Similarly, the difference between the perception confidence of perception node 2 and perception node 1 is calculated as Δ δ =0.25, the difference between the perception confidence of the sensor node 3 is calculated as Δ δ =-0.35, the difference between the perception confidence of the sensor node 4 is calculated as Δ δ =0.85 (since sensor node 4 does not perceive the perception target, the perception confidence of sensor node 4 is considered to be 0), sensor node 3 calculates the perception confidence difference between sensor node 3 and sensor node 2 as Δ δ =-0.35, the difference between the perception confidence of perception node 3 and perception node 4 is calculated as Δ δ =0.35. Perception node 4 matches and associates the perception information of the perception target sent by perception node 2 and perception node 3 with the perception target perceived by perception node 4. It is found that perception node 4 does not perceive the perception target, so the perception confidence of perception node 4 is considered to be 0. The perception confidence difference between perception node 4 and perception node 2 is calculated as Δδ =-0.85, the difference in perception confidence between sensing node 4 and sensing node 3 is Δ δ =-0.5.

[0180] Further, based on the difference between the perception confidences of the above-mentioned sensing nodes, sensing node 1 sends an acceptance merge response instruction ACP and L-1-1 of sensing node 1 and L-2-1 of sensing node 2 of the corresponding perception target to sensing node 2, sensing node 2 sends a rejection merge response instruction REJ, L-2-1 and L-1-1 to sensing node 1, sensing node 2 sends a rejection merge response instruction REJ, L-2-1 and L-3-1 to sensing node 3, sensing node 2 sends a rejection merge response instruction REJ, L-2-1 and L-4-1 to sensing node 4, sensing node 3 sends an acceptance merge response instruction ACP, L-3-1 and L-2-1 to sensing node 2, sensing node 3 sends a rejection merge response instruction REJ, L-3-1 and L-4-1 to sensing node 4, sensing node 4 sends an acceptance merge response instruction ACP, L-4-1 and L-2-1 to sensing node 2, and sensing node 4 sends an acceptance merge response instruction ACP, L-4-1 and L-3-1 to sensing node 3.

[0181] In summary, only the response instructions received by the adjacent sensing nodes from the sensing node 2 are ACP, so the sensing node 2 is used as the fusion sensing node of the sensing target.

[0182] In some embodiments, when sensing node 2 is determined to be a fused sensing node, the fused sensing node sends information that sensing node 2 has become a fused sensing node to its adjacent sensing nodes 1, 3, and 4, and sensing node 1, 3, and 4 send a confirmation instruction ACK to the fused sensing node (sensing node 2).

[0183] Furthermore, sensing nodes 1, 3, 4, 5, and 6 associate their respective UAV local identities with the global identity of the UAV of sensing node 2, while continuing to sense the UAV and record the UAV's trajectory.

[0184] In some embodiments, after the fused sensing node is determined, all sensing nodes adjacent to the fused sensing node become companion sensing nodes of the fused sensing node.

[0185] For example, taking sensing node 2 as a fusion sensing node, sensing node 1, sensing node 3, and sensing node 4 are companion sensing nodes of the fusion sensing node.

[0186] In some embodiments, when a drone moves from position 1, the need to switch fused sensing nodes is determined based on the drone's location. See Figure 7, which illustrates another schematic diagram of sensing node distribution according to some embodiments. As shown in Figure 7, taking the fused sensing node as sensing node 2 and the drone moving from position 1 to position 2 as an example, the sensing network of the sensing fusion system enters the mobility management phase.

[0187] In some embodiments, since the drone leaves the sensing range of sensing node 1, sensing node 1 cannot sense the drone. Sensing node 1 sends the global identity G-1 and fusion request RTC to sensing node 2. However, since there is no target trajectory of the drone, the perception confidence is δ 1-1 = 0, sensing node 3 sends the global identity G-1 and the target trajectory of the UAV to sensing node 2, and the perception confidence δ 3-1 =0.4, and the fusion request RTC, after sensing the drone, the sensing node 4 performs matching and finds that the drone is a drone that has been assigned the global identity G-1, so it sends the global identity G-1 and the target trajectory of the drone to the sensing node 2, and the perception confidence δ 4-1 =0.6, and fusion request RTC; the sensing node 6 detects the drone, assigns it a local identity L-6-1, and obtains the corresponding perception confidence δ 3-1 =0.3, the sensing node 6 sends the sensing information of the sensing target sensed by the sensing node 6 to the adjacent sensing node 4, and sends a fusion request RTC.

[0188] Furthermore, the perception confidence of the fusion perception node (perception node 2) that the drone is perceived is δ 2-1 =0.5, based on the global identity G-1 of the UAV, the difference in perception confidence between the calculated and sensing node 1 is Δδ=0.5, the difference in perception confidence between the calculated and sensing node 3 is Δδ=0.1, and the difference in perception confidence between the calculated and sensing node 4 is Δδ=-0.1.

[0189] In some embodiments, perception node 4 matches and associates the perception information of the drone sent by perception node 6 with the drone perceived by perception node 4, and finds that it is a drone that has been assigned the global identity G-1, and the current fusion perception node of the drone is perception node 2. Therefore, the perception confidence of the perception target of perception node 6 is considered to be 0, and the perception confidence difference calculated by perception node 4 and perception node 6 is Δδ=0.6.

[0190] In some embodiments, whether to switch the fused sensing node may be determined based on the relationship between the sensing confidence differences between the sensing nodes and a preset sensing confidence threshold.

[0191] As an example, the preset perception confidence threshold may be -0.2.

[0192] In some embodiments, sensing node 2 sends a rejection merge response instruction REJ and the global identity G-1 of the drone to sensing node 1; sensing node 2 sends a rejection merge response instruction REJ and the global identity G-1 of the drone to sensing node 3; sensing node 2 sends a rejection merge response instruction REJ and the global identity G-1 of the drone to sensing node 4.

[0193] In some embodiments, if the perception confidence differences between the fused sensing node and all accompanying sensing nodes are greater than a set threshold, the fused sensing node remains unchanged.

[0194] As an example, the companion sensing node (sensing node 4) sends a reject-merge response instruction REJ and the drone's global identities G-1, L-4-1, and L-6-1 to sensing node 6. At this time, the companion sensing node (sensing node 4) believes that the perception confidence of sensing node 6's perception information is 0 and sends a reject-merge response instruction REJ to sensing node 6. Because sensing node 6 and the fused sensing node (sensing node 2) are not adjacent and have no interaction, if sensing node 6 also perceives the drone, if sensing node 4 sends an accept-merge response instruction ACK to sensing node 6, the same sensing target will appear in the perception network with two fused sensing nodes. After receiving the correlation information sent by sensing node 4, sensing node 6 knows that the drone already has a fused sensing node and that it is not adjacent to the fused sensing node. Therefore, there is no need to exchange the drone's perception information in the future.

[0195] It should be noted that the processing method of the accompanying sensing node (such as the sensing node 4) ensures the uniqueness of the fusion sensing node corresponding to the sensing target and the stability of the mobility management strategy.

[0196] In some embodiments, when it is determined that the fused perception node does not need to be switched, the fused perception node (perception node 2) fuses the trajectory information of the drones of all corresponding accompanying perception nodes to obtain fused perception information, and notifies the fused perception information and the global identity G-1 of the drone to the perception center SC. At the same time, the fused perception node (perception node 2) notifies the fused perception information and the global identity G-1 of the drone to all accompanying perception nodes (perception node 1, perception node 3, perception node 4).

[0197] In some embodiments, after receiving the fused sensing information, the companion sensing nodes (sensing nodes 1, 3, and 4) send an acknowledgment (ACK) to the fused sensing node (sensing node 2). Sensing node 6 sends an acknowledgment (ACK) to sensing node 4. All sensing nodes associate the drone's corresponding local identity information with its global identity information, continue sensing the drone, and record its trajectory.

[0198] See Figure 8, which illustrates another schematic diagram of the distribution of sensing nodes according to some embodiments. As shown in Figure 8, taking the fused sensing node as sensing node 2 and a drone moving from location 1 to location 2 and then to location 3 as an example, the sensing network of the sensing fusion system continues the mobility management phase.

[0199] In some embodiments, since the drone leaves the sensing range of sensing node 1, sensing node 1 cannot sense the drone. Sensing node 1 sends the global identity G-1 and fusion request RTC to sensing node 2, but there is no trajectory information of the drone, so the perception confidence is δ 1-1 =0; sensing node 3 sends the global identity G-1 of the drone, the target trajectory of the perceived target, and the fusion request RTC to sensing node 2, and the perception confidence δ 3-1 =0.3, sensing node 4 sends the global identity G-1 of the UAV and the trajectory information of the UAV and the fusion request RTC to sensing node 2, and the perception confidence δ 4-1 =0.9. Although sensing node 6 can sense the UAV at this time, it no longer sends the UAV’s sensing information to the adjacent sensing nodes because it has previously received the sensing information from sensing node 4.

[0200] Furthermore, the perception confidence of the perception information of the fusion perception node (perception node 2) is δ 2-1 =0.3, based on the global identity G-1 of the UAV, the perception confidence difference between perception node 2 and perception node 1 is calculated as Δδ=0.3, the perception confidence difference between perception node 2 and perception node 3 is calculated as Δδ=0, and the perception confidence difference between perception node 2 and perception node 4 is calculated as Δδ=-0.6.

[0201] In some embodiments, whether to switch the fused sensing node may be determined based on the relationship between the sensing confidence differences between the sensing nodes and a preset sensing confidence threshold.

[0202] As an example, the preset perception confidence threshold may be -0.2.

[0203] In some embodiments, the difference in perception confidence between perception node 2 and perception node 4 is less than -0.2, and perception node 4 has the highest perception confidence among all accompanying perception nodes. Therefore, perception node 2 sends an acceptance merge response instruction ACP and the global identity G-1 of the drone to perception node 4, and at the same time sends a rejection merge response instruction REJ and the global identity G-1 of the drone to the remaining accompanying perception nodes (perception node 1, perception node 3). At this time, the fusion perception node is switched, and perception node 4 receives ACP and becomes the new fusion perception node.

[0204] In some embodiments, after determining the sensing node 4 as the new fused sensing node, the original fused sensing node (sensing node 2) fuses the trajectory information of all the corresponding accompanying sensing nodes of the drone to obtain fused sensing information, and notifies the fused sensing information and the global identity G-1 of the drone to the sensing center SC. At the same time, the original fused sensing node (sensing node 2) notifies the fused sensing information, the global identity G-1 of the drone, and the information that the fused sensing node has switched to the new fused sensing node (sensing node 4) to all the accompanying sensing nodes of sensing node 2 (sensing node 1, sensing node 3, and sensing node 4).

[0205] Furthermore, after receiving the fusion information, the companion sensing nodes of sensing node 2 (sensing node 1, sensing node 3, and sensing node 4) send an acknowledgment command ACK to the original fusion sensing node (sensing node 2). All sensing nodes associate the corresponding local identity information of the drone with the global identity information, and continue to perceive the drone and record the drone's trajectory information. The new fusion sensing node (sensing node 4) notifies its adjacent sensing nodes of its own fusion sensing node identity, and all its adjacent sensing nodes become corresponding companion sensing nodes (sensing node 2, sensing node 3, sensing node 5, and sensing node 6).

[0206] In some embodiments, the fusion of trajectory information of the perception target and the construction of the global identity by the perception network in the perception fusion system can also be achieved through the control of the perception center, that is, the perception nodes report the perceived perception information to the perception center, and the perception center designates the fused perception node and can also designate the corresponding companion perception node. As the perception target moves in the perception network, mobility management is implemented by the perception center based on the information of the fused perception node and the companion perception node. The switching of the fused perception node is also decided and implemented by the perception center, and the corresponding perception node is notified.

[0207] It should be noted that the above scenarios are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Ordinary technicians in this field can know that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0208] In some embodiments, steps S201-S203 described above may also be implemented using the process shown in FIG9 . FIG9 is a flow chart illustrating another perception fusion method provided in an embodiment of the present disclosure. As shown in FIG9 , the method includes a fusion request phase, a fusion response phase, a fusion report phase, and a fusion confirmation phase, and includes the following steps: A1 to A6.

[0209] A1. The first perception node sends a first perception message, first local identity information, first perception confidence and a fusion request to the second perception node.

[0210] In some embodiments, in the fusion request stage, the first perception node assigns a first local identity to the perceived perception target, executes the perception signal to perceive the perception target to obtain first perception information, and determines a first perception confidence of the first perception information.

[0211] Furthermore, the first perception node sends a first perception message, first local identity information, first perception confidence, and a fusion request to the second perception node.

[0212] A2. The second perception node sends a second perception message, second local identity information, second perception confidence, and a fusion request to the first perception node.

[0213] In some embodiments, the second perception node assigns a second local identity to the perceived perception target, executes the perception signal to perceive the perception target to obtain second perception information, and determines a second perception confidence of the second perception information.

[0214] In some embodiments, after the first perception node receives the second perception message, second local identity information, second perception confidence and fusion request sent by the second perception node, it matches and associates them with the first perception message, first local identity information and first perception confidence, and calculates the difference between the first perception confidence and the second perception confidence.

[0215] In some implementations, if the first perception node perceives a perception target that the second perception node does not perceive, the second perception confidence of the second perception information is 0; if the second perception node perceives a perception target that the first perception node does not perceive, the first perception confidence of the first perception information is 0.

[0216] A3. The first perception node sends a first message to the second perception node.

[0217] The first message is used to request the first perception node to be used as the fusion perception node corresponding to the perception target. The first message includes first perception information, first perception confidence and first local identity information of the perception target. The fusion perception node is the node responsible for fusing the perception information of the perception target.

[0218] As an example, in the fusion response stage, the first perception node sends a first message of rejection of fusion to all second perception nodes whose difference between the first perception confidence and the second perception confidence is greater than a preset confidence threshold, and sends a first message of agreement to fusion to all second perception nodes whose difference between the first perception confidence and the second perception confidence is less than the preset confidence threshold.

[0219] A4. The second perception node sends a second message to the first perception node.

[0220] The second message is a response message to the first message.

[0221] In some embodiments, if all messages in the second message sent by the second perception node include an approval indication, the first perception node is used as a fused perception node; if the second message sent by the second perception node includes a rejection indication, the first perception node is not used as a fused perception node.

[0222] In some embodiments, after receiving the second message sent by the second sensory node, the first sensory node sends a third message to the second sensory node based on the fact that all the second messages include the consent indication.

[0223] The third message is used to indicate that the first sensing node is a fusion sensing node.

[0224] In some embodiments, the third message includes the global identity information and second local identity information of the sensing target, where the second local identity information is used to locally identify the sensing target at the second sensing node.

[0225] A5. The first perception node sends the fused perception information and global identity information to the second perception node and the perception center.

[0226] In some embodiments, in the fusion reporting stage, after the first perception node is used as the fusion perception node, the first perception node fuses the first perception information and the second perception information to obtain fusion perception information, and assigns a global identity to the perception target, and then sends the fusion perception information and the global identity information to the second perception node and the perception center.

[0227] A6. The second sensing node sends a confirmation instruction to the first sensing node.

[0228] In some embodiments, in the fusion confirmation stage, after the second perception node receives the fusion perception information sent by the first perception node (fusion perception node), the second perception node sends a confirmation instruction to the first perception node (fusion perception node).

[0229] Furthermore, the first sensing node associates the first local identity, the second local identity, and the global identity of the sensing target, and continues to sense the sensing target and record the running trajectory of the sensing target.

[0230] In some embodiments, steps S401-S402 described above may also be implemented using the process shown in FIG10 . Referring to FIG10 , FIG10 is a schematic flow diagram of another perception fusion method according to some embodiments. As shown in FIG10 , the method includes a fusion request phase, a fusion response phase, a fusion report phase, and a fusion confirmation phase, and includes the following steps: B1 to B5.

[0231] B1. The second perception node sends a second perception message, second local identity information, second perception confidence, and a fusion request to the fusion perception node.

[0232] In some embodiments, in the fusion request stage, the second perception node assigns a second local identity to the perceived perception target, executes the perception signal to perceive the perception target to obtain second perception information, and determines a second perception confidence of the second perception information.

[0233] Furthermore, the second perception node sends a second perception message, second local identity information, second perception confidence, and a fusion request to the fusion perception node.

[0234] B2. The fusion sensing node sends a fourth message to the second sensing node.

[0235] The fourth message is used to request the second perception node to serve as a fusion perception node of the perception target, where the fusion perception node is a node responsible for fusing perception information of the perception target.

[0236] In some embodiments, in the fusion response stage, the fusion perception node calculates the difference between the perception confidence of the fusion perception node and the second perception confidence of the second perception node based on the global identity information of the perception target. If the second perception node does not perceive the perception target, the second perception confidence is 0.

[0237] Furthermore, after the second perception node obtains the perception information of its adjacent perception nodes, it matches and associates all the second perception information with the perception target. If the perception target already has a corresponding fusion perception node, the perception confidence of the adjacent perception node is 0.

[0238] B3. The fusion sensing node sends a fifth message to the second sensing node.

[0239] The fifth message is a response message to the fourth message.

[0240] As an example, the first perception node determines whether the second perception node meets the conditions for being a fusion perception node based on the first perception confidence and the second perception confidence.

[0241] The condition for the second perception node to serve as a fusion perception node includes: a difference between the first perception confidence and the second perception confidence is less than a preset confidence threshold.

[0242] In some embodiments, if the second sensing node meets the conditions for being a fused sensing node, a fifth message including an approval indication is sent to the second sensing node; or, if the second sensing node does not meet the conditions for being a fused sensing node, a fifth message including a rejection indication is sent to the second sensing node.

[0243] In some embodiments, if there is a target second perception node among the second perception nodes that has the highest perception confidence and the difference between the perception confidence of the first perception node to the perception target and the perception confidence of the target second perception node to the perception target is less than a preset confidence threshold, the target second perception node is determined as a new fusion perception node.

[0244] B4. The fused sensing node sends fused sensing information, global identity information or fused sensing node switching information to the sensing center and the second sensing node.

[0245] In some embodiments, in the fusion reporting stage, when a new fusion perception node is determined, the fusion perception node assigns a global identity to the perception target, fuses the second perception information of each second perception node to obtain fusion perception information, and sends the fusion perception information and global identity information to the perception center and the second perception node.

[0246] In some embodiments, if the fusion sensing node switches, it is also necessary to send fusion sensing node switching information to the second sensing node.

[0247] B5. The second sensing node sends a confirmation instruction or the identity of the new fused sensing node to the fused sensing node.

[0248] In some embodiments, in the fusion confirmation stage, after the second perception node receives the fusion perception information and global identity information, the second perception node sends a confirmation instruction to the fusion perception node. If the fusion perception node switches, the new fusion perception node identity must also be sent to the fusion perception node.

[0249] The above mainly introduces the solution of the embodiment of the present disclosure from the perspective of method. It can be understood that in order to realize the above functions, the perception fusion device includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiment of the present disclosure.

[0250] It is understandable that, in order to realize the above functions, the perception fusion device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments of the present disclosure, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0251] The embodiment of the present disclosure can divide the perception fusion device into functional modules according to the above-mentioned method embodiment. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above-mentioned integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0252] Figure 11 is a schematic diagram of the structure of a perception fusion device according to some embodiments. The perception fusion device is applied to a first perception node and can execute the perception fusion method provided in the above method embodiment. As shown in Figure 11, the perception fusion device 200 includes an acquisition module 201, a fusion module 202, a sending module 203, a receiving module 204, and a determination module 205.

[0253] In some embodiments, the acquisition module 201 is used to acquire the first perception information and the second perception information.

[0254] In some embodiments, the fusion module 202 is configured to fuse the first perception information and the second perception information to obtain fused perception information of the perception target.

[0255] In some embodiments, the sending module 203 is configured to send the fused perception information and the global identity information of the perception target to the second perception node.

[0256] In some embodiments, the sending module 203 is further configured to send the fused perception information and the global identity information of the perception target to the perception center.

[0257] In some embodiments, the sending module 203 is further configured to send the first message to the second sensing node.

[0258] In some embodiments, the receiving module 204 is configured to receive a second message sent by a second sensing node.

[0259] In some embodiments, the sending module 203 is further configured to send a third message to the second sensing node based on that all second messages include an agreement indication.

[0260] In some embodiments, the receiving module 204 is further configured to receive a fourth message sent by the second sensing node.

[0261] In some embodiments, the sending module 203 is further configured to send a fifth message to the second sensing node.

[0262] In some embodiments, the sending module 203 is further configured to send a fifth message to the second sensing node based on the first perception confidence and the second perception confidence.

[0263] In some embodiments, the determination module 205 is configured to determine whether the second perception node meets the conditions for being a fused perception node based on the first perception confidence and the second perception confidence.

[0264] In some embodiments, the sending module 203 is further configured to send a fifth message including an agreement indication to the second sensing node when the second sensing node meets the conditions for being a fused sensing node.

[0265] In some embodiments, the sending module 203 is further configured to send a fifth message including a rejection indication to the second sensing node when the second sensing node does not meet the conditions for being a fused sensing node.

[0266] In some embodiments, the determination module 205 is further configured to determine whether a target second sensing node exists among all second sensing nodes.

[0267] In some embodiments, the sending module 203 is further used to send a sixth message to all second sensing nodes.

[0268] Figure 12 is a schematic diagram of the structure of another perception fusion device according to some embodiments. Applied to a second perception node, perception fusion device 300 can execute the perception fusion method provided in the aforementioned method embodiment. As shown in Figure 12, perception fusion device 300 includes a sending module 301, a receiving module 302, a determining module 303, and an associating module 304.

[0269] In some embodiments, the sending module 301 is used to send the second perception information to the first perception node.

[0270] In some embodiments, the receiving module 302 is configured to receive the fused perception information and the global identity information of the perception target sent by the first perception node.

[0271] In some embodiments, the receiving module 302 is further used to receive a first message sent by the first perception node.

[0272] In some embodiments, the sending module 301 is further used to send a second message to the first perception node.

[0273] In some embodiments, the sending module 301 is further configured to send a second message to the first sensing node based on the first sensing confidence and the second sensing confidence.

[0274] In some embodiments, the determination module 303 is configured to determine whether the first perception node meets the conditions for being a fused perception node based on the first perception confidence and the second perception confidence.

[0275] In some embodiments, the sending module 301 is further configured to send a second message including an agreement indication to the first sensing node when the first sensing node meets the conditions for being a fused sensing node.

[0276] In some embodiments, the sending module 301 is further configured to send a second message including a rejection indication to the first sensing node when the first sensing node does not meet the conditions for being a fused sensing node.

[0277] In some embodiments, the receiving module 302 is further used to receive a third message sent by the first perception node.

[0278] In some embodiments, the association module 304 is configured to establish an association relationship between the global identity information and the second local identity information.

[0279] In some embodiments, the receiving module 302 is further used to receive a sixth message sent by the first perception node.

[0280] In some embodiments, the receiving module 302 is further configured to receive a seventh message sent by the third sensing node after determining the first sensing node as the fusion sensing node of the sensing target.

[0281] In some embodiments, the sending module 301 is further used to send an eighth message including a rejection indication to the third perception node.

[0282] In the case of implementing the functions of the above-mentioned integrated modules in hardware, the embodiments of the present disclosure provide a structure of the communication device involved in the above-mentioned embodiments. As shown in Figure 13, the communication device 400 includes: a processor 402 and a bus 404. In some embodiments, the communication device 400 may also include a memory 401; in some embodiments, the communication device 400 may also include a communication interface 403.

[0283] Processor 402 may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. Processor 402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Processor 402 may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. Processor 402 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0284] The communication interface 403 is used to connect to other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0285] The memory 401 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0286] As an implementation, memory 401 may exist independently of processor 402 and may be connected to processor 402 via bus 404 for storing instructions or program codes. When processor 402 calls and executes the instructions or program codes stored in memory 401, the perception fusion method provided in the embodiments of the present disclosure can be implemented.

[0287] In another implementation, the memory 401 may also be integrated with the processor 402 .

[0288] Bus 404 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 404 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG13 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0289] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the perception fusion method of any of the above embodiments.

[0290] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0291] An embodiment of the present disclosure provides a computer program product comprising instructions. When the computer program product is run on a computer, the computer is enabled to execute the perception fusion method of any one of the above embodiments.

[0292] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A perception fusion method, wherein: The method is performed by a first sensing node, and the method includes: Acquire first perception information and second perception information, where the first perception information is perception information obtained by the first perception node perceiving a perception target, and the second perception information is perception information obtained by the second perception node perceiving the perception target; fusing the first perception information and the second perception information to obtain fused perception information of the perception target; The fused perception information and the global identity information of the perception target are sent to the second perception node, where the global identity information is used to identify the perception target in the perception network.

2. The method according to claim 1, further comprising: The fused perception information and the global identity information of the perception target are sent to a perception center.

3. The method according to claim 1, further comprising: Sending a first message to the second sensing node, where the first message is used to request that the first sensing node be used as a fusion sensing node corresponding to the sensing target, where the fusion sensing node is a node responsible for fusing sensing information of the sensing target; Receive a second message sent by the second perception node, where the second message is a response message to the first message.

4. The method according to claim 3, further comprising: Based on the fact that all the second messages include the consent indication, a third message is sent to the second sensing node, where the third message is used to indicate that the first sensing node is the fusion sensing node.

5. The method according to claim 3, wherein The first message includes first perception information, first perception confidence, and first local identity information of the perception target; wherein the first perception confidence is the confidence of the first perception information; the first local identity information is used to locally identify the perception target at the first perception node.

6. The method according to claim 4, wherein: The third message includes the global identity information and second local identity information of the sensing target, where the second local identity information is used to locally identify the sensing target at the second sensing node.

7. The method according to claim 6, wherein: The third message also includes the fusion perception information.

8. The method according to claim 1, further comprising: receiving a fourth message sent by the second sensing node, where the fourth message is used to request the second sensing node to serve as a fusion sensing node of the sensing target, where the fusion sensing node is a node responsible for fusing sensing information of the sensing target; A fifth message is sent to the second perception node, where the fifth message is a response message to the fourth message.

9. The method according to claim 8, wherein The fourth message includes the second perception information, the second perception confidence and the second local identity information of the perception target; wherein the second perception confidence is the confidence of the second perception information; the second local identity information is used to locally identify the perception target at the second perception node.

10. The method according to claim 8, wherein The sending the fifth message to the second sensing node includes: The fifth message is sent to the second sensing node based on the first sensing confidence and the second sensing confidence.

11. The method according to claim 10, wherein: The sending the fifth message to the second sensing node based on the first sensing confidence and the second sensing confidence includes: Determining, based on the first perception confidence and the second perception confidence, whether the second perception node meets the condition of serving as the fusion perception node; If the second perception node meets the conditions for being the fused perception node, send a fifth message including an approval indication to the second perception node; or, if the second perception node does not meet the conditions for being the fused perception node, send a fifth message including a rejection indication to the second perception node.

12. The method according to claim 11, wherein The condition for the second sensing node to serve as the fusion sensing node includes: a difference between the first sensing confidence and the second sensing confidence is less than a preset confidence threshold.

13. The method according to claim 12, wherein: In the case that the first perception node is already the fused perception node, the condition for the second perception node to serve as the fused perception node also includes: the second perception confidence of the second perception node is the maximum value among the second perception confidences of all second perception nodes.

14. The method according to claim 3, further comprising: Determining that there is a target second sensing node from all the second sensing nodes, where the target second sensing node is a sensing node having the highest perception confidence of the perception target among all the second sensing nodes, and a difference between the perception confidence of the first sensing node for the perception target and the perception confidence of the target second sensing node for the perception target is less than a preset confidence threshold; Send a sixth message to all the second sensing nodes; wherein the sixth message is used to instruct the fusion sensing node to switch from the first sensing node to the target second sensing node.

15. The method according to claim 14, wherein The sixth message includes: the node identifier of the target second perception node and the global identity information of the perception target.

16. The method according to claim 1, wherein The second sensing node satisfies at least one of the following: The sensing area of ​​the second sensing node overlaps with the sensing area of ​​the first sensing node; The distance between the second sensing node and the first sensing node is less than a preset distance threshold.

17. A perception fusion method, wherein: The method is performed by a second sensing node, and the method includes: Sending second perception information to the first perception node, where the second perception information is perception information obtained by the second perception node sensing the perception target; Receive the fused perception information sent by the first perception node and the global identity information of the perception target; wherein, the fused perception information is obtained by fusing the first perception information and the second perception information; the first perception information is the perception information obtained by the first perception node perceiving the perception target; the global identity information is used to identify the perception target in the perception network.

18. The method according to claim 17, further comprising: receiving a first message sent by the first sensing node, where the first message is used to request that the first sensing node be used as a fusion sensing node corresponding to the sensing target, where the fusion sensing node is a node responsible for fusing sensing information of the sensing target; A second message is sent to the first perception node, where the second message is a response message to the first message.

19. The method according to claim 18, wherein The first message includes first perception information, first perception confidence, and first local identity information of the perception target; wherein the first perception confidence is the confidence of the first perception information; the first local identity information is used to locally identify the perception target at the first perception node.

20. The method according to claim 19, wherein The sending the second message to the first sensing node includes: The second message is sent to the first perception node based on the first perception confidence and the second perception confidence, where the second perception confidence is the confidence of the second perception information.

21. The method according to claim 20, wherein The sending the second message to the first sensing node based on the first sensing confidence and the second sensing confidence includes: Determining, based on the first perception confidence and the second perception confidence, whether the first perception node meets the condition of serving as the fused perception node; If the first perception node meets the conditions for being the fused perception node, send a second message including an approval indication to the first perception node; or if the first perception node does not meet the conditions for being the fused perception node, send a second message including a rejection indication to the first perception node.

22. The method according to claim 21, wherein The condition for the first sensing node to serve as the fusion sensing node includes: a difference between the first sensing confidence and the second sensing confidence is less than a preset confidence threshold.

23. The method of claim 18, further comprising: A third message sent by the first perception node is received, where the third message is used to indicate that the first perception node is the fused perception node.

24. The method according to claim 23, wherein The third message includes the global identity information and second local identity information of the sensing target, where the second local identity information is used to locally identify the sensing target at the second sensing node.

25. The method according to claim 24, further comprising: An association relationship is established between the global identity information and the second local identity information.

26. The method of claim 17, further comprising: Receive a sixth message sent by the first perception node; wherein the sixth message is used to instruct the fusion perception node of the perception target to switch from the first perception node to the target second perception node, and the fusion perception node is a node responsible for fusing the perception information of the perception target.

27. The method according to claim 26, wherein The sixth message includes: the node identifier of the target second perception node and the global identity information of the perception target.

28. The method of claim 17, further comprising: After determining the first sensing node as the fusion sensing node of the sensing target, receiving a seventh message sent by a third sensing node, wherein the seventh message is used to request the third sensing node to be used as the fusion sensing node, where the fusion sensing node is a node responsible for fusing sensing information of the sensing target; An eighth message containing a rejection indication is sent to the third perception node, where the eighth message is a response message to the seventh message.

29. A communication device comprising: memory and processor; The memory is coupled to the processor; The memory is used to store instructions executable by the processor; When the processor executes the instructions, the processor performs the method according to any one of claims 1-16, or the method according to any one of claims 17-28.

30. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 16, or the method according to any one of claims 17 to 28.

Images