A method for backhauling ore analysis data

Abstract

This invention discloses a method for backhauling ore analysis data, comprising: sending ore scene image data collected by each camera to an encoder to form a video stream and dynamically allocating buffer space; the encoder sending the video stream to a multicast group address, and the backend analysis device joining the multicast group address; a multicast router receiving the video stream and forming (S,G) entries; the encoder sending multicast data to the backend analysis device; the multicast router parsing the IGMP (S,G) member report messages sent by the backend analysis device to update the (S,G) entries, and sending PIM (S,G) update messages along the multicast tree; comparing the priorities of the corresponding backend analysis devices using each multicast router, recording the information of the highest priority backend analysis device and forwarding it to the encoder; and the encoder clearing the backup video stream and updating metadata based on the received sequence number confirmation message. This method has high storage utilization and can ensure data integrity while reducing feedback traffic.

Description

Technical Field

[0001] This invention belongs to the field of data backhaul technology, specifically relating to a method for backhauling ore analysis data. Background Technology

[0002] In ore composition analysis scenarios, real-time image data needs to be acquired via cameras as ore analysis data (such as images from ore grade analysis, conveyor belt monitoring, and environmental inspection). This data is then encoded into a video stream by an encoder and transmitted back to backend analysis equipment (such as servers) for image processing. To reduce bandwidth consumption, a multicast + UDP (User Datagram Protocol) transmission scheme is typically used: the encoder continuously sends UDP video packets with sequence numbers to the multicast group address, while simultaneously buffering and backing up the video stream locally. Upon receiving the UDP video packets with sequence numbers, the backend analysis equipment returns a sequence number acknowledgment message (ACK) to the encoder. The encoder clears the corresponding buffered data based on the ACK message; if no ACK message is received, the backup is retained. However, existing technologies still have the following problems:

[0003] 1) Buffer overflow risk: The encoder needs to connect to multiple cameras, each with a fixed buffer space. When backend analysis devices malfunction (such as network packet loss or device failure) and ACKs are missing, the buffer will continue to accumulate until it overflows, causing video data loss.

[0004] 2) Lack of multi-level reception guarantee: A single video stream needs to be distributed to multiple back-end analysis devices with different priorities, but the existing multicast mechanism cannot ensure that all back-end analysis devices receive the data completely, and lacks a priority-based differentiated retransmission mechanism. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by proposing a method for transmitting ore analysis data back to the source, which helps to avoid buffer overflow and data loss, has high storage utilization, and reduces feedback traffic while ensuring data integrity.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention proposes a method for transmitting ore analysis data back to the source, applied to a data transmission device. The data transmission device includes multiple cameras, an encoder, at least one multicast router, and at least one backend analysis device. The method for transmitting ore analysis data back to the source includes the following steps:

[0008] S1. Send the ore scene image data collected by each camera to the encoder, which then forms a video stream and dynamically allocates buffer space for backup.

[0009] S2. Use the encoder to send the video stream to the multicast group address. Different video streams carry corresponding sequence numbers and use different multicast group addresses.

[0010] S3. Send IGMP using backend analysis equipment. G) Member report message requests to join multicast group address, where, For any source, G is the multicast group address;

[0011] S4, by the multicast router according to IGMP ( Member report message creation (G) The (S,G) table entry receives the video stream to form the (S,G) table entry, where S represents the corresponding source address;

[0012] S5. After the encoder sends a UDP multicast message to the backend analysis device according to the (S,G) table entry, the backend analysis device sends an IGMP (S,G) member report message to the corresponding multicast router. The IGMP (S,G) member report message carries the device IP, priority, received sequence number acknowledgment message and timestamp of the backend analysis device.

[0013] S6. The corresponding IGMP(S,G) member report message is parsed through the multicast router to update the (S,G) table entry, and the PIM(S,G) update message is sent to the upstream multicast router along the multicast tree.

[0014] S7. Use each multicast router to compare the priorities of the corresponding backend analysis devices, record the device IP, priority, and received sequence number acknowledgment message of the backend analysis device with the highest priority, and forward it to the encoder. The encoder clears the corresponding backup video stream and updates the metadata of each video stream according to the received sequence number acknowledgment message in the PIM(S,G) update message.

[0015] Preferably, the cache space is dynamically allocated, as follows:

[0016] S11. Divide the encoder's cache pool into corresponding priority areas according to the priority of the backend analysis device;

[0017] S12. Based on the high-priority PIM(S,G) update message received by the encoder, check whether the utilization rate of the high-priority area in the encoder is greater than or equal to the preset threshold. If so, trigger backup video stream recycling, that is, sort all video streams in descending order of priority, release unacknowledged data starting from the lowest priority video stream, allocate the released buffer space to the high-priority video stream, and update the metadata of each video stream. Otherwise, back up the video stream directly to the corresponding buffer space and update the metadata of each video stream.

[0018] Preferably, the metadata includes the video stream's ID, priority, buffer usage, maximum pending confirmation number, maximum confirmed number, and the time when the last confirmation message was received. Unconfirmed data refers to video streams that fall between the maximum pending confirmation number and the maximum confirmed number.

[0019] Preferably, the multicast router parses the corresponding IGMP(S,G) member report message to update the (S,G) table entry, and sends a PIM(S,G) update message to the upstream multicast router along the multicast tree, as follows:

[0020] S61. The multicast router aggregates the IGMP (S,G) member report messages within the period to form an aggregated ACK to update the (S,G) table entries, as follows:

[0021] For IGMP(S,G) member report messages received from the same backend analysis device within the period, record the ACK sequence number of the latest timestamp of the corresponding backend analysis device. The ACK sequence number is the sequence number carried in the received sequence confirmation message.

[0022] IGMP(S,G) member report messages are sorted in descending order of priority and then grouped by priority, that is, IGMP(S,G) member report messages with the same priority are grouped together.

[0023] Determine the number of backend analysis devices corresponding to the highest priority group. If there is only one backend analysis device, directly select the ACK sequence number recorded by that backend analysis device as the aggregation result. If there are multiple backend analysis devices, select the smallest ACK sequence number recorded by all backend analysis devices in the highest priority group as the aggregation result.

[0024] Generate an aggregated ACK based on the aggregation results to update the (S,G) table entries. The format of the aggregated ACK is: [selected ACK sequence number] + [highest priority] + [device IP corresponding to the lowest ACK sequence number] + [timestamp].

[0025] S62. The multicast router generates a PIM(S,G) update message and passes the corresponding computed aggregation ACK along the multicast tree to the upstream multicast router to update the (S,G) table entry until it is passed to the encoder.

[0026] Preferably, the PIM(S,G) update message includes an aggregated ACK, priority, device IP, and timestamp, and the selected ACK sequence number in the aggregated ACK is the receive sequence number confirmation message in the PIM(S,G) update message.

[0027] Preferably, the priorities of the corresponding backend analysis devices are compared using each multicast router, and the device IP, priority, and received sequence number acknowledgment message of the backend analysis device with the highest priority are recorded and forwarded to the encoder, as follows:

[0028] When a PIM(S,G) update message with the highest priority exists, the device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the highest priority are recorded in the (S,G) table entry and forwarded to the encoder. When multiple PIM(S,G) update messages with the same highest priority exist, the PIM(S,G) update messages sent by multiple backend analysis devices are cached. The priority of the latest backend analysis device in each PIM(S,G) update message is compared, and the highest priority PIM(S,G) update message is selected. Then, the received sequence number acknowledgment messages on different backend analysis devices in the highest priority PIM(S,G) update message are compared. The smallest ACK sequence number is selected as the unified received sequence number acknowledgment message. The device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the smallest ACK sequence number of the highest priority are recorded in the (S,G) table entry and forwarded to the upstream multicast router until it is sent to the encoder.

[0029] Preferably, the (S,G) entries include the source address, multicast group address, upstream interface, downstream interface list, downstream device list, and aggregated ACK. The downstream device list includes the device IP, priority, received sequence number acknowledgment message, and timestamp of the backend analysis device.

[0030] Preferably, the method for transmitting ore analysis data back further includes the following steps:

[0031] S8. For cases where the highest priority backend analysis device actively leaves or the multicast router fails to receive IGMP (S,G) member report messages within a preset time, exception handling is performed, the (S,G) table entries are updated, and the encoder is triggered to dynamically allocate cache space, as follows:

[0032] 1) When a highest-priority backend analytics device voluntarily leaves, perform the following operations:

[0033] S811, send an IGMP active leave message carrying the corresponding device IP to the multicast router through the highest priority backend analysis device;

[0034] S812. After receiving the IGMP active leave message, the multicast router determines whether the device IP carried is the device IP corresponding to the aggregated ACK. If so, it is considered to be abnormal, and an IGMP query message is sent to the backend analysis device to execute step S813. Otherwise, it is considered to be normal, the IGMP active leave message is discarded and the (S,G) table entry of the device IP carried is deleted, and the normal process is maintained.

[0035] S813. After receiving the IGMP query message, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router.

[0036] S814. The multicast router determines whether it receives an IGMP (S,G) member report message from a backend analysis device that actively leaves within the period. If yes, it recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry corresponding to the device IP carried and recalculates the aggregated ACK to update the (S,G) entry.

[0037] S815. When any of the device IP, ACK sequence number, or timestamp of the recalculated aggregated ACK changes, a new PIM(S,G) update message is sent to the upstream multicast router. In the case where the device IP of the aggregated ACK changes, the new PIM(S,G) update message also carries the pruning information of the backend analysis device that actively left.

[0038] S816. The upstream multicast router updates the (S,G) table entry based on the new PIM(S,G) update message and determines whether there is pruning information. If there is, it deletes the record of the backend analysis device that actively left, recalculates the aggregated ACK to update the (S,G) table entry, until it is sent to the encoder, triggering the encoder to dynamically allocate the buffer space and end the process.

[0039] 2) When the multicast router does not receive the IGMP(S,G) member report message from the backend analysis device corresponding to the aggregation ACK within the preset time, the following operation is performed:

[0040] S821: Continuously send multiple IGMP query messages to the corresponding backend analysis device. After receiving the IGMP query messages, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router.

[0041] S822. The multicast router determines whether it has received an IGMP (S,G) member report message from the backend analysis device corresponding to the aggregated ACK within the determination period. If yes, it updates the timestamp and ACK sequence number in the (S,G) entry and recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry of the backend analysis device corresponding to the aggregated ACK, takes the second priority group as the highest priority group, and uses the smallest ACK sequence number of the second priority group to recalculate the aggregated ACK to update the (S,G) entry. It then sends a PIM (S,G) update message and carries the pruning information of the original backend analysis device corresponding to the aggregated ACK to the upstream multicast router.

[0042] S823. The upstream multicast router deletes the corresponding (S,G) table entry based on the pruning information of the backend analysis device corresponding to the original aggregated ACK, and recalculates the aggregated ACK to update the (S,G) table entry. It continues to send PIM(S,G) update messages and the pruning information of the backend analysis device corresponding to the original aggregated ACK to the upstream multicast router until they are sent to the encoder, triggering the encoder to dynamically allocate buffer space and maintain the normal process.

[0043] Preferably, the backend analysis device is a server.

[0044] Preferably, the UDP multicast message includes the source address, destination address, sequence number of the video stream, and the video stream itself.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention's multicast router records only the IP address of the highest-priority device on its interface, accepts only its sequence number acknowledgment message, and rejects those of lower-priority devices. This ensures that high-priority backend analysis devices receive multicast data completely and frees up encoder transmit buffers in a timely manner. When multiple highest-priority backend analysis devices exist, the smallest sequence number is selected, ensuring that all highest-priority backend analysis devices receive complete data. This improves the high-priority service buffer guarantee rate, increases storage utilization, and reduces feedback traffic while ensuring data integrity for all high-priority backend analysis devices. Specifically:

[0047] 1) Dynamic cache resource pooling mechanism: Unlike the traditional fixed allocation mode, a unified cache pool is created across camera video streams. The priority-driven preemptive allocation algorithm is implemented, that is, high-priority streams (such as L0 level ore analysis) can preempt the cache space of low-priority streams (such as L3 level environmental monitoring) in real time.

[0048] 2) Multicast protocol stack depth extension: Embed device IP, priority, ACK sequence number and timestamp in IGMP member report messages to achieve in-band fused transmission of control signaling and data stream;

[0049] 3) Enhanced PIM protocol functionality: The PIM join message (PIM update message) is encapsulated with aggregated ACK, priority, device IP and timestamp to build an ACK feedback channel across network domains;

[0050] 4) Intelligent aggregation decision engine: It adopts a two-level dynamic aggregation algorithm, that is, it first judges the priority and then judges the sequence number; and when multiple devices have the same priority, it selects the smallest ACK sequence number to ensure the data integrity of all high-priority devices. Attached Figure Description

[0051] Figure 1This is a flowchart of the ore analysis data back transmission method of the present invention;

[0052] Figure 2 A flowchart illustrating the cache space allocation of this invention;

[0053] Figure 3 This is a schematic diagram of the data return device of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0056] like Figures 1-3 As shown, a method for transmitting ore analysis data is applied to a data transmission device, which includes multiple cameras, an encoder, at least one multicast router, and at least one backend analysis device. The method for transmitting ore analysis data includes the following steps:

[0057] S1. Send the ore scene image data collected by each camera to the encoder, which then forms a video stream and dynamically allocates buffer space for backup.

[0058] In one embodiment, the cache space is dynamically allocated, as follows:

[0059] S11. Divide the encoder's cache pool into corresponding priority areas according to the priority of the backend analysis device;

[0060] S12. Based on the high-priority PIM(S,G) update message received by the encoder, check whether the utilization rate of the high-priority area in the encoder is greater than or equal to the preset threshold. If so, trigger backup video stream recycling, that is, sort all video streams in descending order of priority, release unacknowledged data starting from the lowest priority video stream, allocate the released buffer space to the high-priority video stream, and update the metadata of each video stream. Otherwise, back up the video stream directly to the corresponding buffer space and update the metadata of each video stream.

[0061] In one embodiment, the metadata includes the video stream's ID, priority, buffer usage, maximum pending confirmation number, maximum confirmed number, and the time when the last confirmation message was received. Unconfirmed data refers to video streams that fall between the maximum pending confirmation number and the maximum confirmed number.

[0062] In one embodiment, the back-end analysis device is a server. The back-end analysis device may also be a computer or other device with image processing capabilities.

[0063] like Figure 3 As shown, the data backhaul device in this embodiment includes multiple cameras, an encoder, at least one multicast router, and at least one backend analysis device. The specific number can be determined according to actual needs. All multicast routers form a multicast network. The multicast routers interact with each other through the PIM protocol (Protocol Independent Multicast). The backend analysis device and the corresponding multicast router interact through the IGMP protocol (Internet Group Management Protocol).

[0064] The encoder dynamically shares cache space for ore scene image data (ore analysis data) collected by multiple cameras. For high-priority services (whose priority corresponds to the priority of the backend analysis device), more cache space can be dynamically allocated. Specifically, the encoder determines whether the cache space for the corresponding video stream is sufficient based on the high-priority PIM(S,G) update message received. If insufficient, the priority of the receiving backend analysis device for each cache space is compared. When a cache space with a lower priority than the current video stream exists, the cache space is dynamically coordinated to back up the video stream and update the metadata of each video stream. If sufficient, the video stream is directly backed up to the corresponding cache space, and the metadata of each video stream is updated.

[0065] S2. The encoder sends the video stream to the multicast group address, with different video streams carrying corresponding sequence numbers and using different multicast group addresses.

[0066] S3. Send IGMP using backend analysis equipment. G) Member report message requests to join multicast group address, where, For any source, G is the multicast group address. When the backend analysis device needs to view the video stream, it sends an IGMP (Intense Gain Publication) packet to the multicast router. ,G) member report message (i.e., IGMP report message), IGMP ( The member report message indicates any source. Membership relationship with multicast group address G.

[0067] S4, by the multicast router according to IGMP ( Member report message creation (G) The (S,G) table entry receives the video stream to form the (S,G) table entry, where S represents the corresponding source address.

[0068] S5. After the encoder sends a UDP multicast message to the backend analysis device according to the (S,G) table entry, the backend analysis device sends an IGMP (S,G) member report message to the corresponding multicast router. The IGMP (S,G) member report message carries the device IP, priority, received sequence number acknowledgment message and timestamp of the backend analysis device.

[0069] In one embodiment, the UDP multicast message includes a source address, a destination address, a sequence number of the video stream, and the video stream itself.

[0070] The multicast router first passes through ( The (S,G) table entry receives the video stream and establishes the (S,G) table entry ((S,G) multicast forwarding entry). After the backend analysis device receives the multicast data (UDP multicast message), it sends an IGMP (S,G) membership report message, carrying the device IP, priority, received sequence number acknowledgment message and timestamp of the backend analysis device to the multicast router. The IGMP (S,G) membership report message indicates the membership relationship between the corresponding source address S and multicast group address G.

[0071] S6. The corresponding IGMP(S,G) member report message is parsed by the multicast router to update the (S,G) table entry, and the PIM(S,G) update message is sent to the upstream multicast router along the multicast tree.

[0072] In one embodiment, the multicast router parses the corresponding IGMP(S,G) member report message to update the (S,G) table entry, and sends a PIM(S,G) update message to the upstream multicast router along the multicast tree, as follows:

[0073] S61. The multicast router aggregates the IGMP (S,G) member report messages within the period to form an aggregated ACK to update the (S,G) table entries, as follows:

[0074] For IGMP(S,G) member report messages received from the same backend analysis device within the period, record the ACK sequence number of the latest timestamp of the corresponding backend analysis device. The ACK sequence number is the sequence number carried in the received sequence confirmation message.

[0075] IGMP(S,G) member report messages are sorted in descending order of priority and then grouped by priority, that is, IGMP(S,G) member report messages with the same priority are grouped together.

[0076] Determine the number of backend analysis devices corresponding to the highest priority group. If there is only one backend analysis device, directly select the ACK sequence number recorded by that backend analysis device as the aggregation result. If there are multiple backend analysis devices, select the smallest ACK sequence number recorded by all backend analysis devices in the highest priority group as the aggregation result.

[0077] Generate an aggregated ACK based on the aggregation results to update the (S,G) table entries. The format of the aggregated ACK is: [selected ACK sequence number] + [highest priority] + [device IP corresponding to the lowest ACK sequence number] + [timestamp].

[0078] S62. The multicast router generates a PIM(S,G) update message and passes the corresponding computed aggregation ACK along the multicast tree to the upstream multicast router to update the (S,G) table entry until it is passed to the encoder.

[0079] In one embodiment, the (S,G) entry includes the source address, multicast group address, upstream interface, downstream interface list, downstream device list, and aggregated ACK. The downstream device list includes the device IP, priority, received sequence number acknowledgment message, and timestamp of the backend analysis device.

[0080] In one embodiment, the PIM(S,G) update message includes an aggregated ACK, priority, device IP, and timestamp. The selected ACK sequence number in the aggregated ACK is the receive sequence number acknowledgment message in the PIM(S,G) update message.

[0081] S7. Use each multicast router to compare the priorities of the corresponding backend analysis devices, record the device IP, priority, and received sequence number acknowledgment message of the backend analysis device with the highest priority, and forward it to the encoder. The encoder clears the corresponding backup video stream and updates the metadata of each video stream according to the received sequence number acknowledgment message in the PIM(S,G) update message.

[0082] In one embodiment, the priorities of the corresponding backend analysis devices are compared using each multicast router. The device IP, priority, and received sequence number acknowledgment message of the backend analysis device with the highest priority are recorded and forwarded to the encoder, as follows:

[0083] When a PIM(S,G) update message with the highest priority exists, the device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the highest priority are recorded in the (S,G) table entry and forwarded to the encoder. When multiple PIM(S,G) update messages with the same highest priority exist, the PIM(S,G) update messages sent by multiple backend analysis devices are cached. The priority of the latest backend analysis device in each PIM(S,G) update message is compared, and the highest priority PIM(S,G) update message is selected. Then, the received sequence number acknowledgment messages on different backend analysis devices in the highest priority PIM(S,G) update message are compared. The smallest ACK sequence number is selected as the unified received sequence number acknowledgment message. The device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the smallest ACK sequence number of the highest priority are recorded in the (S,G) table entry and forwarded to the upstream multicast router until it is sent to the encoder.

[0084] When a multicast router receives multiple IGMP(S,G) member report messages or PIM(S,G) update messages from downstream devices, it first compares the priority fields, as follows:

[0085] The multicast router parses the received IGMP(S,G) member report messages or PIM(S,G) update messages, and updates the corresponding (S,G) entries. Each (S,G) entry includes the source address (multicast source address), multicast group address, upstream interface, downstream interface list, and downstream device list (including the device IP, priority, received sequence number acknowledgment message, and timestamp of the backend analysis device). If a (S,G) entry contains a downstream backend analysis device with the highest priority (there is only one highest priority device IP), then the (S,G) entry records the device IP, priority, received sequence number acknowledgment message, and timestamp corresponding to this highest priority backend analysis device, generates an aggregated ACK, places it in a PIM(S,G) update message, and forwards it to the PIM upstream device (upstream multicast router) until it reaches the encoder. Upon receiving the latest received sequence number acknowledgment message, the encoder clears the corresponding backup data (backup video stream).

[0086] If the multicast router's (S,G) table entry contains multiple downstream backend analytics devices with the same high priority (there are multiple highest-priority device IPs), first compare the timestamps of the latest PIM(S,G) update messages from each backend analytics device, take the latest timestamp and its corresponding sequence number, then compare the sequence numbers from different backend analytics devices, and select the smallest sequence number (the sequence numbers sent by the encoder (corresponding to ACK sequence numbers) are from smallest to largest; selecting the smallest sequence number is to clear only the sequence numbers received by all the highest-priority backend analytics devices), as the unified receive sequence number acknowledgment message. Then, the device IP, priority, receive sequence number acknowledgment message, and timestamp corresponding to this highest-priority backend analytics device are recorded in the (S,G) table entry, an aggregated ACK is generated, placed in the PIM(S,G) update message, and forwarded to the upstream PIM device until the encoder. The encoder receives the latest receive sequence number acknowledgment message and clears the corresponding backup data.

[0087] In one embodiment, the method for transmitting ore analysis data back further includes the following steps:

[0088] S8. For cases where the highest priority backend analysis device actively leaves or the multicast router fails to receive IGMP (S,G) member report messages within a preset time, exception handling is performed, the (S,G) table entries are updated, and the encoder is triggered to dynamically allocate cache space, as follows:

[0089] 1) When a highest-priority backend analytics device voluntarily leaves, perform the following operations:

[0090] S811, send an IGMP active leave message carrying the corresponding device IP to the multicast router through the highest priority backend analysis device;

[0091] S812. After receiving the IGMP active leave message, the multicast router determines whether the device IP carried is the device IP corresponding to the aggregated ACK. If so, it is considered to be abnormal, and an IGMP query message is sent to the backend analysis device to execute step S813. Otherwise, it is considered to be normal, the IGMP active leave message is discarded and the (S,G) table entry of the device IP carried is deleted, and the normal process is maintained.

[0092] S813. After receiving the IGMP query message, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router.

[0093] S814. The multicast router determines whether it receives an IGMP (S,G) member report message from a backend analysis device that actively leaves within the period. If yes, it recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry corresponding to the device IP carried and recalculates the aggregated ACK to update the (S,G) entry.

[0094] S815. When any of the device IP, ACK sequence number, or timestamp of the recalculated aggregated ACK changes, a new PIM(S,G) update message is sent to the upstream multicast router. In the case where the device IP of the aggregated ACK changes, the new PIM(S,G) update message also carries the pruning information of the backend analysis device that actively left.

[0095] S816. The upstream multicast router updates the (S,G) table entry based on the new PIM(S,G) update message and determines whether there is pruning information. If there is, it deletes the record of the backend analysis device that actively left, recalculates the aggregated ACK to update the (S,G) table entry, until it is sent to the encoder, triggering the encoder to dynamically allocate the buffer space and end the process.

[0096] 2) When the multicast router does not receive the IGMP(S,G) member report message from the backend analysis device corresponding to the aggregation ACK within the preset time, the following operation is performed:

[0097] S821: Continuously send multiple IGMP query messages to the corresponding backend analysis device. After receiving the IGMP query messages, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router.

[0098] S822. The multicast router determines whether it has received an IGMP (S,G) member report message from the backend analysis device corresponding to the aggregated ACK within the determination period. If yes, it updates the timestamp and ACK sequence number in the (S,G) entry and recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry of the backend analysis device corresponding to the aggregated ACK, takes the second priority group as the highest priority group, and uses the smallest ACK sequence number of the second priority group to recalculate the aggregated ACK to update the (S,G) entry. It then sends a PIM (S,G) update message and carries the pruning information of the original backend analysis device corresponding to the aggregated ACK to the upstream multicast router.

[0099] S823. The upstream multicast router deletes the corresponding (S,G) table entry based on the pruning information of the backend analysis device corresponding to the original aggregated ACK, and recalculates the aggregated ACK to update the (S,G) table entry. It continues to send PIM(S,G) update messages and the pruning information of the backend analysis device corresponding to the original aggregated ACK to the upstream multicast router until they are sent to the encoder, triggering the encoder to dynamically allocate buffer space and maintain the normal process.

[0100] It is particularly important to note that the multicast router only sends an aggregated ACK (PIM(S,G) update message) to the upstream multicast router, containing the device IP, priority, and sequence number acknowledgment message of the highest-priority backend analysis device. This ensures that high-priority analysis devices receive complete multicast data and promptly frees up the encoder's transmission buffer. When multiple highest-priority analysis devices exist, the smallest sequence number is selected to guarantee that all highest-priority analysis devices receive complete data.

[0101] For ease of understanding, the following detailed explanation is provided through specific embodiments.

[0102] 1) Multicast group joining and initial distribution:

[0103] 1.1 The backend analysis device initiates the joining process:

[0104] When the backend analysis device needs to receive a video stream, it sends an IGMP (In-Gross Streaming Protocol) message to the multicast router. A member report message (G) requests to join a multicast group address to receive video streams, such as by sending an IGMPv3 (G) member report message. (G) Member report message, example as follows:

[0105] Device IP3 (priority L1) requests to join multicast group 224.1.1.1.

[0106] IGMPv3 ( The structure of the IGMP member report message (IGMP report message) is as follows:

[0107] | Multicast group address (224.1.1.1) | Number of source addresses (0) | Represents arbitrary source

[0108] 1.2 Multicast router establishes initial entries:

[0109] Multicast router creation ( ,G) table item (( The initial entry (G) is a multicast routing table entry, with the following structure:

[0110] | Multicast group address | Upstream interface | Downstream interface list | Status |

[0111] |-------|-------|-------|-------|

[0112] | 224.1.1.1 | Eth0 | [Eth1, Eth2] | Active |

[0113] Send PIM to upstream multicast router ( (G) Update the message and establish the RPT tree.

[0114] 1.3 Encoder sends video stream:

[0115] The encoder sends sequenced UDP multicast messages at fixed intervals (50ms), with the following structure:

[0116] | Source IP (192.168.1.100) | Destination IP (224.1.1.1) | Sequence Number (Seq=105) | Video Stream |

[0117] The source IP (source address) is the encoder's IP address, and the destination IP (destination address) is the multicast group address. After receiving a UDP multicast packet, the multicast router matches ( The (S,G) entry is sent to the backend analysis device. The multicast router connected to the backend analysis device performs an SPT tree switch, creates the (S,G) entry, sends a PIM(S,G) update message to the upstream multicast router, and establishes the (S,G) multicast forwarding tree. The backend analysis device receives the multicast data, obtains the source address, and sends an IGMP(S,G) member report message carrying information about the (S,G) entry and other extended fields. Specifically:

[0118] 2) Priority ACK feedback and aggregation;

[0119] 2.1 The backend analysis device sends an IGMP extended membership report message (IGMP(S,G) membership report message):

[0120] After receiving the video packet (a sequenced UDP multicast packet), the backend analysis device generates an IGMPv3(S,G) member report message (an IGMP extended member report message, including custom extensions):

[0121] The extended field structure is as follows:

[0122] | Device IP (32-bit) | Priority (4-bit) | ACK Sequence Number (28-bit) | Timestamp |

[0123] Example: | 10.1.1.3 | L1(01) | 105 |12:00:00 |

[0124] Among them, the device IP, priority, and ACK sequence number are the IP address, priority, and sequence number carried in the received sequence confirmation message (corresponding to the video stream sequence number) of the backend analysis device, respectively. Specific extended fields can be adjusted according to actual needs.

[0125] 2.2 Multicast routers process IGMP extended membership report messages:

[0126] Parse the extended fields and update the (S,G) table entries:

[0127] | Multicast source address (S) | Multicast group address (G) | Upstream interface | Downstream interface list | Downstream device list | Aggregated ACK |

[0128] |-------|-------|-------|-------|-------|-------|

[0129] | 192.168.1.100 | 224.1.1.1 | Eth0 | [Eth1] | [IP1(L0,110,12:00:00)]|To be calculated|

[0130] | [Eth2] | [IP3(L1,105,12:00:00)]| |

[0131] The multicast router first aggregates IGMP extended member report messages within a period (e.g., 60 seconds). If it receives device IPs, different ACK sequence numbers, and timestamps from the same backend analysis device within the period, it records the ACK sequence number corresponding to the latest timestamp and updates the multicast router's (S,G) table entries. Specifically:

[0132] After sorting the IGMP Extended Member Report messages in descending order of priority (e.g., L0>L1>L2) and grouping them, the ACK sequence numbers of each backend analysis device within the collection period (e.g., within a 60-second time window) of the highest priority group are determined. The number of backend analysis devices with the same priority in the highest priority group is then determined. If the highest priority group has only one backend analysis device, its ACK sequence number is directly used as the aggregation result. If the highest priority group has multiple backend analysis devices with the same priority, the smallest ACK sequence number is selected as the aggregation result (conservative principle).

[0133] Generate aggregated ACKs in the following format: [Selected ACK sequence number] + [Highest priority] + [Device IP corresponding to the lowest ACK sequence number] + [Time stamp].

[0134] 2.3 Send the aggregated ACK to the upstream multicast router:

[0135] The multicast router generates a PIM(S,G) update message and embeds an aggregated ACK within it. Its PIM header extension structure is as follows:

[0136] | Type (0x06) | Reserved Field (Aggregated ACK) | Priority Field | Device IP | Timestamp |

[0137] Example: | 0x06 | 110 | L0 | IP1 | 12:00:00 |

[0138] The message is passed up the multicast tree to the upstream multicast routers, and an aggregation ACK operation is performed at each multicast router until it reaches the encoder.

[0139] 3) Encoder cache management:

[0140] 3.1 The encoder receives and parses the PIM(S,G) update message, that is, extracts the key fields of the PIM(S,G) update message as follows:

[0141] Selected ACK number in aggregated ACK = 110

[0142] Priority = L0

[0143] Device IP=IP1

[0144] Multicast group address = 224.1.1.1

[0145] 3.2 Perform cache reclamation

[0146] Locate the video stream corresponding to multicast group address 224.1.1.1, delete all backup data (backup video streams) with sequence numbers ≤ 110, and update the metadata of each video stream:

[0147] | Stream ID | Maximum Confirmed Sequence Number | Cache Usage |

[0148] |-------|-------|-------|

[0149] | 001 | 105 → 110 | 200MB → 80MB |

[0150] 3.3 Dynamic Cache Allocation

[0151] The encoder establishes a unified buffer pool to manage the video streams from all cameras. The pool is divided into priority regions (e.g., L0-L3), and higher priority regions have the right to preempt the buffer, realizing dynamic allocation of buffer space, as detailed below:

[0152] When a high-priority video stream needs to be expanded, a priority scan is performed:

[0153] Detect the utilization rate of the high-priority buffer pool. If it is ≥90%, trigger resource reclamation. Sort all video streams in descending order of priority. Start releasing unconfirmed data buffers from the lowest priority video stream (such as L3) and allocate the reclaimed space to the high-priority video streams (such as L0).

[0154] Cache status monitoring:

[0155] Maintain metadata for each video stream, recording:

[0156] | Stream ID | Priority | Cache Usage | Maximum Pending Acknowledgment Number | Maximum Acknowledgment Number | Last ACK Time |

[0157] Stream ID is the ID of the video stream, and the last ACK time is the time when the last sequence number acknowledgment message was received, such as... Figure 2 As shown, when the high-priority cache utilization rate is ≥90%:

[0158] This embodiment of the iron ore analysis system (data feedback device) includes three cameras: camera A, camera B, and camera C, which respectively acquire three video streams.

[0159] Stream A (ore grade analysis, L0 priority, requires 200MB cache);

[0160] Stream B (conveyor belt monitoring, L2 priority, 300MB usage);

[0161] Flow C (Environmental inspection, L3 priority, 150MB usage);

[0162] When the cache requests for stream A reach 180MB (total pool 600MB, stream A cache 200MB, 90% utilization), 100MB of unacknowledged data from stream C will be released and allocated to stream A. Result:

[0163] | Stream ID | Priority | Cache before adjustment | Cache after adjustment |

[0164] |-------|-------|-------|-------|

[0165] | A | L0 | 150MB | 250MB |

[0166] | C | L3 | 150MB | 50MB |

[0167] High-priority streams (such as L0-level ore grade analysis) can preempt the cache space of low-priority streams (such as L3-level environmental monitoring) in real time.

[0168] 4) Exception handling:

[0169] The highest priority backend analysis device may have left voluntarily, or the multicast router may not have received IGMP(S,G) member report messages due to network issues.

[0170] 4.1 The highest priority backend analysis device sends an IGMP active leave message, carrying the device IP.

[0171] When a multicast router receives an IGMP active leave message, it checks whether it is the IP address of the device corresponding to the aggregation ACK. If not, it does not perform any processing; if it is, the multicast router needs to send an IGMP query message downstream.

[0172] After receiving the IGMP query message, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router.

[0173] The multicast router updates the corresponding (S,G) table entry and recalculates the aggregated ACK. First, it checks if an IGMP (S,G) member report message for the backend analysis device corresponding to the original aggregated ACK has been received within the 60-second period. If received, the (S,G) table entry is refreshed, and the calculation is performed according to the aggregated ACK calculation logic described above. If not received, the (S,G) table entry corresponding to the device IP of the backend analysis device is deleted, and the calculation is recalculated according to the aggregated ACK calculation logic described above. If any of the device IP, ACK sequence number, or timestamp in the recalculated aggregated ACK changes, a new PIM (S,G) update message needs to be sent to the upstream multicast router. Furthermore, in cases where the device IP of the aggregated ACK changes, the new PIM (S,G) update message also carries pruning information (including the device IP) of the backend analysis device that actively left the network.

[0174] The upstream multicast router updates the corresponding (S,G) table entry. If pruning information is available, it deletes the record of the backend analysis device corresponding to the original aggregated ACK, recalculates the aggregated ACK, and continues to send the new aggregated ACK to the upstream multicast router. When the device IP of the aggregated ACK changes, it also carries the pruning information of the backend analysis device (carrying the device IP) corresponding to the original aggregated ACK. This process continues until it is sent to the encoder, triggering the encoder to dynamically allocate buffer space.

[0175] 4.2 If the multicast router does not receive the IGMP(S,G) member report message of the backend analysis device corresponding to the current aggregation ACK (i.e. the original aggregation ACK below) within 3 cycles (3 minutes), it will continuously send multiple IGMP query messages in a short period of time (e.g., every 200ms).

[0176] After receiving the IGMP query message, the backend analysis device updates and sends the latest IGMP(S,G) member report message to the multicast router.

[0177] The multicast router first checks whether it has received an IGMP (S,G) member report message from the backend analysis device corresponding to the current aggregated ACK within the period (60 seconds). If it has, it updates the timestamp and ACK sequence number in the (S,G) entry and recalculates the aggregated ACK to update the (S,G) entry. If it has not received, it deletes the (S,G) entry for the backend analysis device corresponding to the current aggregated ACK, recalculates the aggregated ACK using the smallest ACK sequence number of the second-priority group (when there is only one backend analysis device in the second-priority group, the unique sequence number is the smallest ACK sequence number; when there are multiple backend analysis devices, the smallest ACK sequence number is taken), and transmits it to the upstream multicast router, carrying the pruning information of the backend analysis device (carrying the device IP) corresponding to the original aggregated ACK.

[0178] The upstream multicast router updates its (S,G) table entry, deletes the (S,G) table entry corresponding to the original aggregated ACK for the backend analysis device, recalculates the aggregated ACK, and continues to send the new aggregated ACK and pruning information (carrying the device IP) of the original aggregated ACK's corresponding backend analysis device upstream. This is equivalent to the (S,G) table entry carrying the new device IP, priority, received sequence number acknowledgment message, and timestamp, along with the device IP of the original aggregated ACK's corresponding backend analysis device to be pruned. This refreshes the upstream multicast router's (S,G) table entry, deleting the device IP of the original aggregated ACK's corresponding backend analysis device, and recalculating the aggregated ACK with the new device IP, priority, and received sequence number acknowledgment message. This process continues until it reaches the encoder, triggering the encoder to dynamically allocate buffer space.

[0179] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for transmitting ore analysis data back to the source, applied to a data transmission device, the data transmission device comprising multiple cameras, an encoder, at least one multicast router, and at least one backend analysis device, characterized in that: Includes the following steps: S1. Send the ore scene image data collected by each camera to the encoder, which then forms a video stream and dynamically allocates buffer space for backup. S2. Use the encoder to send the video stream to the multicast group address. Different video streams carry corresponding sequence numbers and use different multicast group addresses. S3. Send IGMP using backend analysis equipment. G) Member report message requests to join multicast group address, where, For any source, G is the multicast group address; S4, by the multicast router according to IGMP ( Member report message creation (G) The (S,G) table entry receives the video stream to form the (S,G) table entry, where S represents the corresponding source address; S5. After the encoder sends a UDP multicast message to the backend analysis device according to the (S,G) table entry, the backend analysis device sends an IGMP (S,G) member report message to the corresponding multicast router. The IGMP (S,G) member report message carries the device IP, priority, received sequence number confirmation message and timestamp of the backend analysis device. S6. The corresponding IGMP(S,G) member report message is parsed through the multicast router to update the (S,G) table entry, and the PIM(S,G) update message is sent to the upstream multicast router along the multicast tree. S7. Use each multicast router to compare the priorities of the corresponding backend analysis devices, record the device IP, priority, and received sequence number acknowledgment message of the backend analysis device with the highest priority, and forward it to the encoder. The encoder clears the corresponding backup video stream and updates the metadata of each video stream according to the received sequence number acknowledgment message in the PIM(S,G) update message.

2. The method for transmitting ore analysis data as described in claim 1, characterized in that: The cache space is dynamically allocated as follows: S11. Divide the encoder's cache pool into corresponding priority areas according to the priority of the backend analysis device; S12. Based on the high-priority PIM(S,G) update message received by the encoder, check whether the utilization rate of the high-priority area in the encoder is greater than or equal to the preset threshold. If so, trigger backup video stream recycling, that is, sort all video streams in descending order of priority, release unacknowledged data starting from the lowest priority video stream, allocate the released buffer space to the high-priority video stream, and update the metadata of each video stream. Otherwise, back up the video stream directly to the corresponding buffer space and update the metadata of each video stream.

3. The method for transmitting ore analysis data as described in claim 2, characterized in that: The metadata includes the video stream's ID, priority, buffer usage, maximum pending confirmation number, maximum confirmed number, and the time when the last confirmation message was received. The unconfirmed data refers to the video stream between the maximum pending confirmation number and the maximum confirmed number.

4. The method for transmitting ore analysis data as described in claim 1, characterized in that: The process of parsing the corresponding IGMP(S,G) member report message through the multicast router to update the (S,G) table entry and sending the PIM(S,G) update message along the multicast tree to the upstream multicast router is as follows: S61. The multicast router aggregates the IGMP (S,G) member report messages within the period to form an aggregated ACK to update the (S,G) table entries, as follows: For IGMP(S,G) member report messages received from the same backend analysis device within the period, record the ACK sequence number of the latest timestamp of the corresponding backend analysis device. The ACK sequence number is the sequence number carried in the received sequence confirmation message. IGMP(S,G) member report messages are sorted in descending order of priority and then grouped by priority, that is, IGMP(S,G) member report messages with the same priority are grouped together. Determine the number of backend analysis devices corresponding to the highest priority group. If there is only one backend analysis device, directly select the ACK sequence number recorded by that backend analysis device as the aggregation result. If there are multiple backend analysis devices, select the smallest ACK sequence number recorded by all backend analysis devices in the highest priority group as the aggregation result. An aggregated ACK is generated based on the aggregation result to update the (S,G) table entry. The format of the aggregated ACK is: [selected ACK sequence number] + [highest priority] + [device IP corresponding to the lowest ACK sequence number] + [timestamp]. S62. The multicast router generates a PIM(S,G) update message and passes the corresponding computed aggregation ACK along the multicast tree to the upstream multicast router to update the (S,G) table entry until it is passed to the encoder.

5. The method for transmitting ore analysis data as described in claim 4, characterized in that: The PIM (S,G) update message includes aggregated ACK, priority, device IP and timestamp, and the selected ACK sequence number in the aggregated ACK is the receive sequence number confirmation message in the PIM (S,G) update message.

6. The method for transmitting ore analysis data as described in claim 4, characterized in that: The process involves comparing the priorities of the corresponding backend analysis devices using each multicast router, recording the device IP, priority, and received sequence number acknowledgment message of the highest-priority backend analysis device, and forwarding it to the encoder, as detailed below: When a PIM(S,G) update message with the highest priority exists, the device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the highest priority are recorded in the (S,G) table entry and forwarded to the encoder. When multiple PIM(S,G) update messages with the same highest priority exist, the PIM(S,G) update messages sent by multiple backend analysis devices are cached. The priority of the latest backend analysis device in each PIM(S,G) update message is compared, and the highest priority PIM(S,G) update message is selected. Then, the received sequence number acknowledgment messages on different backend analysis devices in the highest priority PIM(S,G) update message are compared. The smallest ACK sequence number is selected as the unified received sequence number acknowledgment message. The device IP, priority, and received sequence number acknowledgment message of the backend analysis device corresponding to the smallest ACK sequence number of the highest priority are recorded in the (S,G) table entry and forwarded to the upstream multicast router until it is sent to the encoder.

7. The method for transmitting ore analysis data as described in claim 4, characterized in that: The (S,G) entries include source address, multicast group address, upstream interface, downstream interface list, downstream device list, and aggregated ACK. The downstream device list includes the device IP, priority, received sequence number acknowledgment message, and timestamp of the backend analysis device.

8. The method for transmitting ore analysis data as described in claim 4, characterized in that: The method for transmitting ore analysis data back to the source also includes the following steps: S8. For cases where the highest priority backend analysis device actively leaves or the multicast router fails to receive IGMP (S,G) member report messages within a preset time, exception handling is performed, the (S,G) table entries are updated, and the encoder is triggered to dynamically allocate cache space, as follows: 1) When a highest-priority backend analytics device voluntarily leaves, perform the following operations: S811, send an IGMP active leave message carrying the corresponding device IP to the multicast router through the highest priority backend analysis device; S812. After receiving the IGMP active leave message, the multicast router determines whether the device IP carried is the device IP corresponding to the aggregated ACK. If so, it is considered to be abnormal, and an IGMP query message is sent to the backend analysis device to execute step S813. Otherwise, it is considered to be normal, the IGMP active leave message is discarded and the (S,G) table entry of the device IP carried is deleted, and the normal process is maintained. S813. After receiving the IGMP query message, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router. S814. The multicast router determines whether it receives an IGMP (S,G) member report message from a backend analysis device that actively leaves within the period. If yes, it recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry corresponding to the device IP carried and recalculates the aggregated ACK to update the (S,G) entry. S815. When any of the device IP, ACK sequence number, or timestamp of the recalculated aggregated ACK changes, a new PIM(S,G) update message is sent to the upstream multicast router. In the case where the device IP of the aggregated ACK changes, the new PIM(S,G) update message also carries the pruning information of the backend analysis device that actively left. S816. The upstream multicast router updates the (S,G) table entry based on the new PIM(S,G) update message and determines whether there is pruning information. If there is, it deletes the record of the backend analysis device that actively left, recalculates the aggregated ACK to update the (S,G) table entry, until it is sent to the encoder, triggering the encoder to dynamically allocate the buffer space and end the process. 2) When the multicast router does not receive the IGMP(S,G) member report message from the backend analysis device corresponding to the aggregation ACK within the preset time, the following operation is performed: S821: Continuously send multiple IGMP query messages to the corresponding backend analysis device. After receiving the IGMP query messages, the backend analysis device updates the IGMP(S,G) member report message and sends it to the corresponding multicast router. S822. The multicast router determines whether it has received an IGMP (S,G) member report message from the backend analysis device corresponding to the aggregated ACK within the determination period. If yes, it updates the timestamp and ACK sequence number in the (S,G) entry and recalculates the aggregated ACK to update the (S,G) entry. If no, it deletes the (S,G) entry of the backend analysis device corresponding to the aggregated ACK, takes the second priority group as the highest priority group, and uses the smallest ACK sequence number of the second priority group to recalculate the aggregated ACK to update the (S,G) entry. It then sends a PIM (S,G) update message and carries the pruning information of the original backend analysis device corresponding to the aggregated ACK to the upstream multicast router. S823. The upstream multicast router deletes the corresponding (S,G) table entry based on the pruning information of the backend analysis device corresponding to the original aggregated ACK, and recalculates the aggregated ACK to update the (S,G) table entry. It continues to send PIM(S,G) update messages and the pruning information of the backend analysis device corresponding to the original aggregated ACK to the upstream multicast router until they are sent to the encoder, triggering the encoder to dynamically allocate buffer space and maintain the normal process.

9. The method for transmitting ore analysis data as described in claim 1, characterized in that: The backend analysis device is a server.

10. The method for transmitting ore analysis data as described in claim 1, characterized in that: The UDP multicast message includes the source address, destination address, video stream sequence number, and video stream.

Images