Visual processing system, visual data processing method, chip and computer storage medium

Abstract

Embodiments of the present disclosure provide a visual processing system, a visual data processing method, a chip and a computer storage medium. The visual processing system comprises a plurality of nodes connected in a group network, and the plurality of nodes at least comprise a first node and a second node. The second node comprises a second transmission unit, and the second node is configured to obtain first visual data and transmit the first visual data to the first node based on a first transmission protocol through the second transmission unit. The first node comprises a first image signal processing unit and a first transmission unit, and the first node is configured to receive the first visual data from the second node based on the first transmission protocol through the first transmission unit, and perform image signal processing on the first visual data through the first image signal processing unit to obtain second visual data.

Description

[0001] This application claims priority to the following PCT patent applications filed on February 9, 2024, with application number "PCT / CN2024 / 077134" and patent title "Chip, Networking System and Electronic Device"; PCT patent applications filed on February 9, 2024, with application number "PCT / CN2024 / 077133" and patent title "Data Transmission Method, Chip and Storage Medium"; and PCT patent applications filed on July 12, 2024, with application number "PCT / CN2024 / 105328" and patent title "A Data Transmission Method, Chip and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of electronic communication technology, and in particular to a vision processing system, a vision data processing method, a chip, and a computer storage medium. Background Technology

[0003] Currently, vision processing systems are being used more and more widely. Taking automotive vision systems as an example, by processing visual data collected by cameras installed in vehicles, the processed visual data can be used to achieve functions including but not limited to 360° surround view of the vehicle, reversing camera, and autonomous / assisted driving. A typical current automotive vision system solution involves the camera first processing the visual data using its local image signal processor (ISP) before sending the processed visual data to a remote location for application. However, there is still room for improvement in the data transmission and processing performance of current vision processing systems. Summary of the Invention

[0004] In view of the above, embodiments of this disclosure provide a vision processing system, a vision data processing method, a chip, and a computer storage medium to at least partially solve the above problems.

[0005] According to a first aspect of the present disclosure, a vision processing system is provided, comprising: a plurality of nodes connected in a network, wherein the plurality of nodes includes at least a first node and a second node, wherein: the second node includes a second transmission unit, and the second node is configured to: obtain first visual data, and transmit the first visual data to the first node via the second transmission unit based on a first transmission protocol; the first node includes a first image signal processing unit and a first transmission unit, and the first node is configured to: receive the first visual data from the second node via the first transmission unit based on the first transmission protocol, and perform image signal processing on the first visual data via the first image signal processing unit to obtain second visual data.

[0006] According to a second aspect of the present disclosure, a vision processing system is provided, comprising: a plurality of nodes connected in a network, wherein the plurality of nodes includes at least a first node and a second node, wherein: the second node includes a second image signal processing unit and a second transmission unit, the second node being configured to: configure the second image signal processing unit according to functional configuration information; obtain raw visual data, and obtain third visual data through the functionally configured second image signal processing unit, and transmit the third visual data to the first node through the second transmission unit based on a first transmission protocol; the first node includes a first transmission unit, the first node being configured to: receive the third visual data from the second node through the first transmission unit based on the first transmission protocol.

[0007] According to a third aspect of the present disclosure, a visual data processing method is provided for a second node in a network of multiple nodes, the multiple nodes further including a first node. The second node includes a second transmission unit, and the first node includes a first image signal processing unit and a first transmission unit. The method includes: obtaining first visual data; transmitting the first visual data to the first node via the second transmission unit based on a first transmission protocol, so that the first node receives the first visual data via the first transmission unit based on the first transmission protocol, and performs image signal processing on the first visual data via the first image signal processing unit to obtain second visual data.

[0008] According to a fourth aspect of the present disclosure, a visual data processing method is provided for a first node among a plurality of nodes connected in a network, the plurality of nodes further comprising a second node, the second node including a second transmission unit, and the first node including a first image signal processing unit and a first transmission unit. The method includes: receiving first visual data from the second node through the first transmission unit based on a first transmission protocol, wherein the first visual data is transmitted from the second node to the first node through the second transmission unit based on the first transmission protocol; and performing image signal processing on the first visual data through the first image signal processing unit to obtain second visual data.

[0009] According to a fifth aspect of the present disclosure, a visual data processing method is provided for a second node in a network of multiple nodes, the multiple nodes further including a first node. The second node includes a second transmission unit and a second image signal processing unit. The method includes: configuring the second image signal processing unit according to functional configuration information; obtaining raw visual data and obtaining third visual data through the functionally configured second image signal processing unit; and transmitting the third visual data to the first node through the second transmission unit based on a first transmission protocol.

[0010] According to a sixth aspect of the present disclosure, a chip is provided, comprising: a processor and a memory, wherein the processor and the memory communicate with each other; the memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to the visual data processing method as described in any one of the third, fourth, and fifth aspects.

[0011] According to a seventh aspect of the present disclosure, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the visual data processing method as described in any one of the third, fourth, and fifth aspects.

[0012] The vision processing system provided in this embodiment includes multiple nodes connected in a network, with at least a first node and a second node among them. The second node can obtain first visual data and transmit it to the first node via a second transmission unit based on a first transmission protocol. The first node can receive the first visual data from the second node via the first transmission unit based on the first transmission protocol and perform image signal processing on the first visual data using its first image signal processing unit to obtain second visual data. This effectively realizes the transmission and processing of visual data. Furthermore, the first visual data from the second node can be processed centrally at the first node, reducing the need for excessive processing on the second node. This results in better real-time transmission of the first visual data and a lower data processing burden on the second node. When not necessary, image signal processing units can be omitted for one or more nodes, including the second node, reducing system costs. Therefore, this solution effectively improves the data transmission and processing performance of the vision processing system. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1A A schematic diagram of a vision processing system, representing some examples of embodiments of this disclosure, is shown.

[0015] Figure 1B Schematic diagrams of vision processing systems of some other examples in embodiments of this disclosure are shown.

[0016] Figure 2 A schematic diagram is shown in an embodiment of this disclosure, illustrating how the second node obtains the first visual data based on the original visual data.

[0017] Figure 3A The diagrams show some schematics of how the first and second nodes process visual data.

[0018] Figure 3B Further schematic diagrams are shown of the first and second nodes processing visual data.

[0019] Figure 4 This diagram illustrates the time slot division of a preset period and the frame number allocation of child nodes.

[0020] Figure 5 A schematic diagram illustrating an example of the transmission process of visual data is shown.

[0021] Figure 6A A simplified schematic diagram of an in-vehicle vision transmission system, an example of a related technology, is shown.

[0022] Figure 6B An example data packet format from related technologies is shown.

[0023] Figure 7A A schematic diagram of a vision processing system, representing some further examples of embodiments of this disclosure, is shown.

[0024] Figure 7B A schematic diagram of a vision processing system, representing yet another example of an embodiment of this disclosure, is shown.

[0025] Figure 8 A schematic diagram is shown illustrating how the second node obtains third visual data based on the original visual data in an embodiment of this disclosure.

[0026] Figure 9A The diagrams show some schematics of how the first and second nodes process visual data.

[0027] Figure 9B Further schematic diagrams are shown of the first and second nodes processing visual data.

[0028] Figure 9C Further schematic diagrams are shown of the first and second nodes processing visual data.

[0029] Figure 9D Further schematic diagrams are shown of how the first and second nodes process visual data.

[0030] Figure 9E Further schematic diagrams are shown of how the first and second nodes process visual data.

[0031] Figure 10 This diagram illustrates how the master node schedules visual data from multiple child nodes.

[0032] Figure 11 A schematic diagram illustrating an example of the transmission process of visual data is shown.

[0033] Figure 12 A schematic flowchart illustrating some examples of visual data processing methods in embodiments of this disclosure is shown.

[0034] Figure 13 Schematic flowcharts of visual data processing methods for some other examples of embodiments of this disclosure are shown.

[0035] Figure 14 A schematic flowchart illustrating a visual data processing method, representing some further examples of embodiments of this disclosure, is shown.

[0036] Figure 15 A schematic diagram of a chip, representing some examples of embodiments of this disclosure, is shown.

[0037] Explanation of reference numerals in the attached figures: 100, vision processing system; 10, first node; 11, first image signal processing unit; 12, first transmission unit; 20, second node; 21, second image signal processing unit; 22, second transmission unit; 30, vision acquisition unit; 40, host computer; 1000, chip; 1002, processor; 1006, memory; 1010, program. Detailed Implementation

[0038] To enable those skilled in the art to better understand the technical solutions in the embodiments of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art should fall within the protection scope of this disclosure.

[0039] Currently, vision processing systems are being used more and more widely. Taking in-vehicle vision systems as an example, by processing visual data collected by cameras installed in vehicles, the processed visual data can be used to achieve functions including but not limited to 360° surround view of the vehicle body, reversing camera, and autonomous / assisted driving. It should be understood that the terms "vehicle," "in-vehicle," or "in-vehicle" or other similar terms used in this document generally include various private or commercial vehicles such as sedans, SUVs, buses, and trucks, as well as various boats, ships, aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles.

[0040] For example, some typical solutions in current automotive vision systems involve processing the visual data acquired by the camera using its local Image Signal Processor (ISP) before sending the processed data to a remote location for application. The advantage of this approach is that the visual data is processed locally, and the remote end only handles the application of the visual data, simplifying the division of labor. However, this approach also introduces the problem that each camera requires a costly ISP chip, leading to a higher overall system cost. Furthermore, there is still room for improvement in the data transmission and processing performance of current vision processing systems.

[0041] According to a first aspect of the embodiments of this disclosure, a vision processing system 100 is provided. (Refer to...) Figure 1A , Figure 1B As shown, the vision processing system 100 may include: multiple nodes connected in a network, including at least a first node 10 and a second node 20, wherein: the second node 20 includes a second transmission unit 22, and the second node 20 is used to: obtain first visual data, and transmit the first visual data to the first node 10 through the second transmission unit 22 based on a first transmission protocol; the first node 10 includes a first image signal processing unit 11 and a first transmission unit 12, and the first node 10 is used to: receive the first visual data from the second node 20 through the first transmission unit 12 based on the first transmission protocol, and perform image signal processing on the first visual data through the first image signal processing unit 11 to obtain second visual data.

[0042] The vision processing system 100 provided in this embodiment includes multiple nodes connected in a network, with at least a first node 10 and a second node 20 among them. The second node 20 can obtain first visual data and transmit it to the first node 10 via a second transmission unit 22 based on a first transmission protocol. The first node 10 can receive the first visual data from the second node 20 via the first transmission unit 12 based on the first transmission protocol, and perform image signal processing on the first visual data via its first image signal processing unit 11 to obtain second visual data. This effectively realizes the transmission and processing of visual data. Furthermore, the first visual data from the second node 20 can be centrally processed at the first node 10, reducing the need for the second node 20 to perform excessive processing. This results in better real-time transmission of the first visual data and a lower data processing burden on the second node 20. When not necessary, it is not necessary to set up an image signal processing unit for one or more nodes, including the second node 20, thereby reducing system costs. Therefore, this solution can effectively improve the data transmission and processing performance of the vision processing system.

[0043] It should be understood that the visual processing system 100 of the first aspect of this disclosure can be applied to any scenario. For example, it can be adapted to scenarios involving the transmission and processing of visual data from multiple cameras. As an example, it can be applied to scenarios involving the transmission and processing of visual data in vehicles, as well as to scenarios involving the transmission of visual data in homes or security systems.

[0044] In the embodiments of this disclosure, the image signal processing unit and the transmission unit can be separate chips or devices, integrated simultaneously in the node; alternatively, the transmission unit can be a separate chip or device integrating the functions of the image signal processing unit; or the image signal processing unit can be a separate chip or device integrating the functions of the transmission unit. The node can be a physical concept such as a module, electronic device, integrated chip, etc. This disclosure does not limit the physical entities of the node, transmission unit, and image signal processing unit. In some optional embodiments, the image signal processing unit can be implemented as an image signal processor (ISP).

[0045] In some embodiments, a node can be any module, device, or chip capable of implementing the solutions of the embodiments of this disclosure. For example, in some embodiments, each of a plurality of nodes includes at least one chip for data transmission in a network. In some embodiments, a node can also be a chip, that is, a chip can be directly used as a node.

[0046] In this embodiment of the disclosure, visual data may include at least one of image data and video data. For example, visual data may be understood as video data in the following text.

[0047] In this embodiment of the disclosure, the first transmission protocol can be any suitable data transmission protocol. The first transmission protocol can be an existing standard protocol, such as Ethernet or other existing protocols. Alternatively, the first transmission protocol can also be a proprietary protocol.

[0048] In some optional embodiments, the first transmission protocol is a first proprietary transmission protocol, wherein the first proprietary transmission protocol implements data transmission and / or processing based on the physical layer. For example, the second node 20 can transmit the first visual data to the first node 10 based on the first proprietary transmission protocol through the second transmission unit 22. The first node 10 can receive the first visual data from the second node 20 based on the first proprietary transmission protocol through the first transmission unit 12, and perform image signal processing on the first visual data through the first image signal processing unit 11 to obtain the second visual data.

[0049] It should be understood that since the second node 20 in the vision processing system 100 can transmit the first visual data through the second transmission unit 22 based on the first private transmission protocol, and the first node 10 can receive the first visual data through the first transmission unit 12 based on the first private transmission protocol, visual data can be directly transmitted and / or processed at the physical layer. This eliminates the need for data encapsulation based on standard protocols by a processor (e.g., an MCU), meaning processing is not required based on higher-level protocols above the physical layer. Instead, data is transmitted and processed directly at the physical layer, ensuring reliable data transmission. Furthermore, this solution eliminates the need for switches when transmitting visual data at the physical layer, reducing the demand for processors (e.g., MCUs) in the network during visual data transmission. This effectively reduces the transmission latency of visual data and further reduces costs by decreasing the number of switches and processors (e.g., MCUs). Therefore, this solution can further and effectively improve the data transmission and processing performance of the vision processing system.

[0050] The physical layer is the lowest layer in network communication. It should be understood that the data in the embodiments of this disclosure can be transmitted and processed based on a first private transmission protocol, that is, transmitted and / or processed based on the physical layer. However, this does not mean that it only performs the functions that the physical layer can perform as defined in the standard protocol. For example, in the standard protocol, the function of the physical layer is to convert the frame signals of higher layers (e.g., the data link layer) above the physical layer into electrical or optical signals that can be transmitted on the physical transmission medium, and to convert the electrical or optical signals received from the physical transmission medium into bit streams for processing by higher layers (e.g., the data link layer). The data transmission and processing in the vision processing system of this disclosure are implemented based on the physical layer of the node, which means that the data transmission and processing scheme of the embodiments of this disclosure does not go through higher layers (e.g., the data link layer, network layer, etc.) above the physical layer as defined in standard physical communication. Instead, it is based on a set of private protocols defined by the physical layer, which enables it to perform data transmission and processing other than certain functions that the physical layer can perform as defined in the standard protocol.

[0051] For ease of explanation of the embodiments disclosed herein, the first transmission protocol may be referred to as the first proprietary transmission protocol in the following description.

[0052] In this embodiment of the disclosure, multiple nodes can be connected to form any type of network. For example, such as Figure 1A As shown, multiple nodes can be connected to form a daisy-chain network. Figure 1A The example shown is a double daisy-chain topology; in other embodiments, a single daisy-chain topology can also be used. For example... Figure 1B As shown, multiple nodes can be connected to form a ring network. The vision processing system 100 of this embodiment adopts a daisy-chain network or a ring network, resulting in lower data forwarding latency when transmitting visual data based on a first transmission protocol (e.g., a first proprietary transmission protocol).

[0053] Optionally, in the vision processing system 100 of this disclosure, when a node transmits data, including but not limited to visual data, based on a first transmission protocol (e.g., a first proprietary transmission protocol), the data transmission and processing can be implemented in real-time streaming. For example, taking the first transmission protocol as a first proprietary transmission protocol, a target node among multiple nodes (e.g., the second node 20) can perform real-time streaming processing of data packets (e.g., target data packets) based on the first proprietary transmission protocol through its transmission unit (e.g., the second transmission unit 22). It can also write the data to be transmitted (e.g., visual data) into the data packet when needed, and transmit the real-time streaming data packet to the next adjacent node (e.g., the first node 10 or other nodes) in real-time streaming. That is, real-time streaming processing of data packets means processing them immediately when they arrive at the target node, rather than waiting until the entire data packet is completely received before processing or transmission.

[0054] It should be understood that, because the nodes in this embodiment can perform real-time streaming data transmission and processing, for example, the node's transmission unit can transmit and process data packets separately in real-time streaming, enabling real-time processing of data in the received data packets. Data can also be written to the received data packets in real-time as needed; that is, transmission and processing are performed immediately after receiving a portion of the data packet. This effectively reduces data transmission latency and ensures that the data in the data packets can be transmitted and processed in real time. In contrast, related technologies based on switches or other software or wireless data packet transmissions require waiting for all data in the data packet to be completely received before transmission or processing, resulting in significant latency. Therefore, compared to related technologies, the visual processing system 100 of this solution has better data transmission and processing performance.

[0055] It should be noted that, Figure 1A and Figure 1B The number of nodes mentioned is merely an example to illustrate the embodiments of this disclosure, and the specific number can be set as needed. In the embodiments of this disclosure, multiple nodes can be connected through a physical transmission medium (such as shielded twisted pair cable, unshielded twisted pair cable, coaxial cable, etc., which are not specifically limited in this embodiment).

[0056] Optionally, for different types of network topologies, a master node can be specified among multiple nodes, and the nodes other than the master node can be called child nodes. It should be noted that a network can include more than two nodes, i.e., one master node and multiple child nodes; or, a network can also include only two nodes, i.e., one master node and one child node.

[0057] In this embodiment of the disclosure, the first node 10 among multiple nodes can be the master node, while the other nodes besides the first node 10 (including the second node 20) can all be child nodes (e.g., ...). Figure 1A and Figure 1B As shown, for ease of description below, each child node can be referred to as child node a to g). The plurality of nodes in this embodiment may include at least one second node 20. Optionally, refer to... Figure 1A and Figure 1B As shown, all nodes (child nodes) except for the first node 10 (master node) can be the second node 20.

[0058] In this embodiment of the disclosure, the second node 20 may include a second transmission unit 22. The second node 20 can obtain the first visual data and can transmit the first visual data to the first node 10 based on a first transmission protocol (e.g., a first proprietary transmission protocol) through the second transmission unit 22. Optionally, as... Figure 1A and Figure 1B As shown, the second node 20 can be connected to the visual acquisition unit 30. The visual acquisition unit 30 can be used to acquire visual data. The visual data can include at least one of image data and video data. For example, the visual acquisition unit 30 can be a camera or a video image sensor, etc. In the following description, the visual acquisition unit 30 can be exemplified as a camera. Optionally, the first visual data can be obtained from the raw visual data acquired by the visual acquisition unit 30.

[0059] Optionally, the raw visual data can be visual data in Bayer RAW format. Each pixel in the Bayer domain contains only one of the three colors: red (R), green (G), and blue (B). Because the raw visual data is in Bayer RAW format, the visual processing system 100 of this solution can transmit visual data in the Bayer domain, resulting in less data transmission compared to traditional RGB data. Under the same bandwidth, it can support the transmission of visual data from more visual acquisition units 30, or the transmission of higher-resolution visual data.

[0060] In some optional embodiments, the signals sensed by the sensors of the visual acquisition unit 30 (such as a camera) can be analog signals, which can be converted into digital signals through analog-to-digital conversion, and then the raw visual data in Bayer RAW format can be obtained from the digital signals.

[0061] In some optional embodiments, the second node 20 may be used to: obtain raw visual data from the visual acquisition unit 30, and transmit the raw visual data as first visual data to the first node 10 via the second transmission unit 22 based on a first transmission protocol (e.g., a first proprietary transmission protocol).

[0062] like Figure 2 As shown, in some cases, the second node 20 can output the raw visual data directly as the first visual data.

[0063] It should be understood that in the visual processing system 100 of this embodiment, the second node 20 can transmit the raw visual data acquired by the visual acquisition unit 30 (such as a camera) as the first visual data to the first node 10 based on the first transmission protocol through the second transmission unit 22. The first image signal processing unit 11 of the first node 10 can perform centralized processing, and the second node 20 does not need to perform too much processing on the first visual data. This makes the real-time transmission of the first visual data better, and the data processing burden of the second node 20 is also smaller. When not necessary, it is not necessary to set up an image signal processing unit for one or more nodes including the second node 20, so as to reduce the system cost.

[0064] Optionally, the second node 20 can transmit the original visual data as the first visual data to the first node 10 via the second transmission unit 22 based on the first proprietary transmission protocol. It should be understood that transmitting the original visual data (i.e., the first visual data) based on the first proprietary transmission protocol does not require data encapsulation based on standard protocols by a processor (e.g., MCU), that is, it does not require processing based on higher-level protocols above the physical layer, but can be directly transmitted and processed based on the physical layer, which can ensure the reliability of data transmission.

[0065] In some examples, the second node 20 may also include a second image signal processing unit, which may be set to a bypass mode. In the bypass mode, the original visual data is not further processed by the second image signal processing unit. The second node 20 may directly transmit the original visual data as the first visual data to the first node 10 through the second transmission unit 22 based on the first transmission protocol (e.g., the first proprietary transmission protocol).

[0066] In other examples, the second node 20 may not include the second image signal processing unit. The second node 20 can directly transmit the raw visual data as the first visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22.

[0067] In some alternative embodiments, the second node 20 may be used to: obtain raw visual data from the visual acquisition unit 30, compress the raw visual data, and transmit the compressed raw visual data as first visual data to the first node 10 via the second transmission unit 22 based on a first transmission protocol (e.g., a first proprietary transmission protocol).

[0068] like Figure 2 As shown, in some cases, the second node 20 can compress the original visual data and output it as the first visual data.

[0069] It should be understood that in the visual processing system 100 of this embodiment, the second node 20 can compress the raw visual data acquired by the visual acquisition unit 30 (such as a camera), and then transmit the compressed raw visual data as first visual data to the first node 10 via the second transmission unit 22 based on the first transmission protocol. This first visual data can then be centrally processed by the first image signal processing unit 11 of the first node 10, eliminating the need for the second node 20 to perform excessive processing. This results in better real-time transmission of the first visual data and a lower data processing burden on the second node 20. Furthermore, when not necessary, it is not necessary to set up an image signal processing unit for one or more nodes, including the second node 20, to reduce system costs. In addition, since the raw visual data is compressed before transmission, the transmission efficiency of the first visual data can be further improved.

[0070] Optionally, the second node 20 can transmit the compressed raw visual data as first visual data to the first node 10 via the second transmission unit 22 based on the first proprietary transmission protocol. It should be understood that transmitting the raw visual data (i.e., the first visual data) based on the first proprietary transmission protocol does not require data encapsulation by a processor (e.g., an MCU) based on a standard protocol, meaning it does not require processing based on higher-level protocols above the physical layer. Instead, it can be directly transmitted and processed at the physical layer, ensuring the reliability of data transmission.

[0071] In some examples, the second node 20 may also include a second image signal processing unit, which may be configured to compress visual data. In this mode, the original visual data may be compressed by the second image signal processing unit, and the second node 20 may transmit the compressed original visual data as first visual data to the first node 10 through the second transmission unit 22 based on the first transmission protocol (e.g., the first proprietary transmission protocol).

[0072] In other examples, the second node 20 may not include the second image signal processing unit. The second node 20 may compress the original visual data using other compression algorithms. The second node 20 may transmit the compressed original visual data as the first visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22.

[0073] Optionally, such as Figure 2As shown, in some cases, the second node 20 in this embodiment can directly transmit the original visual data as the first visual data to the first node 10 based on the first transmission protocol (e.g., the first private transmission protocol) through the second transmission unit 22, or in other cases, it can also transmit the compressed original visual data as the first visual data to the first node 10 based on the first transmission protocol (e.g., the first private transmission protocol) through the second transmission unit 22. These two cases can be two non-conflicting functions of the second node 20.

[0074] In this embodiment of the disclosure, the first node 10 can receive first visual data from the second node 20 through the first transmission unit 12 based on the first transmission protocol (e.g., the first private transmission protocol), and perform image signal processing on the first visual data through the first image signal processing unit 11 to obtain second visual data.

[0075] Optionally, refer to Figure 1A and Figure 1B As shown, the first node 10 is also connected to the host 40, and can transmit the second visual data obtained after the first image signal processing unit 11 performs image signal processing on the first visual data to the host 40. The host 40 can apply the second visual data to achieve the required functions. For example, 360° surround view of the vehicle body, reversing camera, autonomous driving / assisted driving, etc. Correspondingly, the host 40 can be part of the 360° surround view system, reversing camera system, autonomous driving / assisted driving system, etc.

[0076] Optionally, image signal processing may include any processing method that meets the requirements. Optionally, the image signal processing method may include, but is not limited to, at least one of the following: Black Level (BL) processing (typically used to remove fixed offsets caused by dark current, ensuring consistent image performance in complete black), Lens Shade Correction (LSC) processing (typically used to correct brightness and color differences at image edges caused by lens optical characteristics), White Balance Gain (WB Gain) processing (typically used to adjust color balance in an image, ensuring that photos taken under different lighting conditions exhibit natural white), Bad Pixel Correction (BPC) processing (typically used to repair bad pixels or damaged pixels in an image), Denoising processing (typically used to remove noise from noisy signals or images), High Dynamic Range Fusion (HDR Fusion) processing (typically used to merge multiple images with different exposures into a single high dynamic range (HDR) image), and Chromatic Aberration correction. Color correction (CAC) processing (typically used for color difference correction in images), debayer processing (typically used for reconstructing full-color images from Bayer format), color correction processing (typically used to adjust colors in images or videos to eliminate color deviations caused by devices such as cameras, monitors, etc.), global tone mapping (GTM) processing, local tone mapping (LTM) processing (global / local tone mapping is typically used to display wide dynamic range images on display devices that do not have wide dynamic range), sharpening processing (typically used to enhance image sharpness and detail), chroma noise reduction (CNR) processing (typically used to reduce chroma noise introduced during image processing), gamma adjustment processing (typically used to non-linearly adjust pixel values ​​in images), format change processing (typically used to change the size, format, and color space of an image, etc.), image stabilization processing (typically used to reduce or eliminate jitter or judder in images or videos), compression and output (Compress and output). Output processing (typically used to compress images before output).It should be understood that the above-mentioned processing methods can all be implemented using algorithms in related technologies or other innovative algorithms, and the embodiments disclosed herein do not impose any limitations on them.

[0077] Optionally, the image signal processing performed by the first image signal processing unit 11 on the first visual data includes at least one of the following processing methods: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, HDR fusion processing, color difference correction processing, Debayer processing, color correction processing, global tone mapping processing, local tone mapping processing, sharpening processing, CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing. These various processing methods can make image signal processing more flexible to meet various visual processing needs and improve the processing effect of visual data.

[0078] For example, Figure 3A The diagram illustrates some schematics of the first and second nodes processing visual data. In some embodiments, the first visual data received by the first node 10 is the raw visual data. Figure 3A As shown, a predetermined full image signal processing flow can be performed on the first visual data (i.e., the raw visual data) to obtain the second visual data. The full image signal processing flow can include one or more arbitrary image signal processing methods (for example, refer to the processing methods described above for understanding). For example, in... Figure 3A The example full image signal processing flow can include the following processing methods executed sequentially: black level correction, lens shading correction, white balance gain processing, bad pixel correction, noise reduction, HDR fusion, white balance gain processing, color difference correction, debyer processing, color correction, global tone mapping, local tone mapping, noise reduction, sharpening & CNR processing, gamma correction, format change processing, image stabilization, compression, and output processing. By processing the first visual data according to the example full image signal processing flow, second visual data can be output for transmission to host 40 for application. Figure 3A As shown, in the aforementioned full image signal processing flow, the first six processing methods can be preprocessing flows, which can be part of the full image signal processing flow. However, it should be understood that this is only an example, and the preprocessing flow can also include fewer or more processing methods.

[0079] Optionally, the preprocessing procedure can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing a preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0080] For example, Figure 3B Further schematic diagrams illustrating the processing of visual data by the first and second nodes are shown. In some optional embodiments, the first visual data received by the first node 10 is compressed raw visual data, as follows: Figure 3B As shown, the first visual data can be decompressed to obtain the original visual data, and then a predetermined full image signal processing flow can be performed on the original visual data to obtain the second visual data. For example, in Figure 3B The example full image signal processing flow can include the following processing methods executed sequentially: black level correction, lens shading correction, white balance gain processing, bad pixel correction, denoising, HDR fusion, white balance gain processing, color difference correction, debyer processing, color correction, global tone mapping, local tone mapping, denoising, sharpening & CNR processing, gamma correction, format change processing, image stabilization, compression, and output processing. By decompressing the first visual data to obtain the original visual data, and then following the full image signal processing flow exampled above, second visual data can be output for transmission to host 40 for application. Figure 3B As shown, in the aforementioned full image signal processing flow, the first six processing methods can be preprocessing flows, which can be part of the full image signal processing flow. However, it should be understood that this is only an example, and the preprocessing flow can also include fewer or more processing methods.

[0081] Optionally, the preprocessing procedure can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing a preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0082] Optionally, the predetermined full image signal processing flow may include one or more of the processing methods described above, and each processing method may be used once or multiple times (e.g., Figure 3A and Figure 3B The example shown uses multiple steps for white balance gain processing and noise reduction processing, as long as the processing requirements of the visual data are met.

[0083] In some optional embodiments, the second node 20 and the first node 10 are adjacent nodes. The second node 20 is specifically used to: obtain the target data packet through the second transmission unit 22 based on the first transmission protocol, write at least a portion of the first visual data into the target data packet, and transmit the target data packet to the first node 10 based on the first transmission protocol.

[0084] Optionally, the target data packet obtained by the second node 20 may be a data packet generated by the second node 20, or the target data packet may be a data packet received by the second node 20 based on the first transmission protocol. This embodiment of the present disclosure does not limit this.

[0085] Optionally, the second node 20 can obtain the target data packet based on the first private transmission protocol through the second transmission unit 22, write at least a portion of the first visual data into the target data packet, and transmit the target data packet to the first node 10 based on the first private transmission protocol. Optionally, the target data packet obtained by the second node 20 can be a data packet generated by the second node 20 based on the first private transmission protocol, or the target data packet can be a data packet received by the second node 20 based on the first private transmission protocol. This embodiment of the present disclosure does not limit this.

[0086] It should be understood that the data in the embodiments of this disclosure may be transmitted and processed based on a first private transmission protocol, that is, based on the physical layer for transmission and / or processing. However, this does not mean that it only performs the functions that the physical layer can perform as defined in the standard protocol. For example, in the standard protocol, the function of the physical layer is to convert the frame signals of higher layers (e.g., the data link layer) above the physical layer into electrical or optical signals that can be transmitted on the physical transmission medium, and to convert the electrical or optical signals received from the physical transmission medium into bit streams for processing by higher layers (e.g., the data link layer). The data transmission and processing in the vision processing system of this disclosure are implemented based on the physical layer of the target node, which means that the data transmission and processing scheme in the vision processing system of the embodiments of this disclosure does not go through higher layers (e.g., the data link layer, network layer, etc.) above the physical layer as defined in standard physical communication. Instead, it is based on a set of private protocols defined by the physical layer, which enables it to perform data transmission and processing other than some functions that the physical layer can perform as defined by the standard protocol.

[0087] For example, in some optional embodiments, the second node 20 is the data packet initiating node, and the second node 20 can generate the target data packet based on the first transmission protocol through the second transmission unit 22. Alternatively, in other optional embodiments, the second node 20 is not the data packet initiating node, and the second node 20 can obtain the target data packet transmitted from neighboring nodes other than the first node 10 based on the first transmission protocol through the second transmission unit 22. Thus, the second node 20 can effectively obtain the target data packet based on the first transmission protocol, so as to send the first visual data to the first node 10 through the target data packet based on the first transmission protocol.

[0088] For example, taking the first transmission protocol as a first private transmission protocol, optionally, if the second node 20 is the data packet initiating node, then the second node 20 can generate the target data packet based on the first private transmission protocol through the second transmission unit 22. Alternatively, optionally, if the second node 20 is not the data packet initiating node, then the second node 20 can obtain the target data packet transmitted from adjacent nodes other than the first node 10 through the second transmission unit 22 based on the first private transmission protocol. Thus, the second node 20 can effectively obtain the target data packet based on the first private transmission protocol, so as to send the first visual data to the first node 10 through the target data packet based on the first private transmission protocol, realizing the transmission of visual data based on the physical layer.

[0089] The initiating node of the target data packet can be selected as needed. Under different circumstances, the initiating node of the target data packet can be set to any node other than the first node 10 among multiple nodes, so as to meet the needs of visual data transmission under different conditions (such as different network topologies, different numbers of nodes, etc.).

[0090] For example, optionally, refer to Figure 1A In a daisy-chain network where multiple nodes are connected, the node initiating the data packet can be the last node in the daisy chain. For example... Figure 1A The middle section is a double daisy chain, consisting of a right chain (which includes a master node and child nodes a, b, c, and d) and a left chain (which includes a master node and child nodes g, f, and e). This allows for two packet initiating nodes: child node d and child node e for the right and left chains, respectively. These two initiating nodes accommodate the visual data transmission needs of the nodes in the daisy chain network. Alternatively, if multiple nodes are connected in a single daisy chain network, only one node needs to be designated as the target packet initiating node.

[0091] For example, refer to Figure 1BIn the ring network formed by multiple connected nodes shown, the packet initiating node can be any one or two nodes other than the first node 10 (the master node). In one example, the packet initiating node can be set to one node, such as child node a or child node g. Then, the target packet can be transmitted to the first node in one transmission direction (taking child node g as the packet initiating node as an example, the transmission direction can be: child node g → child node f → child node e → child node d → child node c → child node b → child node a → master node (i.e., the first node 10). If the packet initiating node is set to child node a, then it is child node a → child node b → child node c → child node d → child node e → child node f → child node g → master node). In other examples, the data packet initiating node can be set to two nodes, such as child node d or child node e. The transmission directions between the target data packets initiated by child node d and child node e can be different (for example, the transmission direction of the target data packet initiated by child node d is: child node d → child node c → child node b → child node a → master node (i.e., first node 10), while the transmission direction of the target data packet initiated by child node e is: child node e → child node f → child node g → master node (i.e., first node 10)). This is to better accommodate the visual data transmission needs of each node in a ring network. Of course, the above applies to… Figure 1A and Figure 1B The descriptions provided are for illustrative purposes only and are not intended to limit the scope of the embodiments disclosed herein.

[0092] For example, taking the first transmission protocol as the first private transmission protocol as an example, refer to Figure 1AIn the daisy-chain network shown, in the right chain, child node a (second node 20) and master node (first node 10) are adjacent nodes. Child node a may not be the initiating node of the target data packet. Child node a can obtain the target data packet from the downstream node of the right chain, i.e., child node b, through its second transmission unit 22 based on the first private transmission protocol (for example, the target data packet may be generated by child node d). It writes at least part of the first visual data into the target data packet and transmits the target data packet upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data in it. For example, in the left chain, child node g (second node 20) and master node (first node 10) are adjacent nodes. Child node g may not be the initiating node of the target data packet. Child node g can obtain the target data packet from the downstream node, i.e., child node f, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node e). It writes at least part of the first visual data into the target data packet and transmits the target data packet upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the first visual data in it.

[0093] For example, taking the first transmission protocol as the first proprietary transmission protocol, refer to... Figure 1AAs shown, assuming that there are no child nodes b, c, d, e, and f in this network, and only child nodes a, g, and the master node (first node 10), then in the right chain, child node a (second node 20) and the master node (first node 10) are adjacent nodes, and child node a is the last node. Child node a can be the data packet initiating node of the target data packet. Child node a can initiate a data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least part of the first visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the first visual data in it. In the left chain, the child node g (second node 20) and the master node (first node 10) are adjacent nodes, and the child node g is the last node. The child node g can be the data packet initiating node of the target data packet. The child node g can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, and thus obtain the target data packet. At least part of the first visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the first visual data in it.

[0094] It should be understood that other situations can be deduced by analogy to the above examples, and will not be elaborated further here.

[0095] It is understood that, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are adjacent nodes, at least a portion of the first visual data can be written into the target data packet obtained based on the first transmission protocol through the second transmission unit 22, and the first visual data can be effectively transmitted to the first node 10 by transmitting the target data packet to the first node 10 based on the first transmission protocol.

[0096] Optionally, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are adjacent nodes, at least a portion of the first visual data can be written into the target data packet obtained based on the first private transmission protocol through the second transmission unit 22, and the first visual data can be effectively transmitted to the first node 10 based on the first private transmission protocol by transmitting the target data packet to the first node 10 based on the first private transmission protocol. Therefore, the visual data is directly transmitted based on the physical layer, and it is not necessary to encapsulate the data based on the standard protocol through the processor (e.g., MCU, etc.), that is, it is not necessary to process it based on the higher-level protocol above the physical layer, but to transmit and process it directly based on the physical layer, which can ensure the reliability of data transmission.

[0097] In some alternative embodiments, the second node 20 and the first node 10 are not adjacent nodes. The second node 20 is specifically used to: obtain a target data packet through the second transmission unit 22 based on the first transmission protocol, write at least a portion of the first visual data into the target data packet, and transmit the target data packet to the first node 10 through at least one node between the second node 20 and the first node 10 based on the first transmission protocol.

[0098] Optionally, the second node 20 can obtain the target data packet based on the first private transmission protocol through the second transmission unit 22, write at least a portion of the first visual data into the target data packet, and transmit the target data packet to the first node 10 through at least one node between the second node 20 and the first node 10 based on the first private transmission protocol.

[0099] For example, taking the first transmission protocol as the first private transmission protocol as an example, refer to Figure 1AIn the daisy-chain network shown, in the right chain, child node b (second node 20) is connected to the master node (first node 10) via child node a. Therefore, child node b and the master node are not adjacent nodes. Child node b may not be the initiating node of the target data packet. Child node b can obtain the target data packet from the downstream node, i.e., child node c, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node d). It writes at least part of the first visual data into the target data packet and transmits the target data packet upstream to child node a based on the first private transmission protocol. For example, it can be transmitted to the physical layer of child node a. Then, through the second transmission unit 22 of child node a, it is forwarded to the master node (first node 10) based on the first private transmission protocol. For example, it can be forwarded to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data in it. For example, in the left chain, child node f (second node 20) is connected to the master node (first node 10) by child node g. Therefore, child node f and the master node are not adjacent nodes. Child node f may not be the initiating node of the target data packet. Child node f can obtain the target data packet from the downstream node, i.e., child node e, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node e). It writes at least part of the first visual data into the target data packet and transmits the target data packet upstream to child node g based on the first private transmission protocol. For example, it can be transmitted to the physical layer of child node g. Then, child node g forwards the data packet to the master node (first node 10) through its second transmission unit 22 based on the first private transmission protocol. For example, it can be forwarded to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data in it.

[0100] For example, taking the first transmission protocol as the first proprietary transmission protocol, refer to... Figure 1AIn the daisy-chain network shown, in the right chain, child node d (second node 20) is connected to the master node (first node 10) via child nodes a, b, and c. Therefore, child node d and the master node are not adjacent nodes, and child node d is the last node. Child node d can be the initiating node of the target data packet. Child node d can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least a portion of the first visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the child node based on the first private transmission protocol. The physical layer of node c is forwarded to the physical layer of node b via its second transmission unit 22 based on the first private transmission protocol. Then, it is forwarded to the physical layer of node a via its second transmission unit 22 based on the first private transmission protocol. Finally, it is forwarded to the physical layer of the master node (first node 10) via its second transmission unit 22 based on the first private transmission protocol. The master node (first node 10) receives the target data packet and reads the first visual data in it through its first transmission unit 12 based on the first private transmission protocol. For example, in the left chain, child node e (second node 20) is connected to the master node (first node 10) by child nodes f and g. Therefore, child node e and the master node are not adjacent nodes, and child node e is the last node. Child node e can be the data packet initiating node of the target data packet. Child node e can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least part of the first visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the physical layer of child node f based on the first private transmission protocol. After being forwarded by child node f through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of child node g. Then, after being forwarded by child node g through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data in it.

[0101] For example, taking the first transmission protocol as a first proprietary transmission protocol, refer to... Figure 1BIn the ring network shown, it is assumed that child node d and child node e are the data packet initiating nodes of two target data packets, respectively. Child node d (second node 20) is not an adjacent node to the master node (first node 10). Child node d can initiate data packets based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least a portion of the first visual data can be written into the target data packet, and the target data packet is transmitted upstream to the physical layer of child node c based on the first private transmission protocol. From child node c, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of child node b. Then, from child node b, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of child node a. Finally, from child node a, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data within it. Child node e (second node 20) and master node (first node 10) are not adjacent nodes. Child node e can initiate a data packet based on the first private transmission protocol through its second transmission unit 22, thereby obtaining the target data packet. At least part of the first visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the physical layer of child node f based on the first private transmission protocol. After being forwarded by child node f through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of child node g. Then, after being forwarded by child node g through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of master node (first node 10). Master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the first visual data in it.

[0102] It should be understood that the other cases can be deduced from the above examples, and will not be elaborated further here.

[0103] It is understood that, in the embodiments of this disclosure, when the second node 20 and the first node 10 are not adjacent nodes, at least a portion of the first visual data can be written into the target data packet obtained based on the first transmission protocol through the second transmission unit 22, and the first visual data can be effectively transmitted to the first node by transmitting the target data packet to the first node 10 sequentially through at least one node based on the first transmission protocol.

[0104] Optionally, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are not adjacent nodes, the first visual data can be effectively transmitted to the first node by writing at least a portion of the first visual data into a target data packet obtained based on the first private transmission protocol, and by transmitting the target data packet sequentially to the first node 10 through at least one node based on the first private transmission protocol. Therefore, the visual data is directly transmitted based on the physical layer, and it is not necessary to encapsulate the data based on a standard protocol by a processor (such as an MCU), that is, it is not necessary to process it based on a higher-level protocol above the physical layer, but to transmit and process it directly based on the physical layer, which can ensure the reliability of data transmission.

[0105] In some alternative embodiments, such as Figure 2 As shown, if the second node 20 is disconnected from the visual acquisition unit 30, or if the visual acquisition unit 30 connected to the second node 20 is not enabled, the second node 20 enters a low-power mode.

[0106] The second node 20 is disconnected from the vision acquisition unit 30, meaning the connection between the vision acquisition unit 30 and the second node 20 is severed. In this situation, the second node 20 cannot obtain raw visual data. The vision acquisition unit 30 connected to the second node 20 is not enabled. Although connected to the second node 20, the vision acquisition unit 30 does not acquire visual data; for example, it may be in a disabled, powered-off, or faulty state. Therefore, the second node 20 also cannot obtain raw visual data in this case. In both of these situations, the second node 20 enters a low-power mode, which helps reduce the overall power consumption of the vision processing system 100.

[0107] Optionally, when the second node 20 is in low-power mode, if the second node 20 receives visual data from an adjacent node through the second transmission unit 22 based on the first transmission protocol, then the second node 20 forwards the visual data from the adjacent node to the next adjacent node along a first direction, where the first direction is the direction from the second node 20 to the first node 10. This allows the node in this embodiment to effectively transmit visual data while reducing power consumption in low-power mode.

[0108] For example, taking the first transmission protocol as a first proprietary transmission protocol, when the second node 20 is in low-power mode, if the second node 20 receives visual data from an adjacent node through the second transmission unit 22 based on the first proprietary transmission protocol, then the second node 20 forwards the visual data from the adjacent node to the next adjacent node along the first direction through the second transmission unit 22 based on the first proprietary transmission protocol. Thus, the second node 20 in this embodiment can effectively reduce power consumption in low-power mode while achieving visual data transmission at the physical layer, facilitating reliable data transmission.

[0109] For example, taking the first transmission protocol as the first private transmission protocol as an example, Figure 1A Taking the daisy-chain network as an example, taking the right chain child node a (second node 20) as an example, if it is not connected to the visual acquisition unit 30 (that is, disconnected from the visual acquisition unit 30) or the visual acquisition unit 30 it is connected to is not enabled, then if child node a obtains visual data from the adjacent child node b through the second transmission unit 22 based on the first private transmission protocol (for example, it may be receiving a target data packet containing the first visual data received from child node b), it can forward the visual data to the adjacent next node along the first direction, that is, the master node (first node 10). The first transmission unit 12 of the master node (first node 10) can receive the visual data based on the first private transmission protocol. For example, taking the right-chain child node b (second node 20) as an example, if it is not connected to the visual acquisition unit 30 (that is, disconnected from the visual acquisition unit 30) or the visual acquisition unit 30 it is connected to is not enabled, then if the child node b obtains visual data from the adjacent child node c through the second transmission unit 22 based on the first private transmission protocol (for example, it may be receiving a target data packet containing the first visual data received from the child node b), it can forward the visual data to the adjacent next node, i.e., child node a, along the first direction, so that it can be forwarded again through child node a to the master node (first node 10) for processing.

[0110] The first transmission protocol is a first private transmission protocol. In some optional embodiments, the second node 20 may be assigned at least one node identifier, and the target data packet records the target identifier. The second node 20 may be used to: after obtaining the target data packet through the second transmission unit 22 based on the first private transmission protocol, determine the target identifier in the target data packet, and in response to the existence of a node identifier with the same target identifier among at least one node identifier, write at least a portion of the first visual data into the target data packet through the second transmission unit 22 based on the first private transmission protocol.

[0111] Therefore, in this embodiment of the present disclosure, by assigning at least one node identifier to the second node 20, and when there is a node identifier among the at least one node identifier that is the same as the target identifier recorded in the obtained target data packet, at least a portion of the first visual data is written into the target data packet by the second transmission unit 22 based on the first private transmission protocol. This makes the transmission of visual data from the second node 20 to the first node 10 by the second transmission unit 22 based on the first private transmission protocol more orderly, and also facilitates the first node 10 to schedule and process visual data from multiple nodes.

[0112] Optionally, different nodes receiving data packets initiated by the same data packet initiating node are assigned different node identifiers. This ensures that the visual data written in each target data packet is the visual data of a single node, thereby guaranteeing the orderly transmission of visual data and facilitating the first node 10 to schedule and process visual data from multiple nodes.

[0113] The target identifier and node identifier in the embodiments of this disclosure can be in any form, such as including but not limited to text identifiers, symbol identifiers, etc.

[0114] In some optional embodiments, the node identifier is the node's frame number, and the target identifier is the data packet's frame number, wherein data packets sent by the same data packet initiating node within different time slots divided in the same preset period record different frame numbers.

[0115] Optionally, multiple time slots divided within the same preset period can be of equal length. The preset period can be set as needed, for example, it can be 1 second.

[0116] Optionally, different nodes receiving data packets initiated by the same data packet initiating node are assigned different frame numbers. This ensures that the visual data written in each target data packet is the visual data of a single node, thereby guaranteeing the orderly transmission of visual data and facilitating the first node 10 to schedule and process visual data from multiple nodes.

[0117] To address the latency jitter issue (latency jitter refers to the variable time interval between data packets arriving at the receiving end during network transmission. This variation can cause problems when the receiving end processes data, such as discontinuous video feeds, which can negatively impact user experience), this embodiment allocates a predetermined time slot for transmitting visual data to each child node (which can be the second node 20) except for the master node (first node 10). For example, a preset period (e.g., 1 second) can be divided into N equal time slots, where N can be preset, for example, to 256, 512, 1024, etc. The following example uses N=1024. The initiating node can send one data packet per time slot. The data packet records its frame number, which increments sequentially. For example, in a preset period of N = 1024 time slots, the frame number of a data packet sent in the first time slot is 1, the frame number of a data packet sent in the second time slot is 2, and so on, with the frame number of a data packet sent in the 1024th time slot being 1024. That is, data packets sent in different time slots within the same preset period record different frame numbers. At least one frame number can be assigned to each child node, and different child nodes are assigned different frame numbers.

[0118] For example, for Figure 1A The right chain of the daisy-chain network includes four child nodes: child node a, child node b, child node c, and child node d (in this example, all four child nodes are the second node 20). The preset period can then be divided into N = 1024 time slots. Refer to the frame number allocation table in Table 1 below, and... Figure 4 The diagram shown illustrates the time slot division of a preset period and the frame number allocation to child nodes, which can be used to understand the frame number allocation of nodes:

[0119] Table 1

[0120] Allocated frame number child node a 1，2，3，11，12，13，21，22，23，…… child node b 4，5，14，15，24，25，…… Child node c 6，7，16，17，26，27，…… Child node d 8，9，18，19，28，29，…… Transmission of other information 10，20，30，……

[0121] As mentioned above, Figure 1A In a daisy-chain topology, the right chain can be initiated by the last child node d. Within a preset period of N = 1024 time slots, this child node will periodically initiate target data packets, including the frame number, in sequence from 1 to 1024, with the frame number automatically incrementing. For example... Figure 5 A schematic diagram illustrating an example of visual data transmission is shown, which can be combined with... Figure 5As shown, if the current time slot is the 5th time slot, the frame number of the currently initiated target data packet is 5. According to Table 1, the frame number of child node d does not contain "5". Therefore, child node d can determine that the target data packet is not a data packet used to transmit its own visual data. It does not write the first visual data into the target data packet through the second transmission unit 22, but only sends an empty data packet to child node c through its second transmission unit 22 based on the first private transmission protocol. Child node c receives the empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol. It also finds that the target data packet is not a data packet used to transmit its own visual data. Therefore, it does not write the first visual data into the target data packet through its second transmission unit 22, but instead forwards the target data packet to the physical layer of child node b through its second transmission unit 22 based on the first private transmission protocol. Child node b receives the empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol. If node b can determine that its node identifier contains a node identifier "5" that is the same as the target identifier, then it can write at least part of the first visual data into the target data packet through its second transmission unit 22 based on the first private transmission protocol, and then transmit the target data packet to the physical layer of child node a based on the first private transmission protocol. Child node a receives an empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol and finds that the target data packet is not a data packet used to transmit its own visual data. Therefore, it does not write the first visual data into the target data packet through its second transmission unit 22, but forwards the target data packet to the physical layer of the master node (first node 10) through its second transmission unit 22 based on the first private transmission protocol. After the master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol, it can determine the visual data of child node b transmitted by the target packet according to the frame number recorded in the target data packet, and then perform the corresponding processing. Following a similar process, target data packets with frame numbers 1 to 1024 are transmitted in N=1024 time slots of a preset period. After the maximum value of 1024 is reached, the next preset period begins. In the first time slot of the next preset period, the frame number of the target data packet returns to 1, and the transmission of visual data continues.

[0122] It should be understood that Figure 1A The transmission method of the left chain in the daisy chain network is similar to that of the right chain mentioned above. The difference is that the data packet initiating node is the child node e, and the frame number allocation of each node can be different from that of the right chain.

[0123] It should also be understood that Figure 1B The ring network can also be constructed as described above. Figure 1A The transmission method of the right chain in a daisy-chain network can be understood in the same way. For example, the Figure 1BA ring network can include two data packet initiating nodes, such as child node d and child node e. It can actually be decomposed into a structure resembling... Figure 1A The chrysanthemum chain network consists of right-chain and left-chain forms.

[0124] Based on this, through the above-mentioned optional solutions in this embodiment of the present disclosure, each node of the vision processing system 100 (including the second node 20) can transmit its vision data within a predetermined time slot of a preset period, thereby ensuring the processing latency and timely transmission of vision data of each node, and enabling the main node of vision processing (the first node 10) to conveniently schedule and process multiple channels of vision data. In addition, since each time slot of the preset period corresponds to the vision data transmission of one node, it is not only easier to achieve high-precision synchronization of vision data, but also to effectively improve the latency jitter problem of vision data transmission.

[0125] As shown in Table 1, the transmission of other information also corresponds to frame numbers. That is, the transmission of other information of the child node needs to be transmitted to the master node (first node 10) through data packets with corresponding frame numbers in certain time slots of a preset period. This ensures the orderly transmission of visual data and other information. In this embodiment of the disclosure, other information can be information other than visual data. For example, other information may include, but is not limited to: audio data, control data for reading and writing registers, debugging data, etc.

[0126] In some optional embodiments, the data packet includes a frame header and a data block, with the frame number of the data packet recorded in the frame header. Thus, the second node 20 can read the frame header of the target data packet to determine its frame number, and can, when needed, write at least a portion of the first visual data into the data block via the second transmission unit 22 to facilitate the transmission of the visual data.

[0127] In some optional embodiments, the second node 20 is further configured to: in response to each node identifier being different from the target identifier, transmit the target data packet to the next adjacent node of the second node 20 along a first direction based on a first private transmission protocol via the second transmission unit 22, wherein the first direction is the direction from the second node 20 to the first node 10.

[0128] This optional embodiment can be referred to the foregoing. Figure 1A To understand this, we can use the example of the left chain, which will not be elaborated here. Optionally, if the second node 20 determines that the frame number of each node is different from the frame number of the data packet, the target data packet can be transmitted to the next adjacent node of the second node 20 along the first direction based on the first private transmission protocol.

[0129] Based on this, through the above-mentioned optional solutions in this embodiment of the present disclosure, each node of the vision processing system 100 (including the second node 20) can transmit its vision data within a predetermined time slot of a preset period, thereby ensuring the processing latency and timely transmission of vision data of each node, and enabling the main node of vision processing (the first node 10) to conveniently schedule and process multiple channels of vision data. In addition, since each time slot of the preset period corresponds to the vision data transmission of one node, it is not only easier to achieve high-precision synchronization of vision data, but also to effectively improve the latency jitter problem of vision data transmission.

[0130] In some optional embodiments, the first node 10 is further configured to: transmit frame number configuration information to the second node 20 through the first transmission unit 12 based on the first private transmission protocol; the second node 20 is further configured to: receive the frame number configuration information through the second transmission unit 22 based on the first private transmission protocol, and configure the node's frame number according to the frame number configuration information.

[0131] Therefore, the first node 10 can transmit frame number configuration information to the second node 20 through its first transmission unit 12 based on the first private transmission protocol. This allows the second node 20 to receive the frame number configuration information through its second transmission unit 22 based on the first private transmission protocol. Consequently, the second node 20 can effectively configure its frame number according to the frame number configuration information, enabling it to transmit its visual data within a predetermined time slot within a preset period. This ensures the processing latency and timely transmission of visual data from each node, facilitating the first node 10's scheduling of the second node 20's visual data transmission and improving the latency jitter problem in visual data transmission. Furthermore, transmitting frame number configuration information based on the first private transmission protocol eliminates the need for data encapsulation based on standard protocols by a processor (e.g., an MCU), meaning it does not require processing based on higher-level protocols above the physical layer. Instead, it can be directly transmitted and processed at the physical layer, ensuring the reliability of data transmission.

[0132] For example, a user can configure the first node 10. The first node 10 initiates a downlink data packet including frame number configuration information based on the first private transmission protocol, and transmits the downlink data packet to the second node 20 based on the first private transmission protocol, so that the second node 20 can obtain the frame number configuration information to configure the node's frame number.

[0133] Optionally, the following can be combined with Figure 6A and Figure 6BThis section introduces the advantages of the vision processing system 100 in this solution, which transmits visual data based on a first proprietary transmission protocol. Currently, the mainstream solutions for in-vehicle vision data transmission are based on Ethernet and Controller Area Network (CAN) buses. However, with the development of intelligent technology, the CAN bus cannot meet the requirements for transmission speed and latency. While current in-vehicle Ethernet uses traditional switch-based communication, which is mature and reliable, it requires numerous switches to connect multiple Electronic Control Units (ECUs) within the vehicle, resulting in high cost, complex implementation, and an inability to support more flexible node interconnection scenarios and complex communication structures. Furthermore, it also suffers from significant transmission latency. (Refer to...) Figure 6A As shown, an onboard ECU typically consists of a microcontroller unit (MCU), memory (such as read-only memory (ROM) or random access memory (RAM)), input / output interfaces (I / O interfaces), analog-to-digital converters (A / D converters), and large-scale integrated circuits for shaping and driving functions. For example, refer to... Figure 6A The diagram illustrates a simplified schematic of an example of an in-vehicle vision transmission system in the related art. It shows multiple ECUs connected to a switch (Ethernet switch), with a simplified representation of the structure of one ECU. The MCU within the ECU is responsible for local data processing, such as collecting visual data from cameras, encapsulating it into Ethernet packets, and sending it to a remote regional central processing unit via the switch (which can be used for data forwarding). Conversely, the MCU can receive data from the switch, parse it according to the Ethernet protocol, and perform local control. Figure 6A As shown, both sending and receiving data require passing through the Ethernet physical layer and being processed by the MCU. Data is transmitted using the standard Ethernet protocol, resulting in significant latency due to data processing. With the rapid development of intelligent vehicles, the number of ECUs in vehicles is increasing, with high-end vehicles containing dozens or even hundreds of ECUs. Clearly, the increasing number of ECUs leads to a significant increase in cost, and consequently, a corresponding increase in latency.

[0134] For example, refer to Figure 6B As shown, it illustrates an example data packet format. The above... Figure 6A This is an example of the ECU transmitting the acquired visual data to the central processing unit in a remote area, following the standard Ethernet transmission protocol, and the data needs to be encapsulated. Figure 6B The data packet format shown (this) Figure 6BThe example packet format includes a Media Access Control (MAC) header, an Internet Protocol (IP) header, a User Datagram Protocol / Transmission Control Protocol (UDP / TCP) header, an application layer header, visual data, and Cyclic Redundancy Check (CRC) 32 checksum. In other words, data transmitted via standard Ethernet typically undergoes processing through multiple layers of protocols, including the physical layer, MAC layer, IP layer, UDP / TCP layer, and application layer, to encapsulate the raw visual data into... Figure 6B The data packet format shown is problematic. This incurs significant overhead when dealing with small amounts of visual data. Firstly, it significantly reduces network bandwidth utilization because visual data constitutes a small proportion of the entire data packet. Secondly, it results in substantial transmission latency. For example, a typical transmission latency value includes: the latency for the MCU to receive the visual data (approximately 1µs), the latency for the MCU to encapsulate the visual data into a data packet (the example packet here is a standard Ethernet packet) (approximately 10µs), the latency for the data packet to be transmitted between two switches (approximately 500µs), and the additional latency for the regional central processing unit to parse the data packet and obtain the visual data (approximately 5µs). Such high transmission latency is increasingly failing to meet the demands of modern automotive visual data transmission.

[0135] It is understood that, through the vision processing system 100 provided in the first aspect of the embodiments of this disclosure, since the second node 20 in the vision processing system 100 can transmit the first visual data through the second transmission unit 22 based on the first private transmission protocol, and the first node 10 can receive the first visual data through the first transmission unit 12 based on the first private transmission protocol, the vision data is directly transmitted and / or processed based on the physical layer. This eliminates the need for data encapsulation based on standard protocols by a processor (e.g., MCU), meaning it does not require processing based on higher-level protocols above the physical layer. Instead, it directly transmits and processes the data based on the physical layer, ensuring the reliability of data transmission. Furthermore, when transmitting visual data based on the physical layer, this solution does not require a switch to transmit the visual data, reducing the demand for processors (e.g., MCUs) in the network during visual data transmission. Therefore, it effectively reduces the transmission latency of visual data and further reduces costs by decreasing the number of switches and processors (e.g., MCUs). Thus, this solution can further and effectively improve the data transmission and processing performance of the vision processing system.

[0136] According to a second aspect of the embodiments of this disclosure, a vision processing system 100 is provided. (Refer to...) Figure 7A , Figure 7B As shown, the vision processing system 100 includes: multiple nodes connected in a network, including at least a first node 10 and a second node. The second node 20 includes a second image signal processing unit 21 and a second transmission unit 22. The second node 20 is used to: configure the second image signal processing unit 21 according to functional configuration information; obtain raw visual data; obtain third visual data through the functionally configured second image signal processing unit 21; and transmit the third visual data to the first node 10 through the second transmission unit 22 based on a first transmission protocol. The first node 10 includes a first transmission unit 12, and the first node 10 is used to: receive the third visual data from the second node 20 through the first transmission unit 12 based on the first transmission protocol.

[0137] The vision processing system 100 provided in this embodiment includes multiple nodes connected in a network, with at least a first node 10 and a second node 20 among them. The second node 20 can configure its second image signal processing unit 21 according to functional configuration information. The second node 20 can obtain raw visual data and then obtain third visual data through the configured second image signal processing unit 21. The third visual data can be transmitted to the first node 10 via the second transmission unit 22 based on a first transmission protocol. The first node 10 can receive the third visual data through the first transmission unit 12, thereby effectively realizing the transmission and processing of visual data. Furthermore, the processing function of the second image signal processing unit 21 can be configured as needed, making the vision processing system more flexible and able to meet various visual data processing requirements. Therefore, this solution can effectively improve the data transmission and processing performance of the vision processing system.

[0138] It should be understood that the visual processing system 100 of the second aspect of this disclosure can be applied to any scenario. For example, it can be adapted to scenarios involving the transmission and processing of visual data from multiple cameras. As an example, it can be applied to scenarios involving the transmission and processing of visual data in vehicles, as well as to scenarios involving the transmission of visual data in homes or security systems.

[0139] In the embodiments of this disclosure, the image signal processing unit and the transmission unit can be separate chips or devices that are integrated into the node. Alternatively, the transmission unit can be a separate chip or device that integrates the functions of the image signal processing unit. Or, the image signal processing unit can be a separate chip or device that integrates the functions of the transmission unit. The node can be a physical concept such as a module, electronic device, integrated chip, etc. The embodiments of this disclosure do not limit the physical entities of the node, transmission unit, and image signal processing unit.

[0140] In some alternative embodiments, the image signal processing unit can be implemented as an image signal processor (ISP). In some embodiments, a node can be any module, device, or chip capable of implementing the solutions of the embodiments of this disclosure. For example, in some embodiments, each of a plurality of nodes includes at least one chip for data transmission in the network. In some embodiments, a node can also be a chip, that is, a chip can be directly used as a node.

[0141] In this embodiment of the disclosure, visual data may include at least one of image data and video data. For example, visual data may be understood as video data in the following text.

[0142] In this embodiment of the disclosure, the first transmission protocol can be any suitable data transmission protocol. The first transmission protocol can be an existing standard protocol, such as Ethernet or other existing protocols. Alternatively, the first transmission protocol can also be a proprietary protocol.

[0143] In some optional embodiments, the first transmission protocol is a first proprietary transmission protocol, wherein the first proprietary transmission protocol implements data transmission and / or processing based on the physical layer. For example, the second node 20 can transmit the third visual data to the first node 10 based on the first proprietary transmission protocol through the second transmission unit 22, and the first node 10 can receive the third visual data from the second node 20 based on the first proprietary transmission protocol through the first transmission unit 12.

[0144] It should be understood that since the second node 20 in the vision processing system 100 can transmit third visual data through the second transmission unit 22 based on the first proprietary transmission protocol, and the first node 10 can receive third visual data through the first transmission unit 12 based on the first proprietary transmission protocol, visual data can be directly transmitted and / or processed at the physical layer. This eliminates the need for data encapsulation based on standard protocols by a processor (e.g., an MCU), meaning processing is not required based on higher-level protocols above the physical layer. Instead, data is transmitted and processed directly at the physical layer, ensuring reliable data transmission. Furthermore, this solution eliminates the need for switches when transmitting visual data at the physical layer, reducing the demand for processors (e.g., MCUs) in the network during visual data transmission. This effectively reduces the transmission latency of visual data and further reduces costs by decreasing the number of switches and processors (e.g., MCUs). Therefore, this solution can further and effectively improve the data transmission and processing performance of the vision processing system.

[0145] The physical layer is the lowest layer in network communication. It should be understood that the data in the embodiments of this disclosure can be transmitted and processed based on a first private transmission protocol, that is, transmitted and / or processed based on the physical layer. However, this does not mean that it only performs the functions that the physical layer can perform as defined in the standard protocol. For example, in the standard protocol, the function of the physical layer is to convert the frame signals of higher layers (e.g., the data link layer) above the physical layer into electrical or optical signals that can be transmitted on the physical transmission medium, and to convert the electrical or optical signals received from the physical transmission medium into bit streams for processing by higher layers (e.g., the data link layer). The data transmission and processing in the vision processing system of this disclosure are implemented based on the physical layer of the node, which means that the data transmission and processing scheme of the embodiments of this disclosure does not go through higher layers (e.g., the data link layer, network layer, etc.) above the physical layer as defined in standard physical communication. Instead, it is based on a set of private protocols defined by the physical layer, which enables it to perform data transmission and processing other than certain functions that the physical layer can perform as defined in the standard protocol.

[0146] For ease of explanation of the embodiments disclosed herein, the first transmission protocol may be referred to as the first proprietary transmission protocol in the following description.

[0147] In this embodiment of the disclosure, multiple nodes can be connected to form any type of network. For example, such as Figure 7A As shown, multiple nodes can be connected to form a daisy-chain network. Figure 7A The example shown is a double daisy-chain topology; in other embodiments, a single daisy-chain topology can also be used. For example... Figure 7B As shown, multiple nodes can be connected to form a ring network. The vision processing system 100 of this embodiment adopts a daisy-chain network or a ring network, resulting in lower data forwarding latency when transmitting visual data based on a first transmission protocol (e.g., a first proprietary transmission protocol).

[0148] Optionally, in the vision processing system 100 of this disclosure, when a node transmits data, including but not limited to visual data, based on a first transmission protocol (e.g., a first proprietary transmission protocol), the data transmission and processing can be implemented in real-time streaming. For example, taking the first transmission protocol as a first proprietary transmission protocol, a target node among multiple nodes (e.g., the second node 20) can perform real-time streaming processing of data packets (e.g., target data packets) based on the first proprietary transmission protocol through its transmission unit (e.g., the second transmission unit 22). It can also write the data to be transmitted (e.g., visual data) into the data packet when needed, and transmit the real-time streaming data packet to the next adjacent node (e.g., the first node 10 or other nodes) in real-time streaming. That is, real-time streaming processing of data packets means processing them immediately when they arrive at the target node, rather than waiting until the entire data packet is completely received before processing or transmission.

[0149] It should be understood that, because the nodes in this embodiment can perform real-time streaming data transmission and processing, for example, the node's transmission unit can transmit and process data packets separately in real-time streaming, enabling real-time processing of data in the received data packets. Data can also be written to the received data packets in real-time as needed; that is, transmission and processing are performed immediately after receiving a portion of the data packet. This effectively reduces data transmission latency and ensures that the data in the data packets can be transmitted and processed in real time. In contrast, related technologies based on switches or other software or wireless data packet transmissions require waiting for all data in the data packet to be completely received before transmission or processing, resulting in significant latency. Therefore, compared to related technologies, the visual processing system 100 of this solution has better data transmission and processing performance.

[0150] It should be noted that, Figure 7A and Figure 7B The number of nodes mentioned is merely an example to illustrate the embodiments of this disclosure, and the specific number can be set as needed. In the embodiments of this disclosure, multiple nodes can be connected through a physical transmission medium (such as shielded twisted pair cable, unshielded twisted pair cable, coaxial cable, etc., which are not specifically limited in this embodiment).

[0151] Optionally, for different types of network topologies, a master node can be specified among multiple nodes, and the nodes other than the master node can be called child nodes. It should be noted that a network can include more than two nodes, i.e., one master node and multiple child nodes; or, a network can also include only two nodes, i.e., one master node and one child node.

[0152] In this embodiment of the disclosure, the first node 10 among multiple nodes can be the master node, while the other nodes besides the first node 10 (including the second node 20) can all be child nodes (e.g., ...). Figure 7A and Figure 7B As shown, for ease of description below, each child node can be referred to as child node a to g). The plurality of nodes in this embodiment may include at least one second node 20. Optionally, refer to... Figure 1A and Figure 1B As shown, all nodes (child nodes) except for the first node 10 (master node) can be the second node 20.

[0153] In this embodiment of the disclosure, the second node 20 may include a second image signal processing unit 21 and a second transmission unit 22. The second node 20 can obtain raw visual data. Optionally, as... Figure 7A and Figure 7BAs shown, the second node 20 can be connected to the visual acquisition unit 30. The visual acquisition unit 30 can be used to acquire raw visual data. The visual data may include at least one of image data and video data. The second node 20 can obtain the raw visual data from the visual acquisition unit 30. For example, the visual acquisition unit 30 can be a camera or a video image sensor, etc. In the following description, the visual acquisition unit 30 can be exemplified as a camera. After the raw visual data is input to the second image signal processing unit 21 of the second node 20, third visual data can be obtained according to the functional configuration of the second image signal processing unit 21.

[0154] Optionally, the raw visual data can be visual data in Bayer RAW format. Each pixel in the Bayer domain contains only one of the three colors: red (R), green (G), and blue (B). Because the raw visual data is in Bayer RAW format, the visual processing system 100 of this solution can transmit visual data in the Bayer domain, resulting in less data transmission compared to traditional RGB data. Under the same bandwidth, it can support the transmission of visual data from more visual acquisition units 30, or the transmission of higher-resolution visual data.

[0155] In some optional embodiments, the signals sensed by the sensors of the visual acquisition unit 30 (such as a camera) can be analog signals, which can be converted into digital signals through analog-to-digital conversion, and then the raw visual data in Bayer RAW format can be obtained from the digital signals.

[0156] In this embodiment, the second node 20 can configure the second image signal processing unit 21 according to the functional configuration information, so that the second image signal processing unit 21 can implement different processing functions, or it can also implement a function that does not process visual data. Optionally, the functional configuration information can be stored locally on the second node 20. The functional configuration information can be obtained in any suitable manner, for example, it can be obtained online through a network, or it can be obtained by the user directly setting the second node 20, etc. Alternatively, it can also be transmitted from the first node 10 to the second node 20.

[0157] For example, in some optional embodiments, the first node 10 is further configured to: transmit functional configuration information to the second node 20 via the first transmission unit 12 based on the first transmission protocol; the second node 20 is further configured to: receive functional configuration information via the second transmission unit 22 based on the first transmission protocol.

[0158] Therefore, the first node 10 can transmit function configuration information to the second node 20 through the first transmission unit 12 based on the first transmission protocol. The second node 20 can receive the function configuration information through the second transmission unit 22 based on the first transmission protocol, so that the second node 20 can effectively configure the function of the second image signal processing unit 21 according to the function configuration information. Thus, the first node 10 can effectively schedule the visual data processing of the second node 20, so as to realize the flexible transmission and processing of visual data, making the function of the visual processing system more flexible and able to meet various visual data processing needs.

[0159] For example, in some optional embodiments, the first node 10 is further configured to: transmit function configuration information to the second node 20 via the first transmission unit 12 based on the first private transmission protocol; the second node 20 is further configured to: receive function configuration information via the second transmission unit 22 based on the first private transmission protocol. Thus, the first node 10 can transmit function configuration information to the second node 20 via the first transmission unit 12 based on the first private transmission protocol, and the second node 20 can receive function configuration information via the second transmission unit 22 based on the first private transmission protocol. This allows the second node 20 to effectively configure the function of the second image signal processing unit 21 according to the function configuration information, thereby enabling the first node 10 to effectively schedule the visual data processing of the second node 20. This facilitates flexible transmission and processing of visual data, making the visual processing system more flexible and able to meet various visual data processing needs. Furthermore, in this embodiment, transmitting function configuration information based on the first private transmission protocol does not require data encapsulation via a processor (e.g., MCU) based on a standard protocol, i.e., it does not require processing based on higher-level protocols above the physical layer. Instead, it directly processes and transmits data based on the physical layer, ensuring the reliability of data transmission.

[0160] For example, the user can configure the first node 10. The first transmission unit 12 of the first node 10 initiates a downlink data packet containing functional configuration information based on the first transmission protocol (e.g., the first private transmission protocol), and transmits the downlink data packet to the second node 20 based on the first private transmission protocol, so that the second transmission unit 22 of the second node 20 receives the functional configuration information based on the first transmission protocol (e.g., the first private transmission protocol), so that the second node 20 can perform corresponding functional configuration on the second image signal processing unit 21 according to the functional configuration information.

[0161] In this embodiment of the disclosure, the first node 10 receives third visual data from the second node 20 based on a first transmission protocol (e.g., a first proprietary transmission protocol). In some optional embodiments, the first node 10 may perform image signal processing on the third visual data through the first image signal processing unit 11 to obtain fourth visual data.

[0162] Optionally, refer to Figure 7A and Figure 7B As shown, the first node 10, in addition to including the first transmission unit 12, may further include a first image signal processing unit 11. Optionally, referring to... Figure 7A and Figure 7B As shown, the first node 10 is also connected to the host 40, and can transmit the processed visual data (the fourth visual data as described below) obtained by the first image signal processing unit 11 to the host 40. The host 40 can apply the processed visual data to achieve the required functions. For example, 360° surround view of the vehicle body, reversing camera, autonomous driving / assisted driving, etc. Correspondingly, the host 40 can be part of the 360° surround view system, reversing camera system, autonomous driving / assisted driving system, etc.

[0163] Optionally, image signal processing may include any processing method that meets the requirements. Optionally, the image signal processing methods may include, but are not limited to, at least one of the following: Black Level (BL) processing, Lens Shade Correction (LSC) processing, White Balance Gain (WB Gain) processing, Bad Pixel Correction (BPC) processing, Denoising processing, High Dynamic Range Fusion (HDR Fusion) processing, Chromatic Aberration Correction (CAC) processing, Debayer processing, Color Correction processing, Global Tone Mapping (GTM) processing, Local Tone Mapping (LTM) processing, Sharpen processing, Chroma Noise Reduction (CNR) processing, Gamma Adjustment processing, Format Change processing, Image Stabilization processing, and Compression and Output processing. It should be understood that the above-mentioned processing methods can all be implemented using algorithms in related technologies or other innovative algorithms, and the embodiments disclosed herein do not impose any limitations on them.

[0164] In this embodiment of the disclosure, the functional configuration information may include one or more, and can be arbitrarily selected to configure the function of the second image signal processing unit 21 of the second node 20. Some examples of functional configuration information are described below.

[0165] In some optional embodiments, the functional configuration information may include first functional configuration information; the second node 20 is specifically used to: configure the second image signal processing unit 21 to perform compression processing on visual data according to the first functional configuration information; and compress the original visual data through the second image signal processing unit 21 to obtain third visual data; such as Figure 7A , Figure 7B As shown, the first node 10 also includes a first image signal processing unit 11. The first node 10 is further configured to: decompress the third visual data to obtain the original visual data through the first image signal processing unit 11, and perform a predetermined full image signal processing flow on the original visual data to obtain the fourth visual data.

[0166] like Figure 8 As shown, in some cases, the second node 20 can compress the original visual data and output it as the third visual data through the second image signal processing unit 21.

[0167] It should be understood that, in this embodiment, the second node 20 can configure the second image signal processing unit 21 to perform compression processing on visual data according to the first functional configuration information, thereby effectively configuring the processing function of the second image signal processing unit 21 on demand, making the function of the visual processing system 100 more flexible; then, the second image signal processing unit 21 compresses the original visual data to obtain third visual data, and the third visual data can be transmitted to the first node 10 based on the first transmission protocol through the second transmission unit 22. After receiving the third visual data based on the first transmission protocol, the first transmission unit 12 of the first node 10 can then transmit the third visual data through the first image signal processing unit 22. After decompressing the third visual data, the signal processing unit 11 obtains the original visual data and performs a predetermined full image signal processing flow on the original visual data to obtain the fourth visual data, thereby effectively realizing the transmission and processing of visual data. In addition, in this optional scheme, since the second image signal processing unit 21 of the second node 20 only compresses the original visual data and outputs it as the third visual data, the first node 10 can handle more processing of the third visual data from the second node 20, and the second node 20 does not need to perform too much processing on the original visual data, so that the real-time transmission of the third visual data is better and the data processing burden of the second node 20 is also smaller.

[0168] Optionally, when the first transmission protocol is a first private transmission protocol, the compressed original visual data (i.e., third visual data) transmitted based on the first private transmission protocol does not need to be encapsulated by a processor (e.g., MCU) based on a standard protocol. That is, it does not need to be processed based on a higher-level protocol above the physical layer. Instead, it can be directly transmitted and processed based on the physical layer, which can ensure the reliability of data transmission.

[0169] Optionally, the predetermined full image signal processing flow may include at least one of the following image signal processing methods: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, HDR fusion processing, chromatic aberration correction processing, Debayer processing, color correction processing, global tone mapping processing, local tone mapping processing, sharpening processing, CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing. These various processing methods can make image signal processing more flexible to meet various visual processing needs and improve the processing effect of visual data. In the embodiments of this disclosure, the predetermined full image signal processing flow may include one or more arbitrary image signal processing methods (for example, those described above), and each processing method may be used once or multiple times (e.g., ...). Figure 9A In the example shown, white balance gain processing and noise reduction processing are performed multiple times. As long as the processing requirements of visual data can be met, no limitation is made in this embodiment.

[0170] For example, Figure 9A The diagrams shown illustrate how the first and second nodes process visual data. Figure 9A As shown, the third visual data received by the first node 10 is the compressed original visual data, then as follows: Figure 9A As shown, the third visual data can be decompressed to obtain the original visual data, and then a predetermined full image signal processing flow can be performed on the original visual data to obtain the fourth visual data. For example, in Figure 9A The example full image signal processing flow can include the following processing methods executed sequentially: black level correction, lens shading correction, white balance gain processing, bad pixel correction, noise reduction, HDR fusion, white balance gain processing, color difference correction, debyer processing, color correction, global tone mapping, local tone mapping, noise reduction, sharpening & CNR processing, gamma correction, format change processing, image stabilization, compression, and output processing. By decompressing the third visual data to obtain the original visual data, and then following the full image signal processing flow exampled above, fourth visual data can be output for transmission to host 40 for application. Figure 9AAs shown, in the aforementioned full image signal processing flow, the first six processing methods can be preprocessing flows, which can be part of the full image signal processing flow. However, it should be understood that this is only an example, and the preprocessing flow can also include fewer or more processing methods.

[0171] Optionally, the preprocessing procedure can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing a preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0172] In some optional embodiments, the functional configuration information may include second functional configuration information; the second node 20 is specifically used to: configure the second image signal processing unit 21 to perform image signal preprocessing on the visual data according to the second functional configuration information; and perform an image signal preprocessing process on the original visual data through the second image signal processing unit 21 to obtain third visual data; the first node 10 also includes a first image signal processing unit 11, and the first node 10 is further used to: perform a first image signal processing process on the third visual data through the first image signal processing unit 11 to obtain fourth visual data; wherein, the union of the image signal preprocessing process and the first image signal processing process is equal to the predetermined full image signal processing process.

[0173] like Figure 8 As shown, in some cases, the second node 20 can preprocess the original visual data through the second image signal processing unit 21 and output it as the third visual data.

[0174] It should be understood that, according to the second functional configuration information, the second node 20 in this embodiment can configure the second image signal processing unit 21 to perform image signal preprocessing on the visual data, thereby effectively configuring the processing function of the second image signal processing unit 21 on demand, making the function of the visual processing system 100 more flexible; then, the second image signal processing unit 21 performs image signal preprocessing on the original visual data to obtain third visual data, and can transmit the third visual data to the first node 10 based on the first transmission protocol through the second transmission unit 22. After receiving the third visual data based on the first transmission protocol, the first transmission unit 12 of the first node 10 can then transmit the third visual data through the first node 10. The first image signal processing unit 11 continues to perform the first image signal processing flow on the third visual data to obtain the fourth visual data, thereby effectively realizing the transmission and processing of visual data. In addition, in this optional scheme, since the second image signal processing unit 21 of the second node 20 only preprocesses the original visual data as the third visual data output, the third visual data from the second node 20 can be further processed at the first node 10 to realize the complete full-process processing of the original visual data. Therefore, the second node 20 does not need to perform too much processing on the original visual data, resulting in better real-time transmission of the third visual data and a smaller data processing burden on the second node 20.

[0175] Optionally, when the first transmission protocol is a first private transmission protocol, the third vision data can be transmitted based on the first private transmission protocol without the need for data encapsulation based on a standard protocol by a processor (such as an MCU), that is, without the need for processing based on higher-level protocols above the physical layer, but can be directly transmitted and processed based on the physical layer, which can ensure the reliability of data transmission.

[0176] In any embodiment of the present disclosure, the image signal preprocessing process and the first image signal processing process may include one or more arbitrary image signal processing methods (for example, they can be understood with reference to the processing methods described above), and each processing method may be used once or multiple times, as long as the image signal preprocessing process and the first image signal processing process can form a predetermined full image signal processing process. The present disclosure does not impose any limitations on this.

[0177] Optionally, the image signal preprocessing procedure (i.e., the preprocessing procedure) can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing an image signal preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0178] In some optional embodiments, the image signal preprocessing flow includes at least one of the following image signal processing methods: black level correction, lens shading correction, white balance gain correction, bad pixel correction, noise reduction, and HDR fusion. These various processing methods make image signal preprocessing more flexible, meeting diverse visual processing needs and improving the processing effect of visual data.

[0179] For example, Figure 9B Other schematic diagrams illustrating the processing of visual data by the first and second nodes are shown, such as... Figure 9B As shown, the third visual data received by the first node 10 is the preprocessed raw visual data, then as follows: Figure 9B As shown, a first image signal processing procedure can be performed on the third visual data to obtain the fourth visual data. For example, as... Figure 9B As shown, the image signal preprocessing process executed by the second image signal processing unit 21 of the second node 20 during preprocessing may include the following processing methods executed sequentially: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, and HDR fusion processing. The first image signal processing process executed by the first image signal processing unit 11 of the first node 10 may include the following processing methods executed sequentially: white balance gain processing, color difference correction processing, Debye processing, color correction processing, global tone mapping processing, local tone mapping processing, noise reduction processing, sharpening & CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing. Figure 9B The image signal preprocessing flow and the first image signal processing flow can constitute the predetermined full image signal processing flow exemplified above. After completing the above processing, the first node 10 can output fourth visual data for transmission to the host 40 for application. It should be understood that the various processing flows described herein are merely examples and are not intended to limit any aspect of the embodiments of this disclosure.

[0180] In some optional embodiments, the functional configuration information may include third functional configuration information; the second node 20 is specifically used to: configure the second image signal processing unit 21 to perform image signal preprocessing and compression processing on the visual data according to the third functional configuration information; perform an image signal preprocessing process on the original visual data through the second image signal processing unit 21 to obtain a preprocessing result, and compress the preprocessing result to obtain third visual data; the first node 10 also includes a first image signal processing unit 11, and the first node 10 is further used to: decompress the third visual data through the first image signal processing unit 11 to obtain a preprocessing result, and perform a first image signal processing process on the preprocessing result to obtain fourth visual data; wherein, the union of the image signal preprocessing process and the first image signal processing process is equal to the predetermined full image signal processing process.

[0181] like Figure 8 As shown, in some cases, the second node 20 can output the original visual data as third visual data after preprocessing and compression by the second image signal processing unit 21.

[0182] It should be understood that, according to the third functional configuration information, the second node 20 in this embodiment can configure the second image signal processing unit 21 to perform image signal preprocessing and compression processing on visual data, thereby effectively configuring the processing function of the second image signal processing unit 21 on demand, making the function of the visual processing system 100 more flexible; then, the second image signal processing unit 21 performs an image signal preprocessing process on the original visual data, and compresses the obtained preprocessing result to obtain third visual data, which can be transmitted to the first node 10 based on the first transmission protocol. After receiving the third visual data based on the first transmission protocol, the first transmission unit 12 of the first node 10 can then transmit the data through the third transmission protocol. An image signal processing unit 11 continues to process the preprocessed result obtained after decompressing the third visual data, executes the first image signal processing flow, and obtains the fourth visual data, thereby effectively realizing the transmission and processing of visual data. In addition, in this optional scheme, since the second image signal processing unit 21 of the second node 20 only needs to preprocess and compress the original visual data to output it as the third visual data, the preprocessed result from the second node 20 can be further processed at the first node 10 to realize the complete full-process processing of the original visual data. Therefore, the second node 20 does not need to perform too much processing on the original visual data, resulting in better real-time transmission of the third visual data and a smaller data processing burden on the second node 20.

[0183] Optionally, when the first transmission protocol is a first private transmission protocol, the third vision data can be transmitted based on the first private transmission protocol without the need for data encapsulation based on a standard protocol by a processor (such as an MCU), that is, without the need for processing based on higher-level protocols above the physical layer. Instead, it can be transmitted and processed directly based on the physical layer, which can ensure the reliability of data transmission.

[0184] For example, Figure 9C Further schematic diagrams illustrating the processing of visual data by the first and second nodes are shown, such as... Figure 9C As shown, the third visual data received by the first node 10 is the preprocessed and compressed raw visual data, then as follows: Figure 9C As shown, the first image signal processing flow can be performed on the preprocessed result obtained by decompressing the third visual data to obtain the fourth visual data. For example, as... Figure 9C As shown, the image signal preprocessing process executed by the second image signal processing unit 21 of the second node 20 during preprocessing may include the following processing methods executed sequentially: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, and HDR fusion processing. The first image signal processing process executed by the first image signal processing unit 11 of the first node 10 may include the following processing methods executed sequentially: white balance gain processing, color difference correction processing, Debye processing, color correction processing, global tone mapping processing, local tone mapping processing, noise reduction processing, sharpening & CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing. Figure 9C The image signal preprocessing flow and the first image signal processing flow can constitute the predetermined full image signal processing flow exemplified above. After completing the above processing, the first node 10 can output fourth visual data for transmission to the host 40 for application. It should be understood that the various processing flows described herein are merely examples and are not intended to limit any aspect of the embodiments of this disclosure.

[0185] Optionally, the image signal preprocessing procedure (i.e., the preprocessing procedure) can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing an image signal preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0186] In some optional embodiments, the functional configuration information may include fourth functional configuration information; the second node 20 is specifically used to: configure the second image signal processing unit 21 to perform full-process image signal processing on visual data according to the fourth functional configuration information; and obtain third visual data by performing a predetermined full-process image signal processing on the original visual data through the second image signal processing unit 21.

[0187] like Figure 8 As shown, in some cases, the second node 20 can process the original visual data through the second image signal processing unit 21 and output it as third visual data. Then, the second node 20 can transmit the third visual data to the first node 10 based on the first proprietary transmission protocol through the second transmission unit 22. Optionally, after receiving the third visual data through the first transmission unit 12, the first node 10 can directly output the third visual data as fourth visual data to the host 40 for application, without further processing by the first image signal processing unit 11.

[0188] It should be understood that, according to the fourth functional configuration information, the second node 20 in this embodiment can configure the second image signal processing unit 21 to perform a full-process image signal processing flow on the visual data, thereby effectively configuring the processing function of the second image signal processing unit 21 on demand, making the function of the visual processing system 100 more flexible; then, the second image signal processing unit 21 performs a predetermined full image signal processing flow on the original visual data to obtain the third visual data, and the third visual data can be transmitted to the first node 10 based on the first transmission protocol through the second transmission unit 22. The first transmission unit 12 of the first node 10 can receive the third visual data based on the first transmission protocol, thereby effectively realizing the transmission and processing of visual data; in addition, in this optional scheme, since the second image signal processing unit 21 of the second node 20 performs full-process image signal processing on the original visual data, the data processing burden of the first node 10 can be reduced, allowing the first node 10 to focus more on data scheduling.

[0189] Optionally, when the first transmission protocol is a first private transmission protocol, the third vision data can be transmitted based on the first private transmission protocol without the need for data encapsulation based on a standard protocol by a processor (such as an MCU), that is, without the need for processing based on higher-level protocols above the physical layer. Instead, it can be transmitted and processed directly based on the physical layer, which can ensure the reliability of data transmission.

[0190] For example, Figure 9D Further schematic diagrams illustrating the processing of visual data by the first and second nodes are shown, such as... Figure 9D As shown, the third visual data received by the first node 10 is the raw visual data after the entire image signal processing. For example, in Figure 9DThe example full image signal processing flow may include the following processing methods executed sequentially: black level correction, lens shading correction, white balance gain processing, bad pixel correction, noise reduction, HDR fusion, white balance gain processing, color difference correction, debyer processing, color correction, global tone mapping, local tone mapping, noise reduction, sharpening & CNR processing, gamma correction, format change processing, image stabilization, compression, and output processing. The first node 10 can transmit third-vision data to the host 40 for application. For example... Figure 9D As shown, in the aforementioned full image signal processing flow, the first six processing methods can be preprocessing flows, which can be part of the full image signal processing flow. However, it should be understood that this is only an example, and the preprocessing flow can also include fewer or more processing methods.

[0191] Optionally, the preprocessing procedure can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing a preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0192] In some optional embodiments, the functional configuration information may include fifth functional configuration information; the second node 20 is further configured to: configure the second image signal processing unit 21 to not perform image signal processing on the visual data according to the fifth functional configuration information; and use the original visual data as the third visual data; the first node 10 further includes a first image signal processing unit 11, and the first node 10 is further configured to: perform a predetermined full image signal processing flow on the third visual data through the first image signal processing unit 11 to obtain the fourth visual data.

[0193] like Figure 8 As shown, in some cases, the second node 20 can directly output the original visual data as the third visual data.

[0194] It should be understood that, according to the fifth functional configuration information, the second node 20 in this embodiment can configure the second image signal processing unit 21 to not perform image signal processing on the visual data, thereby effectively configuring the processing function of the second image signal processing unit 21 on demand, making the function of the visual processing system 100 more flexible; then, the original visual data is transmitted to the first node 10 as the third visual data based on the first transmission protocol. After receiving the third visual data based on the first transmission protocol, the first transmission unit 12 of the first node 10 can perform a predetermined full image signal processing flow on the original visual data by the first image signal processing unit 11 of the first node 10, thereby effectively realizing the transmission and processing of visual data; in addition, in this optional scheme, the original visual data acquired by the visual acquisition unit 30 (such as a camera) can be transmitted to the first node 10 as the third visual data based on the first transmission protocol by the second node 20, which can be centrally processed by the first image signal processing unit 11 of the first node 10, without the second node 20 having to perform too much processing on the original visual data, so that the real-time transmission of the original visual data is better and the data processing burden of the second node 20 is also smaller.

[0195] Optionally, when the first transmission protocol is a first private transmission protocol, the transmission of third-vision data based on the first private transmission protocol does not require data encapsulation by a processor (such as an MCU) based on a standard protocol. That is, it does not require processing based on higher-level protocols above the physical layer. Instead, it can be directly transmitted and processed based on the physical layer, which can ensure the reliability of data transmission.

[0196] In some examples, the second image signal processing unit 21 can be set to bypass mode according to the fifth function configuration information. In bypass mode, the original visual data is not further processed by the second image signal processing unit 21. The second node 20 can directly use the original visual data as the third visual data and transmit it to the first node 10 through the second transmission unit 22 based on the first transmission protocol (e.g., the first private transmission protocol).

[0197] For example, Figure 9E Further schematic diagrams illustrating the processing of visual data by the first and second nodes are shown, such as... Figure 9E As shown, the third visual data received by the first node 10 is the original visual data, then as follows: Figure 9E As shown, a predetermined full-image signal processing flow can be performed on the third visual data to obtain the fourth visual data. For example, in Figure 9EThe example full image signal processing flow can include the following processing methods executed sequentially: black level correction, lens shading correction, white balance gain processing, bad pixel correction, noise reduction, HDR fusion, white balance gain processing, color difference correction, debyer processing, color correction, global tone mapping, local tone mapping, noise reduction, sharpening & CNR processing, gamma correction, format change processing, image stabilization, compression, and output processing. By processing the raw visual data (here, the third visual data) according to the example full image signal processing flow, the processed raw visual data (here, the fourth visual data) can be output and transmitted to the host 40 for application. Figure 9E As shown, in the aforementioned full image signal processing flow, the first six processing methods can be preprocessing flows, which can be part of the full image signal processing flow. However, it should be understood that this is only an example, and the preprocessing flow can also include fewer or more processing methods.

[0198] Optionally, the preprocessing procedure can be used to pre-correct and balance the physical differences between different visual acquisition units 30 (e.g., cameras), such as differences in parameter settings, manufacturers, environments, etc. It should be understood that employing a preprocessing procedure in any embodiment of this disclosure is beneficial for improving the processing effect of visual data.

[0199] Optionally, such as Figure 8 As shown, in some cases, the second node 20 in this embodiment can directly transmit the original visual data as third visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22. In other cases, it can also transmit the compressed original visual data as third visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22. In still other cases, it can also transmit the preprocessed original visual data as third visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22. In yet another case, it can also transmit the preprocessed and compressed original visual data as third visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22. In yet another case, it can also transmit the original visual data after full-process image signal processing as third visual data to the first node 10 based on the first transmission protocol (e.g., the first proprietary transmission protocol) through the second transmission unit 22. These cases can be several non-conflicting functions of the second node 20.

[0200] Optionally, in the visual processing system 100 of this embodiment, the main node for visual processing, namely the first node 10, can schedule the visual data and corresponding ISP function configuration information of the visual acquisition units 30 of each sub-node (including the second node 20), so that the first image signal processing unit 11 in the first node 10, located at the far end of the visual acquisition unit 30 (such as a camera), can reasonably schedule and process the visual data of different sub-nodes, and then send the processed visual data to the host 40 for application. Figure 10 As shown, it illustrates Figure 7A This diagram illustrates how the master node (first node 10) schedules visual data from multiple child nodes in the right chain of a double daisy-chain network. Figure 10As shown, the master node (first node 10) can obtain data, including the visual data of child node a (corresponding to the original visual data collected by the visual acquisition unit 30 (referred to as camera 1) connected to child node a, such as the third visual data obtained based on the original visual data), the visual data of child node b (corresponding to the original visual data collected by the visual acquisition unit 30 (referred to as camera 2) connected to child node b, such as the third visual data obtained based on the original visual data), the visual data of child node c (corresponding to the original visual data collected by the visual acquisition unit 30 (referred to as camera 3) connected to child node c, such as the third visual data obtained based on the original visual data), the visual data of child node d (corresponding to the original visual data collected by the visual acquisition unit 30 (referred to as camera 4) connected to child node d, such as the third visual data obtained based on the original visual data), and the ISP function configuration information (i.e., the function configuration information of the second image signal processing unit 21) corresponding to child nodes a to d. Since the ISP function configuration information of each child node in child nodes a to d can be different from each other, the processing performed by the master node (first node 10) on the visual data of each child node can be different. For example, assuming the ISP configuration information corresponding to child node a allows its second image signal processing unit 21 to be configured to perform compression processing on visual data, then when the master node (first node 10) processes the visual data of child node a, it can determine, based on the ISP configuration information corresponding to child node a, that its first image signal processing unit 11 needs to decompress the visual data of child node a and execute a predetermined full image signal processing flow. Thus, the first image signal processing unit 11 can accurately process the visual data of child node a. Similarly, assuming the ISP configuration information corresponding to child node b allows its second image signal processing unit 21 to be configured to perform image signal preprocessing on visual data, when the master node (first node 10) processes the visual data of child node b, it can determine, based on the ISP configuration information corresponding to child node b, that its first image signal processing unit 11 needs to perform a first full image signal processing flow on the visual data of child node b, thereby completing the predetermined full image signal processing flow on the raw visual data acquired by camera 2. Child nodes c and d can be deduced similarly from the descriptions of child nodes a and b, and will not be repeated here. It is understandable that the above information... Figure 10 The exemplary descriptions provided are not intended to limit any aspect of the embodiments disclosed herein.

[0201] In some optional embodiments, the second node 20 and the first node 10 are adjacent nodes. The second node 20 is specifically used to: obtain the target data packet through the second transmission unit 22 based on the first transmission protocol, write at least a portion of the third visual data into the target data packet, and transmit the target data packet to the physical layer of the first node 10 based on the first transmission protocol.

[0202] Optionally, the target data packet obtained by the second node 20 may be a data packet generated by the second node 20 based on the first transmission protocol, or the target data packet may be a data packet received by the second node 20 based on the first transmission protocol. This embodiment of the present disclosure does not limit this.

[0203] Optionally, the second node 20 can obtain the target data packet based on the first private transmission protocol through the second transmission unit 22, write at least a portion of the third visual data into the target data packet, and transmit the target data packet to the first node 10 based on the first private transmission protocol. Optionally, the target data packet obtained by the second node 20 can be a data packet generated by the second node 20 based on the first private transmission protocol, or the target data packet can be a data packet received by the second node 20 based on the first private transmission protocol. This embodiment of the present disclosure does not limit this.

[0204] It should be understood that the data in the embodiments of this disclosure may be transmitted and processed based on a first private transmission protocol, that is, based on the physical layer for transmission and / or processing. However, this does not mean that it only performs the functions that the physical layer can perform as defined in the standard protocol. For example, in the standard protocol, the function of the physical layer is to convert the frame signals of higher layers (e.g., the data link layer) above the physical layer into electrical or optical signals that can be transmitted on the physical transmission medium, and to convert the electrical or optical signals received from the physical transmission medium into bit streams for processing by higher layers (e.g., the data link layer). The data transmission and processing in the vision processing system of this disclosure are implemented based on the physical layer of the target node, which means that the data transmission and processing scheme in the vision processing system of the embodiments of this disclosure does not go through higher layers (e.g., the data link layer, network layer, etc.) above the physical layer as defined in standard physical communication. Instead, it is based on a set of private protocols defined by the physical layer, which enables it to perform data transmission and processing other than some functions that the physical layer can perform as defined by the standard protocol.

[0205] For example, in some optional embodiments, the second node 20 is the data packet initiating node, and the second node 20 can generate the target data packet based on the first transmission protocol through the second transmission unit 22. Alternatively, in other optional embodiments, the second node 20 is not the data packet initiating node, and the second node 20 can obtain the target data packet transmitted from neighboring nodes other than the first node 10 based on the first transmission protocol through the second transmission unit 22. Thus, the second node 20 can effectively obtain the target data packet based on the first transmission protocol, so as to send the third visual data to the first node 10 through the target data packet based on the first transmission protocol.

[0206] For example, taking the first transmission protocol as a first private transmission protocol, optionally, if the second node 20 is the data packet initiating node, then the second node 20 can generate the target data packet based on the first private transmission protocol through the second transmission unit 22. Alternatively, optionally, if the second node 20 is not the data packet initiating node, then the second node 20 can obtain the target data packet transmitted from adjacent nodes other than the first node 10 through the second transmission unit 22 based on the first private transmission protocol. Thus, the second node 20 can effectively obtain the target data packet based on the first private transmission protocol, so as to send the first visual data to the first node 10 through the target data packet based on the first private transmission protocol, realizing the transmission of visual data based on the physical layer.

[0207] The initiating node of the target data packet can be selected as needed. Under different circumstances, the initiating node of the target data packet can be set to any node other than the first node 10 among multiple nodes, so as to meet the needs of visual data transmission under different conditions (such as different network topologies, different numbers of nodes, etc.).

[0208] For example, optionally, refer to Figure 7A In a daisy-chain network where multiple nodes are connected, the node initiating the data packet can be the last node in the daisy chain. For example... Figure 7A The middle section is a dual daisy chain, consisting of a right chain (including a master node and child nodes a, b, c, d) and a left chain (including a master node and child nodes g, f, e). This allows for two packet initiating nodes: child node d and child node e for the right and left chains, respectively. These two initiating nodes accommodate the visual data transmission needs of the nodes in the daisy chain network. Alternatively, if multiple nodes are connected in a single daisy chain network, only one node can be designated as the target packet initiating node.

[0209] For example, refer to Figure 7BIn the ring network formed by multiple connected nodes shown, the packet initiating node can be any one or two nodes other than the first node 10 (the master node). In one example, the packet initiating node can be set to one node, such as child node a or child node g. Then, the target packet can be transmitted to the first node in one transmission direction (taking child node g as the packet initiating node as an example, the transmission direction can be: child node g → child node f → child node e → child node d → child node c → child node b → child node a → master node (i.e., the first node 10). If the packet initiating node is set to child node a, then it is child node a → child node b → child node c → child node d → child node e → child node f → child node g → master node). In other examples, the data packet initiating node can be set to two nodes, such as child node d or child node e. The transmission directions between the target data packets initiated by child node d and child node e can be different (for example, the transmission direction of the target data packet initiated by child node d is: child node d → child node c → child node b → child node a → master node (i.e., first node 10), while the transmission direction of the target data packet initiated by child node e is: child node e → child node f → child node g → master node (i.e., first node 10)). This is to better accommodate the visual data transmission needs of each node in a ring network. Of course, the above applies to… Figure 7A and Figure 7B The descriptions provided are for illustrative purposes only and are not intended to limit the scope of the embodiments disclosed herein.

[0210] For example, taking the first transmission protocol as the first private transmission protocol as an example, refer to Figure 7AIn the daisy-chain network shown, in the right chain, child node a (second node 20) and master node (first node 10) are adjacent nodes. Child node a may not be the initiating node of the target data packet. Child node a can obtain the target data packet from the downstream node of the right chain, i.e., child node b, through its second transmission unit 22 based on the first private transmission protocol (for example, the target data packet may be generated by child node d). It writes at least part of the third visual data into the target data packet and transmits the target data packet upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the third visual data in it. For example, in the left chain, child node g (second node 20) and master node (first node 10) are adjacent nodes. Child node g may not be the initiating node of the target data packet. Child node g can obtain the target data packet from the downstream node, i.e., child node f, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node e). It writes at least part of the third visual data into the target data packet and transmits the target data packet upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the third visual data in it.

[0211] For example, taking the first transmission protocol as the first proprietary transmission protocol, refer to... Figure 7AAs shown, assuming that there are no child nodes b, c, d, e, and f in this network, and only child nodes a, g, and the master node (first node 10), then in the right chain, child node a (second node 20) and the master node (first node 10) are adjacent nodes, and child node a is the last node. Child node a can be the data packet initiating node of the target data packet. Child node a can initiate a data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least part of the third visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the third visual data in it. In the left chain, the child node g (second node 20) and the master node (first node 10) are adjacent nodes, and the child node g is the last node. The child node g can be the data packet initiating node of the target data packet. The child node g can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, and thus obtain the target data packet. At least part of the third visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the master node (first node 10) based on the first private transmission protocol. For example, it can be transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet based on the first private transmission protocol through its first transmission unit 12 and reads the third visual data in it.

[0212] It should be understood that other situations can be deduced by analogy to the above examples, and will not be elaborated further here.

[0213] It is understood that, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are adjacent nodes, at least a portion of the third visual data can be written into the target data packet obtained based on the first transmission protocol through the second transmission unit 22, and the third visual data can be effectively transmitted to the first node 10 by transmitting the target data packet to the first node 10 based on the first transmission protocol.

[0214] Optionally, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are adjacent nodes, at least a portion of the third visual data can be written into the target data packet obtained based on the first private transmission protocol through the second transmission unit 22, and the third visual data can be effectively transmitted to the first node 10 based on the first private transmission protocol by transmitting the target data packet to the first node 10 based on the first private transmission protocol. Therefore, the visual data is directly transmitted based on the physical layer, and it is not necessary to encapsulate the data based on the standard protocol through the processor (e.g., MCU, etc.), that is, it is not necessary to process it based on the higher-level protocol above the physical layer, but to transmit and process it directly based on the physical layer, which can ensure the reliability of data transmission.

[0215] In some alternative embodiments, the second node 20 and the first node 10 are not adjacent nodes. The second node 20 is specifically used to: obtain a target data packet through the second transmission unit 22 based on the first transmission protocol, write at least a portion of the third visual data into the target data packet, and transmit the target data packet to the first node 10 through at least one node between the second node 20 and the first node 10 based on the first transmission protocol.

[0216] Optionally, the second node 20 obtains the target data packet through the second transmission unit 22 based on the first private transmission protocol, writes at least a portion of the third visual data into the target data packet, and transmits the target data packet to the first node 10 through at least one node between the second node 20 and the first node 10 based on the first private transmission protocol.

[0217] For example, taking the first transmission protocol as the first private transmission protocol as an example, refer to Figure 7AIn the daisy-chain network shown, in the right chain, child node b (second node 20) is connected to the master node (first node 10) via child node a. Therefore, child node b and the master node are not adjacent nodes. Child node b may not be the initiating node of the target data packet. Child node b can obtain the target data packet from the downstream node, i.e., child node c, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node d). It writes at least part of the third visual data into the target data packet and transmits the target data packet upstream to child node a based on the first private transmission protocol. For example, it can be transmitted to the physical layer of child node a. Then, through the second transmission unit 22 of child node a, it is forwarded to the master node (first node 10) based on the first private transmission protocol. For example, it can be forwarded to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the third visual data in it. For example, in the left chain, child node f (second node 20) is connected to the master node (first node 10) by child node g. Therefore, child node f and the master node are not adjacent nodes. Child node f may not be the initiating node of the target data packet. Child node f can obtain the target data packet from the downstream node, i.e., child node e, based on the first private transmission protocol through its second transmission unit 22 (for example, the target data packet may be generated by child node e). It writes at least part of the third visual data into the target data packet and transmits the target data packet upstream to child node g based on the first private transmission protocol. For example, it can be transmitted to the physical layer of child node g. Then, child node g forwards the data packet to the master node (first node 10) through its second transmission unit 22 based on the first private transmission protocol. For example, it can be forwarded to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the third visual data in it.

[0218] For example, taking the first transmission protocol as the first proprietary transmission protocol, refer to... Figure 7AIn the daisy-chain network shown, in the right chain, child node d (second node 20) is connected to the master node (first node 10) via child nodes a, b, and c. Therefore, child node d and the master node are not adjacent nodes, and child node d is the last node. Child node d can be the initiating node of the target data packet. Child node d can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least a portion of the third visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the child node based on the first private transmission protocol. The physical layer of node c is forwarded to the physical layer of node b via its second transmission unit 22 based on the first private transmission protocol. Then, it is forwarded to the physical layer of node a via its second transmission unit 22 based on the first private transmission protocol. Finally, it is forwarded to the physical layer of the master node (first node 10) via its second transmission unit 22 based on the first private transmission protocol. The master node (first node 10) receives the target data packet and reads the third visual data in it via its first transmission unit 12 based on the first private transmission protocol. For example, in the left chain, child node e (second node 20) is connected to the master node (first node 10) by child nodes f and g. Therefore, child node e and the master node are not adjacent nodes, and child node e is the last node. Child node e can be the data packet initiating node of the target data packet. Child node e can initiate the data packet based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least part of the third visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the physical layer of child node f based on the first private transmission protocol. After being forwarded by child node f through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of child node g. Then, after being forwarded by child node g through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the third visual data in it.

[0219] For example, taking the first transmission protocol as a first proprietary transmission protocol, refer to... Figure 7BThe ring network shown assumes that child node d and child node e are the data packet initiating nodes for two target data packets. Child node d (second node 20) is not adjacent to the master node (first node 10). Child node d can initiate data packets based on the first private transmission protocol through its second transmission unit 22, thus obtaining the target data packet. At least a portion of the third visual data can be written into the target data packet, which is then transmitted upstream to the physical layer of child node c based on the first private transmission protocol. From child node c, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of child node b. From child node b, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of child node a. From child node a, it is forwarded through its second transmission unit 22 based on the first private transmission protocol to the physical layer of the master node (first node 10). The master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol and reads the third visual data within it. Child node e (second node 20) and master node (first node 10) are not adjacent nodes. Child node e can initiate a data packet based on the first private transmission protocol through its second transmission unit 22, thereby obtaining the target data packet. At least part of the third visual data can be written into the target data packet, and the target data packet can be transmitted upstream to the physical layer of child node f based on the first private transmission protocol. After being forwarded by child node f through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of child node g. Then, after being forwarded by child node g through its second transmission unit 22 based on the first private transmission protocol, it is transmitted to the physical layer of master node (first node 10). Master node (first node 10) receives the target data packet based on the first private transmission protocol and reads the third visual data in it.

[0220] It should be understood that the other cases can be deduced from the above examples, and will not be elaborated further here.

[0221] It is understood that, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are not adjacent nodes, at least a portion of the third visual data can be written into the target data packet obtained based on the first transmission protocol through the second transmission unit 22, and the third visual data can be effectively transmitted to the first node by transmitting the target data packet to the first node 10 sequentially through at least one node based on the first transmission protocol.

[0222] Optionally, in this embodiment of the present disclosure, when the second node 20 and the first node 10 are not adjacent nodes, at least a portion of the third visual data can be written into the target data packet obtained based on the first private transmission protocol through the second transmission unit 22, and the target data packet can be transmitted to the first node 10 sequentially through at least one node based on the first private transmission protocol. This effectively transmits the third visual data to the first node based on the first private transmission protocol, thus realizing the direct transmission of visual data based on the physical layer. It does not require data encapsulation based on standard protocols by a processor (such as an MCU), that is, it does not require processing based on higher-level protocols above the physical layer, but directly transmits and processes based on the physical layer, which can ensure the reliability of data transmission.

[0223] Optionally, the second node 20 is connected to the visual acquisition unit 30 and obtains raw visual data from the visual acquisition unit 30.

[0224] In some alternative embodiments, such as Figure 8 As shown, if the second node 20 is disconnected from the visual acquisition unit 30, or if the visual acquisition unit 30 connected to the second node 20 is not enabled, the second node 20 enters a low-power mode.

[0225] The second node 20 is disconnected from the vision acquisition unit 30, meaning the connection between the vision acquisition unit 30 and the second node 20 is severed. In this situation, the second node 20 cannot obtain raw visual data. The vision acquisition unit 30 connected to the second node 20 is not enabled. Although connected to the second node 20, the vision acquisition unit 30 does not acquire visual data; for example, it may be in a disabled, powered-off, or faulty state. Therefore, the second node 20 also cannot obtain raw visual data in this case. In both of these situations, the second node 20 enters a low-power mode, which helps reduce the overall power consumption of the vision processing system 100.

[0226] Optionally, when the second node 20 is in low-power mode, if the second node 20 receives visual data from an adjacent node through the second transmission unit 22 based on the first transmission protocol, then the second node 20 forwards the visual data from the adjacent node to the next adjacent node along a first direction, where the first direction is the direction from the second node 20 to the first node 10. This allows the node in this embodiment to effectively transmit visual data while reducing power consumption in low-power mode.

[0227] For example, taking the first transmission protocol as a first proprietary transmission protocol, when the second node 20 is in low-power mode, if the second node 20 receives visual data from an adjacent node through the second transmission unit 22 based on the first proprietary transmission protocol, then the second node 20 forwards the visual data from the adjacent node to the next adjacent node along the first direction through the second transmission unit 22 based on the first proprietary transmission protocol. Thus, the second node 20 in this embodiment can effectively reduce power consumption in low-power mode while achieving visual data transmission at the physical layer, facilitating reliable data transmission.

[0228] For example, taking the first transmission protocol as the first private transmission protocol as an example, Figure 7A Taking the daisy-chain network as an example, taking the right chain child node a (second node 20) as an example, if it is not connected to the visual acquisition unit 30 (that is, disconnected from the visual acquisition unit 30) or the visual acquisition unit 30 it is connected to is not enabled, then if child node a obtains visual data from the adjacent child node b through the second transmission unit 22 based on the first private transmission protocol (for example, it may be receiving a target data packet containing third visual data received from child node b), it can forward the visual data to the adjacent next node along the first direction, that is, the master node (first node 10). The first transmission unit 12 of the master node (first node 10) can receive the visual data based on the first private transmission protocol. For example, taking the right-chain child node b (second node 20) as an example, if it is not connected to the visual acquisition unit 30 (that is, disconnected from the visual acquisition unit 30) or the visual acquisition unit 30 it is connected to is not enabled, then if the child node b obtains visual data from the adjacent child node c through the second transmission unit 22 based on the first private transmission protocol (for example, it may be receiving a target data packet containing third visual data received from the child node b), it can forward the visual data to the adjacent next node, i.e., child node a, along the first direction, so that it can be forwarded again through child node a to the master node (first node 10) for processing.

[0229] The first transmission protocol is a first private transmission protocol. In some optional embodiments, the second node 20 is assigned at least one node identifier, and the target data packet records the target identifier. The second node 20 can be used to: after obtaining the target data packet through the second transmission unit 22 based on the first private transmission protocol, determine the target identifier in the target data packet, and in response to the existence of a node identifier with the same target identifier among at least one node identifier, write at least a portion of the third visual data into the target data packet through the second transmission unit 22 based on the first private transmission protocol.

[0230] Therefore, in this embodiment of the present disclosure, by assigning at least one node identifier to the second node 20, and when there is a node identifier among the at least one node identifier that is the same as the target identifier recorded in the obtained target data packet, at least a portion of the third visual data is written into the target data packet by the second transmission unit 22 based on the first private transmission protocol. This makes the transmission of visual data from the second node 20 to the first node 10 by the second transmission unit 22 based on the first private transmission protocol more orderly, and also facilitates the first node 10 to schedule and process visual data from multiple nodes.

[0231] Optionally, different nodes receiving data packets initiated by the same data packet initiating node are assigned different node identifiers. This ensures that the visual data written in each target data packet is the visual data of a single node, thereby guaranteeing the orderly transmission of visual data and facilitating the first node 10 to schedule and process visual data from multiple nodes.

[0232] The target identifier and node identifier in the embodiments of this disclosure can be in any form, such as including but not limited to text identifiers, symbol identifiers, etc.

[0233] In some optional embodiments, the node identifier is the node's frame number, and the target identifier is the data packet's frame number, wherein data packets sent by the same data packet initiating node within different time slots divided in the same preset period record different frame numbers.

[0234] Optionally, multiple time slots divided within the same preset period can be of equal length. The preset period can be set as needed, for example, it can be 1 second.

[0235] Optionally, different nodes receiving data packets initiated by the same data packet initiating node are assigned different frame numbers. This ensures that the visual data written in each target data packet is the visual data of a single node, thereby guaranteeing the orderly transmission of visual data and facilitating the first node 10 to schedule and process visual data from multiple nodes.

[0236] To address the latency jitter issue (latency jitter refers to the variable time interval between data packets arriving at the receiving end during network transmission. This variation can cause problems when the receiving end processes data, such as discontinuous video feeds, which can negatively impact user experience), this embodiment allocates a predetermined time slot for transmitting visual data to each child node (which can be the second node 20) except for the master node (first node 10). For example, a preset period (e.g., 1 second) can be divided into N equal time slots, where N can be preset, for example, to 256, 512, 1024, etc. The following example uses N=1024. The initiating node can send one data packet per time slot. The data packet records its frame number, which increments sequentially. For example, in a preset period of N = 1024 time slots, the frame number of a data packet sent in the first time slot is 1, the frame number of a data packet sent in the second time slot is 2, and so on, with the frame number of a data packet sent in the 1024th time slot being 1024. That is, data packets sent in different time slots within the same preset period record different frame numbers. At least one frame number can be assigned to each child node, and different child nodes are assigned different frame numbers.

[0237] For example, for Figure 7A The right chain of the daisy-chain network includes four child nodes: child node a, child node b, child node c, and child node d (in this example, all four child nodes are the second node 20). The preset period can then be divided into N = 1024 time slots. Referring to the frame number allocation table in Table 1 above, and... Figure 4 The diagram shown illustrates the time slot division of a preset period and the frame number allocation to child nodes, which can be used to understand the frame number allocation of nodes:

[0238] As mentioned above, Figure 7A In a daisy-chain topology, the right chain can be initiated by the last child node d. Within a preset period of N = 1024 time slots, this child node will periodically initiate target data packets, including the frame number, in sequence from 1 to 1024, with the frame number automatically incrementing. For example... Figure 11 A schematic diagram illustrating an example of visual data transmission is shown, which can be combined with... Figure 11As shown, if the current time slot is the 5th time slot, the frame number of the currently initiated target data packet is 5. According to Table 1, the frame number of child node d does not contain "5". Therefore, child node d can determine that the target data packet is not a data packet used to transmit its own visual data. Thus, it does not write third visual data into the target data packet through the second transmission unit 22, but only sends an empty data packet to child node c through its second transmission unit 22 based on the first private transmission protocol. Child node c receives the empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol. It also finds that the target data packet is not a data packet used to transmit its own visual data. Therefore, it does not write third visual data into the target data packet through its second transmission unit 22, but instead forwards the target data packet to the physical layer of child node b through its second transmission unit 22 based on the first private transmission protocol. Child node b receives the empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol. If node b can determine that its node identifier contains a node identifier "5" that is the same as the target identifier, then it can write at least part of the third visual data into the target data packet through its second transmission unit 22 based on the first private transmission protocol, and then transmit the target data packet to the physical layer of child node a based on the first private transmission protocol. Child node a receives an empty target data packet with frame number 5 through its second transmission unit 22 based on the first private transmission protocol and finds that the target data packet is not a data packet used to transmit its own visual data. Therefore, it does not write the third visual data into the target data packet through its second transmission unit 22, but forwards the target data packet to the physical layer of the master node (first node 10) through its second transmission unit 22 based on the first private transmission protocol. After the master node (first node 10) receives the target data packet through its first transmission unit 12 based on the first private transmission protocol, it can determine the visual data of child node b transmitted by the target packet according to the frame number recorded in the target data packet, and then perform the corresponding processing. Following a similar process, target data packets with frame numbers 1 to 1024 are transmitted in N=1024 time slots of a preset period. After the maximum value of 1024 is reached, the next preset period begins. In the first time slot of the next preset period, the frame number of the target data packet returns to 1, and the transmission of visual data continues.

[0239] It should be understood that Figure 7A The transmission method of the left chain in the daisy chain network is similar to that of the right chain mentioned above. The difference is that the data packet initiating node is the child node e, and the frame number allocation of each node can be different from that of the right chain.

[0240] It should also be understood that Figure 7B The ring network can also be constructed as described above. Figure 7A The transmission method of the right chain in a daisy-chain network can be understood in the same way. For example, the Figure 7BA ring network can include two data packet initiating nodes, such as child node d and child node e. It can actually be decomposed into a structure resembling... Figure 7A The chrysanthemum chain network consists of right-chain and left-chain forms.

[0241] Based on this, through the above-mentioned optional solutions in this embodiment of the present disclosure, each node of the vision processing system 100 (including the second node 20) can transmit its vision data within a predetermined time slot of a preset period, thereby ensuring the processing latency and timely transmission of vision data of each node, and enabling the main node of vision processing (the first node 10) to conveniently schedule and process multiple channels of vision data. In addition, since each time slot of the preset period corresponds to the vision data transmission of one node, it is not only easier to achieve high-precision synchronization of vision data, but also to effectively improve the latency jitter problem of vision data transmission.

[0242] As shown in Table 1, the transmission of other information also corresponds to frame numbers. That is, the transmission of other information of the child node needs to be transmitted to the master node (first node 10) through data packets with corresponding frame numbers in certain time slots of a preset period. This ensures the orderly transmission of visual data and other information. In this embodiment of the disclosure, other information can be information other than visual data. For example, other information may include, but is not limited to: audio data, control data for reading and writing registers, debugging data, etc.

[0243] In some optional embodiments, the data packet includes a frame header and a data block, with the frame number of the data packet recorded in the frame header. Thus, the second node 20 can read the frame header of the target data packet to determine its frame number, and can, when needed, write at least a portion of the third visual data into the data block via the second transmission unit 22 to facilitate the transmission of the visual data.

[0244] In some optional embodiments, the second node 20 is further configured to: in response to each node identifier being different from the target identifier, transmit the target data packet via the second transmission unit 22 along a first direction based on a first private transmission protocol to the next adjacent node of the second node 20, wherein the first direction is from the second node 20 to the next adjacent node.

[0245] This optional embodiment can be referred to the foregoing. Figure 7A To understand this, we can use the example of the left chain, which will not be elaborated here. Optionally, if the second node 20 determines that the frame number of each node is different from the frame number of the data packet, the target data packet can be transmitted to the next adjacent node of the second node 20 along the first direction based on the first private transmission protocol.

[0246] Based on this, through the above-mentioned optional solutions in this embodiment of the present disclosure, each node of the vision processing system 100 (including the second node 20) can transmit its vision data within a predetermined time slot of a preset period, thereby ensuring the processing latency and timely transmission of vision data of each node, and enabling the main node of vision processing (the first node 10) to conveniently schedule and process multiple channels of vision data. In addition, since each time slot of the preset period corresponds to the vision data transmission of one node, it is not only easier to achieve high-precision synchronization of vision data, but also to effectively improve the latency jitter problem of vision data transmission.

[0247] In some optional embodiments, the first node 10 is further configured to: transmit frame number configuration information to the second node 20 through the first transmission unit 12 based on the first private transmission protocol; the second node 20 is further configured to: receive the frame number configuration information through the second transmission unit 22 based on the first private transmission protocol, and configure the node's frame number according to the frame number configuration information.

[0248] Therefore, the first node 10 can transmit frame number configuration information to the second node 20 through its first transmission unit 12 based on the first private transmission protocol. This allows the second node 20 to receive the frame number configuration information through its second transmission unit 22 based on the first private transmission protocol. Consequently, the second node 20 can effectively configure its frame number according to the frame number configuration information, enabling it to transmit its visual data within a predetermined time slot within a preset period. This ensures the processing latency and timely transmission of visual data for each node, facilitating the first node 10 to schedule the visual data transmission of the second node 20 and improving the latency jitter problem in visual data transmission. Furthermore, transmitting frame number configuration information based on the first private transmission protocol eliminates the need for data encapsulation based on standard protocols by a processor (e.g., MCU), meaning it does not require processing based on higher-level protocols above the physical layer. Instead, it directly transmits and processes data at the physical layer, ensuring the reliability of data transmission.

[0249] For example, a user can configure the first node 10. The first node 10 initiates a downlink data packet including frame number configuration information based on the first private transmission protocol, and transmits the downlink data packet to the second node 20 based on the first private transmission protocol, so that the second node 20 can obtain the frame number configuration information to configure the node's frame number.

[0250] Optionally, in conjunction with the above Figure 6A and Figure 6BAs can be understood from the relevant content, the visual processing system 100 provided in the second aspect of the present disclosure allows the second node 20 to transmit third visual data via the second transmission unit 22 based on the first proprietary transmission protocol, and the first node 10 to receive third visual data via the first transmission unit 12 based on the first proprietary transmission protocol. This achieves direct transmission and / or processing of visual data at the physical layer, eliminating the need for data encapsulation via a processor (e.g., MCU) based on standard protocols, and thus eliminating the need for processing via higher-level protocols above the physical layer. This ensures reliable data transmission. Furthermore, this solution eliminates the need for switches when transmitting visual data via the first proprietary transmission protocol, reducing the demand for processors (e.g., MCUs) in the network during visual data transmission. This effectively reduces the transmission latency of visual data and further reduces costs by decreasing the number of switches and processors (e.g., MCUs). Therefore, this solution effectively improves the data transmission and processing performance of the visual processing system.

[0251] It is understood that the description of the visual processing system 100 in the embodiments of this disclosure above is only some optional embodiments of this disclosure and is not a limitation on the embodiments of this disclosure.

[0252] According to a third aspect of the present disclosure, a visual data processing method is provided for connecting a second node 20 in a network of multiple nodes, wherein the multiple nodes further include a first node 10, the second node 20 includes a second transmission unit 22, and the first node 10 includes a first image signal processing unit 11 and a first transmission unit 12. (Refer to...) Figure 12 The flowchart shown illustrates that the method includes steps S102 and S104, specifically:

[0253] S102: Obtain first visual data;

[0254] S104: The first visual data is transmitted to the first node through the second transmission unit based on the first transmission protocol, so that the first node receives the first visual data through the first transmission unit based on the first transmission protocol, and performs image signal processing on the first visual data through the first image signal processing unit to obtain the second visual data.

[0255] It should be understood that the visual data processing method of the third aspect in the embodiments of this disclosure has been specifically described in the visual processing system 100 embodiment of the first aspect above. Its various optional implementations and beneficial effects can be understood with reference to the system embodiment above, and will not be repeated here.

[0256] According to a fourth aspect of the present disclosure, a visual data processing method is provided for connecting a first node 10 of a plurality of nodes in a network, the plurality of nodes further including a second node 20, the second node 20 including a second transmission unit 22, and the first node 10 including a first image signal processing unit 11 and a first transmission unit 12, referring to... Figure 13 The flowchart shown illustrates that the method includes steps S202 and S204, specifically:

[0257] S202: Receive first visual data from the second node through the first transmission unit based on the first transmission protocol, wherein the first visual data is transmitted from the second node to the first node through the second transmission unit based on the first transmission protocol;

[0258] S204: The first image signal processing unit performs image signal processing on the first visual data to obtain the second visual data.

[0259] It should be understood that the visual data processing method of the fourth aspect in the embodiments of this disclosure has been specifically described in the visual processing system 100 embodiment of the first aspect above. Its various optional implementations and beneficial effects can be understood with reference to the system embodiment above, and will not be repeated here.

[0260] According to a fifth aspect of the present disclosure, a visual data processing method is provided for connecting a second node 20 in a network of multiple nodes, wherein the multiple nodes further include a first node 10, and the second node 20 includes a second transmission unit 22 and a second image signal processing unit 21, as shown below. Figure 14 The flowchart shown illustrates that the method includes steps S302, S304, and S306, specifically:

[0261] S302: Configure the second image signal processing unit according to the function configuration information;

[0262] S304: Obtain raw visual data and obtain third visual data through the second image signal processing unit after functional configuration;

[0263] S306: The third visual data is transmitted to the first node based on the first transmission protocol through the second transmission unit.

[0264] It should be understood that the visual data processing method of the fifth aspect in the present disclosure has been specifically described in the visual processing system 100 embodiment of the second aspect above. Its various optional implementations and beneficial effects can be understood with reference to the system embodiment above, and will not be repeated here.

[0265] According to a sixth aspect of the present disclosure, a chip 1000 is provided. Figure 15This is a schematic block diagram of a chip provided in an embodiment of this disclosure. Specific embodiments of this disclosure do not limit the specific implementation of the chip. Figure 15 As shown, the chip 1000 may include a processor 1002 and a memory 1006. The processor 1002 and the memory 1006 communicate with each other. The processor 1002 is used to execute program 1010, specifically performing the relevant steps in any of the aforementioned visual data processing method embodiments.

[0266] Specifically, program 1010 may include program code that includes computer operation instructions.

[0267] The processor 1002 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure. The smart device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0268] RISC-V is an open-source instruction set architecture based on the Reduced Instruction Set Computing (RISC) principle. It can be applied to various aspects of microcontrollers and FPGA chips, specifically in areas such as IoT security, industrial control, mobile phones, and personal computers. Because its design considers small size, speed, and low power consumption, it is particularly suitable for modern computing devices such as warehouse-scale cloud computers, high-end mobile phones, and tiny embedded systems. With the rise of AIoT (Artificial Intelligence of Things), the RISC-V instruction set architecture is receiving increasing attention and support and is expected to become the next generation of widely used CPU architecture.

[0269] The computer operation instructions in this embodiment can be computer operation instructions based on the RISC-V instruction set architecture. Correspondingly, the processor 1002 can be designed based on the RISC-V instruction set. Specifically, the chip provided in this embodiment can be a chip designed using the RISC-V instruction set. This chip can execute executable code based on the configured instructions, thereby implementing the visual data processing method in the above embodiments.

[0270] Memory 1006 is used to store program 1010. Memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0271] Specifically, program 1010 can be used to cause processor 1002 to execute the visual data processing method in any of the foregoing embodiments.

[0272] The specific implementation of each step in program 1010 can be found in the corresponding steps and units described in any of the aforementioned visual data processing method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0273] According to a seventh aspect of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the visual data processing method as described in any of the foregoing embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0274] According to an eighth aspect of the present disclosure, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the visual data processing method as described in any of the foregoing embodiments.

[0275] The chip 1000, computer storage medium, and computer program product embodiments in this disclosure have been described in detail in the aforementioned vision processing system 100 embodiment. Therefore, their related content and beneficial effects can be understood by referring to the above embodiments, and will not be repeated here.

[0276] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0277] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this disclosure can be broken down into more components / steps, or two or more components / steps or parts of the operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this disclosure. It should be understood that the various technical features in the technical solutions of the embodiments of this disclosure can be combined or broken down in any suitable manner.

[0278] The methods described above according to embodiments of this disclosure can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded over a network. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for performing the methods shown herein.

[0279] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of the embodiments disclosed herein.

[0280] The above embodiments are only used to illustrate the embodiments of this disclosure, and are not intended to limit the embodiments of this disclosure. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this disclosure. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this disclosure, and the patent protection scope of the embodiments of this disclosure should be defined by the claims.

[0281] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". It should be noted that the concepts of "first", "second", etc., mentioned in the embodiments of this disclosure are only used to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should be noted that the modifications of "a" and "a plurality" mentioned in the embodiments of this disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0282] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of this disclosure, and are not intended to limit them. Although the embodiments of this disclosure have been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A vision processing system, comprising: Multiple nodes connected to form a network, wherein the multiple nodes include at least a first node and a second node, wherein: The second node includes a second transmission unit, which is used to: obtain first visual data and transmit the first visual data to the first node based on a first transmission protocol through the second transmission unit; The first node includes a first image signal processing unit and a first transmission unit. The first node is configured to: receive first visual data from the second node through the first transmission unit based on a first transmission protocol, and perform image signal processing on the first visual data through the first image signal processing unit to obtain second visual data. The second node is specifically used for: obtaining a target data packet through the second transmission unit based on a first transmission protocol, writing at least a portion of the first visual data into the target data packet, and transmitting the target data packet to the first node based on the first transmission protocol, wherein the first transmission protocol is a first private transmission protocol; The second node is assigned at least one node identifier, and the target data packet records the target identifier; the second node is specifically used to: after obtaining the target data packet through the second transmission unit based on the first private transmission protocol, determine the target identifier in the target data packet, and in response to the existence of a node identifier with the same target identifier among the at least one node identifier, write at least a portion of the first visual data into the target data packet through the second transmission unit based on the first private transmission protocol; The node identifier is the node's frame number, and the target identifier is the data packet's frame number. The data packets sent by the same data packet initiating node within different time slots divided in the same preset period have different recorded frame numbers.

2. The system according to claim 1, wherein, The second node is connected to the visual acquisition unit; The second node is specifically used for: The original visual data is obtained from the visual acquisition unit and transmitted to the first node as the first visual data through the second transmission unit based on the first transmission protocol. And / or, The original visual data is obtained from the visual acquisition unit, compressed, and transmitted to the first node as the first visual data through the second transmission unit based on the first transmission protocol.

3. The system according to claim 2, wherein, The raw visual data is in Bayer RAW format.

4. The system according to claim 2, wherein, If the second node is disconnected from the visual acquisition unit, or if the visual acquisition unit connected to the second node is not enabled, the second node enters a low-power mode. In the low-power mode, if the second node receives visual data from a neighboring node through the second transmission unit based on the first transmission protocol, the second node forwards the visual data from the neighboring node to the next neighboring node along a first direction through the second transmission unit based on the first transmission protocol, wherein the first direction is the direction from the second node to the first node.

5. The system according to claim 1, wherein, The image signal processing performed by the first image signal processing unit on the first visual data includes at least one of the following processing methods: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, HDR fusion processing, color difference correction processing, Debayer processing, color correction processing, global tone mapping processing, local tone mapping processing, sharpening processing, CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing.

6. The system according to any one of claims 1-5, wherein, The second node and the first node are adjacent nodes. The second node is specifically used to: transmit the target data packet to the first node adjacent to the second node based on the first transmission protocol; or, The second node and the first node are not adjacent nodes. The second node is specifically used to: transmit the target data packet to the first node through at least one node between the second node and the first node based on the first transmission protocol.

7. The system according to claim 6, wherein, If the second node is the data packet initiating node, then the second node generates the target data packet based on the first transmission protocol through the second transmission unit; or, If the second node is not the data packet initiating node, then the second node obtains the target data packet transmitted from the adjacent nodes other than the first node through the second transmission unit based on the first transmission protocol.

8. The system according to claim 6, wherein, The first private transmission protocol implements data transmission and / or processing based on the physical layer.

9. A vision processing system, comprising: Multiple nodes connected to form a network, wherein the multiple nodes include at least a first node and a second node, wherein: The second node includes a second image signal processing unit and a second transmission unit. The second node is used to: configure the second image signal processing unit according to the function configuration information; obtain the original visual data, and obtain the third visual data through the second image signal processing unit after function configuration; and transmit the third visual data to the first node through the second transmission unit based on the first transmission protocol. The first node includes a first transmission unit, which is configured to: receive third visual data from the second node through the first transmission unit based on the first transmission protocol; The second node is specifically used for: obtaining a target data packet through the second transmission unit based on a first transmission protocol, writing at least a portion of the third visual data into the target data packet, and transmitting the target data packet to the first node based on the first transmission protocol, wherein the first transmission protocol is a first private transmission protocol; The second node is assigned at least one node identifier, and the target data packet records the target identifier; the second node is specifically used to: after obtaining the target data packet through the second transmission unit based on the first private transmission protocol, determine the target identifier in the target data packet, and in response to the existence of a node identifier with the same target identifier among the at least one node identifier, write at least a portion of the third visual data into the target data packet through the second transmission unit based on the first private transmission protocol; The node identifier is the node's frame number, and the target identifier is the data packet's frame number. The data packets sent by the same data packet initiating node within different time slots divided in the same preset period have different recorded frame numbers.

10. The system according to claim 9, wherein, The functional configuration information includes first functional configuration information; The second node is specifically used to: configure the second image signal processing unit to perform compression processing on visual data according to the first functional configuration information; The original visual data is compressed by the second image signal processing unit to obtain the third visual data; The first node further includes a first image signal processing unit, which is also configured to: decompress the third visual data to obtain the original visual data through the first image signal processing unit, and perform a predetermined full image signal processing flow on the original visual data to obtain the fourth visual data.

11. The system according to claim 9, wherein, The functional configuration information includes second functional configuration information; The second node is specifically used to: configure the second image signal processing unit to perform image signal preprocessing on visual data according to the second functional configuration information; and perform the image signal preprocessing process on the original visual data through the second image signal processing unit to obtain the third visual data. The first node further includes a first image signal processing unit, and the first node is further configured to: perform a first image signal processing procedure on the third visual data through the first image signal processing unit to obtain fourth visual data; The union of the image signal preprocessing process and the first image signal processing process is equal to the predetermined full image signal processing process.

12. The system according to claim 9, wherein, The functional configuration information includes third-function configuration information; The second node is specifically used to: configure the second image signal processing unit to perform image signal preprocessing and compression processing on visual data according to the third functional configuration information; The second image signal processing unit performs an image signal preprocessing process on the original visual data to obtain a preprocessing result, and then compresses the preprocessing result to obtain the third visual data. The first node further includes a first image signal processing unit, and the first node is further configured to: decompress the third visual data through the first image signal processing unit to obtain the preprocessing result, and perform a first image signal processing flow on the preprocessing result to obtain the fourth visual data; The union of the image signal preprocessing process and the first image signal processing process is equal to the predetermined full image signal processing process.

13. The system according to claim 9, wherein, The functional configuration information includes fourth functional configuration information; The second node is specifically used to: configure the second image signal processing unit to perform full-process image signal processing on visual data according to the fourth function configuration information; and obtain the third visual data by performing a predetermined full-process image signal processing on the original visual data through the second image signal processing unit.

14. The system according to claim 9, wherein, The functional configuration information includes the fifth functional configuration information; The second node is further configured to: configure the second image signal processing unit to not perform image signal processing on the visual data according to the fifth function configuration information; and use the original visual data as the third visual data; The first node further includes a first image signal processing unit, which is also configured to: perform a predetermined full image signal processing procedure on the third visual data through the first image signal processing unit to obtain fourth visual data.

15. The system according to claim 9, wherein, The first node is further configured to: transmit functional configuration information to the second node via the first transmission unit based on the first transmission protocol; The second node is also configured to: receive the functional configuration information based on the first transmission protocol via the second transmission unit.

16. The system according to claim 9, wherein, The raw visual data is in Bayer RAW format.

17. The system according to claim 9, wherein, The second node is connected to the visual acquisition unit and obtains raw visual data from the visual acquisition unit; If the second node is disconnected from the visual acquisition unit, or if the visual acquisition unit connected to the second node is not enabled, the second node enters a low-power mode. In the low-power mode, if the second node receives visual data from a neighboring node through the second transmission unit based on the first transmission protocol, the second node forwards the visual data from the neighboring node to the next neighboring node along a first direction through the second transmission unit based on the first transmission protocol, wherein the first direction is the direction from the second node to the first node.

18. The system according to any one of claims 10-14, wherein, The predetermined full image signal processing flow includes at least one of the following image signal processing methods: black level correction processing, lens shading correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, HDR fusion processing, color difference correction processing, Debayer processing, color correction processing, global tone mapping processing, local tone mapping processing, sharpening processing, CNR processing, gamma correction processing, format change processing, image stabilization processing, compression and output processing.

19. The system according to claim 11 or 12, wherein, The image signal preprocessing procedure is used to pre-correct and balance the physical differences between different visual acquisition units.

20. The system according to claim 19, wherein, The image signal preprocessing process includes at least one of the following image signal processing methods: black level correction processing, lens shadow correction processing, white balance gain processing, bad pixel correction processing, noise reduction processing, and HDR fusion processing.

21. The system according to any one of claims 9-17, wherein, The second node and the first node are adjacent nodes. The second node is specifically used to: transmit the target data packet to the first node adjacent to the second node based on the first transmission protocol; or, The second node and the first node are not adjacent nodes. The second node is specifically used to: transmit the target data packet to the first node through at least one node between the second node and the first node based on the first transmission protocol.

22. The system according to claim 21, wherein, If the second node is the data packet initiating node, then the second node generates the target data packet based on the first transmission protocol through the second transmission unit; or, If the second node is not the data packet initiating node, then the second node obtains the target data packet transmitted from the adjacent nodes other than the first node through the second transmission unit based on the first transmission protocol.

23. The system according to claim 21, wherein, The first private transmission protocol implements data transmission and / or processing based on the physical layer.

24. A visual data processing method for a second node in a network of multiple nodes, the multiple nodes further comprising a first node, the second node comprising a second transmission unit, the first node comprising a first image signal processing unit and a first transmission unit, the method comprising: Obtain first-person visual data; The first visual data is transmitted to the first node through the second transmission unit based on the first transmission protocol, so that the first node receives the first visual data through the first transmission unit based on the first transmission protocol, and performs image signal processing on the first visual data through the first image signal processing unit to obtain the second visual data. The step of transmitting the first visual data to the first node through the second transmission unit based on the first transmission protocol includes: obtaining a target data packet through the second transmission unit based on the first transmission protocol, writing at least a portion of the first visual data into the target data packet, and transmitting the target data packet to the first node based on the first transmission protocol, wherein the first transmission protocol is a first private transmission protocol; The second node is assigned at least one node identifier, and the target data packet records the target identifier; the step of obtaining the target data packet through the second transmission unit based on the first transmission protocol and writing at least a portion of the first visual data into the target data packet includes: after obtaining the target data packet through the second transmission unit based on the first private transmission protocol, determining the target identifier in the target data packet, and in response to the existence of a node identifier with the same name as the target identifier among the at least one node identifier, writing at least a portion of the first visual data into the target data packet through the second transmission unit based on the first private transmission protocol; The node identifier is the node's frame number, and the target identifier is the data packet's frame number. The data packets sent by the same data packet initiating node within different time slots divided in the same preset period have different recorded frame numbers.

25. A visual data processing method for a first node in a network of multiple nodes, the multiple nodes further comprising a second node, the second node including a second transmission unit, the first node including a first image signal processing unit and a first transmission unit, the method comprising: The first visual data is received from the second node through the first transmission unit based on the first transmission protocol, wherein the first visual data is transmitted from the second node to the first node through the second transmission unit based on the first transmission protocol; The first image signal processing unit performs image signal processing on the first visual data to obtain the second visual data; The first visual data is obtained by the second node through the second transmission unit based on the first transmission protocol to obtain the target data packet. At least a portion of the first visual data is written into the target data packet, and the target data packet is transmitted to the first node based on the first transmission protocol, wherein the first transmission protocol is a first private transmission protocol. The second node is assigned at least one node identifier, and the target data packet records the target identifier; after the second node obtains the target data packet through the second transmission unit based on the first private transmission protocol, it determines the target identifier in the target data packet, and in response to the existence of a node identifier with the same target identifier among the at least one node identifier, it writes at least a portion of the first visual data into the target data packet through the second transmission unit based on the first private transmission protocol; The node identifier is the node's frame number, and the target identifier is the data packet's frame number. The data packets sent by the same data packet initiating node within different time slots divided in the same preset period have different recorded frame numbers.

26. A visual data processing method for a second node in a network of multiple nodes, wherein the multiple nodes further include a first node, and the second node includes a second transmission unit and a second image signal processing unit, the method comprising: The second image signal processing unit is configured with the corresponding functions according to the function configuration information. The raw visual data is obtained, and the third visual data is obtained through the second image signal processing unit after functional configuration. The third visual data is transmitted to the first node via the second transmission unit based on the first transmission protocol. The step of transmitting the third visual data to the first node through the second transmission unit based on the first transmission protocol includes: obtaining a target data packet through the second transmission unit based on the first transmission protocol, writing at least a portion of the third visual data into the target data packet, and transmitting the target data packet to the first node based on the first transmission protocol, wherein the first transmission protocol is a first private transmission protocol; The second node is assigned at least one node identifier, and the target data packet records the target identifier; the step of obtaining the target data packet through the second transmission unit based on the first transmission protocol and writing at least a portion of the third visual data into the target data packet includes: after obtaining the target data packet through the second transmission unit based on the first private transmission protocol, determining the target identifier in the target data packet, and in response to the existence of a node identifier identical to the target identifier among the at least one node identifier, writing at least a portion of the third visual data into the target data packet through the second transmission unit based on the first private transmission protocol; The node identifier is the node's frame number, and the target identifier is the data packet's frame number. The data packets sent by the same data packet initiating node within different time slots divided in the same preset period have different recorded frame numbers.

27. A chip, comprising: A processor and a memory, wherein the processor and the memory communicate with each other; The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 24-26.

28. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 24-26.

Images