An image processing method, an electronic device, and a computer-readable storage medium

Abstract

This application discloses an image processing, electronic device, and computer-readable storage medium. In response to receiving an image serial differential signal sent by a data acquisition terminal, this application performs conversion processing on the image serial differential signal to obtain a parallel data stream; a bridging chip performs protocol parsing processing on the parallel data stream to obtain a parsed data stream; and a main chip performs data analysis processing on the parsed data stream to obtain the target image. This improves the stability of image processing.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image processing method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In the current field of industrial machine vision, the continuous development of applications such as intelligent manufacturing, automated inspection, and precision assembly has brought new challenges to image transmission technology in various dimensions, including bandwidth, latency, reliability, and cost. Image processing, in particular, is a significant challenge in image transmission.

[0003] Current image processing methods typically employ a single FPGA (Field-Programmable Gate Array) to process the received data. However, with the increase in the number of camera channels and the improvement in single-channel bandwidth, the receiver needs to simultaneously handle the parsing of multiple high-speed protocols and the transfer of large amounts of image data. A single FPGA often faces the problem of limited pins and logic resources, resulting in poor system timing and difficulty in ensuring operational stability. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide an image processing method, electronic device, and computer-readable storage medium that can improve stability.

[0005] To address the aforementioned technical problems, this application provides an image processing method applied to a data processing end. The data processing end is communicatively connected to a data acquisition end. The data processing end includes a bridging chip and a main chip. The method includes: in response to receiving an image serial differential signal sent by the data acquisition end, performing conversion processing on the image serial differential signal to obtain a parallel data stream; performing protocol parsing processing on the parallel data stream through the bridging chip to obtain a parsed data stream; and performing data analysis processing on the parsed data stream through the main chip to obtain a target image.

[0006] In one embodiment, the bridging chip stores a preset protocol mapping table. The step of performing protocol parsing processing on the parallel data stream through the bridging chip to obtain the parsed data stream includes: mapping the parallel data stream according to the preset protocol mapping table to obtain a low-voltage differential signal format data stream corresponding to the parallel data stream; and determining the low-voltage differential signal format data stream as the parsed data stream.

[0007] In one embodiment, the main chip stores an image processing clock. The step of performing data analysis and processing on the parsed data stream through the main chip to obtain a target image includes: calibrating the parsed data stream based on the image processing clock to obtain a calibrated data stream; and determining the target image based on the calibrated data stream and the boundary information corresponding to the calibrated data stream.

[0008] In one embodiment, the data processing end includes a deserializer. After the step of performing data analysis and processing on the parsed data stream through the main chip to obtain the target image, the method further includes: sending the acquired global synchronization pulse signal to the control interface of the deserializer through the main chip; and modulating the global synchronization pulse signal through the deserializer.

[0009] To address the aforementioned technical problems, this application provides another image processing method applied to a data transmission system. The data transmission system includes a data acquisition end and a data processing end, which are communicatively connected. The data processing end includes a bridging chip and a main chip. The image processing method includes: acquiring an image serial differential signal through the data acquisition end and sending the image serial differential signal to the data processing end; performing conversion processing on the image serial differential signal through the data processing end to obtain a parallel data stream; performing protocol parsing processing on the parallel data stream through the bridging chip to obtain a parsed data stream; and performing data analysis processing on the parsed data stream through the main chip to obtain a target image.

[0010] In one embodiment, the data acquisition end and the data processing end are connected via a coaxial cable and transmit data via the GMSL2 protocol.

[0011] In one embodiment, the step of acquiring the image serial differential signal through the data acquisition terminal includes: preprocessing the acquired image data stream to obtain a preprocessed image data stream; converting the preprocessed image data stream to obtain a standard data packet; and encoding the standard data packet and the acquired reverse road control signal to obtain the image serial differential signal.

[0012] In one embodiment, the data transmission system includes an onboard unit and a protocol processing controller. One end of the protocol processing controller is connected to the main chip, and the other end is connected to the onboard unit. The data transmission system is connected to a host computer via a bus. The method further includes: in response to the protocol processing controller detecting that the bus of the data transmission system is busy, writing the target image to the onboard unit; and in response to the protocol processing controller detecting that the bus of the data transmission system is idle, transmitting the target image in the onboard unit to the host computer via the bus.

[0013] To address the aforementioned technical problems, this application provides an electronic device, including a memory and a processor. The memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the aforementioned image processing method.

[0014] To address the aforementioned technical problems, this application provides a computer-readable storage medium, comprising: storing program data, which, when executed by a processor, is used to implement the aforementioned image processing method.

[0015] The above scheme, in response to receiving the image serial differential signal sent by the data acquisition end, converts the image serial differential signal to obtain a parallel data stream; a bridging chip performs protocol parsing on the parallel data stream to obtain a parsed data stream; and the main chip performs data analysis on the parsed data stream to obtain the target image. Therefore, by using the bridging chip to handle protocol parsing, the resource pressure on the main chip is reduced, which helps improve the stability and efficiency of image processing. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a schematic flowchart of an exemplary embodiment of the image processing method shown in this application; Figure 2 This is a schematic diagram of an exemplary embodiment of the data processing terminal shown in this application; Figure 3 This is a schematic flowchart of yet another exemplary embodiment of the image processing method shown in this application; Figure 4 This is a schematic diagram of an exemplary embodiment of the data acquisition terminal shown in this application; Figure 5This is a schematic diagram of an exemplary embodiment of the data transmission system shown in this application; Figure 6 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of this application; Figure 7 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] First, it's important to note that in the current field of industrial machine vision, the continuous development of applications such as intelligent manufacturing, automated inspection, and precision assembly has brought new challenges to image transmission technology across various dimensions, including bandwidth, latency, reliability, and cost. Image processing, in particular, is a significant challenge. Current image processing methods typically employ an FPGA to process the received data. However, with the increase in the number of camera channels and the improvement in single-channel bandwidth, the receiving end needs to simultaneously handle the parsing of multiple high-speed protocols and the transfer of large amounts of image data. A single FPGA often faces issues with limited pins and logic resources, leading to poor system timing and difficulty in ensuring operational stability.

[0019] Based on this, this application provides an image processing method, an electronic device, and a computer-readable storage medium. For details, please refer to [link / reference needed]. Figure 1 , Figure 1 This is a schematic flowchart of an exemplary embodiment of an image processing method shown in this application.

[0020] The executing entity of an image processing method can be a terminal device, a server, or other processing device. The terminal device can be a computer, mobile device, terminal, computing device, vehicle-mounted device, etc. The executing entity of an image processing method can also be an image processing device. In some possible implementations, the image processing method can be implemented by a processor calling computer-readable instructions stored in memory. The executing entity of an image processing method can also be a big data cluster. A big data cluster is a computer system architecture formed by multiple computers connected through a network. The big data cluster can be deployed on a private cloud built with K8S (Kubernetes, a container orchestration engine).

[0021] Specifically, in this embodiment, an image processing method is applied to a data processing end, which is communicatively connected to a data acquisition end. The data processing end includes a bridge chip and a main chip.

[0022] A data acquisition terminal refers to a device capable of data acquisition. Data acquisition terminals can include high-resolution industrial cameras, webcams, etc. A data processing terminal refers to a device capable of data processing. A data processing terminal includes at least two FPGA acquisition boards. These at least two FPGA acquisition boards include a bridge chip for protocol parsing and a main chip for data analysis and processing.

[0023] The image processing method in this embodiment includes the following steps: Step S110: In response to receiving the image serial differential signal sent by the data acquisition terminal, the image serial differential signal is converted to obtain a parallel data stream.

[0024] Image serial differential signal refers to image signal transmitted using a serial communication method that uses two transmission lines to transmit binary data.

[0025] Parallel data streams refer to at least one MIPI CSI-2 (Mobile Industry Processor Interface Camera Serial Interface 2) data stream.

[0026] The image processing device converts the serial differential signal of the image to obtain a parallel data stream. Specifically, the image processing device performs a deserialization operation on the serial differential signal of the image to obtain a parallel data stream. For example, the image processing device shifts the serial differential signal of the image into a shift register one bit at a time, and after receiving N bits, outputs N bits of parallel data through parallel latching to obtain a parallel data stream.

[0027] Step S120: The parallel data stream is processed by protocol parsing through the bridging chip to obtain the parsed data stream.

[0028] Protocol parsing processing refers to extracting valid information from the received parallel data stream and performing operations such as frame synchronization, field splitting, error detection, and data reassembly according to the protocol specifications.

[0029] The parsed data stream refers to the valid information extracted from the parallel data stream.

[0030] The image processing device uses a bridging chip to perform protocol parsing on the parallel data stream, obtaining the parsed data stream. Specifically, the bridging chip integrates a finite state machine; the image processing device inputs the parallel data stream into the finite state machine and obtains the parsed data stream output by the finite state machine.

[0031] Step S130: The main chip performs data analysis and processing on the parsed data stream to obtain the target image.

[0032] The target image refers to an image that meets the preset quality requirements.

[0033] The main chip analyzes and processes the parsed data stream to obtain the target image. The image preprocessing module integrated into the main chip then denoises the parsed data stream to obtain the target image. This denoising process includes median filtering and Gaussian filtering.

[0034] As can be seen, in response to the received image serial differential signal sent by the data acquisition end, the image serial differential signal is converted to obtain a parallel data stream; the parallel data stream is then processed by the bridging chip to obtain a parsed data stream; finally, the parsed data stream is processed by the main chip to obtain the target image. Therefore, by using the bridging chip to handle protocol parsing, the resource pressure on the main chip is reduced, which helps improve the stability and efficiency of image processing.

[0035] Combination Figure 2 As shown, the data processing end includes a GMSL2 deserialization and driver module, a bridging FPGA, and a main FPGA. One end of the GMSL2 deserialization and driver module is connected to the data acquisition end via a coaxial cable, and the other end is connected to one end of the bridging FPGA. The other end of the bridging FPGA is connected to the main FPGA. The bridging FPGA integrates a MIPI receiver core, a transmission protocol conversion module, and an 8-channel LVDS interface. The MIPI (Mobile Industry Processor Interface) receiver core is used to translate the complex MIPI CSI-2 protocol data stream and convert it into a pixel data stream that the FPGA can easily process. The transmission protocol conversion module stores a preset protocol mapping table. The 8-channel LVDS interface is an eight-channel independent LVDS interface used for simultaneously transmitting and receiving multiple independent signals.

[0036] The GMSL2 deserialization and driver module is a physical layer interface used to receive high-speed serial signals from coaxial cables and convert them into standard parallel data streams. The GMSL2 deserialization and driver module is integrated onto the GMSL2MAX96717 deserializer chip.

[0037] The data processing end uses the GMSL2 MAX96717 deserializer chip to perform clock data recovery, decoding, and deserialization on the image serial differential signal to obtain a parallel data stream. Each line of the data stream includes at least one MIPI CSI-2 data stream and embedded synchronization and control information. The synchronization and control information includes a frame start flag, a frame end flag, a line start flag, and a line end flag.

[0038] In one embodiment, such as Figure 2 As shown, the data processing end is connected to two data acquisition ends. Each data acquisition end sends the image serial differential signal to the GMSL2 deserialization and driving module. The GMSL2 MAX96717 deserializer chip processes each image serial differential signal to obtain a parallel data stream.

[0039] In some other embodiments, the data processing end is connected to four data acquisition ends. The GMSL2 deserialization and driving module may include two deserializer chips. Each data acquisition end sends an image serial differential signal to the data processing end. The data processing end inputs two of the image serial differential signals into one deserializer chip and inputs the other two image serial differential signals into another deserializer chip to obtain a parallel data stream output by each deserializer chip.

[0040] The bridging chip stores a preset protocol mapping table. The image processing device performs protocol parsing processing on the parallel data stream through the bridging chip to obtain the parsed data stream. The steps include: mapping the parallel data stream according to the preset protocol mapping table to obtain the low-voltage differential signal format data stream corresponding to the parallel data stream; and determining the low-voltage differential signal format data stream as the parsed data stream.

[0041] The preset protocol mapping table includes multiple preset protocols and the correspondence between the data before and after parsing for each preset protocol. The preset protocols include at least the MIPI CSI-2 protocol.

[0042] Low-Voltage Differential Signaling (LVDS) refers to a format that follows low-voltage differential signaling. LVDS is a high-speed, low-power, low-noise serial data transmission interface standard. It uses the voltage difference between two transmission lines to transmit information, rather than the absolute voltage between a single signal line and ground.

[0043] The image processing device finds the parsed data corresponding to the parallel data stream from the preset protocol mapping table and obtains the parsed data stream.

[0044] In one embodiment, the image processing device sends a received parallel data stream to the bridging FPGA, translates the parallel data stream through the MIPI receiving core and converts it into a pixel data stream that the FPGA can easily process; the pixel data stream is converted into a low-voltage differential signal format data stream, i.e., the parsed data stream, through the transmission protocol conversion module; and the parsed data stream is transmitted to the main FPGA through the 8chLVDS interface.

[0045] It should be noted that a bridged FPGA can be configured with at least two data parsing modules. Each data parsing module includes a MIPI receiver core, a transmission protocol conversion module, and an 8-channel LVDS interface. Therefore, implementing a bridged FPGA allows for real-time protocol parsing of serial differential image signals from multiple data acquisition terminals, further improving image processing efficiency.

[0046] It can be seen that by separating the protocol conversion task of the data processing end from the main chip to the bridge chip, the main chip is effectively avoided from being occupied by a large number of high-speed I / O (Input / Output) interface management and underlying protocol parsing logic, thus improving the efficiency of image processing.

[0047] The main chip stores an image processing clock. The steps of obtaining the target image by analyzing and processing the parsed data stream through the main chip include: calibrating the parsed data stream based on the image processing clock to obtain a calibrated data stream; and determining the target image based on the calibrated data stream and the boundary information corresponding to the calibrated data stream.

[0048] The parsed data stream includes image pixels, frame start flag, frame end flag, line start flag, and line end flag.

[0049] The image processing clock, also known as the ISP (Image Signal Processor) clock, is the clock signal that drives the synchronous operation of all image processing pipelines within the main chip.

[0050] Boundary information includes frame valid signals and line valid signals. The frame valid signal marks the beginning and end of a frame of image data. The line valid signal marks the beginning and end of each row of pixels.

[0051] The main chip includes an 8-channel LVDS interface, a dynamic phase calibration module, and a protocol parsing and preprocessing module.

[0052] The image processing device receives the parsed data stream through an 8-channel LVDS interface, performs dynamic phase adjustment on the parsed data stream through a dynamic phase calibration module to obtain a phase-adjusted data stream, and performs clock calibration on the parsed data stream through a protocol parsing and preprocessing module to obtain a calibrated data stream. The target image is determined based on the boundary information of the calibrated data stream.

[0053] The main chip integrates a dynamic phase calibration module, including an IDDR (Input Double Data Rate register) and a dynamic phase adjustment logic training sequence. The image processing device automatically adjusts the sampling clock in the parsed data stream through this module. This ensures the sampling clock is accurate, thereby determining the optimal data sampling window and ensuring the stability of high-speed data transmission. The sampling clock is the clock used by the data acquisition end to acquire images.

[0054] IDDR is used to convert two bits of data received in one clock cycle into single-edge data that is easier for the FPGA's internal logic to process.

[0055] In one embodiment, the phase adjustment process of the dynamic phase adjustment logic training sequence includes: after receiving the adjustment instruction, the IDDR repeatedly sends preset adjustment data; the phase of the sampling clock is set to an extreme value; the phase is gradually increased; and the preset adjustment data sent by the IDDR is sampled using the sampling clock of this phase, and the sampled result is compared with the standard answer corresponding to the preset adjustment data to obtain the phase adjustment interval; the center point of the phase adjustment interval is determined as the adjusted sampling clock to obtain the optimal data sampling window.

[0056] In one embodiment, the protocol parsing and preprocessing module determines the difference between the adjusted sampling clock and the image processing clock as the time deviation; it adds the time deviation to the clock of the parsed data stream to obtain the calibrated data stream; and it determines the positions of the frame start identifier and frame end identifier in the calibrated data stream as frame valid signals, and the positions of the line start identifier and line end identifier in the calibrated data stream as line valid signals. The clocks of the frame valid signals and line valid signals are strictly synchronized with the image processing clock in the main chip.

[0057] The protocol parsing and preprocessing module includes extracting image pixels from the calibrated data stream according to the frame valid signal and the line valid signal to obtain the target image.

[0058] The protocol parsing and preprocessing module also includes extracting image pixels from the calibrated data stream according to the frame valid signal and the line valid signal to obtain the initial image; and preprocessing the initial image to obtain the target image. Preprocessing includes denoising, white balance, color correction, sharpening, scaling, etc.

[0059] After the data processing device performs data analysis and processing on the parsed data stream through the main chip to obtain the target image, the method further includes: sending the acquired global synchronization pulse signal to the control interface of the deserializer through the main chip; and modulating the global synchronization pulse signal through the deserializer.

[0060] The data acquisition end and the data processing end are connected through a reverse control channel. The data processing end sends control signals to the data acquisition end through the reverse control channel to achieve synchronous acquisition of multiple data acquisition ends.

[0061] A global synchronization pulse signal is a signal used for synchronization control. Specifically, the data processing terminal's internal logic soft trigger generates a global synchronization pulse signal; or, an external hard trigger connected to the data processing terminal generates a global synchronization pulse signal and sends it to the data processing terminal.

[0062] In one embodiment, after receiving the global synchronization pulse signal, the main chip sends the global synchronization pulse signal to the control interface of the deserializer. The deserializer's control interface then modulates the global synchronization pulse signal into the reverse control channel of the GMSL2 link and sends it to each data acquisition terminal. This triggers each data acquisition terminal to simultaneously begin image acquisition the instant it receives the global synchronization pulse signal. Therefore, by sending the global synchronization pulse signal to each data acquisition terminal via the reverse control channel, all image sensors are ensured to trigger exposure simultaneously, eliminating the uncertain delay caused by software command issuance and improving the accuracy of image acquisition.

[0063] Combination Figure 3 As shown, this embodiment provides another image processing method applied to a data transmission system. The data transmission system includes a data acquisition end and a data processing end, which are communicatively connected. The data processing end includes a bridge chip and a main chip. The image processing method includes: Step S310: Acquire the image serial differential signal through the data acquisition terminal and send the image serial differential signal to the data processing terminal.

[0064] Combination Figure 4 As shown, the data acquisition end includes an image sensor, an acquisition chip, a GMSL2 serialization and driver module, and a high-speed physical transmission interface. The acquisition chip integrates a processor, a sensor driver configuration module, an LVDS receiver core, a preprocessing module, and a transmission protocol conversion module.

[0065] The sensor driver configuration module is connected to one end of the processor, and the other end of the processor is connected to one end of the image sensor. The other end of the image sensor is connected to one end of the LVDS receiver core, and the other end of the LVDS receiver core is connected to one end of the preprocessing module. The other end of the preprocessing module is connected to one end of the transmission protocol conversion module. One end of the transmission protocol conversion module is connected to one end of the GMSL2 serialization and driver module, and the other end of the GMSL2 serialization and driver module is connected to the high-speed physical transmission interface.

[0066] The sensor driver configuration module is used to perform precise initialization and runtime control of the high-resolution CMOS image sensor through the processor. The image processing device writes initialization operations and configuration parameters through the sensor driver configuration module, controls the image sensor to perform initialization operations through the processor, and configures the image sensor according to the configuration parameters. The initialization operations include power-on sequence, clock setting, and operating mode selection, while the configuration parameters include, but are not limited to, resolution, frame rate, exposure time, analog gain, and digital gain.

[0067] For example, the sensor driver configuration module, processor, and image sensor are connected via an I2C bus. In response to the completion of configuration parameters and initialization, the image sensor acquires images from a preset acquisition area, obtaining a RAW (Raw Image Format) image data stream.

[0068] The LVDS receiver core is used to convert high-speed, serial, weak differential signals into low-speed, parallel, standard ordinary digital signals so that other logic modules inside the FPGA can easily process them.

[0069] The steps of the image processing device acquiring image serial differential signals through the data acquisition terminal include: preprocessing the acquired image data stream to obtain a preprocessed image data stream; converting the preprocessed image data stream to obtain a standard data packet; and encoding the standard data packet and the acquired reverse road control signal to obtain the image serial differential signal.

[0070] The image processing device preprocesses the acquired image data stream through a preprocessing module to obtain a preprocessed image data stream. Specifically, the image processing device performs quality enhancement processing on the image data stream through the preprocessing module to obtain a preprocessed image data stream. This quality enhancement processing includes, but is not limited to, IIF (Inverse Image Frequency), PRNU (Photo Response Non-Uniformity), FFC (Flat-Field Correction), SPC (Static Defect Pixel Correction), and BOC (Baseline Offset Correction). Thus, the image data stream is enhanced, improving image quality and correcting inherent sensor defects.

[0071] The image processing device converts the preprocessed image data stream into standard data packets. Specifically, the image processing device converts the preprocessed image data stream into standard data packets at the hardware logic level through a transmission protocol conversion module. These standard data packets are standard MIPI CSI-2 data packets containing short synchronization packets and long data packets, including start-of-frame, start-of-line, end-of-line, and end-of-frame markers.

[0072] In one embodiment, the transmission protocol conversion module stores a preset protocol mapping table, and determines the standard data packet corresponding to the preprocessed image data stream from the preset protocol mapping table; or, the transmission protocol conversion module segments the preprocessed image data stream based on the packet structure of the target protocol to obtain synchronization short packets and data long packets; and encapsulates the synchronization short packets and data long packets to obtain standard data packets.

[0073] The image processing device encodes the standard data packets and the acquired reverse road control signals to obtain the image serial differential signal. The image processing device uses the GMSL2 serialization and driving module to encode the standard data packets and the reverse road control signals to obtain the image serial differential signal.

[0074] Among them, the reverse road control signal refers to the signal used to control the reverse control channel between the data acquisition end and the data processing end. The reverse channel control signal can be the status information, fault status, etc. of each image sensor in each data acquisition end.

[0075] In some embodiments, the GMSL2 serialization and driving module is integrated on the MAX96716 serializer chip. The image processing device uses the MAX96716 serializer chip to encode and modulate the standard data packet and the reverse road control signal into a high-speed serial differential signal with strong anti-interference capability, thus obtaining the image serial differential signal. Therefore, by adopting GMSL2 technology, the signal attenuation and interference problems of long-distance transmission are overcome, and the robustness of the signal is significantly improved.

[0076] High-speed physical transmission interfaces include FAKRA (Fachkreis Automobil) connectors. FAKRA connectors are automotive-grade standardized interfaces for transmitting radio frequency signals.

[0077] In one embodiment, the image processing device sends an image serial differential signal to a FAKRA connector, which then transmits the image serial differential signal to a data processing terminal via a coaxial cable.

[0078] Step S320: The image serial differential signal is converted and processed by the data processing terminal to obtain a parallel data stream.

[0079] For detailed steps of step S320, please refer to step S110, which will not be repeated here.

[0080] Step S330: The parallel data stream is processed by protocol parsing through the bridging chip to obtain the parsed data stream.

[0081] For detailed steps of step S330, please refer to step S120, which will not be repeated here.

[0082] Step S340: The main chip performs data analysis and processing on the parsed data stream to obtain the target image.

[0083] For detailed steps of S340, please refer to step S130, which will not be repeated here.

[0084] The data acquisition end and the data processing end are connected via a coaxial cable and transmit data via the GMSL2 protocol.

[0085] The coaxial cable has a transmission distance of 15-20 meters, a data rate of 3Gbps-6Gbps, supports 4 independent camera inputs, and PoC (Power over Coax) power supply. During data transmission, the coaxial cable exhibits low loss and full shielding, achieving a maximum data rate of 6 Gbps, thus improving data transmission security and efficiency.

[0086] GMSL2 (Gigabit Multimedia Serial Link 2) is a high-speed serial interface technology. GMSL2 can simultaneously transmit high-definition video, audio, control signals, and power over a single coaxial cable or twisted pair.

[0087] In some embodiments, the data acquisition end is connected to the data processing end via a single coaxial cable, and the data acquisition end uses the GMSL2 protocol to transmit data with the data processing end.

[0088] It should be noted that there are contradictions between transmission bandwidth, distance, and reliability in existing technologies. For example, USB 3.0 has a short transmission distance, typically <3-5 meters, poor connector locking, lack of hardware-level retransmission, and weak resistance to electromagnetic interference, making it unsuitable for complex industrial environments. While Camera Link offers excellent performance, its cables are bulky, connectors are complex, and it does not support remote power supply, resulting in high deployment costs and limited bandwidth increases. Although GigE Vision has a long distance, its UDP (User Datagram Protocol)-based packet loss mechanism can lead to frame drops or data errors during network congestion. The retransmission mechanism may cause a "self-amplification effect," leading to a decrease in system performance, and the software protocol stack increases CPU load and latency. The FPD-Link III (Flat Panel Display Link III) solution has limited bandwidth (approximately 4.16Gbps), making it difficult to support the current high-resolution, high-frame-rate sensor data throughput.

[0089] This embodiment uses FAKRA connectors and automotive-grade coaxial cables at the data acquisition and data processing ends, and adopts the GMSL2 protocol to achieve high-speed data transmission of up to 6Gbps over a distance of up to 15 meters with an end-to-end latency of less than 1 millisecond. This improves the transmission distance and bandwidth, as well as the reliability and efficiency of the transmission.

[0090] The data transmission system includes an onboard unit and a protocol processing controller. One end of the protocol processing controller is connected to the main chip, and the other end is connected to the onboard unit. The data transmission system is connected to a host computer via a bus. The method further includes: writing the target image to the onboard unit in response to the protocol processing controller detecting that the bus of the data transmission system is busy; and transmitting the target image on the onboard unit to the host computer via the bus in response to the protocol processing controller detecting that the bus of the data transmission system is idle.

[0091] Onboard is a device used for storing data.

[0092] The protocol processing controller is a processor with read and write capabilities. It can be a DDR4 SDRAM (Double Data Rate 4 Synchronous Dynamic Random-Access Memory) controller.

[0093] A busy state refers to a situation where the amount of traffic processed by the bus exceeds a preset first standard value. An idle state refers to a situation where the amount of traffic processed by the bus is less than or equal to a preset second standard value. The first standard value is greater than the second standard value.

[0094] In one embodiment, the data transmission system detects the current transmission traffic volume of the bus in real time; if the current transmission traffic volume is greater than the first standard volume, the target image is written to the onboard; if the current transmission traffic volume is less than or equal to the second standard volume and the target image exists on the onboard, the target image on the onboard is read and transmitted to the host computer via the bus.

[0095] Combination Figure 5 As shown, the data transmission system may further include a first transmission branch and a second transmission branch. In the first transmission branch, the data acquisition end is connected to one end of the GMSL2 deserialization and driver module, and the other end of the GMSL2 deserialization and driver module is connected to one end of the main FPGA. In the second transmission branch, the data acquisition end is connected to one end of the GMSL2 deserialization and driver module, the other end of the GMSL2 deserialization and driver module is connected to one end of the bridging FPGA, and the other end of the bridging FPGA is connected to one end of the main FPGA. The other end of the main FPGA is connected to a DDR4 module, which is connected to the host computer via PCIe Gen3 x4 gold fingers.

[0096] In the first transmission branch, the main chip integrates a MIPI receiver core, a protocol parsing and preprocessing module, a stream parsing and framing module, and an image signal processing module. The image processing device receives the parallel data stream sent by the GMSL2 deserialization and driving module through the MIPI receiver core; it parses the parallel data stream through the protocol parsing and preprocessing module to obtain a data stream in low-voltage differential signal format; and it performs data analysis and processing on the parsed data stream through the stream parsing and framing module to obtain the target image.

[0097] In the second transmission branch, the main chip integrates an 8-channel LVDS interface, a dynamic phase calibration module, and a protocol parsing and preprocessing module.

[0098] The DDR4 (Double Data Rate 4) module is used to store the target image.

[0099] In one embodiment, the data transmission system integrates a high-performance DMA (Direct Memory Access) engine. This engine directly writes the target image from DDR4 memory to the host computer's memory via a Scatter-Gather method through a PCIe Gen3 x4 bus. The data transmission system also integrates a high-efficiency DC-DC (Direct Current to Direct Current Converter) power module and a PoC (Power over Coax) injection circuit, enabling a stable and reliable power supply to the data acquisition end through a single coaxial cable. This significantly simplifies system wiring, reduces installation complexity and overall cost, and improves the overall reliability of the system.

[0100] As can be seen, by using POC power supply on the system cabling and power supply side, the data acquisition end no longer needs to be powered independently, which reduces the installation complexity and maintenance cost.

[0101] Figure 6 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of this application. Figure 6 As shown, the exemplary image processing apparatus 600 includes: a conversion module 610, a parsing module 620, and an analysis module 630. Specifically: The conversion module 610 is used to convert the image serial differential signal received from the data acquisition terminal to obtain a parallel data stream.

[0102] The parsing module 620 is used to perform protocol parsing processing on the parallel data stream through the bridging chip to obtain the parsed data stream.

[0103] The analysis module 630 is used to perform data analysis and processing on the parsed data stream through the main chip to obtain the target image.

[0104] In this exemplary image processing apparatus, in response to receiving an image serial differential signal sent by a data acquisition terminal, the image serial differential signal is converted to obtain a parallel data stream; a bridging chip performs protocol parsing on the parallel data stream to obtain a parsed data stream; and the main chip performs data analysis on the parsed data stream to obtain the target image. Therefore, by using a bridging chip for protocol parsing, the resource pressure on the main chip is reduced, which helps improve the stability and efficiency of image processing.

[0105] The functions of each module can be found in the image processing method implementation examples, and will not be repeated here.

[0106] To implement the image processing method of the above embodiments, this application proposes another electronic device, please refer to [link / reference needed]. Figure 7 , Figure 7 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0107] Electronic device 700 includes memory 701 and processor 702, wherein memory 701 and processor 702 are coupled together.

[0108] The memory 701 is used to store program data, and the processor 702 is used to execute the program data to implement the image processing method of the above embodiment.

[0109] In this embodiment, processor 702 can also be referred to as CPU (Central Processing Unit). Processor 702 may be an integrated circuit chip with signal processing capabilities. Processor 702 can also be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 702 can be any conventional processor.

[0110] This application also provides a computer-readable storage medium, such as Figure 8 As shown, the computer-readable storage medium 800 is used to store program data 801, which, when executed by a processor, is used to implement the image processing method as described in the method embodiments of this application.

[0111] The methods involved in the image processing method embodiments of this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0113] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "and / or" is merely a description of the association of related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. In addition, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements, such as including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.

Claims

1. An image processing method, characterized by, Applied to a data processing end, the data processing end is communicatively connected to a data acquisition end, the data processing end includes a bridging chip and a main chip, and the method includes: In response to receiving the image serial differential signal sent by the data acquisition terminal, the image serial differential signal is converted to obtain a parallel data stream; The parallel data stream is processed by the bridging chip to obtain the parsed data stream; The main chip performs data analysis and processing on the parsed data stream to obtain the target image.

2. The method of claim 1, wherein, The bridging chip stores a preset protocol mapping table. The step of performing protocol parsing processing on the parallel data stream through the bridging chip to obtain the parsed data stream includes: The parallel data stream is mapped according to the preset protocol mapping table to obtain a data stream in low-voltage differential signal format corresponding to the parallel data stream. The low-voltage differential signal format data stream is determined as the parsed data stream.

3. The method of claim 1, wherein, The main chip stores an image processing clock. The step of performing data analysis and processing on the parsed data stream through the main chip to obtain the target image includes: The parsed data stream is calibrated based on the image processing clock to obtain a calibrated data stream. The target image is determined based on the calibrated data stream and the boundary information corresponding to the calibrated data stream.

4. The method of claim 1, wherein, The data processing end includes a deserializer. After the step of performing data analysis and processing on the parsed data stream through the main chip to obtain the target image, the method further includes: The main chip sends the acquired global synchronization pulse signal to the control interface of the deserializer. The global synchronization pulse signal is modulated by the deserializer.

5. An image processing method characterized by, The image processing method is applied to a data transmission system, which includes a data acquisition end and a data processing end. The data acquisition end and the data processing end are communicatively connected. The data processing end includes a bridge chip and a main chip. The image processing method includes: The image serial differential signal is acquired through the data acquisition terminal and sent to the data processing terminal. The image serial differential signal is converted and processed by the data processing terminal to obtain a parallel data stream; the parallel data stream is parsed by the bridging chip to obtain a parsed data stream; and the parsed data stream is analyzed and processed by the main chip to obtain the target image.

6. The method of claim 5, wherein, The data acquisition terminal and the data processing terminal are connected via a coaxial cable and transmit data via the GMSL2 protocol.

7. The method of claim 5, wherein, The step of acquiring the image serial differential signal through the data acquisition terminal includes: The acquired image data stream is preprocessed to obtain a preprocessed image data stream. The preprocessed image data stream is converted to obtain a standard data packet; The standard data packet and the acquired reverse road control signal are encoded to obtain the image serial differential signal.

8. The method according to claim 5, characterized in that, The data transmission system includes an onboard unit and a protocol processing controller. One end of the protocol processing controller is connected to the main chip, and the other end is connected to the onboard unit. The data transmission system is connected to a host computer via a bus. The method further includes: In response to the protocol processing controller detecting that the bus of the data transmission system is busy, the target image is written to the onboard. In response to the protocol processing controller detecting that the bus of the data transmission system is in an idle state, the target image on the board is transmitted to the host computer via the bus.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, include: The system stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images