Multi-camera preview and control method and system

Abstract

The embodiment of the invention provides a multi-camera preview and control method and system, and belongs to the technical field of image acquisition and embedded control. The method comprises the following steps: acquiring equipment identification information of multi-camera equipment and link state data of a multi-camera link; constructing multi-camera parameter configuration data based on the equipment identification information and the link state data; establishing a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data, and generating preview image data for representing real-time preview content of each camera; and generating a control instruction corresponding to each camera working mode based on the preview image data, and outputting the control instruction to control the running state of the multi-camera system. According to the scheme of the invention, a unified processing chain from camera identification, link analysis and parameter configuration to acquisition and control is constructed, so that preview, configuration and operation adjustment of multiple cameras are kept consistent and traceable under the same parameter system.

Description

Technical Field

[0001] This invention relates to the field of image acquisition and embedded control technology, specifically to a multi-camera preview and control method and a multi-camera preview and control system. Background Technology

[0002] In recent years, multi-camera systems have been widely used in autonomous driving, intelligent security, robot vision, and industrial inspection. These systems typically consist of multiple image sensors, transmission links, serializers / deserializers, and image processing modules. The debugging process involves camera preview, parameter configuration, link management, status monitoring, and synchronization verification. Due to the complexity of the system structure, the development and debugging of multi-camera systems usually rely on a combination of various tools or scripts.

[0003] In existing technologies, image preview is generally achieved through command-line tools. For example, tools such as gst-launch-1.0 and v4l2-ctl are often used for previewing video streams from a single camera. This approach usually requires setting up a separate preview pipeline for each camera and is difficult to display multiple video streams simultaneously, making it difficult to conduct unified monitoring in multi-camera scenarios.

[0004] In terms of parameter configuration, camera parameters such as exposure, gain, frame rate, resolution, and trigger mode are often set through I2C (Inter-Integrated Circuit) scripts, register lists, or driver interfaces. For systems using serializers / deserializers such as MAX9295A / 96712, multiple sets of register write scripts are also required to complete link initialization, virtual channel allocation, or mode switching. Because these configuration methods are scattered across multiple scripts or instructions, they typically lack a unified interface, making it inconvenient to quickly adjust parameters in multi-camera scenarios.

[0005] In terms of link management, bandwidth planning, virtual channel configuration, and link startup and shutdown for multiple transmission links such as MIPI (Mobile Industry Processor Interface) and GMSL (Gigabit Multimedia Serial Link) typically rely on manual operation. Link configuration files are often distributed across different modules or directories, requiring engineers to check each link parameter to ensure it matches the system architecture. In such operations, parameter inconsistencies or link conflicts are common, and troubleshooting usually requires a combination of methods, including register reading and log analysis.

[0006] In terms of status monitoring, existing debugging methods mostly rely on driver logs or third-party monitoring tools to obtain information such as frame rate, link bandwidth usage, error count, and frame loss, lacking a centralized display method. Engineers often need to switch between multiple interfaces to obtain the complete link status, which is not conducive to timely identification of link anomalies or verification of whether the configuration is effective.

[0007] Furthermore, multi-camera systems often involve externally triggered synchronization or time synchronization. Existing technologies are typically verified through external hardware devices or separate synchronization test scripts, and there is a lack of comprehensive debugging tools that can be used in conjunction with preview interfaces and parameter configuration interfaces.

[0008] Because different camera models and different link protocols (such as MIPI and GMSL) have differences in register format, data path structure and control interface, existing tools usually require redeveloping scripts or re-integrating driver interfaces to adapt to cross-platform or multi-type cameras, which is not convenient for system expansion and migration.

[0009] In summary, the preview, configuration, link management, and status monitoring of multi-camera systems usually rely on multiple tools and scripts to complete, resulting in a relatively fragmented process that is not conducive to unified debugging and maintenance. Summary of the Invention

[0010] The purpose of this invention is to provide a multi-camera preview and control method and system to at least solve the problems of scattered previews, fragmented parameter configurations, complex link management, and inconsistent status monitoring in the existing multi-camera development and debugging process.

[0011] To achieve the above objectives, a first aspect of the present invention provides a multi-camera preview and control method, the method comprising: acquiring device identification information of a multi-camera device and link status data of a multi-camera link; constructing multi-camera parameter configuration data based on the device identification information and the link status data; establishing a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data, and generating preview image data characterizing the real-time preview content of each camera; generating control instructions corresponding to the working mode of each camera based on the preview image data, and outputting the control instructions to control the operating state of the multi-camera system.

[0012] Optionally, acquiring device identification information of multi-camera devices and link status data of multi-camera links includes: performing segmented link probing operations on the links where each camera is located; extracting device identification information fragments and link status data fragments based on multiple preset sampling nodes of the multi-camera links; reorganizing the device identification information fragments according to the camera topology order to form device identification information used to characterize the attribute association relationship of multi-camera devices; and sorting the link status data fragments according to the link transmission direction to construct link status data used to characterize the link continuity characteristics.

[0013] Optionally, constructing multi-camera parameter configuration data based on the device identification information and the link status data includes: performing structured parsing on the operating mode parameters of each camera based on the device identification information to generate a set of mode parameters characterizing the camera's transmission capabilities; performing segmented bandwidth estimation on multiple preset bandwidth segments of the link based on the link status data to obtain bandwidth estimation data characterizing the link's available transmission characteristics; and constructing multi-camera parameter configuration data based on the set of mode parameters and the bandwidth estimation data.

[0014] Optionally, based on the device identification information, structured parsing is performed on the operating mode parameters of each camera to generate a set of mode parameters characterizing the camera's transmission capabilities. This includes: performing a bit-domain-level decomposition operation on the pixel structure field in the device identification information to extract pixel composition parameters characterizing the camera's output unit size; performing a frequency domain mapping operation on the clock configuration field in the device identification information to generate timing parameters characterizing the camera's internal timing characteristics; and combining the pixel composition parameters and the timing parameters according to a preset organization rule for the camera's data path to form a set of mode parameters characterizing the camera's transmission capabilities.

[0015] Optionally, based on the link status data, a segmented bandwidth estimation operation is performed on multiple preset bandwidth segments of the link to obtain bandwidth estimation data characterizing the available transmission characteristics of the link. This includes: performing a segment mapping operation on the bit error statistics field in the link status data to extract bit error segment data characterizing the bit error level of each preset bandwidth segment; performing a segment mapping operation on the delay jitter field in the link status data to obtain jitter segment data characterizing the delay fluctuation characteristics of each preset bandwidth segment; and combining the bit error segment data and the jitter segment data according to the segment order in the link transmission direction to form bandwidth estimation data characterizing the available transmission characteristics of the link.

[0016] Optionally, a multi-camera image acquisition pipeline is established based on the multi-camera parameter configuration data, and preview image data representing the real-time preview content of each camera is generated. This includes: performing a pixel path parsing operation on the image format parameters in the multi-camera parameter configuration data to extract pixel path data representing the data path mapping relationship of each camera; performing a time slice division operation on the frame timing parameters in the multi-camera parameter configuration data to generate time-series segment data representing the data output rhythm of each camera; constructing a multi-camera image acquisition pipeline based on the pixel path data and the time-series segment data, and performing frame-level organization operations on the image data output by each camera according to the multi-camera image acquisition pipeline to form preview image data representing the real-time preview content of each camera.

[0017] Optionally, a time-slice division operation is performed on the frame timing parameters in the multi-camera parameter configuration data to generate timing segment data that characterizes the output rhythm of each camera. This includes: performing a period quantization operation on the line period field in the frame timing parameters to extract line period quantization data that characterizes the line-level output rhythm; performing a synchronization phase splitting operation on the frame synchronization field in the frame timing parameters to generate phase splitting data that characterizes the frame-level synchronization relationship; and combining the line period quantization data and the phase splitting data according to a preset timing structure of the multi-camera data output to form timing segment data that characterizes the output rhythm of each camera.

[0018] Optionally, generating control commands corresponding to the operating modes of each camera based on the preview image data, and outputting the control commands to control the operating state of the multi-camera system, includes: performing a regionalized statistical operation on the brightness distribution field in the preview image data to extract brightness statistics data characterizing the brightness response characteristics of each camera; performing a directional analysis operation on the edge contour field in the preview image data to generate directional analysis data characterizing the imaging direction characteristics of each camera; constructing a control command parameter group corresponding to the operating modes of each camera based on the brightness statistics data and the directional analysis data; generating control commands according to the control command parameter group, and outputting the control commands to control the operating state of the multi-camera system.

[0019] Optionally, a control command parameter set corresponding to each camera operating mode is constructed based on the brightness statistics data and the directional analysis data, including: performing an interval discretization operation on the brightness statistics data to extract brightness discretized data for characterizing the camera brightness response level; performing a directional tilt angle quantization operation on the directional analysis data to generate directional quantization data for characterizing the camera imaging direction offset relationship; and combining the brightness discretized data and the directional quantization data according to a preset parameter mapping structure corresponding to the camera operating mode to form a control command parameter set corresponding to each camera operating mode.

[0020] A second aspect of the present invention provides a multi-camera preview and control system, the system comprising: an acquisition unit for acquiring device identification information of a multi-camera device and link status data of a multi-camera link; a processing unit for constructing multi-camera parameter configuration data based on the device identification information and the link status data; a preview unit for establishing a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data and generating preview image data characterizing the real-time preview content of each camera; and a control unit for generating control commands corresponding to the working modes of each camera based on the preview image data and outputting the control commands to control the operating state of the multi-camera system.

[0021] Through the above technical solution, the present invention acquires device identification information of multi-camera devices and link status data of multi-camera links, thereby forming input data to characterize the basic operating conditions of the system. Based on this input data, multi-camera parameter configuration data is constructed, allowing the working modes, image output structures, and link transmission characteristics of each camera to be described under a unified parameter system. Based on this parameter system, a multi-camera image acquisition pipeline is established and preview image data is generated, enabling the image content output by each camera to be organized and displayed in the same processing flow. Then, control commands corresponding to the camera working modes are generated based on the preview image data, establishing a correspondence between subsequent control actions and real-time imaging status, achieving continuous adjustment and updating of the multi-camera operating status. Overall, this method forms a continuous processing chain from link status acquisition, parameter construction, acquisition pipeline establishment to command output, providing a consistent data foundation and processing path for the preview, configuration, and operation control of the multi-camera system.

[0022] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0024] Figure 1 This is a flowchart of the steps of a multi-camera preview and control method provided in one embodiment of the present invention;

[0025] Figure 2 This is a system structure diagram of a multi-camera preview and control system provided in one embodiment of the present invention. Detailed Implementation

[0026] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0027] like Figure 1 As shown, embodiments of the present invention provide a multi-camera preview and control method, the method comprising:

[0028] Step S10: Obtain the device identification information of the multi-camera device and the link status data of the multi-camera link.

[0029] Specifically, a segmented link detection operation is performed on the link where each camera is located. Based on multiple preset sampling nodes of the multi-camera link, device identification information fragments and link status data fragments are extracted respectively. The device identification information fragments are reorganized according to the camera topology order to form device identification information used to characterize the association relationship of multi-camera device attributes. The link status data fragments are sorted according to the link transmission direction to construct link status data used to characterize the link continuity characteristics.

[0030] Furthermore, acquiring device identification information and link status data of multi-camera devices is the starting point of the entire method and the foundation for subsequent parameter construction and acquisition pipeline design. To avoid information contamination caused by a single coarse-grained scan, this implementation adopts a segmented link probing approach, dividing the multi-camera link into several segments based on spatial or logical location. Within each segment, a preset sampling node is selected, and the camera-related register groups, channel configuration fields, and link status registers are read using a unified probing rule. In this way, device identification information and link status data are separated by segment during the acquisition phase, eliminating the need for additional data splitting in subsequent processing stages.

[0031] In the specific execution of segmented link probing, each preset sampling node on a multi-camera link corresponds to a physical or logical location, such as an interface location near the camera, a serializer output location, or a deserializer input location. During probing, a read operation is initiated sequentially for each node to obtain device identification information fragments containing camera model, resolution capability, and supported operating mode markers, as well as link status data fragments containing phase-locked state, bit error count, and link reset flags. The arrangement of sampling nodes is not limited to endpoints and can also cover intermediate transition locations, facilitating the observation of link status changes along the transmission path in subsequent analysis. Through this node-segmented acquisition method, the spatial correlation between device-side information and link-side information is relatively clear.

[0032] For the device identification information fragments, further reorganization according to the camera topology order is required. A multi-camera topology typically consists of multiple cameras, multiple virtual channels, and one or more physical links; the logical arrangement order may not be entirely consistent with the physical wiring order. To avoid channel misalignment during subsequent parameter configuration, this implementation introduces a topology mapping table during the reorganization stage, matching each device identification information fragment with its corresponding camera logical location, virtual channel number, and physical port number. After matching, the identification information fragments are arranged sequentially according to the topology order and spliced ​​into a continuous device identification information data structure to represent the attribute relationships between multiple camera devices. This reorganization process presents the position and attributes of each camera in the overall structure as a traceable entry, facilitating subsequent parameter configuration and status checks performed in the same order.

[0033] The processing method for link state data segments differs slightly, focusing more on continuity analysis along the transmission path. Multi-camera links typically have a clear transmission direction, for example, starting from the camera side, passing through a serializer, transmission medium, and deserializer before reaching the master control end. To reflect this directional characteristic, this implementation sorts all link state data segments according to the link transmission direction, placing segments closer to the camera end at the beginning and segments closer to the master control end sequentially at the end. The sorted link state data not only records the local state of each node but also implicitly contains the trend of state changes along the transmission path. Based on this sorting result, a set of link state data can be constructed to characterize the link continuity characteristics, providing a structured description of issues such as whether there are intermediate breakpoints, sudden increases in local bit errors, and abnormal phase-locked state.

[0034] When constructing link continuity features, a set of simple analysis rules can be defined for the sorted link status data. For example, comparing whether the phase-locked loop (PLL) flags of adjacent nodes remain consistent, comparing whether the difference in bit error counts is within a preset threshold range, and comparing whether the reset flag changes abruptly at a certain node. As long as the order information of each sampling node is reliable during the initial acquisition phase, the analysis rules can make a relatively clear judgment on link continuity without adding complex algorithms. For cases where multiple links exist in parallel, a link number field can be added to the data structure to group and manage the status information of different links.

[0035] Through the aforementioned segmented link detection, topology reordering, and transmission direction sorting, multi-camera device identification information and multi-camera link status data are organized into two types of structured data. One type of data clearly expresses the position and attribute relationship of each camera in the overall topology, while the other type of data centrally expresses the continuity characteristics of the link along the transmission path. This processing method technically provides a unified data foundation for subsequent multi-camera parameter configuration, bandwidth estimation, and link health diagnosis, and also reduces the workload of repeatedly comparing logs and manually verifying channel correspondences during debugging.

[0036] Step S20: Construct multi-camera parameter configuration data based on the device identification information and the link status data.

[0037] Specifically, based on the device identification information, structured parsing is performed on the operating mode parameters of each camera to generate a set of mode parameters characterizing the camera's transmission capabilities; based on the link status data, segmented bandwidth estimation is performed on multiple preset bandwidth segments of the link to obtain bandwidth estimation data characterizing the available transmission characteristics of the link; and multi-camera parameter configuration data is constructed based on the set of mode parameters and the bandwidth estimation data.

[0038] Furthermore, based on the device identification information, structured parsing is performed on the operating mode parameters of each camera to generate a set of mode parameters characterizing the camera's transmission capabilities. This includes: performing bit-domain level decomposition on the pixel structure field in the device identification information to extract pixel composition parameters characterizing the camera's output unit size; performing frequency domain mapping on the clock configuration field in the device identification information to generate timing parameters characterizing the camera's internal timing characteristics; and combining the pixel composition parameters and the timing parameters according to preset organization rules of the camera's data path to form a set of mode parameters characterizing the camera's transmission capabilities.

[0039] Furthermore, based on the link status data, a segmented bandwidth estimation operation is performed on multiple preset bandwidth segments of the link to obtain bandwidth estimation data characterizing the available transmission characteristics of the link. This includes: performing a segment mapping operation on the bit error statistics field in the link status data to extract bit error segment data characterizing the bit error level of each preset bandwidth segment; performing a segment mapping operation on the delay jitter field in the link status data to obtain jitter segment data characterizing the delay fluctuation characteristics of each preset bandwidth segment; and combining the bit error segment data and the jitter segment data according to the segment order in the link transmission direction to form bandwidth estimation data characterizing the available transmission characteristics of the link.

[0040] In this embodiment of the invention, this step does not simply merge the two types of data, but rather, through a series of structured processes, links information about the camera's capabilities with data about the link's transmission characteristics into a parameter set that can be directly used for configuration decisions. This resulting set will play a fundamental role in subsequent image acquisition pipeline establishment, preview content generation, and control command output.

[0041] Structured parsing of camera operating mode parameters is the first step in constructing multi-camera parameter configuration data. The core of this step is extracting the elements that truly impact subsequent data path design from the device identification information and expressing these elements in a more explicit and engineering-friendly format. For example, pixel structure fields typically exist as register bit fields, containing information such as pixel depth, channel combination method, and subsampling structure. Without further decomposition, this information is difficult to use directly for data path planning. Therefore, a bit-field-level decomposition operation is used here to break down the pixel structure field into several basic units according to preset bit segment parsing rules, forming pixel composition parameters. This set of parameters provides a more intuitive understanding of the size and composition of camera output units and facilitates the determination of pixel processing paths in the image acquisition pipeline.

[0042] The clock configuration field is handled slightly differently. This field typically describes the camera's internal timing characteristics, such as pixel clock, line period, and frame period. Directly using the raw field limits the applicability of the parameter system because register structures often differ between manufacturers. To present timing information within a more general framework, the raw field needs to be mapped to the frequency domain. The goal of frequency domain mapping is not to perform complex signal analysis, but rather to make timing parameters more comparable and composable through a fixed mapping rule. The mapped timing parameters can express information such as the frequency, relative relationship, or multiplier of each clock signal, providing a fundamental temporal description for subsequent acquisition pipeline construction.

[0043] After processing the pixel structure field and clock configuration field, the two types of parameters need to be merged into a single mode parameter set. The merging rules typically depend on the camera's data path layout. For example, for some cameras, pixel depth and output format may directly determine the number of channels, while timing parameters affect the data output rhythm. To avoid inherent contradictions caused by arbitrary splicing, preset organizational rules are used during merging to standardize the parameter arrangement order, interrelationships, and expression granularity. The resulting mode parameter set can more realistically reflect the camera's transmission capabilities and express the logical correspondence between its pixel structure and timing definitions in the data link.

[0044] After forming the mode parameter set, another type of data that needs to be processed comes from link state information. Link state is usually more complex in multi-camera scenarios because multiple links may exist in parallel, and each link contains different bandwidth segments. In order to obtain a practical description of the available transmission characteristics of the link, segmented bandwidth estimation needs to be performed on the link state data according to bandwidth segments. Here, "segment" is not only a division of physical location, but can also be a logical region defined according to different virtual channels, transmission rate levels, or path structures. After segmentation, the bit error rate, latency fluctuation, and other characteristics of each segment are analyzed separately.

[0045] The bit error statistics field provides the bit error count or bit error rate at each sampling node of the link. Without organizing these values ​​by segment, it's difficult to determine the bit error trend. The segment mapping operation filters the bit error statistics field, retaining only the bit error data corresponding to that segment, forming bit error segment data. This data represents the bit error level within that segment, rather than the overall bit error value of the entire link, making subsequent judgments more targeted.

[0046] The latency jitter field records the time fluctuations of the link across different nodes. For multi-camera systems, latency fluctuations not only affect the synchronization of the link itself but also the continuity of data acquisition. Therefore, a segment mapping operation needs to be performed on the latency jitter field to map it to the corresponding analysis format for each segment, forming jitter segment data. This data expresses the latency fluctuation characteristics within each segment, providing time-dimensional information for bandwidth estimation.

[0047] Bandwidth estimation is not simply an aggregation of bit error and latency fluctuations; rather, it combines two data sequences from different segments according to the link's transmission direction. This combination process typically involves piecing together segments along the link direction and organizing the bit error and latency trends into a single data structure using linear or regularized methods. The resulting bandwidth estimation data describes the available transmission characteristics of the link at different locations, providing a basis for subsequent parameter configuration. This two-dimensional estimation method preserves the state differences between different segments, which is beneficial for rationally allocating bandwidth requirements based on link capabilities in multi-camera applications.

[0048] The mode parameter set and bandwidth estimation data describe two key features from the device side and the link side, respectively. To construct directly applicable multi-camera parameter configuration data, these two types of data need to be further combined. In the combination strategy, the basic bandwidth requirements of the camera on the link are typically determined first based on the camera's pixel structure and timing parameters, and then the link's capacity to support these requirements is assessed based on the bandwidth estimation data. If mismatches exist, conflicting locations need to be marked at the combination level, or alternative parameter paths need to be provided. For example, for certain high frame rate or high pixel depth camera modes, if the bandwidth estimation data indicates that a certain segment cannot be supported, this conflict should be recorded in the configuration data. The goal of the combination process is not to resolve conflicts, but to provide a one-time processed description, enabling subsequent processes to continue on the same parameter set.

[0049] In the final stage of parameter construction, the multi-camera parameter configuration data forms a dataset containing structured mode parameters and link capability information. This dataset can contain multiple subfields, such as camera capability fields, link capability fields, and conflict flag fields, with the logical relationships between these fields already determined in previous steps. This parameter dataset provides complete input for subsequent acquisition pipeline construction, ensuring clear rules for generating preview images and control commands.

[0050] In another possible implementation, a perturbation-based auxiliary processing method is introduced during the parameter construction stage to observe the implicit coupling relationship between device identification information and link status data. Specifically, small-amplitude parameter perturbations are applied to some key fields; for example, a controllable displacement is artificially inserted into the pixel structure field, and a low-amplitude virtual frequency offset is injected into the clock configuration field. The data before and after the perturbation can be parsed according to the same rules, forming two separate sets of mode parameters. Subsequently, by comparing the changing trends of corresponding parameters in the two sets, fields sensitive to perturbations are extracted to determine the degree of dependence of the camera mode on link conditions.

[0051] Similar perturbation injection can be performed on the link side. For example, a simulated bit error event can be constructed near the bit error statistics field, or a slight jitter with a fixed period can be added to the delay jitter field. Then, a perturbation-based bandwidth estimation data can be generated using a segmented bandwidth estimation method. By observing the magnitude of the changes in this data, sensitive nodes in the bandwidth segment can be identified, providing a more granular information basis for subsequent configuration. This perturbation-based analysis method does not directly change the parameter configuration process, but it forms an additional sensitivity description during the data construction stage, making the parameter system more complete in terms of expressive power.

[0052] Step S30: Establish a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data, and generate preview image data to characterize the real-time preview content of each camera.

[0053] Specifically, a pixel path parsing operation is performed on the image format parameters in the multi-camera parameter configuration data to extract pixel path data that characterizes the data path mapping relationship of each camera; a time slice division operation is performed on the frame timing parameters in the multi-camera parameter configuration data to generate time-series segment data that characterizes the data output rhythm of each camera; a multi-camera image acquisition pipeline is constructed based on the pixel path data and the time-series segment data, and a frame-level organization operation is performed on the image data output by each camera according to the multi-camera image acquisition pipeline to form preview image data that characterizes the real-time preview content of each camera.

[0054] Furthermore, a time-slice division operation is performed on the frame timing parameters in the multi-camera parameter configuration data to generate timing segment data that characterizes the output rhythm of each camera. This includes: performing a period quantization operation on the line period field in the frame timing parameters to extract line period quantization data that characterizes the line-level output rhythm; performing a synchronization phase splitting operation on the frame synchronization field in the frame timing parameters to generate phase splitting data that characterizes the frame-level synchronization relationship; and combining the line period quantization data and the phase splitting data according to a preset timing structure of multi-camera data output to form timing segment data that characterizes the output rhythm of each camera.

[0055] In this embodiment of the invention, the goal of this stage is not to construct a fixed acquisition framework, but rather to generate an acquisition structure that more closely resembles the actual data flow based on the format, timing, and path characteristics of the data defined by the aforementioned parameter configuration. Since different cameras have different pixel formats, output rhythms, and transmission capabilities, it is necessary to first clarify the mapping relationship between each camera in the data path when constructing the acquisition pipeline, and then determine the output method of these relationships in the time dimension. This approach makes subsequent frame-level organization clearer and also helps reduce structural conflicts in the real-time preview stage.

[0056] Pixel path parsing is a necessary step before constructing the acquisition pipeline. Image format parameters in multi-camera parameter configuration data often include pixel depth, number of output channels, and format type. Without structured parsing, these fields are difficult to directly reflect the data's path in the pipeline. Therefore, it is necessary to parse the format fields based on preset mapping rules to extract pixel path data. During parsing, each format field is broken down into several subfields according to its bit segment meaning, such as effective pixel bit width, channel number, and subsampling marker, and then assigned to different data path positions according to mapping rules. The pixel path data ultimately presents as a set of structures describing the data arrangement between the camera and the acquisition end, making subsequent path construction more operational.

[0057] While processing pixel path data, temporal information also needs to be organized, and time-slice partitioning is an operation designed to address this goal. Frame timing parameters typically contain multiple levels of periodic information; the row period field corresponds to the row-level output rhythm, and the frame synchronization field describes the phase relationship of the same frame at different nodes. If this timing information is still expressed as raw register values, it becomes difficult to organize multi-camera output on a unified timeline. Therefore, time-slice partitioning decomposes timing parameters into more comparable representations at a granular level, enabling the output rhythm to be assembled in a fragmented manner.

[0058] The line period field first undergoes period quantization. This operation converts the raw line period value into a standardized quantized value, allowing period differences between different cameras to be represented on a relatively uniform scale. For example, by dividing the line period by a preset baseline period, quantized line period data can be obtained. This quantized data clearly expresses the line output density of the camera, making the synchronization determination of multi-camera structures at the line level more direct. The quantization operation does not rely on complex algorithms; it simply performs a proportional transformation on the raw period value, facilitating engineering implementation.

[0059] The frame synchronization field requires phase splitting. Frame synchronization information is generally represented by the trigger phase of an internal clock signal, used to express frame-level synchronization relationships. Directly processing the raw phase values ​​makes it difficult to establish a comparison between multiple cameras. Therefore, a splitting operation maps the raw phase values ​​to several phase segments, each representing a specific time position within a frame period. This approach makes the synchronization relationship clearer at the representation level, such as whether they are within the same phase window or whether there is a fixed offset. The split phase structure also facilitates the insertion of synchronization reference points when constructing the acquisition pipeline.

[0060] Line periodic quantization data and phase splitting data do not exist in isolation; they need to be combined according to a preset temporal structure to form complete temporal segment data. This combination structure typically includes line-level segments, subframe segments, and full-frame segments, with each segment assembled according to a preset arrangement. The combined temporal segment data can express the internal rhythm of the camera and also serve as a time template for subsequent acquisition pipelines, giving the data output process a clearer temporal basis.

[0061] Once the pixel path data and temporal segment data have been processed, a multi-camera image acquisition pipeline can be built upon this foundation. The construction process includes three steps: path allocation, tempo stitching, and path matching. Path allocation maps different pixel segments to the channel structure of the acquisition end based on the pixel path data. Tempo stitching inserts output segments into the global timeline based on the temporal segment data. Path matching confirms whether the correspondence between paths and tempos conforms to the parameter definitions. The resulting acquisition pipeline can be viewed as a set of rules used to guide the organization of real-time data.

[0062] After the acquisition pipeline is constructed, performing frame-level organization on the image data output by each camera becomes a natural next step. The core of frame-level organization is to arrange the raw output content at the frame level according to the path mapping and time segments of the acquisition pipeline, forming structured image data that can be used for real-time preview. This organization method typically includes two criteria: first, determining the correspondence between pixel paths in spatial layout; and second, determining whether the position of the time segment on the time axis meets the pipeline requirements. Only image content that meets these two criteria will be assembled into preview frames, thus ensuring the logical consistency between the preview image and the parameter configuration.

[0063] Once organized, the preview image data can be directly used for higher-level display or control. Its high degree of structure includes frame indexes, timestamps, pixel format markers, and corresponding acquisition pipeline descriptions, making it suitable not only for real-time display but also for facilitating subsequent control command derivation. To ensure the consistency of the preview image, the acquisition pipeline must adhere to the aforementioned pixel path and timing segment rules when organizing frame content. This results in preview image data that can reliably represent the real-time output status of each camera.

[0064] Step S40: Generate control commands corresponding to the working modes of each camera based on the preview image data, and output the control commands to control the operating status of the multi-camera system.

[0065] Specifically, a regional statistical operation is performed on the brightness distribution field in the preview image data to extract brightness statistics data that characterize the brightness response characteristics of each camera; a directional analysis operation is performed on the edge contour field in the preview image data to generate directional analysis data that characterizes the imaging direction characteristics of each camera; a control instruction parameter set corresponding to the working mode of each camera is constructed based on the brightness statistics data and the directional analysis data; control instructions are generated according to the control instruction parameter set, and the control instructions are output to control the operating state of the multi-camera system.

[0066] Furthermore, an interval discretization operation is performed on the brightness statistics data to extract brightness discretization data used to characterize the camera brightness response level; an orientation tilt quantization operation is performed on the directional analytical data to generate orientation quantization data used to characterize the camera imaging orientation offset relationship; the brightness discretization data and the orientation quantization data are combined according to a preset parameter mapping structure corresponding to the camera operating mode to form a control command parameter group corresponding to each camera operating mode.

[0067] In this embodiment of the invention, the preceding steps have already output structured preview image data, which includes not only pixel content but also frame-related format markers and time information. Based on this, this implementation does not directly set parameters based on experience, but instead first extracts statistical features from the image itself, and then compresses these features into control quantities related to the operating mode. The control chain constructed in this way relies more on objective image information, reducing the reliance on default register values.

[0068] For processing the brightness distribution field, regionalized statistical operations are a starting point. Preview image data is often organized frame by frame, but the brightness distribution within a single frame is not spatially uniform. To reflect this spatial difference, the target image is first divided into several regions, which can be a regular grid or divided according to the region of interest. For each region, the mean brightness, brightness variance, or median is calculated, and these are combined to form brightness statistics. The resulting statistics are no longer just a global average, but include the brightness response of local areas, more closely reflecting the actual scene's requirements for camera exposure. The results of regionalized statistics will directly participate in the generation of control parameters, so the region division method can be flexibly adjusted according to the specific application scenario.

[0069] Directional resolution of edge contour fields focuses on structural information in the image. Many multi-camera applications require attention to the sharpness and directional distribution of target boundaries, such as lane lines, building edges, and equipment outlines. Edge contour fields in preview image data can be extracted by front-end operators, such as gradient-based edge detection or kernel-based contour recognition. The directional resolution operation classifies and statistically analyzes these edge points according to their orientation angles, forming directional resolution data. The statistical method can be to divide the orientation angles into several directional intervals and statistically analyze the edge intensity or edge length within each interval, thereby obtaining a general expression of the imaging direction characteristics. This expression facilitates the determination of whether a particular camera has been misaligned or whether there is a field-of-view rotation deviation in a multi-camera layout.

[0070] After obtaining brightness statistics and directional resolution data, it is necessary to construct a set of control command parameters corresponding to each camera's operating mode. The operating mode here includes not only exposure mode and gain level, but may also be associated with resolution configuration, frame rate selection, and image rotation or cropping strategies. The process of constructing the parameter set generally involves first determining whether the current exposure response deviates from the expected range based on the brightness statistics, and then combining this with the directional resolution data to determine whether the image orientation is consistent with the calibrated pose. In a simpler implementation, when the brightness is too low, a set of higher exposure or gain configurations can be written into the parameter set; when the directional deviation exceeds the allowable range, a set of mode markers specifically for rotation or geometric adjustment can be written. The final control command parameter set can be viewed as a set of parameter candidates driven by various image features.

[0071] In the control command generation stage, control commands are generated based on the control command parameter set and then output to the multi-camera operating environment. The generation process typically maps each field in the parameter set to specific register write actions or driver call information, such as exposure register write values, frame rate switching commands, and flip configuration flags. The output path can be an I2C control channel or a downward transmission via the driver interface. As long as the mapping rules between the parameter set and the command format are predefined, the generation and output of control commands can be completed within a unified framework. This establishes a more direct connection between the status information in the preview screen and the configuration activities on the control end.

[0072] Furthermore, to make the control behavior more stable and less sensitive to single-frame fluctuations, this implementation performs interval discretization on the brightness statistics. The basic idea of ​​interval discretization is to divide the continuous brightness index into several levels, such as dark, slightly dark, normal, slightly bright, and overly bright, or even more granular level intervals can be defined. Each interval corresponds to a discrete value, used to represent the brightness response level of the current image. The brightness discretization data generated after discretization is no longer easily affected by single-point anomalies or local noise, making it more suitable as an input to the control logic. For multi-camera collaborative scenarios, a set of discretized brightness levels is also more convenient for comparing the relative states of different cameras.

[0073] Directional analysis data is further compressed through directional tilt quantization. The edge directions in an image are themselves a continuous angular distribution. To represent this offset relationship in the control command parameter set, the continuous angles need to be quantized into a finite number of levels. Quantization can be performed using equal intervals, or a finer directional window can be set based on calibration results. The quantized directional data can express the offset range of the camera's imaging direction relative to the reference direction, without needing to be precise to a single angle value. This allows the control logic to handle slight and significant offsets differently and also reduces the impact of numerical jitter.

[0074] Brightness discretization data and orientation quantization data do not drive independent control channels separately. Instead, they are combined according to a preset parameter mapping structure corresponding to the camera's operating mode. This mapping structure can be understood as a mapping table or a set of rules, specifying which operating mode configuration should be used under different combinations of brightness levels and orientation offset levels. For example, when the brightness is at a low level and the orientation quantization data shows that the camera attitude is normal, a mode to increase exposure can be selected; when the brightness is normal but the orientation offset is large, a mode to adjust the field of view can be selected; when both deviate from the expected values, two types of adjustment parameters can be used in combination. The combined result is a set of control command parameters corresponding to each camera's operating mode, with clear triggering conditions and adjustment directions.

[0075] A control variable inference method based on local structural consistency is incorporated into the preview image data processing stage to assist in determining whether the camera's operating mode needs adjustment. This method does not rely on common image features such as brightness or edge direction, but instead focuses on the degree of structural repetition between multiple consecutive preview frames. The degree of structural repetition can be obtained by calculating the texture consistency index of local blocks, for example, by extracting the gradient co-occurrence matrix in each sampling region and then defining a consistency score based on the concentration of matrix elements. Changes in consistency across consecutive frames can reflect scene dynamics and also indicate whether the camera's current mode matches the environmental conditions.

[0076] During processing, several local regions are first selected from the preview image data, and a texture consistency index is calculated for each region to form a consistency vector. The trend of this vector changing over time can describe the stability of the preview content. For example, if the texture consistency of multiple regions suddenly decreases in a short period of time, it may mean the loss of detail due to underexposure, or it may mean local perturbation caused by noise enhancement. Collecting the changes in these regions uniformly can form a set of local structural feature data.

[0077] Subsequently, based on the preset structural stability level classification rules, the aforementioned feature data is discretized into several levels, such as stable, slightly fluctuating, and strongly fluctuating. Then, combined with the camera's operating mode characteristics, a set of structural stability parameters is constructed. For example, if the stability level is high, the current mode can be maintained; if the fluctuation level is in the medium range, the gain can be adjusted to obtain clearer textures; if the fluctuation level is too high, the scene can be considered to determine whether to switch to high dynamic range mode.

[0078] Finally, control commands are generated based on the structural stability parameter set. This embodiment indirectly determines camera configuration requirements through microscopic changes in texture consistency, providing a set of auxiliary criteria for mode switching independent of brightness and geometric features.

[0079] In one specific implementation, the multi-camera preview and control method is organized and executed according to a pre-defined process mechanism, with each step forming a continuous processing chain through clearly defined data structures and triggering rules. During the startup phase, the device identification information and link status data of the connected camera devices are first acquired. This acquisition process is completed by the protocol adaptation module through low-level register interaction, returning information including the image sensor number, output format encoding, link status flags of the serializer and deserializer, and error statistics for each node. After receiving the above data, the task flow management module registers the device identification information with the camera management module and generates a preliminary status description for judging the link health based on the link status data.

[0080] Once the link status description is generated, further analysis will be performed according to the link configuration and diagnostic rules. This includes segmented reading of the status registers of each node, checking the link timing markers, and comparing the virtual channel status. The analysis results will generate a link continuity judgment table to indicate whether each link meets the basic conditions for subsequent data transmission. If a link experiences a phase-locked loop anomaly, a sudden increase in bit errors, or the channel is not available at a certain node, the task flow management module will switch to the link anomaly handling branch according to the preset state machine, prompting the user to check the connection or perform an automatic reset operation to restore the link to a state where it can continue to operate.

[0081] Once the link is stable, multi-camera parameter configuration data can be constructed based on previously acquired device identification information and link status data. The camera management module estimates bandwidth requirements according to the number of cameras, resolution, frame rate, pixel depth, and virtual channel mapping relationship, and compares this estimate with the bandwidth data. If the expected bandwidth is close to the link's capacity limit, parameter adjustment suggestions are provided, including modifying the resolution level, reducing the frame rate, or changing channel allocation, to maintain an acceptable match between link resources and data output. After parameter confirmation, the task flow management module executes camera initialization in a predetermined order, including power supply control, reset pulse, clock start, sensor initialization, MIPI / GMSL configuration writing, and loading of basic parameters such as exposure and gain. The execution result of each step is recorded as a status flag to allow for rollback actions in case of initialization failure.

[0082] After initialization, a multi-camera image acquisition pipeline can be constructed based on multi-camera parameter configuration data. The image processing module determines the multi-channel input mapping relationship based on pixel path data, and then establishes a global temporal structure based on time-series segment data, enabling the image content output by each camera to be organized within a unified processing framework. After the acquisition pipeline is generated, the video processing module begins to receive multiple image data streams and performs real-time stitching according to the user interface layout to form preview image data. The stitching process follows frame-level organization rules to ensure that the frame content of each stream remains consistent with the temporal information.

[0083] When a user needs to switch the operating mode of any camera, such as changing the resolution, enabling HDR, or changing the output format, the task flow management module executes a control command generation process based on the preview image data. First, it performs a regional statistical operation on the brightness distribution field in the preview image data to extract brightness statistics; then, it performs a directional analysis operation on the edge contour field to form directional analysis data. Based on these two types of data, a control command parameter set is constructed, and control commands are generated according to the parameter mapping relationship. Before issuing the command, a bandwidth assessment is performed again to prevent a sudden increase in link load caused by mode switching. After confirming feasibility, the module pauses the video stream, writes the new operating mode parameters, and then resumes the preview display. The entire process does not require the user to manually input multiple commands.

[0084] If the user enables the multi-camera synchronization trigger function, the task flow management module will generate synchronization control commands based on synchronization parameters and write them to the camera or serializer through the protocol adapter module, ensuring that the exposure start times of multiple cameras are consistent. The synchronization process can be set to hard synchronization or soft synchronization, which can be flexibly switched according to application requirements. During system operation, operation logs will be continuously recorded, and situations such as link drops, frame rate fluctuations, or bandwidth overruns will be monitored and marked. When an anomaly occurs, the task flow management module will locate the abnormal link based on the logs and provide prompts to enable users to quickly troubleshoot problems.

[0085] Through this streamlined execution approach, the steps involved in the method, such as device identification acquisition, link analysis, parameter construction, acquisition pipeline creation, preview image generation, and control command output, maintain consistency through clear data transmission relationships, ensuring that the multi-camera system maintains a traceable and structured processing path during debugging and operation.

[0086] like Figure 2 As shown, this invention provides a multi-camera preview and control system, the system comprising: an acquisition unit for acquiring device identification information of a multi-camera device and link status data of the multi-camera link; a processing unit for constructing multi-camera parameter configuration data based on the device identification information and the link status data; a preview unit for establishing a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data and generating preview image data characterizing the real-time preview content of each camera; and a control unit for generating control commands corresponding to the working modes of each camera based on the preview image data and outputting the control commands to control the operating state of the multi-camera system.

[0087] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0088] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0089] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A multi-camera preview and control method, characterized in that, The method includes: Obtain device identification information of multi-camera devices and link status data of multi-camera links; Multi-camera parameter configuration data is constructed based on the device identification information and the link status data; A multi-camera image acquisition pipeline is established based on the multi-camera parameter configuration data, and preview image data is generated to characterize the real-time preview content of each camera. Based on the preview image data, control commands corresponding to the working modes of each camera are generated, and the control commands are output to control the operating status of the multi-camera system.

2. The method according to claim 1, characterized in that, Obtain device identification information for multi-camera devices and link status data for multi-camera links, including: Perform segmented link detection operation on the link where each camera is located, and extract device identification information fragments and link status data fragments based on multiple preset sampling nodes of the multi-camera link; The device identification information fragments are reassembled according to the camera topology order to form device identification information used to characterize the attribute association relationship of multiple camera devices; The link state data segments are sorted according to the link transmission direction to construct link state data that characterizes the link continuity features.

3. The method according to claim 1, characterized in that, Multi-camera parameter configuration data is constructed based on the device identification information and the link status data, including: Based on the device identification information, the working mode parameters of each camera are subjected to structured parsing to generate a set of mode parameters that characterize the camera's transmission capabilities. Based on the link status data, segmented bandwidth estimation operations are performed on multiple preset bandwidth segments of the link to obtain bandwidth estimation data that characterizes the available transmission characteristics of the link. Multi-camera parameter configuration data is constructed based on the set of mode parameters and the bandwidth estimation data.

4. The method according to claim 3, characterized in that, Based on the device identification information, structured parsing is performed on the operating mode parameters of each camera to generate a set of mode parameters characterizing the camera's transmission capabilities, including: A bit-domain level decomposition operation is performed on the pixel structure field in the device identification information to extract pixel composition parameters that characterize the size of the camera output unit; A frequency domain mapping operation is performed on the clock configuration field in the device identification information to generate timing parameters that characterize the internal timing features of the camera; The pixel composition parameters and the timing parameters are combined according to the preset organization rules of the camera data path to form a set of mode parameters used to characterize the camera's transmission capabilities.

5. The method according to claim 3, characterized in that, Based on the link state data, segment-based bandwidth estimation is performed on multiple preset bandwidth segments of the link to obtain bandwidth estimation data characterizing the available transmission characteristics of the link, including: Perform a segment correspondence operation on the bit error statistics field in the link status data to extract bit error segment data that characterizes the bit error level of each preset bandwidth segment; Perform a segment mapping operation on the latency jitter field in the link status data to obtain jitter segment data that characterizes the latency fluctuation characteristics of each preset bandwidth segment; The error segment data and the jitter segment data are combined in the segment order according to the link transmission direction to form bandwidth estimation data that characterizes the available transmission characteristics of the link.

6. The method according to claim 1, characterized in that, A multi-camera image acquisition pipeline is established based on the multi-camera parameter configuration data, and preview image data is generated to characterize the real-time preview content of each camera, including: A pixel path parsing operation is performed on the image format parameters in the multi-camera parameter configuration data to extract pixel path data that characterizes the data path mapping relationship of each camera. A time-slice division operation is performed on the frame timing parameters in the multi-camera parameter configuration data to generate time-series segment data that characterizes the data output rhythm of each camera; A multi-camera image acquisition pipeline is constructed based on the pixel path data and the temporal segment data. Frame-level organization operations are then performed on the image data output by each camera according to the multi-camera image acquisition pipeline to form preview image data that characterizes the real-time preview content of each camera.

7. The method according to claim 6, characterized in that, Perform time-slice division on the frame timing parameters in the multi-camera parameter configuration data to generate time-series segment data characterizing the data output rhythm of each camera, including: Perform periodic quantization on the line period field in the frame timing parameters to extract line periodic quantization data that characterizes the line-level output rhythm; Perform a synchronization phase splitting operation on the frame synchronization field in the frame timing parameters to generate phase splitting data that characterizes the frame-level synchronization relationship; The line period quantization data and the phase split data are combined according to a preset time sequence structure of multi-camera data output to form time sequence segment data that characterizes the data output rhythm of each camera.

8. The method according to claim 1, characterized in that, Based on the preview image data, control commands corresponding to the operating modes of each camera are generated, and the control commands are output to control the operating state of the multi-camera system, including: Perform a regionalized statistical operation on the brightness distribution field in the preview image data to extract brightness statistics data that characterize the brightness response characteristics of each camera; Perform a directional parsing operation on the edge contour field in the preview image data to generate directional parsing data that characterizes the imaging direction characteristics of each camera; Based on the brightness statistics and the directionality analysis data, a control command parameter group corresponding to the working mode of each camera is constructed; Control commands are generated based on the control command parameter set, and the control commands are output to control the operating status of the multi-camera system.

9. The method according to claim 8, characterized in that, Based on the brightness statistics and the directional analysis data, a set of control command parameters corresponding to the operating modes of each camera is constructed, including: Perform interval discretization on the brightness statistics to extract brightness discretization data that characterizes the camera's brightness response level; Perform orientation tilt quantization on the orientation analytical data to generate orientation quantization data that characterizes the camera imaging orientation offset relationship; The brightness discretized data and the direction quantization data are combined according to a preset parameter mapping structure corresponding to the camera's operating mode to form a control command parameter group corresponding to each camera's operating mode.

10. A multi-camera preview and control system, characterized in that, The system includes: The acquisition unit is used to acquire device identification information of multi-camera devices and link status data of multi-camera links; The processing unit is used to construct multi-camera parameter configuration data based on the device identification information and the link status data; The preview unit is used to establish a multi-camera image acquisition pipeline based on the multi-camera parameter configuration data, and generate preview image data to characterize the real-time preview content of each camera. The control unit is used to generate control commands corresponding to the working modes of each camera based on the preview image data, and output the control commands to control the operating status of the multi-camera system.

Images