Design method and system of vehicle exterior monitoring system based on panoramic camera

Abstract

The invention relates to the technical field of road engineering, in particular to a design method and system for a vehicle exterior monitoring system based on a panoramic camera, and the method comprises the steps: recognizing and determining key function demands of the vehicle exterior monitoring system, and obtaining a demand analysis result; according to a demand analysis result, a multi-camera layout is adopted, the positions of the cameras are configured, and a complete system design drawing is generated; combining the images captured by the plurality of cameras to form a complete 360-degree view; establishing a deep learning model, performing real-time detection and identification on the dynamic target in the 360-degree view, and recording behavior data of the dynamic target in combination with a target tracking algorithm; and constructing an automatic monitoring parameter adjusting mechanism, and carrying out adaptive adjustment on related parameters of the camera according to real-time illumination and weather conditions. According to the invention, the problems of visual angle limitation, monitoring blind areas and insufficient environmental adaptability in a traditional vehicle exterior monitoring system are effectively solved, and the adaptive capacities of real-time monitoring, target identification and complex environments are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road engineering, and in particular to a design method and system for an off-vehicle monitoring system based on a panoramic camera. Background Art

[0002] In the field of intelligent transportation and vehicle safety, the vehicle exterior monitoring system has gradually become an important means to improve driving safety. Traditional vehicle exterior monitoring technologies mostly rely on a single camera for image capture, which has disadvantages such as viewing angle limitation, monitoring blind spots and data processing delays. These systems often cannot meet the safety requirements in complex traffic environments, leading to potential safety hazards. In recent years, the rapid development of panoramic camera technology has provided new solutions for the improvement of vehicle exterior monitoring systems. Panoramic cameras can cover a wider field of view and eliminate the blind spots in traditional monitoring.

[0003] However, existing panoramic monitoring systems still face challenges in image processing and real-time monitoring capabilities, especially in target detection and recognition in dynamic scenes. At the same time, current systems often lack sufficient environmental adaptability and cannot automatically adjust monitoring parameters according to different weather and lighting conditions, resulting in unstable monitoring effects. Summary of the invention

[0004] The present invention provides a design method and system for an off-vehicle monitoring system based on a panoramic camera, thereby effectively solving the problems pointed out in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A design method for an exterior vehicle monitoring system based on a panoramic camera includes: Conduct demand analysis on the vehicle exterior monitoring system, identify and determine the key functional requirements of the vehicle exterior monitoring system, and obtain demand analysis results; According to the requirements analysis results, a multi-camera layout is adopted, camera positions are configured, and a complete system design diagram is generated; Based on the system design and image stitching algorithm, the images captured by multiple cameras are combined to form a complete 360-degree view; Establishing a deep learning model to detect and identify dynamic targets in the 360-degree view in real time, generating target detection results, and combining with a target tracking algorithm to record the behavior data of the dynamic targets; Build an automatic monitoring parameter adjustment mechanism to adaptively adjust the relevant parameters of the camera according to real-time lighting and weather conditions, and obtain the final monitoring system design plan.

[0006] Furthermore, the requirements of the off-vehicle monitoring system are analyzed to identify and determine the key functional requirements of the off-vehicle monitoring system, and the requirements analysis results are obtained, including: Conduct targeted research through a systematic approach to collect specific needs and expectations for using the off-vehicle monitoring system; Analyze the functional characteristics and technical performance of the vehicle exterior monitoring system, and identify demand gaps by comparing the advantages and disadvantages of different products; According to the target survey and demand analysis results, determine the core functions of the off-vehicle monitoring system and generate a functional requirements list; Conduct a feasibility analysis on the functional requirements list, identify the required hardware and software support, and obtain the requirements analysis results.

[0007] Furthermore, according to the demand analysis results, a multi-camera layout is adopted, the camera positions are configured, and a complete system design diagram is generated, including: Based on the demand analysis results, a dynamic optimization algorithm is used to divide the monitoring area and determine the optimal layout of multiple cameras; Based on 3D modeling technology, different camera layout schemes are simulated, the position of each camera is configured according to the determined layout, and a complete system design diagram is generated.

[0008] Furthermore, based on the system design and image stitching algorithm, the images captured by multiple cameras are combined to form a complete 360-degree view, including: Based on the system design diagram, image data captured by each camera is obtained; Apply image stitching algorithms to pre-process images captured by multiple cameras; The preprocessed images are combined, a seamless 360-degree view is generated by the image stitching algorithm, and the 360-degree view is output.

[0009] Furthermore, the image stitching algorithm includes: Preprocessing the acquired image data, wherein the preprocessing includes denoising, color correction and geometric correction; A feature point detection algorithm is used to identify key feature points in each image, and descriptors of the key feature points are extracted to perform matching between images; The transformation matrix is ​​calculated using the matched feature points to align the images, and the aligned images are fused to generate a complete stitched image.

[0010] Furthermore, a deep learning model is established to detect and identify dynamic targets in the 360-degree view in real time and generate target detection results, including: Establishing a deep learning model and training the deep learning model using a labeled training data set; Deploy the deep learning model in the 360-degree view image data stream acquired in real time, use the deep learning model to extract high-dimensional features in the image, and aggregate different feature information through a feature fusion algorithm; Based on the feature extraction and analysis results, the detection results of dynamic targets are generated.

[0011] Furthermore, in combination with the target tracking algorithm, the behavior data of the dynamic target is recorded, including: Acquire dynamic targets in a 360-degree view, apply multi-target tracking algorithms to track targets in real time, and record the dynamic characteristics of targets; Accurately model the target's movement behavior and generate multi-dimensional data related to the target's behavior; Use deep learning algorithms to analyze behavioral data, identify the target's behavioral patterns through time series modeling, predict the target's future behavior, and detect in real time whether the target has abnormal behavior or potential threats, and generate behavioral trend reports.

[0012] Furthermore, an automatic monitoring parameter adjustment mechanism is constructed to adaptively adjust the relevant parameters of the camera according to the real-time lighting and weather conditions, and the final monitoring system design is obtained, including: By integrating multiple sensors to monitor the light intensity and weather conditions of the external environment in real time, real-time environmental data can be obtained; Based on the environmental data acquired in real time, a data analysis algorithm is used to evaluate various environmental parameters in real time to determine the impact of the current environment on the camera imaging quality; According to the evaluation results, the relevant parameters of the camera are automatically adjusted and optimized through the feedback mechanism to generate the final monitoring system design plan.

[0013] A design system for an exterior vehicle monitoring system based on a panoramic camera, the system comprising: The demand analysis result acquisition module performs demand analysis on the off-vehicle monitoring system, identifies and determines the key functional requirements of the off-vehicle monitoring system, and obtains the demand analysis results; A system design diagram generation module, which adopts a multi-camera layout, configures camera positions, and generates a complete system design diagram according to the demand analysis results; A 360-degree view generation module, based on the system design diagram and image stitching algorithm, combines the images captured by multiple cameras to form a complete 360-degree view; A target detection result generation module is used to establish a deep learning model, detect and identify dynamic targets in the 360-degree view in real time, generate target detection results, and record the behavior data of the dynamic targets in combination with a target tracking algorithm; The system design scheme generation module builds an automatic monitoring parameter adjustment mechanism, adaptively adjusts the relevant parameters of the camera according to real-time lighting and weather conditions, and obtains the final monitoring system design scheme.

[0014] Furthermore, the system design scheme generating module includes: The environmental data acquisition unit monitors the light intensity and weather conditions of the external environment in real time by integrating multiple sensors to obtain real-time environmental data; A real-time parameter evaluation unit, based on the environmental data acquired in real time, uses a data analysis algorithm to perform real-time evaluation on various environmental parameters to determine the impact of the current environment on the camera imaging quality; The feedback mechanism optimization unit automatically adjusts the relevant parameters of the camera according to the evaluation results, and optimizes through the feedback mechanism to generate the final monitoring system design plan.

[0015] The technical solution of the present invention can achieve the following technical effects: It effectively solves the problems of viewing angle limitation, monitoring blind spots and insufficient environmental adaptability existing in traditional off-vehicle monitoring systems, and improves the ability of real-time monitoring, target recognition and adaptability in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a design method for an exterior vehicle monitoring system based on a panoramic camera; Figure 2 A flowchart for obtaining the demand analysis results; Figure 3 A flow chart to generate a complete system design diagram; Figure 4 A schematic diagram of the process to form a complete 360-degree view; Figure 5 It is a flowchart of the image stitching algorithm; Figure 6 A schematic diagram of the process of generating target detection results; Figure 7 A schematic diagram of the process of recording the behavioral data of dynamic targets in combination with the target tracking algorithm; Figure 8 A flow chart showing the final design of the monitoring system is provided. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0020] Embodiment 1 like Figure 1 As shown, the present invention provides a design method for an exterior vehicle monitoring system based on a panoramic camera, the method comprising: S1: Conduct demand analysis on the vehicle exterior monitoring system, identify and determine the key functional requirements of the vehicle exterior monitoring system, and obtain demand analysis results; Specifically, this step deeply understands and clarifies the specific application scenarios and operating environment of the off-vehicle monitoring system, and extracts the core functions and performance standards that the system must have. Through demand analysis, it is possible to identify monitoring requirements under different environmental conditions (such as day and night, bad weather, and complex traffic), including technical indicators such as field of view coverage, image clarity, target recognition accuracy, and response speed. In addition, this step also involves an in-depth understanding of potential safety hazards, user convenience, and system scalability to ensure that the generated demand analysis results not only meet the basic requirements of users.

[0021] S2: Based on the demand analysis results, adopt a multi-camera layout, configure the camera positions, and generate a complete system design diagram; Specifically, this step formulates a reasonable camera layout plan based on the results of the demand analysis to achieve full coverage of the monitoring area and eliminate blind spots to the maximum extent. By accurately configuring the camera position, the viewing angle, installation height and coverage of each camera are optimized, so that the monitoring system can obtain high-quality image data at any angle and distance. This layout design also takes into account the impact of environmental factors (such as lighting, obstructions, weather conditions, etc.) on the camera effect, thereby ensuring that the system has stable imaging capabilities in complex environments. The final system design diagram clarifies the specific location, viewing angle coverage and equipment layout of each camera.

[0022] S3: Based on the system design and image stitching algorithm, the images captured by multiple cameras are combined to form a complete 360-degree view; Specifically, the purpose of this step is to use the camera layout in the system design diagram and integrate the independent images collected by each camera into a seamless 360-degree panoramic view through an image stitching algorithm to achieve full coverage imaging of the monitoring area. This process eliminates the edge seams and viewing angle differences between cameras through an efficient image stitching algorithm, allowing images from different cameras to blend naturally to form a complete monitoring screen with no blind spots. This 360-degree view provides a unified monitoring basis for subsequent target detection and tracking, ensuring that the system can capture and process any dynamic targets in the monitoring area in real time, significantly improving the system's monitoring effectiveness and visual integrity.

[0023] S4: Establish a deep learning model to detect and identify dynamic targets in the 360-degree view in real time, generate target detection results, and combine with the target tracking algorithm to record the behavior data of dynamic targets; Specifically, the purpose of this step is to use the camera layout in the system design diagram and integrate the independent images collected by each camera into a seamless 360-degree panoramic view through an image stitching algorithm to achieve full coverage imaging of the monitoring area. This process eliminates the edge seams and viewing angle differences between cameras through an efficient image stitching algorithm, allowing images from different cameras to blend naturally to form a complete monitoring screen with no blind spots. This 360-degree view provides a unified monitoring basis for subsequent target detection and tracking, ensuring that the system can capture and process any dynamic targets in the monitoring area in real time, significantly improving the system's monitoring effectiveness and visual integrity.

[0024] S5: Build an automatic monitoring parameter adjustment mechanism to adaptively adjust the relevant parameters of the camera according to real-time lighting and weather conditions, and obtain the final monitoring system design plan.

[0025] Specifically, this step establishes an intelligent adaptive adjustment mechanism, which enables the monitoring system to automatically optimize the camera's imaging parameters according to real-time lighting and weather changes, thereby maintaining the consistency and clarity of image quality under changing environmental conditions. By introducing real-time environmental perception modules, automatic adjustment algorithms and feedback mechanisms, the system can dynamically adapt to different light intensities, weather conditions (such as rain, fog, snow) and other external factors to avoid the degradation of imaging quality due to environmental changes. This mechanism not only improves the adaptability and stability of the monitoring system in harsh environments, but also ensures that the monitoring screen always has high-quality visual information.

[0026] The present invention effectively solves the problems of viewing angle limitation, monitoring blind spots and insufficient environmental adaptability in traditional off-vehicle monitoring systems, and improves the capabilities of real-time monitoring, target recognition and adaptability in complex environments.

[0027] As a preferred embodiment of the above, Figure 2As shown, step S1 is to perform a demand analysis on the vehicle exterior monitoring system, identify and determine the key functional requirements of the vehicle exterior monitoring system, and obtain the demand analysis results, including: S11: Conduct targeted research through a systematic approach to collect specific needs and expectations for using the off-board monitoring system; S12: Analyze the functional characteristics and technical performance of the vehicle exterior monitoring system, and identify demand gaps by comparing the advantages and disadvantages of different products; S13: Determine the core functions of the off-vehicle monitoring system based on the target survey and demand analysis results, and generate a functional requirement list; S14: Conduct feasibility analysis on the functional requirements list, identify the required hardware and software support, and obtain the requirements analysis results.

[0028] Specifically, through methods such as surveys and interviews, we systematically collect users' needs and expectations for off-vehicle monitoring systems, including their specific requirements for monitoring range, target recognition accuracy, response time, and system operation convenience, and gain an in-depth understanding of users' pain points in actual applications and the functions they hope to achieve; we conduct research on the off-vehicle monitoring systems currently on the market, compare and analyze their functional characteristics, technical indicators, advantages and disadvantages, and identify possible deficiencies and demand gaps in actual applications. For example, by comparing the monitoring range, imaging clarity, environmental adaptability, and real-time performance of different products, we find that existing products may need to be improved in terms of viewing angle coverage, target recognition stability, or adaptability to complex environments. This step clarifies the direction of product improvement. Direction, provide reference and optimization basis for design; Comprehensive target survey and analysis results, determine the core functions that the system must have, such as 360-degree monitoring coverage, real-time target detection and recognition, dynamic target tracking, and adaptive lighting and weather adjustment capabilities, and include these functions in the requirements list to ensure that the functions of the system design are closely connected with the actual needs of users. At the same time, this core function confirmation also helps the team focus on the most prioritized needs and avoid excessive redundant designs; Conduct feasibility assessments for confirmed functional requirements, analyze the implementation conditions of these functions, including hardware (such as camera configuration, sensor type) and software (image processing algorithms, deep learning models, etc.) requirements, and identify the difficulties or resources required for the implementation of key technologies. Through feasibility analysis, ensure that the functional requirements have a technical basis for implementation, and the final generated requirements analysis results provide a clear and executable direction for system design.

[0029] As a preferred embodiment of the above, Figure 3 As shown, step S2, according to the demand analysis results, adopt a multi-camera layout, configure the camera positions, and generate a complete system design diagram, including: S21: Based on the demand analysis results, a dynamic optimization algorithm is used to divide the monitoring area and determine the optimal layout of multiple cameras; S22: Based on 3D modeling technology, simulate different camera layout schemes, configure the position of each camera according to the determined layout, and generate a complete system design diagram.

[0030] Specifically, according to the results of demand analysis, the dynamic optimization algorithm is applied to divide the entire monitoring area in detail to ensure that the camera's viewing angle coverage can achieve all-round monitoring without blind spots. First, the monitoring area is subdivided into multiple small areas, and the characteristics of each area (such as monitoring priority, potential obstructions, monitoring blind spots, etc.) are considered to calculate the number, position, viewing angle and relative spacing of the cameras. The goal of the dynamic optimization algorithm is to find the best combination layout of multiple cameras through scientific calculations, so that the overall system achieves the best balance between coverage, image quality and cost, ensuring that there are no blind spots in the covered area and that the monitoring accuracy meets the requirements; after determining the layout of multiple cameras, different layouts are visualized and simulated based on 3D modeling technology. 3D modeling can intuitively show the viewing angle and coverage of each camera at different positions, and help predict and identify possible occlusion problems in the layout. Through 3D simulation, the installation height, tilt angle and relative position of each camera can be accurately evaluated to ensure that they can work together without overlapping or generating too much redundancy. Through repeated iterations and adjustments, the camera layout is continuously optimized in a 3D simulation environment, and a complete system design diagram is finally generated. The system design diagram includes the precise location, orientation, monitoring coverage area and corresponding cable layout of each camera to ensure smooth implementation during actual installation. This step not only focuses on the physical layout of the camera, but also considers technical details such as power supply and network connection to ensure that each camera can work stably and transmit video data in real time.

[0031] As a preferred embodiment of the above, Figure 4 As shown, step S3 combines the images captured by multiple cameras based on the system design diagram and the image stitching algorithm to form a complete 360-degree view, including: S31: Based on the system design diagram, obtain image data captured by each camera; S32: applying an image stitching algorithm to pre-process images captured by multiple cameras; S33: combining the preprocessed images, generating a seamless 360-degree view through an image stitching algorithm, and outputting the 360-degree view.

[0032] Specifically, according to the system design drawing, the system will obtain the image data of each camera in real time. This process involves extracting image frames from the real-time video streams of different cameras to ensure that the image data captured by each camera has accurate timestamps and location identifiers. The acquisition of image data needs to consider the working status of the camera, network transmission stability and image quality to ensure that the collected images can provide clear and stable visual information in subsequent processing. After obtaining the original image data, preprocessing is an important step to ensure the quality of image stitching. Preprocessing includes removing image noise, adjusting brightness and contrast, correcting color distortion, etc., to improve the effect of subsequent stitching. This step will be carried out: Image denoising: Use filtering algorithms to remove noise from images and improve image clarity; Brightness and contrast adjustment: unify the brightness and contrast of images captured by each camera to ensure visual consistency of images from different cameras; Perspective Correction: Geometric correction of the images captured by each camera to ensure that the edges of adjacent images can be accurately matched when stitching.

[0033] These pre-processing steps ensure that the image data input to the stitching algorithm is of high quality and consistency, laying the foundation for generating a 360-degree view.

[0034] After pre-processing, image stitching algorithms are applied to combine these images to generate a seamless 360-degree panoramic view. This process usually includes the following key steps: Feature matching: Use feature point matching algorithms (such as SIFT, SURF, etc.) to identify common feature points between adjacent images to achieve image alignment; Image transformation: Calculate the transformation matrix based on the matched feature points, and rotate, translate, and scale the image to achieve the best stitching effect; Stitching and fusion: The aligned images are seamlessly merged through weighted averaging or image fusion algorithms to eliminate obvious traces of seams and overlaps, generating a natural and smooth panoramic view.

[0035] Finally, the generated 360-degree view is output to ensure that it can provide a comprehensive and clear monitoring picture in subsequent dynamic target detection and behavior analysis.

[0036] As a preferred embodiment of the above, Figure 5 As shown, the image stitching algorithm includes: A10: preprocess the acquired image data, including denoising, color correction and geometric correction; A20: Use feature point detection algorithm to identify key feature points in each image, extract descriptors of key feature points, and match between images; A30: Use the matched feature points to calculate the transformation matrix, perform image alignment, and fuse the aligned images to generate a complete stitched image.

[0037] Specifically, before stitching, the acquired image data needs to undergo a series of preprocessing to improve the subsequent processing effect: Denoising: Remove noise in the image through filtering and other denoising algorithms (such as Gaussian filtering, median filtering, etc.) to make the image clearer and avoid noise interference with feature point detection; Color correction: Uniformly adjust the image's hue, saturation, brightness, etc. to ensure the color consistency of each image and prevent obvious color differences during the stitching process; Geometric correction: Perform perspective and distortion correction on the image to correct the geometric distortion caused by differences in camera position and viewing angle, so as to ensure that the image edges can be more accurately aligned with other images. Feature point detection algorithms (such as SIFT, SURF, ORB, etc.) are used to detect significant feature points in each image. These feature points are usually located in key areas of the image (such as edges and corners) and can effectively represent the local features of the image. For the detected feature points, their feature descriptors (such as direction, scale, position, etc.) are extracted as the unique identifier of the feature points. Using the feature descriptors, algorithms (such as KNN matching, RANSAC, etc.) are used to find matching feature points between adjacent images and determine the correspondence between images. The accuracy of feature point matching determines the accuracy of the stitching effect. Based on the matched feature points, an algorithm (such as least squares method or RANSAC) is used to calculate the transformation matrix, which describes the rotation, scaling, and translation relationship between images so that the images can be aligned. The calculated transformation matrix is ​​applied to the image to achieve precise alignment of position and angle. The image is adjusted according to the transformation matrix so that each image can be seamlessly joined to the corresponding position of the adjacent image to form a complete view. The aligned images are edge-fused using a fusion algorithm (such as gradient fusion, weighted averaging, or multi-band fusion) to eliminate the stitching seams so that the entire stitched image presents a natural and continuous visual effect.

[0038] As a preferred embodiment of the above, Figure 6 As shown, a deep learning model is established to detect and identify dynamic targets in a 360-degree view in real time and generate target detection results, including: B10: Build a deep learning model and train it using annotated training data sets; B20: Deploy the deep learning model in the real-time 360-degree view image data stream, use the deep learning model to extract high-dimensional features in the image, and aggregate different feature information through feature fusion algorithm; B30: Generate dynamic target detection results based on feature extraction and analysis results.

[0039] Specifically, first, collect and annotate a large number of image data sets, which should cover various targets in the vehicle's external environment, such as pedestrians, vehicles, and obstacles. In order to enhance the robustness of the model, data augmentation technology is essential, including transformations such as rotation, scaling, and translation of the image to ensure that the model can effectively detect under a variety of weather and lighting conditions. It is crucial to choose a suitable target detection model. YOLO (You Only Look Once) is a common choice because it strikes a good balance between real-time and accuracy. The YOLO model mainly consists of three parts: feature extraction network (such as ResNet or Darknet), feature pyramid network (FPN) for enhancing the fusion of features of different scales, and head network responsible for predicting the category, location, and confidence of each target. After data preparation and enhancement, the data is used to train the model and optimized using appropriate loss functions (such as cross entropy and regression loss). Adam or SGD optimizers are usually used to iteratively optimize the model. Next, in step B20, the target detection model needs to be deployed to the real-time image data stream. The on-board camera continuously captures images of the environment outside the car. These images may be single pictures or video streams. If a multi-camera system is used, the images may be spliced ​​360-degree views. Before the image enters the model, it must be preprocessed, usually including scaling to a fixed size required by the model (such as 416x416 or 608x608) and normalizing the pixel values ​​to ensure that the model It can process images efficiently. The preprocessed images will be sent to the trained target detection model for inference. The model generates the category, location and confidence of the target through forward propagation. For the YOLO model, it will generate several candidate boxes for each image. Each box includes category information, confidence and location coordinates of the target. In order to reduce repeated detection, it is necessary to remove redundant bounding boxes through non-maximum suppression (NMS) to retain the best prediction results; finally, in step B30, the generated target detection results will be processed and output. Each detected target will be assigned a category label (such as "pedestrian", "car" or "obstacle") and a location coordinate box. The four values ​​of the box (x1, y1, x2, y2) indicates the exact location of the target in the image. In addition, each target will have a confidence value, which indicates how sure the model is about the detection result. In order to reduce false positives, a confidence threshold is usually set. Detection results below this value will be filtered out to prevent irrelevant targets from affecting subsequent decisions. Finally, the target detection results will be output in a structured data format (such as JSON), containing information such as the category, location, and confidence of each target for subsequent applications such as target tracking, automatic alarm, or obstacle avoidance systems.

[0040] As a preferred embodiment of the above, Figure 7As shown, combined with the target tracking algorithm, the behavior data of dynamic targets is recorded, including: C10: Acquire dynamic targets in a 360-degree view, apply multi-target tracking algorithms to track targets in real time, and record the dynamic characteristics of targets; C20: Accurately model the target's movement behavior and generate multi-dimensional data related to the target's behavior; C30: Use deep learning algorithms to analyze behavioral data, identify the target's behavioral patterns through time series modeling, predict the target's future behavior, and detect in real time whether the target has abnormal behavior or potential threats, and generate behavioral trend reports.

[0041] Specifically, in step C10, the system first obtains a 360-degree view through multiple on-board cameras or panoramic cameras. The real-time image streams provided by these cameras will be integrated and spliced ​​to form a complete image of the external environment. In order to accurately track each target in this panoramic view, the system needs to apply multi-target tracking algorithms. These algorithms can detect and track multiple targets in real time. Usually, the initial position of each target is obtained through a target detection model (such as YOLOv8), and the target is continuously tracked through a tracking algorithm. Common multi-target tracking methods include Kalman filtering, SORT, and Deep SORT, these algorithms can dynamically update the state of the target according to the position and motion information of the target, including the position information, speed and acceleration of the target; in step C20, the system needs to model the motion behavior of each tracked target. First, the historical motion data of the target (such as position, speed, acceleration, etc.) needs to be used to fit the motion model of the target. For example, the dynamic model in physics (such as Newton's motion equation) can be used to describe the motion trend of the target, or the regression model based on machine learning can be used to predict the next action of the target. In the modeling process, the system needs to consider the target's motion mode, such as straight-line driving, curve driving or stagnation. Through multi-dimensional data recording, the system can capture the target's behavioral characteristics. For example, the target's speed change, direction change, trajectory smoothness, acceleration mode, etc. can all be used as feature data for further analysis. These data will help the system accurately describe the target's behavior and provide a data basis for the next behavior prediction and anomaly detection;In step C30, the system uses deep learning algorithms, especially time series modeling algorithms (such as LSTM, GRU, etc.) to conduct in-depth analysis of the target's behavior data. The time series modeling algorithm can effectively process data with time series characteristics, such as the target's motion trajectory, speed, and acceleration. By learning historical behavior data, the deep learning model can identify the target's behavior pattern and predict the target's future behavior. The system first inputs the target's motion data (including position, speed, acceleration, etc.) into the time series neural network (such as LSTM or GRU) for training. After training, the model can predict the target's future motion trajectory and behavior pattern. For example, if the target's motion trajectory in the past few frames shows a certain fixed pattern, the model can predict that the target will continue to move along this trajectory. In addition to conventional behavior pattern recognition, the system can also use behavior anomaly detection algorithms such as Isolation Forest. Forest) or autoencoder-based anomaly detection to detect in real time whether the target has abnormal behavior or potential threats. For example, if a target's behavior suddenly deviates from its normal motion trajectory, or the target has unreasonable movement (such as drastic changes in speed or unconventional direction), the system can mark the behavior as abnormal. Based on these behavioral analyses, the system can generate a behavioral trend report, provide the target's behavior prediction, abnormal behavior detection results and its potential threat information, and help users or autonomous driving systems make corresponding decisions. In each frame of the image, the tracking algorithm calculates the target's motion trajectory based on the target's position (such as the bounding box) and records the target's dynamic features (such as position, speed, acceleration, direction, etc.). These dynamic features provide important data support for subsequent behavioral analysis. ;

[0042] As a preferred embodiment of the above, Figure 8 As shown, step S5, constructing an automatic monitoring parameter adjustment mechanism, adaptively adjusting the relevant parameters of the camera according to the real-time lighting and weather conditions, and obtaining the final monitoring system design scheme, including: S51: By integrating multiple sensors to monitor the light intensity and weather conditions of the external environment in real time, real-time environmental data can be obtained; S52: Based on the environmental data acquired in real time, a data analysis algorithm is used to perform real-time evaluation on various environmental parameters to determine the impact of the current environment on the camera imaging quality; S53: According to the evaluation results, the relevant parameters of the camera are automatically adjusted, and optimized through the feedback mechanism to generate a final monitoring system design plan.

[0043] Specifically, in step S51, by integrating light sensors and meteorological sensors to monitor the light intensity and weather conditions of the external environment in real time, the system can collect data such as light intensity, temperature, humidity, wind speed, etc. to provide a basis for subsequent adjustments; in step S52, based on real-time environmental data, the system uses data analysis algorithms to evaluate the impact of the environment on the camera imaging quality. For example, excessive light may lead to overexposure, insufficient light may cause the image to be too dark, and bad weather (such as haze and rain) will affect the clarity of the camera. The evaluation results use an analysis model to determine whether the current environmental conditions affect the image quality; in step S53, the system automatically adjusts the camera parameters such as exposure time, gain, white balance and focal length according to the evaluation results to adapt to different lighting and weather conditions. In addition, the system continuously optimizes these adjustments through a feedback mechanism to ensure that the image quality is always stable. Finally, the system generates a monitoring plan to ensure that the camera maintains the best monitoring effect in different environments, thereby improving the adaptability and stability of the overall monitoring system.

[0044] Embodiment 2 Based on the same inventive concept as the design method of an exterior vehicle monitoring system based on a panoramic camera in the aforementioned embodiment, the present invention further provides a design system of an exterior vehicle monitoring system based on a panoramic camera, the system comprising: The demand analysis result acquisition module performs demand analysis on the off-vehicle monitoring system, identifies and determines the key functional requirements of the off-vehicle monitoring system, and obtains the demand analysis results; The system design diagram generation module adopts a multi-camera layout, configures the camera positions, and generates a complete system design diagram based on the demand analysis results; The 360-degree view generation module combines images captured by multiple cameras to form a complete 360-degree view based on the system design diagram and image stitching algorithm; The target detection result generation module builds a deep learning model to detect and identify dynamic targets in the 360-degree view in real time, generates target detection results, and combines the target tracking algorithm to record the behavior data of dynamic targets; The system design scheme generation module builds an automatic monitoring parameter adjustment mechanism, adaptively adjusts the relevant parameters of the camera according to real-time lighting and weather conditions, and obtains the final monitoring system design scheme.

[0045] The above-mentioned design system in the present invention can effectively implement the design method of the vehicle exterior monitoring system based on the panoramic camera, and the technical effects that can be achieved are as described in the above-mentioned embodiments, which will not be repeated here.

[0046] As a preferred embodiment of the above, the system design solution generation module includes: The environmental data acquisition unit monitors the light intensity and weather conditions of the external environment in real time by integrating multiple sensors to obtain real-time environmental data; The real-time parameter evaluation unit uses data analysis algorithms to evaluate various environmental parameters in real time based on the environmental data obtained in real time, and determines the impact of the current environment on the camera imaging quality; The feedback mechanism optimization unit automatically adjusts the relevant parameters of the camera according to the evaluation results, and optimizes through the feedback mechanism to generate the final monitoring system design plan.

[0047] Similarly, the above-mentioned optimization schemes for the system can also respectively achieve the optimization effects corresponding to the method in Example 1, which will not be repeated here.

[0048] Although the present application has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application as defined herein, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these modifications and variations.

Claims

1. A design method for an exterior vehicle monitoring system based on a panoramic camera, characterized in that: include: Conduct demand analysis on the vehicle exterior monitoring system, identify and determine the key functional requirements of the vehicle exterior monitoring system, and obtain demand analysis results; According to the requirements analysis results, a multi-camera layout is adopted, camera positions are configured, and a complete system design diagram is generated; Based on the system design and image stitching algorithm, the images captured by multiple cameras are combined to form a complete 360-degree view; Establishing a deep learning model to detect and identify dynamic targets in the 360-degree view in real time, generating target detection results, and combining with a target tracking algorithm to record the behavior data of the dynamic targets; Build an automatic monitoring parameter adjustment mechanism to adaptively adjust the relevant parameters of the camera according to real-time lighting and weather conditions, and obtain the final monitoring system design plan.

2. The design method of the vehicle exterior monitoring system based on the panoramic camera according to claim 1 is characterized in that: Conduct a requirements analysis on the vehicle exterior monitoring system, identify and determine the key functional requirements of the vehicle exterior monitoring system, and obtain the requirements analysis results, including: Conduct targeted research through a systematic approach to collect specific needs and expectations for using the off-vehicle monitoring system; Analyze the functional characteristics and technical performance of the vehicle exterior monitoring system, and identify demand gaps by comparing the advantages and disadvantages of different products; According to the target survey and demand analysis results, determine the core functions of the off-vehicle monitoring system and generate a functional requirements list; Conduct a feasibility analysis on the functional requirements list, identify the required hardware and software support, and obtain the requirements analysis results.

3. The design method of the vehicle exterior monitoring system based on the panoramic camera according to claim 1 is characterized in that: According to the requirements analysis results, a multi-camera layout is adopted, the camera positions are configured, and a complete system design diagram is generated, including: Based on the demand analysis results, a dynamic optimization algorithm is used to divide the monitoring area and determine the optimal layout of multiple cameras; Based on 3D modeling technology, different camera layout schemes are simulated, the position of each camera is configured according to the determined layout, and a complete system design diagram is generated.

4. The design method of the vehicle exterior monitoring system based on the panoramic camera according to claim 1 is characterized in that: Based on the system design and image stitching algorithm, the images captured by multiple cameras are combined to form a complete 360-degree view, including: Based on the system design diagram, image data captured by each camera is obtained; Apply image stitching algorithms to pre-process images captured by multiple cameras; The preprocessed images are combined, a seamless 360-degree view is generated by the image stitching algorithm, and the 360-degree view is output.

5. The design method of the vehicle exterior monitoring system based on the panoramic camera according to claim 4 is characterized in that: The image stitching algorithm includes: Preprocessing the acquired image data, wherein the preprocessing includes denoising, color correction and geometric correction; A feature point detection algorithm is used to identify key feature points in each image, and descriptors of the key feature points are extracted to perform matching between images; The transformation matrix is ​​calculated using the matched feature points to align the images, and the aligned images are fused to generate a complete stitched image.

6. The design method of the vehicle exterior monitoring system based on a panoramic camera according to claim 1, characterized in that: A deep learning model is established to detect and identify dynamic targets in the 360-degree view in real time, and generate target detection results, including: Establishing a deep learning model and training the deep learning model using a labeled training data set; Deploy the deep learning model in the 360-degree view image data stream acquired in real time, use the deep learning model to extract high-dimensional features in the image, and aggregate different feature information through a feature fusion algorithm; Based on the feature extraction and analysis results, the detection results of dynamic targets are generated.

7. The design method of the vehicle exterior monitoring system based on a panoramic camera according to claim 1, characterized in that: Combined with the target tracking algorithm, the behavior data of the dynamic target is recorded, including: Acquire dynamic targets in a 360-degree view, apply multi-target tracking algorithms to track targets in real time, and record the dynamic characteristics of targets; Accurately model the target's movement behavior and generate multi-dimensional data related to the target's behavior; Use deep learning algorithms to analyze behavioral data, identify the target's behavioral patterns through time series modeling, predict the target's future behavior, and detect in real time whether the target has abnormal behavior or potential threats, and generate behavioral trend reports.

8. The design method of the vehicle exterior monitoring system based on a panoramic camera according to claim 1, characterized in that: Build an automatic monitoring parameter adjustment mechanism to adaptively adjust the relevant parameters of the camera according to real-time lighting and weather conditions, and obtain the final monitoring system design, including: By integrating multiple sensors to monitor the light intensity and weather conditions of the external environment in real time, real-time environmental data can be obtained; Based on the environmental data acquired in real time, a data analysis algorithm is used to evaluate various environmental parameters in real time to determine the impact of the current environment on the camera imaging quality; According to the evaluation results, the relevant parameters of the camera are automatically adjusted and optimized through the feedback mechanism to generate the final monitoring system design plan.

9. A design system for an exterior vehicle monitoring system based on a panoramic camera, characterized in that: The system comprises: The demand analysis result acquisition module performs demand analysis on the off-vehicle monitoring system, identifies and determines the key functional requirements of the off-vehicle monitoring system, and obtains the demand analysis results; A system design diagram generation module, which adopts a multi-camera layout, configures camera positions, and generates a complete system design diagram according to the demand analysis results; A 360-degree view generation module, based on the system design diagram and image stitching algorithm, combines the images captured by multiple cameras to form a complete 360-degree view; A target detection result generation module is used to establish a deep learning model, detect and identify dynamic targets in the 360-degree view in real time, generate target detection results, and record the behavior data of the dynamic targets in combination with a target tracking algorithm; The system design scheme generation module builds an automatic monitoring parameter adjustment mechanism, adaptively adjusts the relevant parameters of the camera according to real-time lighting and weather conditions, and obtains the final monitoring system design scheme.

10. The design system of the vehicle exterior monitoring system based on the panoramic camera according to claim 9, characterized in that: The system design scheme generating module comprises: The environmental data acquisition unit monitors the light intensity and weather conditions of the external environment in real time by integrating multiple sensors to obtain real-time environmental data; A real-time parameter evaluation unit, based on the environmental data acquired in real time, uses a data analysis algorithm to perform real-time evaluation on various environmental parameters to determine the impact of the current environment on the camera imaging quality; The feedback mechanism optimization unit automatically adjusts the relevant parameters of the camera according to the evaluation results, and optimizes through the feedback mechanism to generate the final monitoring system design plan.

Images