Accident scene real-time image rescue system in network connection environment and use method

Abstract

The invention discloses an accident scene real-time image rescue system in a network connection environment and a use method, and belongs to the field of intelligent network connection rescue, and the system comprises a rescue command center, an intelligent network connection vehicle image acquisition system, and a wireless network communication system. The method comprises the steps that a rescue command center obtains alarm information, an area with an accident occurrence point as the center is locked, all intelligent network connection vehicle cameras in the area are accessed, multi-angle real-time images of the accident point are collected and uploaded to the rescue command center through the V2X technology, and the rescue command center synthesizes all the vehicle images to restore an accident site. And the accident type and severity are judged, and an efficient and reasonable rescue strategy is formulated. According to the accident scene real-time image rescue system in the network connection environment and the use method, accident information can be rapidly obtained, the accident scene is accurately analyzed through the multi-angle camera, real-time transmission and accurate analysis are ensured through the V2X technology, and the accident scene real-time image rescue system in the network connection environment and the use method are particularly suitable for complex or high-risk accidents.

Description

Technical Field

[0001] The present invention relates to the field of intelligent network rescue, and in particular to a real-time image rescue system for an accident scene in a networked environment and a method for using the system. Background Art

[0002] With the development of intelligent connected technology, interoperability between vehicles has become possible. This is particularly true during emergency response following a traffic accident, where the ability to quickly and efficiently obtain detailed information from the accident scene is a pressing challenge. Existing technologies typically rely on on-site personnel or video equipment, but these issues can lead to time lags in information transmission and incomplete information, hindering the rapid development of appropriate rescue plans and often hindering their efficiency and effectiveness. Summary of the Invention

[0003] The purpose of the present invention is to provide a real-time image rescue system and method for accident scenes in a networked environment, which can quickly obtain accident information, accurately analyze the accident scene through multi-angle cameras, and use V2X technology to ensure real-time transmission and accurate analysis. It is particularly suitable for complex or high-risk accidents.

[0004] To achieve the above objectives, the present invention provides a real-time image rescue system for accident scenes in a networked environment, comprising the following structure:

[0005] Base Station: Receives and analyzes GPS coordinate data packets in the accident area, dynamically defines the target acquisition range based on electronic fencing technology, and identifies and locks on intelligent connected vehicles equipped with V2X communication modules in the area. It also serves as a physical node in the wireless network, providing a low-latency communication link to support data exchange between terminal vehicles and the cloud.

[0006] Wireless network platform: includes base station controller and wireless network control center; establishes encrypted two-way communication channel with vehicle terminals, and realizes secure command transmission through dynamic token verification and device identity binding;

[0007] Data processing center: includes memory and image data synthesis modules; receives multi-source image data, repairs dynamic blurred images based on motion blur restoration algorithms, and generates panoramic images of the accident scene through feature matching and multi-band fusion technology; combines the initial GPS coordinate correction algorithm to optimize the positioning accuracy of the accident center point;

[0008] Rescue Command Center: This includes a rescue request receiving platform, an accident scene positioning and control platform, and an accident assessment and decision-making center. It generates rescue strategies through panoramic images and real-time data analysis, dynamically coordinates rescue resource dispatch, optimizes decision-making logic based on updated image data, and provides a visual human-computer interaction interface to verify operational instructions.

[0009] Preferably, the base station collects information from the image acquisition terminal; after receiving the rescue request, the rescue request receiving platform transmits the information to the accident scene positioning and control platform; the accident scene positioning and control platform performs on-site positioning and control according to the rescue request, and feeds back the information to the rescue command center and the accident assessment and decision center; the accident assessment and decision center assesses the accident situation and makes a decision, and sends the decision result to the rescue command center; the rescue command center sends instructions to the wireless network control center according to the decision result; the wireless network control center controls the wireless network platform through the base station controller, and transmits the image data to the data processing center; the data processing center stores and processes the image data, and performs data synthesis through the image data synthesis module.

[0010] The present invention provides a method for using a real-time image rescue system for an accident scene in a networked environment, comprising the following steps:

[0011] S1. Obtaining accident alarm information: The rescue command center receives accident alarm information, which includes the type, location, and time of the accident. The information is obtained from telephone or the Internet.

[0012] S2. Marking the GPS coordinates of the accident site on an electronic map: The rescue command center uses positioning technology or the caller's description to determine the specific location of the accident and marks it on an electronic map, with the initial positioning point set as A0.

[0013] S3. Locate the accident area and determine the image collection zone: Use GPS or Beidou positioning to plan a circular image collection zone at the accident scene with the GPS coordinates of the accident site as the center and a specified length as the radius. At the same time, all GPS coordinate data within the image collection zone is aggregated into a data packet and sent to the base station management platform of the wireless communication operator;

[0014] S4. Locking the target terminal: The base station establishes an electronic fence area after receiving the coordinate data packet of the collection area;

[0015] S5. Accessing connected vehicle cameras: When a terminal vehicle enters the area inside the electronic fence and is identified, the terminal vehicle's central control system will receive a remote access request from the authorized rescue command center to the vehicle terminal camera. Vehicles that leave the electronic fence area automatically have their remote access rights disabled.

[0016] S6. Image analysis and scene reconstruction: The data processing center receives images from multiple vehicle terminals at multiple angles and performs corrections after acquisition;

[0017] S7. Develop a rescue strategy: Based on the image analysis results, rescue personnel assess the type of accident and the extent of damage. The rescue command center quickly develops targeted and effective rescue strategies and resource deployment, including rescue vehicles and personnel, and arranges traffic diversion. At the same time, it continues to update the images to understand the latest accident situation.

[0018] Preferably, the processing flow in S5 is as follows:

[0019] S5.1, Permission Access: The vehicle system opens the camera remote access permission and activates the linkage protocol between the camera and the communication module;

[0020] S5.2, Device Binding: Send an identity authentication request to the cloud through the vehicle's built-in 4G / 5G module to complete device binding;

[0021] S5.3. Encrypted communication channel generation: The cloud generates an encrypted communication channel that supports two-way command transmission, including starting and stopping the camera. By default, it obtains permissions for all vehicle cameras except the in-vehicle camera and generates a bird's-eye view.

[0022] S5.4. Video stream transmission: Video stream transmission uses AES-256 encryption, and command interaction requires dynamic token verification.

[0023] Preferably, the correction step in S6 is as follows:

[0024] S6.1. With the initial positioning point as the circle center A0, analyze the acquired image within a certain radius r0, and update the new accident scene location to the circle center A0 on the accident scene positioning operation platform. l , radius r l Update according to the situation, l is the number of updates, and the most accurate accident location is obtained through multiple positioning updates;

[0025] S6.2. Model the blurred image B as the convolution of the clear image I and the blur kernel K, and superimpose the noise ρ:

[0026]

[0027] Among them, u and v are variables in the frequency domain coordinate system, Represents the convolution operation;

[0028] S6.3. Use the vehicle-mounted camera equipped with an inertial measurement unit (IMU) or GPS to obtain camera motion information and accurately estimate the blur kernel. The inertial measurement unit (IMU) includes a combination of accelerometers and gyroscopes, which provides the angular velocity at time t, ω(t) = [ω x ,ω y ,ω z ] T and acceleration a(t) = [a x ,ay ,a z ] T , get the camera pose, the camera pose update method is:

[0029] R(t+Δt)=R(t)·exp(ω(t)Δt);

[0030]

[0031] Where R(t) is the rotation matrix obtained by the inertial measurement unit (IMU), R(t+Δt) is the updated rotation matrix, Δt is the sampling interval, D(t) is the position of the camera at time t, D(t+Δt) is the updated translation, v(t) is the velocity at time t, and exp(·) is the Lie group exponential map used for rotation update. Finally, the integral error is corrected by Kalman filtering.

[0032] S6.4. Map the pose sequence to the pixel displacement in the image plane. The displacement x of pixel p caused by the camera motion at time t is p (t) is:

[0033] x p (t) = T(R(t)X+D(t))-T(X);

[0034] Where X is the 3D coordinate of the object in the camera coordinate system, T(·) is the camera projection function;

[0035] S6.5. Generate a global blur kernel by integrating all pixel point trajectories and use Wiener filtering to deblur. The function of Wiener filtering is to minimize the mean square error in the frequency domain. The formula is:

[0036]

[0037] Among them, S n is the noise power spectrum, S I is the image power spectrum, * represents the complex conjugate; finally, it is converted back to the spatial domain through inverse Fourier transform to obtain the restored image;

[0038] S6.6. After obtaining images from multiple angles, use image stitching and fusion technology to stitch the images from multiple angles to obtain a 360° panoramic image of the accident scene, and update it in real time.

[0039] Preferably, in S6.6, the specific steps of image processing are as follows:

[0040] S6.6.1. Data Collection and Preprocessing: Connected vehicles in the accident area upload onboard camera data. The camera data includes a deblurred image at a specific timestamp and the vehicle or camera position obtained by the inertial measurement unit (IMU) and GPS. Align the timestamps of multiple vehicle cameras using the NTP protocol to ensure low time error, and use histogram matching to standardize the brightness and color temperature of images from different vehicles.

[0041] S6.6.2. Multi-view feature matching and registration: Feature extraction uses the SuperPoint network to extract high-density feature points and output the feature point coordinates p i,n =(u i,n ,v i,n ) and its descriptor d i,n ∈R 256 , i represents image i, similarly, j represents image j as the feature point matching; input the feature point set of the two images and Among them, N and M represent the number of feature points of images i and j respectively, n and m represent the feature point sequence of images i and j respectively, and the matching score matrix between feature points is calculated by the SuperGlue algorithm through the graph neural network GNN The Hungarian algorithm is used to select the best matching pair, complete cross-view matching, and finally the Levenberg-Marquardt algorithm is used to solve it;

[0042] S6.6.3 Panoramic image stitching and fusion: Seamlessly integrate wide-angle fisheye images captured by multiple vehicle cameras into a globally consistent panorama;

[0043] S6.6.4. Dynamic target processing and repair: Eliminate ghosting, missing occluded areas, and inconsistent lighting caused by dynamic targets.

[0044] Preferably, the feature matching loss function in S6.6.2 is:

[0045]

[0046] Where Ψ is the set of true matching pairs, representing all true matching point pairs formed in the matching images i and j, and λ is the balance factor;

[0047] Perform image registration and use the random sampling consensus algorithm RANSAC to remove outliers from the matching feature points, that is, to solve the homography matrix H∈R 3×3 , so that the set of matching point pairs Ψ in image i and image j satisfies:

[0048] p i,n (u i,n ,v i,n )=Hp j,m (uj,m ,v j,m );

[0049] Perform RANSAC robust estimation, the process is: randomly sample k pairs of matching points, calculate the initial H, calculate the reprojection error e i,n :

[0050] e i,n =||p i,n (u i,n ,v i,n )-T(Hp j,m (u j,m ,v j,m ))|| 2 ;

[0051] If the reprojection error e i,n <ε, where ε is the pixel error threshold, then the repaired area is considered consistent;

[0052] Select the inliers to re-estimate H and iterate until the inlier ratio is maximized. At this point, the image registration is complete. The final optimization result is expressed as:

[0053]

[0054] Preferably, the splicing and fusion method in S6.6.3 is as follows:

[0055] First, map the multi-vehicle perspective images to a unified coordinate system, and project the camera images of each vehicle from their respective local coordinate systems to the global coordinate system. Vehicle environment cameras are mostly fisheye wide-angle cameras, which are mapped to a spherical coordinate system:

[0056]

[0057] Among them, (c u ,c v ) is the pixel coordinate of the center of the image, f is the focal length, θ and φ are the polar angle and azimuth angle in the spherical coordinate system;

[0058] Using Laplacian pyramid decomposition, each image is decomposed into multi-scale frequency bands to eliminate chromatic aberration and retain high-frequency details; the image conversion formula is:

[0059]

[0060] Among them, G q is the Gaussian image of the qth layer, G q+1 is the Gaussian image of the q+1th layer, κ h×hrepresents an h×h Gaussian kernel, Up represents the upsampling operation, and represents the mapping relationship between the two layers of images. The upsampling steps are: first, the image is enlarged to twice its original size in each direction, the newly added rows and columns are filled with 0, and then the same kernel is used to convolve with the enlarged image to obtain the approximate value of the newly added pixels;

[0061] Dynamically adjust the fusion weight w according to the distance between the overlapping area and the image center k :

[0062]

[0063] Among them, σ represents the control weight decay rate;

[0064] Poisson fusion is used to solve the seam problem and minimize the gradient difference in the splicing area. The formula is:

[0065]

[0066] Where I is the fused image to be solved, Ω is the target fusion area, is the boundary of the target area, I src is the source image, I t is the target image, where the constraint means that at the boundary, the pixel value of the fused image must be equal to the pixel value of the target image. is the gradient field of the fused image, is the gradient field of the source image.

[0067] Preferably, the repair method in S6.6.4 includes performing time domain median filtering on the dynamic target area, taking the median of the static background in multiple consecutive frames, using the PatchMatch algorithm to search for similar texture blocks from the static background, filling large missing areas, and performing histogram matching on the repaired area to make it consistent with the surrounding background lighting. Finally, the rescue command center manually screens the on-site panoramic views with different time stamps. If the specific location of the accident cannot be locked, other perspective data is reselected or the radius of the electronic fence is expanded.

[0068] Therefore, the present invention adopts the above-mentioned accident scene real-time image rescue system and method of use in a networked environment, which has the following advantages:

[0069] (1) Improve rescue efficiency: It can obtain comprehensive information on the accident scene in the shortest time, help the rescue command center respond quickly, and reduce rescue delays.

[0070] (2) Multi-angle image acquisition: By collecting data through cameras of multiple vehicles, the accident scene can be analyzed from multiple angles, which is more comprehensive and accurate.

[0071] (3) Real-time and accuracy: The use of V2X technology ensures the real-time transmission and analysis of image data, which helps to accurately determine the type and severity of the accident.

[0072] (4) Wide scope of application: Applicable to various types of traffic accidents, especially complex multi-vehicle accidents or accident handling in high-risk environments.

[0073] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a system structure diagram of an embodiment of a real-time image rescue system for an accident scene in a networked environment and a method for using the system;

[0075] Figure 2 This is a communication link diagram of an embodiment of a real-time image rescue system and method for using an accident scene in a networked environment according to the present invention;

[0076] Figure 3 This is a system operation flow chart of an embodiment of a real-time image rescue system for an accident scene in a networked environment and a method for using the same;

[0077] Figure 4 This is a schematic diagram of image acquisition at an accident scene according to an embodiment of a real-time image rescue system and method for using an accident scene in a networked environment of the present invention;

[0078] Figure 5 This is an image processing flow chart of an embodiment of a real-time image rescue system and a method for using an accident scene in a networked environment according to the present invention.

[0079] Reference numerals

[0080] 1. Base station; 2. Wireless network platform; 3. Data processing center; 4. Rescue command center. DETAILED DESCRIPTION

[0081] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0082] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0083] Example 1

[0084] like Figures 1 to 2 As shown, the present invention provides a real-time image rescue system for accident scenes in a networked environment, including the following structures:

[0085] Base Station 1: Receives and parses GPS coordinate data packets in the accident area, dynamically defines the target acquisition range based on electronic fencing technology, and identifies and locks on intelligent connected vehicles equipped with V2X communication modules in the area. It also serves as a physical node in the wireless network, providing a low-latency communication link to support data exchange between terminal vehicles and the cloud.

[0086] Wireless network platform 2: includes base station 1 controller and wireless network control center; establishes an encrypted two-way communication channel with the vehicle terminal, and realizes secure command transmission through dynamic token verification and device identity binding.

[0087] Data Processing Center 3: Includes a memory and image data synthesis module; receives multi-source image data, repairs dynamically blurred images based on a motion blur restoration algorithm, and generates a panoramic image of the accident scene through feature matching and multi-band fusion technology; and optimizes the positioning accuracy of the accident center point by combining a correction algorithm with the initial GPS coordinates.

[0088] Rescue Command Center 4: Includes a rescue request receiving platform, an accident scene positioning and control platform, and an accident assessment and decision-making center; generates rescue strategies through panoramic images and real-time data analysis, dynamically coordinates rescue resource scheduling, optimizes decision-making logic based on updated image data, and provides a visual human-computer interaction interface to verify operational instructions.

[0089] Base station 1 collects information from the image acquisition terminal; after receiving the rescue request, the rescue request receiving platform transmits the information to the accident scene positioning and control platform; the accident scene positioning and control platform performs on-site positioning and control according to the rescue request, and feeds back the information to the rescue command center 4 and the accident assessment and decision center; the accident assessment and decision center assesses the accident situation and makes a decision, and sends the decision result to the rescue command center 4; the rescue command center 4 sends instructions to the wireless network control center based on the decision result; the wireless network control center controls the wireless network platform 2 through the base station 1 controller, and transmits the image data to the data processing center 3; the data processing center 3 stores and processes the image data, and synthesizes the data through the image data synthesis module.

[0090] like Figure 3 As shown, the present invention provides a method for using a real-time image rescue system for an accident scene in a networked environment, comprising the following steps:

[0091] S1. Obtain accident alarm information;

[0092] The rescue command center 4 receives the accident alarm information, which includes the type, location and time of the accident. The source of the information includes various information channels such as telephone and Internet.

[0093] S2. Mark the GPS coordinates of the accident site on the electronic map;

[0094] Based on the alarm information, the rescue center uses positioning technology (such as mobile phone positioning or vehicle GPS) or the caller's description to determine the exact location of the accident. This location is then mapped on an electronic map, with the initial location set as A0. If the exact location of the accident is difficult to pinpoint, it can be set directly to the center of the road, with subsequent corrections made by the data processing center 3.

[0095] S3. Locate the accident area and determine the image acquisition area;

[0096] Using GPS or Beidou positioning, with the GPS coordinates of the accident site as the center and a specific length as the radius, a circular accident scene image collection area is planned. At the same time, all GPS coordinate data within the image collection area are collected as a data packet and sent to the base station 1 management platform of the wireless communication operator. This operation can be performed once at a specified time.

[0097] S4. Lock the target terminal;

[0098] After receiving the coordinate data packet from the collection area, the Base Station 1 management platform establishes an "e-fence" area. An e-fence is a virtual boundary set electronically, triggering a response when a target enters or leaves the area. Its core principle is based on GPS, Base Station 1 positioning technology, and boundary settings. An algorithm compares the terminal's real-time location data with the preset boundary to determine whether it is inside or outside the fence.

[0099] The terminal of this invention specifically refers to a vehicle terminal equipped with a V2N system. The internal hardware of such a vehicle typically includes: a 5G / 4G communication module, a high-performance domain controller, multimodal sensors, a vehicle status sensor, and other configurations. Therefore, the terminal of this invention is a set of in-vehicle devices equipped with a low-latency wireless communication system and a camera system.

[0100] S5. Access the connected vehicle camera;

[0101] When a terminal vehicle enters the area inside the electronic fence and is identified, the terminal vehicle's central control system will receive a remote access request from the vehicle terminal camera initiated by the authorized rescue center. The subsequent specific process is as follows:

[0102] S5.1. Permission access: The vehicle system enables remote access rights for the camera and activates the linkage protocol between the camera and the communication module (such as the RTSP video streaming protocol).

[0103] S5.2. Device binding: The vehicle's built-in 4G / 5G module sends an identity authentication request to the cloud to complete device binding.

[0104] S5.3. Encrypted communication channel generation: An encrypted communication channel is generated in the cloud, which supports two-way command transmission (such as starting / stopping the camera). By default, it obtains permissions for all vehicle-mounted cameras except the in-vehicle camera and generates a bird's-eye view.

[0105] S5.4. Video streaming transmission: Video streaming transmission uses AES-256 encryption, and command interaction requires dynamic token verification (such as HTTPS+OAuth 2.0 protocol).

[0106] At this point, the remote rescue command center 4 will control the camera of the terminal vehicle entering the electronic fence to upload real-time images and save them to the data processing center 3. The terminal's cameras include front cameras, rear cameras, and environmental cameras.

[0107] Through V2N remote control of the car camera, the hardware compatibility (4G / 5G module + camera), cloud service integration (encrypted channel + command parsing) and user-end interaction design (rescue command center 4 control interface) technology are guaranteed to complete the extraction of real-time images from the entire camera.

[0108] Vehicles that leave the electronic fence area automatically close their remote access rights, reducing the data transmission load on base station 1.

[0109] S6. Image Analysis and Scene Reconstruction:

[0110] The data processing center 3 receives images from multiple vehicle terminals at multiple angles. These images include images of the road accident scene and images of the road surroundings. After acquiring the images, since the previous positioning operation target was the person who reported the incident, their location may not be the accident location, and there is an error in the positioning. Figure 4 As shown, set its position as the initial positioning point, and the correction operation is as follows:

[0111] S6.1. With the initial positioning point as the circle center A0, analyze the acquired image within a certain radius r0, and update the new accident scene location to the circle center A0 on the accident scene positioning operation platform. l , radius r l Update according to the situation, l is the number of updates, and the most accurate accident location is obtained through multiple positioning updates. The specific update steps are as follows:

[0112] The present invention designs a dynamic electronic fence contraction strategy. The initial electronic fence radius is set according to the road type, such as 500 meters on highways and 200 meters in urban areas. The radius is gradually contracted to the core area of the accident through real-time vehicle trajectory clustering (DBSCAN algorithm). The final radius is determined by the rescue command center 4 at the accident scene positioning operation platform based on the accident situation.

[0113] Manual screening method: When the initial location point A0 of the accident is difficult to determine, such as only a certain road, the relevant base station 1 can randomly select connected vehicles covering the accident area. The connected vehicles upload images from various environmental cameras, and the rescue command center 4 selects images that meet the accident description and updates the location point A accordingly. n .

[0114] Image types include static images and moving images. Moving images are often blurry, making them difficult for rescue center operators to accurately analyze and judge with the naked eye. Here, we use "motion blurred image restoration technology" to restore moving images.

[0115] S6.2. Model the blurred image B as the convolution of the clear image I and the blur kernel K, and superimpose the noise ρ:

[0116]

[0117] Among them, u and v are variables in the frequency domain coordinate system, Represents the convolution operation;

[0118] S6.3. Use the vehicle-mounted camera equipped with an inertial measurement unit (IMU) or GPS to obtain camera motion information and accurately estimate the blur kernel. The inertial measurement unit (IMU) includes a combination of accelerometers and gyroscopes, which provides the angular velocity at time t, ω(t) = [ω x ,ω y ,ω z ] T and acceleration a(t) = [a x ,a y ,a z ] T , get the camera pose, the camera pose update method is:

[0119] R(t+Δt)=R(t)·exp(ω(t)Δt);

[0120]

[0121] Where R(t) is the rotation matrix obtained by the inertial measurement unit (IMU), R(t+Δt) is the updated rotation matrix, Δt is the sampling interval, D(t) is the position of the camera at time t, D(t+Δt) is the updated translation, v(t) is the velocity at time t, and exp(·) is the Lie group exponential map used for rotation update. Finally, the integral error is corrected by Kalman filtering.

[0122] S6.4. Map the pose sequence to the pixel displacement in the image plane. The displacement x of pixel p caused by the camera motion at time t is p (t) is:

[0123] x p (t) = T(R(t)X+D(t))-T(X);

[0124] Where X is the 3D coordinate of the object in the camera coordinate system, T(·) is the camera projection function;

[0125] S6.5. Generate a global blur kernel by integrating all pixel trajectories and use Wiener filtering to deblur the image. The formula is:

[0126]

[0127] Among them, S n is the noise power spectrum, S I is the image power spectrum, * represents the complex conjugate; finally, it is converted back to the spatial domain through inverse Fourier transform to obtain the restored image;

[0128] S6.6. After obtaining images from multiple angles, use image stitching and fusion technology to stitch the images from multiple angles to obtain a 360° panoramic image of the accident scene, and update it in real time; Figure 5 The specific steps are as follows:

[0129] S6.6.1. Data collection and preprocessing;

[0130] Connected vehicles in the accident area upload on-board camera data (front-view, side-view, rear-view cameras, and other environmental cameras). The data includes a timestamp, deblurred images, and the vehicle or camera position obtained by the inertial measurement unit (IMU) and GPS.

[0131] The NTP protocol is used to align the timestamps of multiple vehicle cameras, ensuring low time errors, typically less than 10ms. Histogram matching is used to unify the brightness and color temperature of images from different vehicles.

[0132] In rainy and foggy conditions, dark channel priors are used to process rainy and foggy images, combined with BM3D algorithm for denoising. This step can be decided by technical personnel. Defogging and denoising operations may affect on-site accident judgment.

[0133] S6.6.2. Multi-view feature matching and registration: Feature extraction uses the SuperPoint network to extract high-density feature points and output the feature point coordinates p i,n =(u i,n ,v i,n ) and its descriptor d i,n ∈R 256 , i represents image i, similarly, j represents image j as the feature point matching; input the feature point set of the two images and Among them, N and M represent the number of feature points of images i and j respectively, n and m represent the feature point sequence of images i and j respectively, and the matching score matrix between feature points is calculated by the SuperGlue algorithm through the graph neural network GNN The Hungarian algorithm is used to select the optimal matching pair to complete cross-view matching, and finally the Levenberg-Marquardt algorithm is used to solve it; the feature matching loss function is:

[0134]

[0135] Where Ψ is the set of true matching pairs, representing all true matching point pairs formed in the matching images i and j, and λ is the balance factor;

[0136] Perform image registration and use the random sampling consensus algorithm RANSAC to remove outliers from the matching feature points, that is, to solve the homography matrix H∈R 3×3 , so that the set of matching point pairs Ψ in image i and image j satisfies:

[0137] p i,n (u i,n ,v i,n )=Hp j,m (u j,m ,v j,m );

[0138] Perform RANSAC robust estimation, the process is: randomly sample k pairs of matching points, calculate the initial H, calculate the reprojection error e i,n :

[0139] e i,n =||p i,n (u i,n ,v i,n )-T(Hp j,m (u j,m ,v j,m ))|| 2 ;

[0140] If the reprojection error e i,n <ε, where ε is the pixel error threshold, then the repaired area is considered consistent;

[0141] Select the inliers to re-estimate H and iterate until the inlier ratio is maximized. At this point, the image registration is complete. The final optimization result is expressed as:

[0142]

[0143] S6.6.3. Panoramic image stitching and fusion;

[0144] First, map the multi-vehicle perspective images to a unified coordinate system, and project the camera images of each vehicle from their respective local coordinate systems to the global coordinate system. Vehicle environment cameras are mostly fisheye wide-angle cameras, which are mapped to a spherical coordinate system:

[0145]

[0146] Among them, (c u ,c v ) is the pixel coordinate of the center of the image, f is the focal length, θ and φ are the polar angle and azimuth angle in the spherical coordinate system;

[0147] Using Laplacian pyramid decomposition, each image is decomposed into multi-scale frequency bands to eliminate chromatic aberration and retain high-frequency details; the image conversion formula is:

[0148]

[0149] Among them, G q is the Gaussian image of the qth layer, G q+1 is the Gaussian image of the q+1th layer, κ h×h represents an h×h Gaussian kernel, Up represents the upsampling operation, and represents the mapping relationship between the two layers of images. The upsampling steps are: first, the image is enlarged to twice its original size in each direction, the newly added rows and columns are filled with 0, and then the same kernel is used to convolve with the enlarged image to obtain the approximate value of the newly added pixels;

[0150] Dynamically adjust the fusion weight w according to the distance between the overlapping area and the image center k :

[0151]

[0152] Among them, σ represents the control weight decay rate;

[0153] Poisson Blending is used to solve the seam problem and minimize the gradient difference in the splicing area. The formula is:

[0154]

[0155] satisfy:

[0156] Where I is the fused image to be solved, Ω is the target fusion area, is the boundary of the target area, I src is the source image, I t is the target image, where the constraint means that at the boundary, the pixel value of the fused image must be equal to the pixel value of the target image. is the gradient field of the fused image, ▽I srcis the gradient field of the source image.

[0157] S6.6.4, Dynamic target processing and repair;

[0158] In the synthesis of panoramic images of accident scenes, dynamic targets (such as moving vehicles and pedestrians) can cause the following problems:

[0159] Ghosting: The position of a dynamic target changes in consecutive frames, resulting in ghosting when stitching.

[0160] Missing occluded areas: Dynamic targets occlude key areas (such as scattered objects), resulting in incomplete information.

[0161] Inconsistent lighting: Dynamic targets from different perspectives are affected by lighting, resulting in visual inconsistency after fusion.

[0162] The repair method includes performing time-domain median filtering on the dynamic target area, taking the median of the static background in multiple consecutive frames, using the PatchMatch algorithm to search for similar texture blocks from the static background, filling in large missing areas, and performing histogram matching on the repaired area to make it consistent with the surrounding background lighting. Finally, the rescue command center 4 manually screens the scene panorama images with different time stamps. If the specific location of the accident cannot be locked, other perspective data will be reselected or the radius of the electronic fence will be expanded.

[0163] S7. Develop rescue strategies: Based on the image analysis results, rescue personnel assess the accident type and damage severity. The rescue command center 4 quickly develops targeted and effective rescue strategies and resource allocation, including rescue vehicles and personnel, and arranges traffic diversion. Meanwhile, the center continues to update images to understand the latest accident situation.

[0164] Therefore, the present invention adopts the above-mentioned real-time image rescue system and method for the accident scene in a networked environment, which can quickly obtain accident information, accurately analyze the accident scene through multi-angle cameras, and use V2X technology to ensure real-time transmission and accurate analysis, which is particularly suitable for complex or high-risk accidents.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time image rescue system for accident scenes in a networked environment, characterized by: Includes the following structures: Base station: Receives and analyzes GPS coordinate data packets in the accident area, dynamically defines the target acquisition range based on electronic fence technology, and identifies and locks on intelligent connected vehicles equipped with V2X communication modules in the area; At the same time, as a physical node in the wireless network, it provides a low-latency communication link to support data interaction between terminal vehicles and the cloud; Wireless network platform: including base station controller and wireless network control center; Establish an encrypted two-way communication channel with the vehicle terminal and realize secure command transmission through dynamic token verification and device identity binding; Data processing center: includes memory and image data synthesis modules; receives multi-source image data, repairs dynamic blurred images based on motion blur restoration algorithms, and generates panoramic images of the accident scene through feature matching and multi-band fusion technology; combines the initial GPS coordinate correction algorithm to optimize the positioning accuracy of the accident center point; Rescue Command Center: This includes a rescue request receiving platform, an accident scene positioning and control platform, and an accident assessment and decision-making center. It generates rescue strategies through panoramic images and real-time data analysis, dynamically coordinates rescue resource dispatch, optimizes decision-making logic based on updated image data, and provides a visual human-computer interaction interface to verify operational instructions.

2. The real-time image rescue system for accident scenes in a networked environment according to claim 1, characterized in that: The base station collects information from the image acquisition terminal; after receiving the rescue request, the rescue request receiving platform transmits the information to the accident scene positioning control platform; The accident scene positioning and control platform performs on-site positioning and control according to the rescue request, and feeds back the information to the rescue command center and the accident assessment and decision center; the accident assessment and decision center evaluates the accident situation and makes a decision, and sends the decision results to the rescue command center; the rescue command center sends instructions to the wireless network control center based on the decision results; the wireless network control center controls the wireless network platform through the base station controller and transmits the image data to the data processing center; the data processing center stores and processes the image data, and synthesizes the data through the image data synthesis module.

3. A method for using a real-time image rescue system for an accident scene in a networked environment according to any one of claims 1 to 2, characterized in that: The following steps are involved: S1. Obtaining accident alarm information: The rescue command center receives accident alarm information, which includes the type, location, and time of the accident. The information is obtained from telephone or the Internet. S2. Marking the GPS coordinates of the accident site on an electronic map: The rescue command center uses positioning technology or the caller's description to determine the specific location of the accident and marks it on an electronic map, with the initial positioning point set as A0. S3. Locate the accident area and determine the image collection zone: Use GPS or Beidou positioning to plan a circular image collection zone at the accident scene with the GPS coordinates of the accident site as the center and a specified length as the radius. At the same time, all GPS coordinate data within the image collection zone is aggregated into a data packet and sent to the base station management platform of the wireless communication operator; S4. Locking the target terminal: The base station establishes an electronic fence area after receiving the coordinate data packet of the collection area; S5. Accessing connected vehicle cameras: When a terminal vehicle enters the area inside the electronic fence and is identified, the terminal vehicle's central control system will receive a remote access request from the authorized rescue command center to the vehicle terminal camera. Vehicles that leave the electronic fence area automatically have their remote access rights disabled. S6. Image analysis and scene reconstruction: The data processing center receives images from multiple vehicle terminals at multiple angles and performs corrections after acquisition; S7. Develop a rescue strategy: Based on the image analysis results, rescue personnel assess the type of accident and the extent of damage. The rescue command center quickly develops targeted and effective rescue strategies and resource deployment, including rescue vehicles and personnel, and arranges traffic diversion. At the same time, it continues to update the images to understand the latest accident situation.

4. The method for using the real-time image rescue system for accident scenes in a networked environment according to claim 3, characterized in that: The processing flow in S5 is as follows: S5.1, Permission Access: The vehicle system opens the camera remote access permission and activates the linkage protocol between the camera and the communication module; S5.2, Device Binding: Send an identity authentication request to the cloud through the vehicle's built-in 4G / 5G module to complete device binding; S5.

3. Encrypted communication channel generation: The cloud generates an encrypted communication channel that supports two-way command transmission, including starting and stopping the camera. By default, it obtains permissions for all vehicle cameras except the in-vehicle camera and generates a bird's-eye view. S5.

4. Video stream transmission: Video stream transmission uses AES-256 encryption, and command interaction requires dynamic token verification.

5. The method for using the real-time image rescue system for accident scenes in a networked environment according to claim 3, characterized in that: The correction steps in S6 are as follows: S6.

1. With the initial positioning point as the circle center A0, analyze the acquired image within a certain radius r0, and update the new accident scene location to the circle center A0 on the accident scene positioning operation platform. l , radius r l Update according to the situation, l is the number of updates, and the most accurate accident location is obtained through multiple positioning updates; S6.

2. Model the blurred image B as the convolution of the clear image I and the blur kernel K, and superimpose the noise ρ: Among them, u and v are variables in the frequency domain coordinate system, Represents the convolution operation; S6.

3. Use the vehicle-mounted camera equipped with an inertial measurement unit (IMU) or GPS to obtain camera motion information and accurately estimate the blur kernel. The inertial measurement unit (IMU) includes a combination of accelerometers and gyroscopes, which provides the angular velocity at time t, ω(t) = [ω x ,ω y ,ω z ] T and acceleration a(t) = [a x ,a y ,a z ] T , get the camera pose, the camera pose update method is: R(t+Δt)=R(t)·exp(ω(t)Δt); Where R(t) is the rotation matrix obtained by the inertial measurement unit (IMU), R(t+Δt) is the updated rotation matrix, Δt is the sampling interval, D(t) is the position of the camera at time t, D(t+Δt) is the updated translation, v(t) is the velocity at time t, and exp(·) is the Lie group exponential map used for rotation update. Finally, the integral error is corrected by Kalman filtering. S6.

4. Map the pose sequence to the pixel displacement in the image plane. The displacement x of pixel p caused by the camera motion at time t is p (t) is: x p (t)=T(R(t)X+D(t))-T(X); Where X is the 3D coordinate of the object in the camera coordinate system, T(·) is the camera projection function; S6.

5. Generate a global blur kernel by integrating all pixel trajectories and use Wiener filtering to deblur the image. The formula is: Among them, S n is the noise power spectrum, S I is the image power spectrum, * represents the complex conjugate; finally, it is converted back to the spatial domain through inverse Fourier transform to obtain the restored image; S6.

6. After obtaining images from multiple angles, use image stitching and fusion technology to stitch the images from multiple angles to obtain a 360° panoramic image of the accident scene, and update it in real time.

6. The method for using the real-time image rescue system for accident scenes in a networked environment according to claim 5, characterized in that: In S6.6, the specific steps of image processing are as follows: S6.6.

1. Data collection and preprocessing: Connected vehicles in the accident area upload onboard camera data. The camera data includes a deblurred image at a certain timestamp and the vehicle or camera position obtained by the inertial measurement unit (IMU) and GPS. The NTP protocol is used to align the timestamps of multiple vehicle cameras to ensure low time error, and histogram matching is used to unify the image brightness and color temperature of different vehicles; S6.6.

2. Multi-view feature matching and registration: Feature extraction uses the SuperPoint network to extract high-density feature points and output the feature point coordinates p i,n =(u i,n ,v i,n ) and its descriptor d i,n ∈R 256 , i represents image i, similarly, j represents image j as the feature point matching; input the feature point set of the two images and Among them, N and M represent the number of feature points of images i and j respectively, n and m represent the feature point sequence of images i and j respectively, and the matching score matrix between feature points is calculated by the SuperGlue algorithm through the graph neural network GNN The Hungarian algorithm is used to select the best matching pair, complete cross-view matching, and finally the Levenberg-Marquardt algorithm is used to solve it; S6.6.3 Panoramic image stitching and fusion: Seamlessly integrate wide-angle fisheye images captured by multiple vehicle cameras into a globally consistent panorama; S6.6.

4. Dynamic target processing and repair: Eliminate ghosting, missing occluded areas, and inconsistent lighting caused by dynamic targets.

7. The method for collecting real-time image of an accident scene rescue system in a networked environment according to claim 6, characterized in that: The feature matching loss function in S6.6.2 is: Where Ψ is the set of true matching pairs, representing all true matching point pairs formed in the matching images i and j, and λ is the balance factor; Perform image registration and use the random sampling consensus algorithm RANSAC to remove outliers from the matching feature points, that is, to solve the homography matrix H∈R 3×3 , so that the set of matching point pairs Ψ in image i and image j satisfies: p i,n (u i,n ,v i,n )=Hp j,m (u j,m ,v j,m ); Perform RANSAC robust estimation, the process is: randomly sample k pairs of matching points, calculate the initial H, calculate the reprojection error e i,n : e i,n =||p i,n (u i,n ,v i,n )-T(Hp j,m (u j,m ,v j,m ))|| 2 ; If the reprojection error e i,n <ε, where ε is the pixel error threshold, then the repaired area is considered consistent; Select the inliers to re-estimate H and iterate until the inlier ratio is maximized. At this point, the image registration is complete. The final optimization result is expressed as:

8. The method for collecting real-time image of an accident scene rescue system in a networked environment according to claim 6, characterized in that: The splicing and fusion methods in S6.6.3 are as follows: First, map the multi-vehicle perspective images to a unified coordinate system, and project the camera images of each vehicle from their respective local coordinate systems to the global coordinate system. Vehicle environment cameras are mostly fisheye wide-angle cameras, which are mapped to a spherical coordinate system: Among them, (c u ,c v ) is the pixel coordinate of the center of the image, f is the focal length, θ and φ are the polar angle and azimuth angle in the spherical coordinate system; Using Laplacian pyramid decomposition, each image is decomposed into multi-scale frequency bands to eliminate chromatic aberration and retain high-frequency details; the image conversion formula is: Among them, G q is the Gaussian image of the qth layer, G q+1 is the Gaussian image of the q+1th layer, κ h×h represents an h×h Gaussian kernel, Up represents the upsampling operation, and represents the mapping relationship between the two layers of images. The upsampling steps are: first, the image is enlarged to twice its original size in each direction, the newly added rows and columns are filled with 0, and then the same kernel is used to convolve with the enlarged image to obtain the approximate value of the newly added pixels; Dynamically adjust the fusion weight w according to the distance between the overlapping area and the image center k : Among them, σ represents the control weight decay rate; Poisson fusion is used to solve the seam problem and minimize the gradient difference in the splicing area. The formula is: satisfy: Where I is the fused image to be solved, Ω is the target fusion area, is the boundary of the target area, I src is the source image, I t is the target image, where the constraint means that at the boundary, the pixel value of the fused image must be equal to the pixel value of the target image. is the gradient field of the fused image, is the gradient field of the source image.

9. The method for collecting real-time image of an accident scene rescue system in a networked environment according to claim 8, characterized in that: The repair method in S6.6.4 includes performing time-domain median filtering on the dynamic target area, taking the median of the static background in multiple consecutive frames, using the PatchMatch algorithm to search for similar texture blocks from the static background, filling large missing areas, and performing histogram matching on the repaired area to make it consistent with the surrounding background lighting. Finally, the rescue command center manually screens the scene panorama images with different time stamps. If the specific location of the accident cannot be locked, other perspective data will be reselected or the radius of the electronic fence will be expanded.

Images