Vehicle window living body probe detection method and system based on vehicle-mounted camera

Abstract

The invention relates to a vehicle window living body detection method and system based on a vehicle-mounted camera, and the method comprises the steps: M1, obtaining the image data information in a vehicle in real time based on an in-vehicle camera, obtaining the image data information of a vehicle window region in real time based on an out-vehicle camera, and carrying out the preprocessing, the preprocessed image data information in the vehicle and the vehicle window area is obtained; and M2, based on the preprocessed image data information in the vehicle and the vehicle window area, adopting a yolov8n target detection model to identify the image of the living body in the vehicle and the obstacle image of the vehicle window area, and obtaining the identified data information of the image of the living body in the vehicle and the obstacle image of the vehicle window area. According to the invention, whether a child or other passengers stretch the head, the arms and other body parts out of the car window can be detected, and an alarm is automatically given or automatic intervention measures are taken when dangerous behaviors are identified, so that traffic accidents are effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety technology, and in particular to a method and system for detecting live objects protruding from vehicle windows based on an in-vehicle camera. Background Technology

[0002] As vehicles become increasingly intelligent, safety measures both inside and outside the vehicle are being strengthened. Especially regarding child passenger safety, while traditional safety devices (such as anti-pinch windows and physical mesh) can provide some protection, they typically cannot monitor in real time whether children or passengers are extending their heads, arms, or other body parts outside the window. Real-time detection of living objects protruding from the vehicle window can significantly reduce the probability of traffic accidents. Summary of the Invention

[0003] In view of the above problems, the present invention provides a method and system for detecting live objects protruding from vehicle windows based on vehicle-mounted cameras. It can not only detect whether children or other passengers have their heads, arms or other body parts protruding out of the window, but also automatically issue an alarm or take automatic intervention measures when dangerous behavior is identified, thereby effectively avoiding traffic accidents.

[0004] To achieve the above and other related objectives, the present invention provides the following technical solution: A method for detecting live objects protruding from vehicle windows based on an in-vehicle camera, the method comprising: M1. When the vehicle is driving on the road, it acquires real-time image data of the interior of the vehicle based on the in-vehicle camera and real-time image data of the window area based on the external camera. The data is then preprocessed to obtain preprocessed image data of the interior and window areas. M2. Based on the preprocessed image data information of the vehicle interior and window areas, the yolov8n target detection model is used to identify the images of living beings inside the vehicle and the obstacle images in the window areas, respectively, to obtain the data information of the identified images of living beings inside the vehicle and the obstacle images in the window areas; M3. Based on the data information of the identified live body image inside the vehicle and the obstacle image in the window area, a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism is used to predict the number of overlapping pixels in the live body image inside the vehicle and the obstacle image in the window area, and the data information of the number of overlapping pixels in the predicted live body image inside the vehicle and the obstacle image in the window area is obtained. M4. Based on the data information of the number of overlapping pixels of the predicted images of the living person inside the vehicle and the obstacles in the window area, a preset threshold is set. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is less than the preset threshold, no danger will occur. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

[0005] Furthermore, in step M2, the process of using the YOLOv8n target detection model to identify images of living beings inside the vehicle and obstacle images in the window area includes: M21. Based on the preprocessed image data information of the vehicle interior and window areas, construct the pixel matrix of the images of the vehicle interior and window areas to obtain the data information of the pixel matrix of the images of the vehicle interior and window areas; M22. Input the pixel matrix data of the images of the vehicle interior and window areas into the trained yolov8n target detection model to identify the images of living beings inside the vehicle and obstacles in the window area, and obtain the data of the identified images of living beings inside the vehicle and obstacles in the window area.

[0006] Furthermore, in step M22, the trained yolov8n object detection model includes: M221. Based on the pixel matrix of the images of the vehicle interior and the window area, construct a pixel matrix dataset of the images of the vehicle interior and a pixel matrix dataset of the images of the window area; M222. Input the pixel matrix dataset of the images inside the vehicle and the pixel matrix dataset of the images in the window area into the yolov8n object detection model for training and learning, and obtain the trained yolov8n object detection model.

[0007] Furthermore, the preprocessing includes image cropping, image noise reduction, image smoothing and blurring, sharpening, and enhancement.

[0008] Furthermore, in step M3, the prediction of the number of overlapping pixels in the images of the living person inside the vehicle and the obstacle area in the window region using a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism includes: M31. Input the data information of the identified live body image inside the vehicle and the obstacle image in the window area into the multi-objective regression prediction model of the convolutional neural network-attention mechanism, initialize the parameters of the model, and obtain the data information of the initialized model parameters; M32. Based on the parameter data of the initialized model, the parameters of the model are optimized using the Newton-Raphson algorithm to obtain an optimized multi-objective regression prediction model based on the convolutional neural network-attention mechanism. M33. Based on the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, input the data information of the identified live object inside the vehicle and the obstacle image in the window area, predict the number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area, and obtain the data information of the predicted number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area.

[0009] Furthermore, in step M32, the optimization of the model parameters using the Newton-Raphson algorithm includes: M321. Based on the parameter data of the initialized model, construct an initialization equation system P. , Where x represents the weights in the parameters of the initialized model, y represents the biases in the parameters of the initialized model, x0 represents the initial value of the weights, and y0 represents the initial value of the biases. M322. Construct an iterative system of equations P+Δp. , Where Δp is the iteration step size; M323. If Where ε is a random number between 0 and 1, the iteration ends, the parameters of the model are optimized to obtain the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, otherwise return to step M322.

[0010] Furthermore, the iteration step size Δp is any constant parameter between 0 and 1.

[0011] To achieve the above and other related objectives, the present invention also provides a vehicle window live object detection system based on an in-vehicle camera, used to implement the aforementioned vehicle window live object detection method based on an in-vehicle camera, the system comprising: The data acquisition and preprocessing module is used to acquire real-time image data information of the vehicle interior based on the in-vehicle camera and real-time image data information of the window area based on the external camera, and to perform preprocessing to obtain preprocessed image data information of the vehicle interior and window areas. An image recognition module, connected to the data acquisition and preprocessing module, is used to identify images of living people inside the vehicle and images of obstacles in the window area using the yolov8n target detection model. A pixel overlap prediction module for images of living people inside the vehicle and obstacles in the window area is connected to the image recognition module. It is used to predict the pixel overlap of images of living people inside the vehicle and obstacles in the window area using a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism, and obtain the data information of the pixel overlap of images of living people inside the vehicle and obstacles in the window area after prediction. The threshold judgment module is connected to the pixel overlap prediction module of the image of the living person inside the vehicle and the obstacle in the window area. It is used to set a preset threshold. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is less than the preset threshold, no danger will occur. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

[0012] Furthermore, the system also includes a voice broadcast module connected to the threshold judgment module, used to broadcast warning information in real time.

[0013] Furthermore, the system also includes an alarm light module, which is connected to the threshold judgment module and is used to provide alarm reminders at night.

[0014] The present invention has the following positive effects: This invention employs the YOLOv8N object detection model to identify images of living persons inside a vehicle and images of obstacles in the window area. It also combines a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm and an optimized convolutional neural network-attention mechanism to predict the number of overlapping pixels in the images of living persons inside the vehicle and obstacles in the window area. This not only detects whether children or other passengers have their heads, arms, or other body parts extended outside the window, but also automatically issues an alarm or takes automatic intervention measures when dangerous behavior is detected, thereby effectively preventing traffic accidents. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the method flow of the yolov8n target detection model of the present invention; Figure 3 This is a flowchart illustrating the multi-objective regression prediction algorithm based on the Newton-Raphson algorithm to optimize the convolutional neural network-attention mechanism of the present invention. Figure 4 This is a schematic diagram of the system framework of the present invention. Detailed Implementation

[0016] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0017] Example 1: As Figure 1 As shown, a method for detecting live objects protruding from vehicle windows based on an in-vehicle camera, the method comprising: M1. When the vehicle is driving on the road, it acquires real-time image data of the interior of the vehicle based on the in-vehicle camera and real-time image data of the window area based on the external camera. The data is then preprocessed to obtain preprocessed image data of the interior and window areas. M2. Based on the preprocessed image data information of the vehicle interior and window areas, the yolov8n target detection model is used to identify the images of living beings inside the vehicle and the obstacle images in the window areas, respectively, to obtain the data information of the identified images of living beings inside the vehicle and the obstacle images in the window areas; M3. Based on the data information of the identified live body image inside the vehicle and the obstacle image in the window area, a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism is used to predict the number of overlapping pixels in the live body image inside the vehicle and the obstacle image in the window area, and the data information of the number of overlapping pixels in the predicted live body image inside the vehicle and the obstacle image in the window area is obtained. M4. Based on the data information of the number of overlapping pixels of the predicted images of the living person inside the vehicle and the obstacles in the window area, a preset threshold is set. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is less than the preset threshold, no danger will occur. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

[0018] In this embodiment, as Figure 2 As shown, in step M2, the process of using the yolov8n target detection model to identify images of living beings inside the vehicle and obstacle images in the window area includes: M21. Based on the preprocessed image data information of the vehicle interior and window areas, construct the pixel matrix of the images of the vehicle interior and window areas to obtain the data information of the pixel matrix of the images of the vehicle interior and window areas; M22. Input the pixel matrix data of the images of the vehicle interior and window areas into the trained yolov8n target detection model to identify the images of living beings inside the vehicle and obstacles in the window area, and obtain the data of the identified images of living beings inside the vehicle and obstacles in the window area.

[0019] In this embodiment, in step M22, the trained yolov8n object detection model includes: M221. Based on the pixel matrix of the images of the vehicle interior and the window area, construct a pixel matrix dataset of the images of the vehicle interior and a pixel matrix dataset of the images of the window area; M222. Input the pixel matrix dataset of the images inside the vehicle and the pixel matrix dataset of the images in the window area into the yolov8n object detection model for training and learning, and obtain the trained yolov8n object detection model.

[0020] In this embodiment, the preprocessing includes image cropping, image noise reduction, image smoothing and blurring, sharpening, and enhancement.

[0021] In this embodiment, as Figure 3 As shown, in step M3, the multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism is used to predict the number of overlapping pixels in the images of the living person inside the vehicle and the obstacle area in the window region, which includes: M31. Input the data information of the identified live body image inside the vehicle and the obstacle image in the window area into the multi-objective regression prediction model of the convolutional neural network-attention mechanism, initialize the parameters of the model, and obtain the data information of the initialized model parameters; M32. Based on the parameter data of the initialized model, the parameters of the model are optimized using the Newton-Raphson algorithm to obtain an optimized multi-objective regression prediction model based on the convolutional neural network-attention mechanism. M33. Based on the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, input the data information of the identified live object inside the vehicle and the obstacle image in the window area, predict the number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area, and obtain the data information of the predicted number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area.

[0022] In this embodiment, step M32, which involves optimizing the model parameters using the Newton-Raphson algorithm, includes: M321. Based on the parameter data of the initialized model, construct an initialization equation system P. , Where x represents the weights in the parameters of the initialized model, y represents the biases in the parameters of the initialized model, x0 represents the initial value of the weights, and y0 represents the initial value of the biases. M322. Construct an iterative system of equations P+Δp. , Where Δp is the iteration step size; M323. If Where ε is a random number between 0 and 1, the iteration ends, the parameters of the model are optimized to obtain the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, otherwise return to step M322.

[0023] In this embodiment, the iteration step size Δp is any constant parameter between 0 and 1.

[0024] In this embodiment, a compact SUV of a certain brand is used as an example. It is equipped with a forward-facing ADAS camera (exterior) and a top-mounted DMS camera (interior), with the main control chip being an NVIDIA Orin-NX (8GB RAM). 1200 sets of real-world road scene videos (covering sunny / rainy / foggy / nighttime conditions, vehicle speeds 5-60 km / h) were collected. The annotation team consisted of three senior annotators, achieving a Kappa consistency coefficient of 0.93. The training / validation / test sets were divided in a 7:1.5:1.5 ratio. YOLOv8n was fine-tuned using transfer learning (learning rate 1e−4, Cosine annealing), and the CNN-Attention regression network used the AdamW optimizer (weight decay=0.05). After 200 epochs of training, the following results were achieved on the test set: Average absolute error of coincidence count (MAE): 1.87 pixels; Recall rate for N>T: 98.3% (false negative rate: 1.7%). False alarm rate: 2.4% (mainly due to false detection of obstacles under strong backlight conditions); End-to-end average latency: 46.3 ms (meets ISO 26262 ASIL-B real-time requirements).

[0025] In this embodiment, during a congested morning rush hour, a 6-year-old child leans out of the right rear window of a car to grab a floating balloon. The system detects the child's head and right shoulder within 212ms of the child's movement, identifying that their head and right shoulder fall within the ROI of the billboard as seen by the external camera (N=12.7, T=8.3), triggering a level two warning (flashing red on the dashboard). The driver immediately retracts the window to avoid contact.

[0026] In this embodiment, during heavy rain, water droplets condense on the window glass, causing the traditional method to misidentify water marks as obstacles. Through Guided Filter preprocessing + homography correction of the external ROI of the vehicle, the present invention successfully suppresses artifacts, with N = 0.3 < T = 4.1, and no false alarms are triggered.

[0027] In this embodiment, during night driving, the interior顶灯 of the vehicle is turned off, and only ambient dim light is relied on. The system enables a low-light enhancement module (based on the Retinex theory), and the confidence of key point detection still remains above 0.82, and the coincidence point counting error is controlled within ±2.1 pixels.

[0028] Embodiment 2: Based on the method for detecting living body protrusions from the window of a vehicle-mounted camera in Embodiment 1, the present invention will be further described and described below.

[0029] As Figure 1 shown, a method for detecting living body protrusions from the window of a vehicle-mounted camera, the method includes: M1. When the vehicle is driving on the road, based on the in-vehicle camera, the image data information inside the vehicle is obtained in real time, and based on the external vehicle camera, the image data information of the window area is obtained in real time, and preprocessing is performed to obtain the preprocessed image data information of the inside of the vehicle and the window area. M2. Based on the preprocessed image data information of the inside of the vehicle and the window area, the yolov8n object detection model is used to respectively identify the images of living bodies inside the vehicle and the obstacle images in the window area, and the data information of the identified images of living bodies inside the vehicle and the obstacle images in the window area is obtained. M3. Based on the data information of the identified images of living bodies inside the vehicle and the obstacle images in the window area, a multi-object regression prediction algorithm based on the Newton-Raphson algorithm to optimize the convolutional neural network-attention mechanism is used to predict the number of pixel coincidence points of the living bodies inside the vehicle and the obstacle images in the window area, and the data information of the predicted number of pixel coincidence points of the living bodies inside the vehicle and the obstacle images in the window area is obtained. M4. Based on the data information of the predicted number of pixel coincidence points of the living bodies inside the vehicle and the obstacle images in the window area, a preset threshold is set. If the number of pixel coincidence points of the living bodies inside the vehicle and the obstacles in the window area is less than the preset threshold, no danger will occur. If the number of pixel coincidence points of the living bodies inside the vehicle and the obstacles in the window area is greater than the preset threshold, a collision will occur, and a warning will be issued.

[0030] In this embodiment, as Figure 4 shown, the present invention provides a system for detecting living body protrusions from the window of a vehicle-mounted camera, which is used to implement the method for detecting living body protrusions from the window of a vehicle-mounted camera, and the system includes: The data acquisition and preprocessing module is used to acquire real-time image data information of the vehicle interior based on the in-vehicle camera and real-time image data information of the window area based on the external camera, and to perform preprocessing to obtain preprocessed image data information of the vehicle interior and window areas. An image recognition module, connected to the data acquisition and preprocessing module, is used to identify images of living people inside the vehicle and images of obstacles in the window area using the yolov8n target detection model. A pixel overlap prediction module for images of living people inside the vehicle and obstacles in the window area is connected to the image recognition module. It is used to predict the pixel overlap of images of living people inside the vehicle and obstacles in the window area using a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism, and obtain the data information of the pixel overlap of images of living people inside the vehicle and obstacles in the window area after prediction. The threshold judgment module is connected to the pixel overlap prediction module of the image of the living person inside the vehicle and the obstacle in the window area. It is used to set a preset threshold. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is less than the preset threshold, no danger will occur. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

[0031] In this embodiment, the system further includes a voice broadcast module connected to the threshold judgment module, used to broadcast warning information in real time.

[0032] In this embodiment, the system also includes an alarm light module connected to the threshold judgment module, used to provide alarm reminders at night.

[0033] In this embodiment, dual-view simultaneous acquisition and preprocessing are performed: an in-vehicle camera (FOV 120°, resolution 1280×720, mounted in the center of the roof) captures full-body images of the occupants; an external camera (FOV 85°, same resolution, mounted on the outside of the B-pillar) captures the side-rear scene including the complete window outline. Preprocessing includes: (a) adaptive contrast enhancement based on CLAHE; (b) eliminating glass reflection noise using a Guided Filter (filter radius r=8, ε=0.01); (c) mapping the quadrilateral ROI of the window in the external image to a standard rectangle (1024×768) using the homography matrix H, while simultaneously normalizing the scale of the in-vehicle image accordingly.

[0034] YOLOv8n dual-path parallel detection: A lightweight YOLOv8n model (2.9M parameters, 8.1G FLOPs) is run on two preprocessed images. The in-vehicle path outputs human detection bounding boxes and 17 COCO keypoints; the out-of-vehicle path outputs detection bounding boxes and class confidence scores for all obstacles (including pedestrians, tree branches, billboards, etc.) within the vehicle window area. The timestamp alignment error between the two detection results is ≤3 ms.

[0035] A Newton-Raphson optimized CNN-Attention multi-objective regression: A dual-branch shared-weight CNN backbone (including an SE attention module) is constructed. The inputs are: (i) a heatmap of human key points inside the vehicle (17 channels); (ii) an obstacle mask within the ROI outside the vehicle window (1 channel); and (iii) relative pose priors (yaw rate γ and lateral acceleration ay obtained from the vehicle's CAN bus, encoded as a 2D vector). The network output is a 6D vector: [u1, v1, u2, v2, u3, v3], corresponding to the standard coordinates of the top of the head, left shoulder, and right shoulder on the window plane, respectively. The Newton-Raphson method is used to iteratively solve the problem of minimizing the reprojection error.

[0036] Pixel overlap point counting and dynamic threshold warning: The predicted 3 2D coordinate points are intersected pixel-by-pixel with an external obstacle mask to count the total number of overlap points N. A dynamic threshold T = α*A + β*v + γ is set, where A is the area of ​​the human detection box (pixels), v is the vehicle speed (km / h), α = 0.0012, β = 0.18, and γ = 3.5. When N > T, a three-level warning is triggered: N∈(T, 1.5T] triggers an audible alert; N∈(1.5T, 2.0T] triggers a flashing dashboard indicator; and N>2.0T automatically lowers the windows by 5 cm and locks the windows.

[0037] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to perform the described method for detecting live objects protruding from vehicle windows based on an in-vehicle camera.

[0038] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0039] In summary, this invention can not only detect whether children or other passengers have extended their heads, arms, or other body parts out of the car window, and automatically issue an alarm or take automatic intervention measures when dangerous behavior is identified, but also effectively prevent traffic accidents from occurring.

[0040] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting live objects protruding from vehicle windows based on an in-vehicle camera, characterized in that, The method includes: M1. When the vehicle is driving on the road, it acquires real-time image data of the interior of the vehicle based on the in-vehicle camera and real-time image data of the window area based on the external camera. The data is then preprocessed to obtain preprocessed image data of the interior and window areas. M2. Based on the preprocessed image data information of the vehicle interior and window areas, the yolov8n target detection model is used to identify the images of living beings inside the vehicle and the obstacle images in the window areas, respectively, to obtain the data information of the identified images of living beings inside the vehicle and the obstacle images in the window areas; M3. Based on the data information of the identified live body image inside the vehicle and the obstacle image in the window area, a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism is used to predict the number of overlapping pixels in the live body image inside the vehicle and the obstacle image in the window area, and the data information of the number of overlapping pixels in the predicted live body image inside the vehicle and the obstacle image in the window area is obtained. M4. Based on the data information of the number of overlapping pixels of the predicted images of the living person inside the vehicle and the obstacles in the window area, a preset threshold is set. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is less than the preset threshold, no danger will occur. If the number of overlapping pixels of the living person inside the vehicle and the obstacles in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

2. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 1, characterized in that, In step M2, the process of using the yolov8n target detection model to identify images of living beings inside the vehicle and obstacle images in the window area includes: M21. Based on the preprocessed image data information of the vehicle interior and window areas, construct the pixel matrix of the images of the vehicle interior and window areas to obtain the data information of the pixel matrix of the images of the vehicle interior and window areas; M22. Input the pixel matrix data of the images of the vehicle interior and window areas into the trained yolov8n target detection model to identify the images of living beings inside the vehicle and obstacles in the window area, and obtain the data of the identified images of living beings inside the vehicle and obstacles in the window area.

3. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 2, characterized in that, In step M22, the trained yolov8n object detection model includes: M221. Based on the pixel matrix of the images of the vehicle interior and the window area, construct a pixel matrix dataset of the images of the vehicle interior and a pixel matrix dataset of the images of the window area; M222. Input the pixel matrix dataset of the images inside the vehicle and the pixel matrix dataset of the images in the window area into the yolov8n object detection model for training and learning, and obtain the trained yolov8n object detection model.

4. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 1, characterized in that: The preprocessing includes image cropping, image noise reduction, image smoothing and blurring, sharpening, and enhancement.

5. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 1, characterized in that, In step M3, the prediction of the number of overlapping pixels in the images of the living person inside the vehicle and the obstacle area in the window region using a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism includes: M31. Input the data information of the identified live body image inside the vehicle and the obstacle image in the window area into the multi-objective regression prediction model of the convolutional neural network-attention mechanism, initialize the parameters of the model, and obtain the data information of the initialized model parameters; M32. Based on the parameter data of the initialized model, the parameters of the model are optimized using the Newton-Raphson algorithm to obtain an optimized multi-objective regression prediction model based on the convolutional neural network-attention mechanism. M33. Based on the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, input the data information of the identified live object inside the vehicle and the obstacle image in the window area, predict the number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area, and obtain the data information of the predicted number of overlapping pixels in the live object inside the vehicle and the obstacle image in the window area.

6. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 5, wherein in step M32, the optimization of the model parameters using the Newton-Raphson algorithm includes: M321. Based on the parameter data of the initialized model, construct an initialization equation system P. ， Where x represents the weights in the parameters of the initialized model, y represents the biases in the parameters of the initialized model, x0 represents the initial value of the weights, and y0 represents the initial value of the biases. M322. Construct an iterative system of equations P+Δp. ， Where Δp is the iteration step size; M323. If Where ε is a random number between 0 and 1, the iteration ends, the parameters of the model are optimized to obtain the optimized convolutional neural network-attention mechanism multi-objective regression prediction model, otherwise return to step M322.

7. The method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera according to claim 6, characterized in that: The iteration step size Δp is any constant parameter between 0 and 1.

8. A vehicle window live object detection system based on an in-vehicle camera, characterized in that, For implementing the method for detecting live objects protruding from vehicle windows based on a vehicle-mounted camera as described in any one of claims 1-7, the system comprises: The data acquisition and preprocessing module is used to acquire real-time image data information of the vehicle interior based on the in-vehicle camera and real-time image data information of the window area based on the external camera, and to perform preprocessing to obtain preprocessed image data information of the vehicle interior and window areas. An image recognition module, connected to the data acquisition and preprocessing module, is used to identify images of living people inside the vehicle and images of obstacles in the window area using the yolov8n target detection model. A pixel overlap prediction module for images of living people inside the vehicle and obstacles in the window area is connected to the image recognition module. It is used to predict the pixel overlap of images of living people inside the vehicle and obstacles in the window area using a multi-objective regression prediction algorithm based on the Newton-Raphson algorithm-optimized convolutional neural network-attention mechanism, and obtain the data information of the pixel overlap of images of living people inside the vehicle and obstacles in the window area after prediction. The threshold judgment module is connected to the pixel overlap prediction module of the image of the living person inside the vehicle and the obstacle in the window area. It is used to set a preset threshold. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is less than the preset threshold, no danger will occur. If the number of pixel overlaps between the living person inside the vehicle and the obstacle in the window area is greater than the preset threshold, a collision will occur and a warning will be issued.

9. The vehicle window live object detection system based on a vehicle-mounted camera according to claim 8, characterized in that, The system also includes a voice broadcast module, which is connected to the threshold judgment module and is used to broadcast warning information in real time.

10. The vehicle window live object detection system based on a vehicle-mounted camera according to claim 8, characterized in that, The system also includes an alarm light module, which is connected to the threshold judgment module and is used to provide alarm reminders at night.

Images