Method and system for detecting in-vehicle human key points with high efficiency filtering out inter-frame jitter
By combining an in-vehicle camera with a convolutional network model, the bounding boxes of human bodies are determined and smoothed based on the number of seats in the vehicle. This solves the problem of inter-frame jitter in in-vehicle human body detection and achieves efficient and accurate detection of key points of human bodies in vehicles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MOMENTA (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2021-12-31
- Publication Date
- 2026-08-04
AI Technical Summary
Existing methods for detecting human bodies inside vehicles suffer from high complexity, large computational load, and low accuracy when dealing with inter-frame jitter of human body key points across multiple frames, especially when human bodies overlap or are occluded in front and behind seats, resulting in a high error rate.
Images of the vehicle interior are captured by an in-vehicle camera. A convolutional network model is used to perform human body regression based on a preset number of seats to determine independent bounding boxes. Smoothing filtering is then applied between multiple frames to reduce data processing volume and improve detection accuracy.
It effectively reduces the difficulty of model processing, reduces the amount of data processing, improves the accuracy of human key point detection, and efficiently eliminates inter-frame jitter.
Smart Images
Figure CN116434195B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and in particular to a method and system for detecting key human points inside a vehicle by efficiently filtering out inter-frame jitter. Background Technology
[0002] During vehicle operation, to fully understand the situation of people inside the vehicle and prevent safety issues such as driver or passenger errors or children being left behind, it is necessary to detect the people inside the vehicle. Existing methods for detecting people inside vehicles use general object detection techniques. After acquiring images of the vehicle interior using an in-vehicle camera, a detection model identifies and fuses these images to detect potential human key points. However, this method processes a large amount of data, and when detecting people in the front and rear seats, overlapping and occlusion of people in these seats often lead to errors in the detection results, resulting in incorrect findings. Furthermore, for multiple frames, key points belonging to the same person need to be smoothed to eliminate inter-frame jitter. A common approach is to use tracking algorithms to identify the same human target across multiple frames through matching and fusion, followed by smoothing. However, the process of tracking and matching key points across multiple frames to identify the same target is highly complex and computationally intensive, and the error rate is high in scenes where people overlap in the vehicle interior images. Summary of the Invention
[0003] To address the problems of existing tracking algorithms being complex, computationally intensive, and inaccurate in determining the same target across multiple frames when filtering out inter-frame jitter in human body key points, this application proposes an efficient method and system for detecting human body key points inside a vehicle by filtering out inter-frame jitter.
[0004] One technical solution of this application provides an efficient method for detecting human key points inside a vehicle by filtering out inter-frame jitter, comprising: taking pictures of the vehicle interior using an in-vehicle camera to obtain multiple frames of images of the vehicle interior; using a convolutional network model to perform human regression processing on the current frame image according to a preset number of seats, and determining the preset number of human bounding boxes on the current frame image; detecting human bodies in the current frame image and determining human key points; and performing smoothing filtering processing on the human key points in each human bounding box of the current frame image and the human key points in the corresponding human bounding boxes of historical frame images.
[0005] Optionally, a convolutional network model is used to perform human regression processing on the current frame image according to a preset number of seats, and to determine the human bounding boxes of the preset number of seats on the current frame image. This includes: using the Backbone layer in the pre-trained convolutional network model to extract features from the current frame image to obtain the feature map corresponding to the current frame image; and using a single detector head in the convolutional network model to perform human regression processing on the feature map according to the preset number of seats, and to determine the human bounding boxes of the preset number of seats on the current frame image.
[0006] Optionally, feature extraction of the current frame image can be performed using the Backbone layer in the pre-trained convolutional network model, including: reducing the number of data channels in each semantic layer of the Backbone layer, and using a lower number of data channels to extract features from the in-vehicle image.
[0007] Optionally, a single detection head in the convolutional network model is used to perform human regression processing on the feature map according to a preset number of seats, and a preset number of human bounding boxes are determined on the current frame image. This includes: performing human detection on the feature map based on human features using a single detection head in the convolutional network model, and outputting a preset number of labeled boxes, wherein the labeled boxes are independent of each other and are not fused.
[0008] Optionally, the feature map is processed by a single detector head in the convolutional network model to perform human body regression according to a preset number of seats, and the preset number of human body bounding boxes are determined on the current frame image. This also includes: determining the center point position of the human body on the current frame image as the center position of the corresponding human body bounding box; determining the height and width of the human body bounding box according to the size of the human body in the current frame image, and thus obtaining the human body bounding box.
[0009] Optionally, the human key points in each human bounding box of the current frame image and the corresponding human key points in the human bounding boxes of the historical frame images are subjected to smoothing filtering, including: obtaining the number corresponding to each human bounding box in the current frame image and the historical frame images; and performing smoothing filtering on the human key points corresponding to two human bounding boxes with the same number in the current frame image and the historical frame images.
[0010] Optionally, human body detection is performed in the current frame image to determine human body key points, including: using a convolutional network model to detect the image within the current frame image based on human body features, and determining the probability that a human body exists within the human body bounding box; if the probability value is greater than a preset threshold, it is determined that a human body exists within the human body bounding box, and the human body within the human body bounding box is detected to determine human body key points.
[0011] One technical solution of this application provides an efficient in-vehicle human key point detection system for filtering out inter-frame jitter, comprising: an in-vehicle camera that captures images of the in-vehicle interior to obtain multiple frames of images; a convolutional network model module that performs human regression processing on the current frame image according to a preset number of seats, determines the preset number of human bounding boxes on the current frame image, and detects the human body in the current frame image to determine human key points; and a smoothing filtering module that performs smoothing filtering processing on the human key points in each human bounding box of the current frame image and the human key points in the corresponding human bounding boxes of historical frame images.
[0012] In one technical solution of this application, a computer-readable storage medium is provided, which stores computer instructions that are operated to execute the efficient in-vehicle human body key point detection method for filtering inter-frame jitter in Solution 1.
[0013] In one technical solution of this application, a computer device is provided, which includes a processor and a memory. The memory stores computer instructions, wherein the processor operates the computer instructions to execute the efficient in-vehicle human key point detection method for filtering inter-frame jitter in Solution 1.
[0014] The beneficial effects of this application are: by utilizing the number of seats inside the vehicle, when the model detects various human key points in the images inside the vehicle, the application constrains the number of human bodies detected by the model, obtains a certain number of human body bounding boxes, reduces the difficulty of model processing, reduces the amount of data processing, and improves the accuracy of human body key point detection. At the same time, by using a certain number of independent human body bounding boxes, the human body key points belonging to the same human body in multiple frames are associated, effectively eliminating inter-frame jitter. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart illustrating one implementation of the in-vehicle human body key point detection method for efficiently filtering inter-frame jitter according to this application.
[0017] Figure 2 This is a schematic diagram of the basic structure of a convolutional neural network commonly used for object detection;
[0018] Figure 3 This is a schematic diagram of one embodiment of the in-vehicle human body key point detection system for efficiently filtering inter-frame jitter, as described in this application.
[0019] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0021] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a product or device comprising a series of steps or units is not necessarily limited to those units explicitly listed, but may include other units not explicitly listed or inherent to such products or devices.
[0022] During vehicle operation, to fully understand the situation of people inside the vehicle and prevent safety issues such as driver or passenger errors or children being left behind, it is necessary to detect the people inside the vehicle. Existing methods for detecting people inside vehicles use general object detection techniques. After acquiring images of the vehicle interior using an in-vehicle camera, a detection model identifies and fuses these images to detect potential human key points. However, this method involves processing a large amount of data. Furthermore, when detecting people in the front and rear seats, overlapping and occlusion of people in these seats often lead to incorrect detection results. Additionally, for multiple frames, key points belonging to the same person need to be smoothed to eliminate inter-frame jitter. A common approach is to use tracking algorithms to identify the same human target across multiple frames through matching and fusion, followed by smoothing. However, the process of tracking and matching key points across multiple frames to identify the same target is highly complex and computationally intensive. Moreover, in scenes where people overlap in the vehicle interior images, the error rate is very high.
[0023] Therefore, this application redefines the problem of human detection inside vehicles. Based on the premise that the number of human bodies inside a vehicle will not exceed the number of seats, when performing human detection on images of the vehicle interior captured by an onboard camera, the detection model directly returns the number of human detection boxes equal to the number of seats. By determining whether a human body is contained within each box, the detection of human bodies inside the vehicle is completed. Compared to existing detection methods that detect and judge each pixel in the image and then fuse them, this significantly reduces the computational load and speeds up the detection process. Furthermore, the detected human detection boxes are independent and not fused, improving the accuracy of human detection. Additionally, the pre-determined human body bounding boxes are used to smooth key points of the same human body across frames, avoiding the need for complex tracking algorithms to determine the same target across multiple frames, thus making the process more efficient.
[0024] To address the aforementioned issues, this application proposes an efficient method and system for detecting human key points inside a vehicle by filtering out inter-frame jitter. The method includes: capturing multiple frames of images of the vehicle interior using an onboard camera; performing human regression processing on the current frame image according to a preset number of seats using a convolutional network model to determine the preset number of human bounding boxes on the current frame image; detecting human bodies in the current frame image to determine human key points; and performing smoothing filtering on the human key points in each human bounding box of the current frame image and the corresponding human bounding boxes in historical frame images to reduce inter-frame jitter.
[0025] This application presents an efficient method for detecting human key points inside a vehicle by filtering out inter-frame jitter. This method utilizes the number of seats inside the vehicle to constrain the number of human bodies detected by the model when detecting various human key points in images inside the vehicle. This results in a certain number of human body bounding boxes, reducing the difficulty of model processing, decreasing data processing volume, and improving the accuracy of human key point detection. Furthermore, by using a certain number of independent human body bounding boxes, it associates human key points belonging to the same human body across multiple frames, effectively eliminating inter-frame jitter.
[0026] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0027] Figure 1 This paper illustrates one embodiment of the in-vehicle human key point detection method for efficiently filtering inter-frame jitter, as described in this application.
[0028] exist Figure 1In the embodiment shown, the efficient method for detecting human key points inside a vehicle by filtering out inter-frame jitter in this application includes process S101, which involves taking pictures of the inside of the vehicle using an onboard camera to obtain multiple frames of images inside the vehicle.
[0029] In this embodiment, the in-vehicle human body detection method of this application is also based on the detection of in-vehicle images captured by an in-vehicle camera. By detecting human body parts in the in-vehicle images, the detection and judgment of human bodies in the vehicle are completed. The in-vehicle camera is generally installed at the upper front of the vehicle to effectively detect information from all seats while minimizing obstruction between people. The installation position of the in-vehicle camera and the resolution of the in-vehicle images can be reasonably set according to the actual vehicle conditions, and this application does not impose specific limitations.
[0030] exist Figure 1 In the embodiment shown, the efficient method for detecting human body key points inside a vehicle by filtering out inter-frame jitter in this application includes process S102, which uses a convolutional network model to perform human body regression processing on the current frame image according to a preset number of seats, and determines the human body bounding boxes of the preset number of seats on the current frame image.
[0031] In this embodiment, during the detection of key points of human bodies inside the vehicle, this application utilizes the relationship between the number of seats and the number of human bodies inside the vehicle, where the number of human bodies generally does not exceed the number of seats. Using this relationship, the convolutional network model is constrained, requiring only that the convolutional network model regress the bounding boxes of human bodies for a preset number of seats, reducing the difficulty of model processing, improving computational speed, and increasing the accuracy of the final obtained human body bounding boxes.
[0032] Optionally, a convolutional network model is used to perform human regression processing on the current frame image according to a preset number of seats, and to determine the human bounding boxes of the preset number of seats on the current frame image. This includes: using the Backbone layer in the pre-trained convolutional network model to extract features from the current frame image to obtain the feature map corresponding to the current frame image; and using a single detector head in the convolutional network model to perform human regression processing on the feature map according to the preset number of seats, and to determine the human bounding boxes of the preset number of seats on the current frame image.
[0033] In this optional embodiment, when performing human regression processing on the current frame image captured by the in-vehicle camera using a convolutional network model, the main process involves extracting features from the current frame image using the Backbone layer in the convolutional network model to obtain the feature map corresponding to the current frame image. Then, a single detector head in the convolutional network model is used to perform human regression processing on the feature map, ultimately obtaining the human bounding boxes for the preset number of seats.
[0034] Figure 2 This is a schematic diagram of the basic structure of a convolutional neural network commonly used for object detection.
[0035] like Figure 2 As shown, a typical convolutional neural network module used for object detection generally includes a backbone layer, a feature pyramid, and a detection head. The backbone layer comprises multiple semantic layers, extracting higher-level semantic features from the in-vehicle image through layer-by-layer convolutional processing, from layer C3 to layer C4, and finally to layer C5. Then, the extracted speech feature map is fused with the corresponding part of the feature pyramid, and the final result is input into the corresponding detection head. The detection head plays a crucial role in the object recognition task.
[0036] Optionally, feature extraction of the current frame image can be performed using the Backbone layer in the pre-trained convolutional network model, including: reducing the number of model channels in each semantic layer of the Backbone layer, and using a lower number of model channels to extract features from the in-vehicle image.
[0037] In this optional embodiment, the detection of people inside the vehicle using in-vehicle images does not require excessively precise image information to complete the detection task. Furthermore, the more data model channels analyzed in the image, the greater the corresponding data processing volume, increasing the model's data processing burden. Here, model channels correspond to specific image features for detection, such as texture and color. Therefore, when performing in-vehicle human detection, only the number of channels corresponding to features useful for human detection needs to be retained, reducing the data processing volume while accurately detecting people inside the vehicle. Thus, unlike existing technologies that use standard convolutional network models for in-vehicle human detection, this application reduces the number of model channels in each semantic layer of the Backbone layer of the convolutional network model, utilizing a lower number of model channels to complete feature extraction from in-vehicle images.
[0038] Specifically, in the implementation of the seat-based in-vehicle human detection method of this application, the number of model channels in each semantic layer C3-C5 of the Backbone base layer can be halved. For example, the number of channels in 32 convolutional layers can be halved to 16. By reducing the number of model channels, the amount of data processing for the convolutional network model is greatly reduced, and the data processing speed is improved.
[0039] In this embodiment, with Figure 2In existing technologies, after layer-by-layer processing of semantic layers in the Backbone base layer, the resulting feature maps are input into the feature pyramid for feature expansion and fusion. The expanded and fused feature maps are then input into multiple corresponding detector heads for classification and regression processing. Unlike existing technologies, the seat-number-based in-vehicle human detection method of this application skips the feature pyramid after determining the feature map corresponding to the in-vehicle image in the Backbone base layer, inputting the feature map into a single detector for classification and regression processing. By omitting the feature expansion and fusion part of the feature pyramid and using only one detector head to complete the detection of in-vehicle humans, the method of this application is more efficient, resulting in a smaller convolutional network model and significantly improved data processing speed.
[0040] Optionally, a single detection head in the convolutional network model is used to perform human regression processing on the feature map according to a preset number of seats, and a preset number of human bounding boxes are determined on the current frame image. This includes: performing human detection on the feature map based on human features using a single detection head in the convolutional network model, and outputting a preset number of labeled boxes, wherein the labeled boxes are independent of each other and are not fused.
[0041] In this optional embodiment, when existing technologies perform regression processing for human bodies inside a vehicle using a detection head, the detection head detects various parts of the image and fuses the detected results to obtain the final human body bounding box. Since the model does not know the exact number of targets to be regressed during processing, the process is complex. Furthermore, because the detection results need to be fused, errors can easily occur, especially when considering front and rear seats in the image. Unlike existing technologies that use detection heads to regress human bodies in vehicle images, the solution in this application pre-knows the number of seats inside the vehicle, which is the maximum number of human bodies. Under normal circumstances, the number of people inside the vehicle does not exceed the number of seats. Therefore, the detection head in this application's seat-based human body detection method only needs to regress a preset number of bounding boxes. This application, through a change in logic, informs the model in advance of the number of bounding boxes to be regressed, thereby reducing the processing difficulty for the model to obtain the final result. Simultaneously, the regressed bounding boxes are independent of each other and are not fused, allowing for accurate detection even when front and rear seats overlap. This improves the accuracy and speed of human body detection inside the vehicle.
[0042] Optionally, the feature map is processed by a single detector head in the convolutional network model to perform human body regression according to a preset number of seats, and the preset number of human body bounding boxes are determined on the current frame image. This also includes: determining the center point position of the human body on the current frame image as the center position of the corresponding human body bounding box; determining the height and width of the human body bounding box according to the size of the human body in the current frame image, and thus obtaining the human body bounding box.
[0043] In this optional embodiment, when determining the bounding box of the human body inside the vehicle, a single detection head in the convolutional network model detects the feature map corresponding to the image to determine the center position of the human body inside the vehicle. Based on the size of the human body in the image, the height and width of the corresponding size of the bounding box are determined, and finally the bounding box of the human body is determined on the image, thereby determining the position of the human body inside the vehicle.
[0044] exist Figure 1 In the embodiment shown, the efficient method for detecting human body key points inside a vehicle by filtering out inter-frame jitter in this application includes process S103, which detects the human body in the current frame image and determines the human body key points.
[0045] In this embodiment, after processes S101 and S102, the bounding boxes of the preset number of human bodies in the current frame image are determined, and then the human body key points are detected within the human body annotation to finally determine the human body key points.
[0046] Optionally, human body detection is performed in the current frame image to determine human body key points, including: using a convolutional network model to detect the image within the current frame image based on human body features, and determining the probability that a human body exists within the human body bounding box; if the probability value is greater than a preset threshold, it is determined that a human body exists within the human body bounding box, and the human body within the human body bounding box is detected to determine human body key points.
[0047] In this optional embodiment, to accelerate the speed and efficiency of human keypoint detection, a convolutional network model is used to determine the probability of whether a human body is contained within the bounding box. The convolutional network model is used to classify whether a human body is contained within the bounding box, calculating the probability that a human body is contained within the standard bounding box. If this probability value is greater than a preset threshold, it is determined that a human body exists within the bounding box. At this point, human keypoint detection is performed on the current frame image to ultimately determine the human keypoints.
[0048] Specifically, the preset threshold can be selected as 50%, and the specific threshold can be determined and reasonably selected according to the actual judgment requirements. This application does not impose specific restrictions.
[0049] It should be noted that in the specific process of vehicle interior image detection using the convolutional network model, the regression of the bounding box of the human body inside the vehicle and the calculation of the classification probability of whether there is a human body in the labeled box can be performed simultaneously. As two branches of the detection head, the order in which they are processed is not restricted in this application, and the final results can be output simultaneously.
[0050] exist Figure 1 In the embodiment shown, the efficient method for detecting human body key points inside a vehicle by filtering out inter-frame jitter in this application includes process S104, which performs smoothing filtering on the human body key points in each human body bounding box of the current frame image and the human body key points in the corresponding human body bounding boxes of the historical frame images.
[0051] In this implementation, after obtaining the corresponding human bounding boxes in each frame, the next step is to perform smoothing processing on the key points of the same human body across multiple frames. When determining the same target across multiple frames, existing techniques often employ tracking algorithms, such as IoU (Intersection of Union), to iterate through the detection results of each frame and ultimately identify human targets in different frames that meet the requirements as the same target. This method involves computationally intensive and complex tracking algorithms. Furthermore, if there is overlap between people in a single frame, the accuracy of this method will be significantly reduced. It may mistakenly identify two human bodies as a single target during tracking. After tracking is complete, the same target is assigned the same number for differentiation and recording.
[0052] The method in this application directly smooths the human key points within the corresponding human key points in the historical frames based on the independent human bounding boxes determined in the current frame. In other words, the human bounding boxes in each frame determined by this application have a one-to-one correspondence, allowing for direct smoothing of key points between different frames. Therefore, it eliminates the need for traversal in the tracking algorithm, where smoothing is only performed after determining that the target belongs to the same human body. By smoothing and filtering human key points belonging to the same human body across frames, incorrectly identified human key points are filtered out or corrected, resulting in smoother and more stable human key points across multiple frames.
[0053] Optionally, the human key points in each human bounding box of the current frame image and the corresponding human key points in the human bounding boxes of the historical frame images are subjected to smoothing filtering, including: obtaining the number corresponding to each human bounding box in the current frame image and the historical frame images; and performing smoothing filtering on the human key points corresponding to two human bounding boxes with the same number in the current frame image and the historical frame images.
[0054] In this optional embodiment, after the human body bounding box is determined in the current frame, it is assigned a corresponding number. When performing smoothing processing of human body key points across multiple frames, the numbers corresponding to each human body bounding box in the current frame image and historical frame images are obtained. By determining whether the numbers are the same, if the numbers of human body bounding boxes in different frames are the same, then they belong to the same human target, and smoothing processing of human body key points between different frames can be directly performed. This simplifies the process by omitting the determination of which human body bounding boxes in different frame images belong to the same human body, while improving accuracy.
[0055] Specifically, when detecting key points of human bodies inside a vehicle with 5 seats, a convolutional network model is used to perform classification and regression processing on the first frame image, resulting in 5 human body bounding boxes. These 5 bounding boxes are independent of each other and are not fused. They are then assigned the numbers A, B, C, D, and E. Similarly, when detecting key points in the second frame image, 5 bounding boxes with the same numbers are obtained. For key point smoothing across multiple frames, key points within the bounding box A in the first frame are directly fused with those within the bounding box A in the second frame. In contrast, existing technologies involve tracking algorithms to obtain the corresponding bounding boxes before smoothing. This method, however, already knows the bounding box numbers in advance and does not require tracking, thus making it more efficient.
[0056] The efficient in-vehicle human keypoint detection method for filtering inter-frame jitter in this application utilizes the number of seats in the vehicle to constrain the number of human bodies detected by the model when detecting various human keypoints in the in-vehicle images. This results in a certain number of human body bounding boxes, reducing the difficulty of model processing, reducing data processing volume, and improving the accuracy of human keypoint detection. At the same time, a certain number of independent human body bounding boxes are used to associate human keypoints belonging to the same human body across multiple frames, effectively eliminating inter-frame jitter.
[0057] Figure 3 This paper illustrates one embodiment of the in-vehicle human key point detection system that efficiently filters out inter-frame jitter according to this application.
[0058] exist Figure 3 In the embodiments shown, the efficient in-vehicle human key point detection system for filtering inter-frame jitter of this application includes: an in-vehicle camera 301, which captures images of the in-vehicle interior to obtain multiple frames of images; a convolutional network model module 302, which performs human regression processing on the current frame image according to a preset number of seats, determines the preset number of human bounding boxes on the current frame image, and detects the human bodies within the human bounding boxes to determine human key points; and a smoothing filter module 303, which performs smoothing filter processing on the human key points in each human bounding box of the current frame image and the human key points in the corresponding human bounding boxes of historical frame images.
[0059] Optionally, in the convolutional network model module, the Backbone layer in the pre-trained convolutional network model is used to extract features from the current frame image to obtain the feature map corresponding to the current frame image; a single detector head in the convolutional network model is used to perform human regression processing on the feature map according to the preset number of seats to determine the human bounding boxes of the preset number of seats on the current frame image.
[0060] Optionally, in the convolutional network model module, the number of model channels in each semantic layer of the Backbone base layer can be reduced to use a lower number of model channels for feature extraction from in-vehicle images.
[0061] Optionally, in the convolutional network model module, a single detection head in the convolutional network model performs human detection on the feature map based on human features, and outputs a preset number of bounding boxes, where the bounding boxes are independent of each other and are not fused.
[0062] Optionally, in the convolutional network model module, the center point of the human body in the current frame image is determined as the center point of the corresponding human body bounding box; the height and width of the human body bounding box are determined according to the size of the human body in the current frame image, thus obtaining the human body bounding box.
[0063] Optionally, a smoothing filter module obtains the numbers corresponding to each human bounding box in the current frame image and historical frame images; and performs smoothing filter processing on the human key points corresponding to two human bounding boxes with the same number in the current frame image and historical frame images.
[0064] In one specific embodiment of this application, a computer-readable storage medium stores computer instructions, wherein the computer instructions are operated to perform an efficient in-vehicle human keypoint detection method for filtering inter-frame jitter as described in any embodiment. The storage medium may be located directly in hardware, in a software module executed by a processor, or in a combination of both.
[0065] Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in this art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium.
[0066] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in the user terminal. Alternatively, the processor and storage medium can reside as discrete components in the user terminal.
[0067] In one specific embodiment of this application, a computer device includes a processor and a memory, the memory storing computer instructions, wherein the processor operates the computer instructions to execute the efficient in-vehicle human key point detection method for filtering inter-frame jitter as described in any embodiment.
[0068] In the embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0069] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0070] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.
Claims
1. A highly efficient method for detecting key human figures inside a vehicle by filtering out inter-frame jitter, characterized in that, include: Multiple frames of images of the car's interior were obtained by taking pictures of the interior using an in-vehicle camera. Using a pre-trained convolutional network model, human regression processing is performed on the current frame image according to a preset number of seats, and the preset number of human bounding boxes are determined on the current frame image. This process of using a pre-trained convolutional network model to perform human regression processing on the current frame image according to a preset number of seats and determining the preset number of human bounding boxes on the current frame image includes: The Backbone layer in the pre-trained convolutional network model is used to extract features from the current frame image to obtain the feature map corresponding to the current frame image. The number of data channels in the semantic layers C3-C5 of the Backbone layer is halved. The feature map is processed by a single detection head in the convolutional network model to perform human body regression according to a preset number of seats. The preset number of human body bounding boxes are determined on the current frame image. The single detection head includes a bounding box regression branch and a human body presence probability classification branch set in parallel. The human body bounding box regression result and the human body presence probability classification result are output synchronously. Human bodies are detected in the current frame image to determine key human body points; After determining the preset number of human body bounding boxes on the current frame image, assign a number to the preset number of human body bounding boxes, and perform smoothing filtering on the human body key points in each human body bounding box of the current frame image and the human body key points in the corresponding human body bounding boxes of historical frame images according to the number. The human body bounding boxes in each frame image have a one-to-one correspondence, and human body bounding boxes with the same number in different frame images belong to the same human body target.
2. The in-vehicle human keypoint detection method with high efficiency of filtering out inter-frame jitter according to claim 1, characterized in that, Using a single detector head in a convolutional network model, human regression is performed on the feature map according to a preset number of seats. This process determines the preset number of human bounding boxes on the current frame image, including: The convolutional network model uses a single detector head to perform human detection on the feature map based on human features, and outputs a preset number of human bounding boxes, wherein the human bounding boxes are independent of each other and are not fused.
3. The in-vehicle human keypoint detection method of claim 2, wherein, The step of using a single detector head in a convolutional network model to perform human regression processing on the feature map according to a preset number of seats, and determining the preset number of human bounding boxes on the current frame image, further includes: The center point of the human body on the current frame image is determined and used as the center position of the corresponding human body bounding box. The height and width of the human body bounding box are determined based on the size of the human body in the current frame image, thereby obtaining the human body bounding box.
4. The in-vehicle human keypoint detection method with high efficiency of filtering out inter-frame jitter according to claim 1, characterized in that, The step of performing smoothing filtering on the human body key points in each human body bounding box of the current frame image and the corresponding human body key points in the human body bounding boxes of the historical frame images includes: Obtain the numbers corresponding to the human body bounding boxes in the current frame image and the historical frame images; The human body key points corresponding to the two human body bounding boxes with the same number in the current frame image and the historical frame image are subjected to smoothing filtering.
5. The in-vehicle human keypoint detection method of claim 1, wherein, The step of detecting the human body in the current frame image and determining key points of the human body includes: Using the convolutional network model, the image within the current frame is detected based on human body features to determine the probability that a human body exists within the bounding box of the human body. If the probability value is greater than a preset threshold, it is determined that a human body exists within the human body bounding box, and the human body within the human body bounding box is detected to determine the key points of the human body.
6. An in-vehicle human keypoint detection system with high efficiency of filtering out inter-frame jitter, characterized in that, include: The vehicle-mounted camera captures images of the vehicle's interior, producing multiple frames of pictures. The convolutional network model module utilizes a pre-trained convolutional network model to perform human regression processing on the current frame image according to a preset number of seats, and determines the preset number of human bounding boxes on the current frame image. The step of using the pre-trained convolutional network model to perform human regression processing on the current frame image according to a preset number of seats and determining the preset number of human bounding boxes on the current frame image includes: The Backbone layer in the pre-trained convolutional network model is used to extract features from the current frame image to obtain the feature map corresponding to the current frame image. The number of data channels in the semantic layers C3-C5 of the Backbone layer is halved. Using a single detection head in the convolutional network model, the feature map is processed by human regression according to a preset number of seats. A preset number of human bounding boxes are determined on the current frame image, and human bodies in the current frame image are detected to determine human body key points. The single detection head includes a bounding box regression branch and a human body presence probability classification branch set in parallel, and the human body bounding box regression result and human body presence probability classification result are output synchronously. The smoothing filter module determines the preset number of human body bounding boxes on the current frame image, assigns a number to the preset number of human body bounding boxes, and performs smoothing filter processing on the human body key points in each human body bounding box of the current frame image and the human body key points in the corresponding human body bounding boxes of historical frame images according to the number. The human body bounding boxes in each frame image have a one-to-one correspondence, and human body bounding boxes with the same number in different frames images belong to the same human body target.
7. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions that are operated to perform the in-vehicle human key point detection method for efficiently filtering inter-frame jitter as described in any one of claims 1-5.
8. A computer device comprising a processor and a memory, the memory storing computer instructions, wherein: The processor operates computer instructions to execute the in-vehicle human key point detection method for efficiently filtering inter-frame jitter as described in any one of claims 1-5.