Method, device and equipment for detecting passengers in cabin, medium and product

Abstract

The invention relates to the technical field of vehicles, and provides a method, device and equipment for detecting passengers in a cabin, a medium and a product. The method for detecting the passengers in the cabin comprises the steps of obtaining signal data collected by a radar in the cabin; taking the signal data corresponding to the plurality of sampling points and the plurality of acquisition periods as a group, and converting each group of signal data into pixel data; subtracting pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; and processing the target image sequence through a convolutional neural network to obtain a detection result of whether a passenger exists in the cabin. According to the passenger detection scheme in the cockpit, the problem that small-amplitude breathing and heartbeat and adult high-speed movement are difficult to distinguish due to complex background noise can be effectively solved, the detection precision of the passenger in the cockpit based on the radar is improved, the passenger detection result is accurate and reliable, and the passenger detection scheme can serve application scenes such as passenger retention response and the like.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a method, device, equipment, medium and product for detecting passengers in a cabin. Background Art

[0002] As the demand for vehicle safety and controllability continues to increase, detecting occupants in the cabin, such as detecting whether there are living organisms in the cabin to trigger the vehicle's safety response mechanism, is currently a hot research technology.

[0003] Among the current mainstream technical means, radar in the cockpit is used to collect signals, convert the signals from the time domain to the frequency domain, and then based on the frequency characteristics of human breathing and heartbeat, filter out background noise and extract signals related to human breathing and heartbeat to determine whether there are any living organisms in the cockpit.

[0004] However, the background noise in the environment is relatively complex and is usually divided into two types: low-frequency and high-frequency. High-frequency noise is equivalent to the signal frequency of a person's rapid movement, and low-frequency noise is equivalent to the slight breathing frequency of a baby or pet sleeping. Such background noise is mixed in the signal collected by the radar and is difficult to distinguish and extract, resulting in low accuracy in the current radar-based detection of cabin occupants.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In view of this, the present invention provides a method, device, equipment, medium and product for detecting occupants in the cabin, which effectively solves the problem that it is difficult to distinguish small breathing and heartbeats and high-speed movements of adults due to complex background noise, improves the accuracy of radar-based occupant detection in the cabin, and makes the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0007] According to one aspect of the present invention, there is provided a method for detecting occupants in a cockpit, comprising: obtaining signal data collected by a radar in the cockpit; taking signal data corresponding to a plurality of sampling points and a plurality of collection cycles as a group, and converting each group of signal data into pixel data; subtracting the pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; and processing the target image sequence through a convolutional neural network to obtain a detection result of whether there are occupants in the cockpit.

[0008] In some embodiments, converting each group of signal data into pixel data includes: converting each signal data in a group of signal data into amplitude data to obtain a group of amplitude data, wherein each signal data corresponds to a sampling point and an acquisition cycle; using the sampling point and the acquisition cycle as pixel coordinates, converting each amplitude data in the group of amplitude data into pixel data to obtain pixel data corresponding to the group of signal data.

[0009] In some embodiments, converting each amplitude data in the group of amplitude data into pixel data includes: normalizing each amplitude data to obtain corresponding pixel data.

[0010] In some embodiments, the radar is an ultra-wideband radar, which is arranged in front of the cabin and has a scanning range covering the cabin space.

[0011] In some embodiments, subtracting pixel data corresponding to two adjacent groups of signal data includes: subtracting pixel data corresponding to the adjacent previous group of signal data from pixel data corresponding to the latter group of signal data in a time sequence of the acquisition cycle to obtain a target image.

[0012] In some embodiments, before processing the target image sequence through a convolutional neural network, it also includes: converting the dimension of each target image into a preset dimension through image scaling; and converting each target image into a three-channel target image through channel replication.

[0013] In some embodiments, the detection result of whether there is an occupant in the cabin includes that there is no occupant in the cabin and there are first-category occupants and / or second-category occupants in the cabin.

[0014] In some embodiments, the convolutional neural network is deployed on a central processing unit of a vehicle.

[0015] According to another aspect of the present invention, there is provided an in-cabin occupant detection device for implementing the in-cabin occupant detection method described in any of the above-mentioned embodiments, the in-cabin occupant detection device comprising: a signal data acquisition module, configured to obtain signal data collected by a radar in the cabin; a signal data conversion module, configured to group signal data corresponding to multiple sampling points and multiple acquisition cycles, and convert each group of signal data into pixel data; a target image acquisition module, configured to subtract pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; and a target image processing module, configured to process the target image sequence through a convolutional neural network to obtain an occupant detection result.

[0016] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor; a memory, wherein the memory stores executable instructions; wherein when the executable instructions are executed by the processor, the method for detecting an occupant in the cabin described in any of the above embodiments is implemented.

[0017] According to yet another aspect of the present invention, a computer-readable storage medium is provided for storing a program, wherein when the program is executed by a processor, the method for detecting an occupant in a cabin as described in any of the above embodiments is implemented.

[0018] According to yet another aspect of the present invention, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for detecting an occupant in a cabin as described in any of the above embodiments is implemented.

[0019] The beneficial effects of the present invention compared with the prior art include at least:

[0020] The cabin occupant detection scheme of the present invention collects signal data through the radar in the cabin so as to detect and analyze whether there are occupants (living organisms) in the cabin; by converting the signal data corresponding to multiple sampling points and multiple acquisition cycles into pixel data, combining the characteristics of time and space, enriching the image dimension; by subtracting the pixel data corresponding to two adjacent groups of signal data, effectively removing the background noise with small fluctuations, retaining the signal data of human breathing, heartbeat, movement and other signal data with large fluctuations, and realizing the accurate capture of the signal data representing the vital signs of the occupants. Specifically, the high-frequency and low-frequency background noise is usually relatively constant, which is manifested in a small change in the two adjacent groups of pixel data, while the vital signs data such as human breathing, heartbeat, movement and so on have obvious changes in the two adjacent groups of pixel data. Therefore, by subtracting the two adjacent groups of pixel data, the target image sequence obtained effectively removes the background noise, accurately captures the image changes, and combines the multi-dimensional characteristics of time and space to accurately reflect the relevant characteristics of the occupant's vital signs. Further, the features of the target image are extracted and analyzed by the convolutional neural network, and the detection result of whether there are occupants in the cabin is accurately obtained.

[0021] Therefore, the in-cabin occupant detection solution of the present invention effectively solves the problem that it is difficult to distinguish between small breathing and heartbeats and high-speed movements of adults due to complex background noise, improves the accuracy of radar-based in-cabin occupant detection, and makes the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0024] Figure 1 A schematic diagram showing the steps of a method for detecting an occupant in a cabin according to an embodiment of the present invention;

[0025] Figure 2 A schematic diagram showing how to convert a set of signal data into pixel data in an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a module of a cabin occupant detection device according to an embodiment of the present invention is shown;

[0027] Figure 4 A schematic diagram showing the structure of an electronic device in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art.

[0029] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0030] The processes shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps can be decomposed, some steps can be combined or partially combined, and the order of actual execution may change according to the actual situation. The words "first", "second" and similar words used in the specific description do not indicate any order, quantity or importance, but are only used to distinguish different components. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically limited.

[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and features in different embodiments may be combined with each other.

[0032] Figure 1 The main steps of the method for detecting occupants in the cabin are shown in Fig. Figure 1 As shown, the method for detecting an occupant in a cabin provided by an embodiment of the present invention includes:

[0033] S110, obtaining signal data collected by the radar in the cockpit.

[0034] The radar periodically collects signal data from multiple sampling points, and its collection period and number of sampling points can be configured as needed. For example, in one example, the radar is configured with 35 sampling points and a collection period of 0.05s; that is, the radar collects signal data from 35 sampling points every 0.05s.

[0035] The radar in the cockpit collects signal data to facilitate subsequent detection and analysis of whether there are passengers (living organisms) in the cockpit. Specifically, the radar emits electromagnetic waves and receives the reflected signal data. Based on the Doppler effect of electromagnetic waves and the micro-movements of the human body surface caused by vital signs such as breathing and heartbeat, the vital signs can be detected by analyzing the changes in signal data.

[0036] S120, taking signal data corresponding to a plurality of sampling points and a plurality of acquisition cycles as a group, and converting each group of signal data into pixel data.

[0037] The radar collects signal data from multiple sampling points in each acquisition cycle. The signal data collected by the radar in multiple acquisition cycles is taken to form a group of signal data. For example, the radar collects signal data from 35 sampling points in each acquisition cycle. The signal data collected by the radar in 200 acquisition cycles is taken to form a group of signal data. The number of acquisition cycles corresponding to each group of signal data is the same. A group of signal data contains the spatial features of multiple sampling points and the temporal features of multiple acquisition cycles. By converting the signal data corresponding to multiple sampling points and multiple acquisition cycles into pixel data, the image dimension is enriched by combining the characteristics of time and space.

[0038] Among them, the signal data collected by the radar is data in the form of analog signals, and the pixel data is data in the form of digital signals. The process of converting the signal data into pixel data mainly includes converting the signal data in the form of analog signals into pixel data in the form of digital signals, which will be explained in detail below.

[0039] S130, subtracting pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence.

[0040] Wherein, subtracting pixel data corresponding to two adjacent groups of signal data includes: according to the time sequence of the acquisition cycle, subtracting pixel data corresponding to the latter group of signal data from pixel data corresponding to the adjacent previous group of signal data to obtain a target image. For example, a group of signal data consists of signal data collected by the radar in 200 acquisition cycles (signal data of 35 sampling points are collected in each acquisition cycle); the pixel data corresponding to a group of signal data collected by the radar in the 201st acquisition cycle to the 400th acquisition cycle is subtracted from the pixel data corresponding to a group of signal data collected by the radar in the 1st acquisition cycle to the 200th acquisition cycle, that is, the first target image is obtained; the pixel data corresponding to a group of signal data collected by the radar in the 401st acquisition cycle to the 600th acquisition cycle is subtracted from the pixel data corresponding to a group of signal data collected by the radar in the 201st acquisition cycle to the 400th acquisition cycle, that is, the second target image is obtained; and so on, a target image sequence is obtained.

[0041] In the vehicle's internal and external environment, the background noise is usually relatively constant, which is reflected in the small changes in the two adjacent sets of pixel data; while the vital signs data such as human breathing, heartbeat, and movement have obvious changes in the two adjacent sets of pixel data. Specifically, the radar emits electromagnetic waves and collects the echoes after the electromagnetic waves are reflected by obstacles to form signal data; when there are objects around the radar, a peak will appear at the corresponding position of the echo, and when the object moves, the peak at the corresponding position will change due to the Doppler effect. Therefore, when the object to be detected is a living organism, the signal data corresponding to the limb movement, breathing, heartbeat and other behaviors of the living organism will fluctuate significantly in the radar's acquisition cycle before and after; while the background noise is relatively static, and its corresponding signal data has almost no fluctuations in the radar's acquisition cycle before and after. By subtracting the two adjacent sets of pixel data, the background noise with small fluctuations is effectively removed, and the signal data of human breathing, heartbeat, movement and other signals with large fluctuations are retained, so as to accurately capture the signal data representing the vital signs of the occupants. In this way, the obtained target image sequence has the characteristics of effectively removing background noise, accurately capturing image changes, and combining the multi-dimensional characteristics of time and space, accurately reflecting the relevant characteristics of the occupants' vital signs.

[0042] S140, processing the target image sequence through a convolutional neural network to obtain a detection result of whether there is an occupant in the cabin.

[0043] Convolutional Neural Networks (CNN) can use any appropriate CNN network model. Convolutional neural networks can capture various subtle image features, input the target image sequence into the convolutional neural network for feature extraction and analysis, and accurately obtain the detection result of whether there are passengers in the cabin.

[0044] The convolutional neural network is pre-trained. During training, the sample data is collected by using the same steps as the above steps S110, S120, and S130, collecting signal data in the cockpit through radar, converting the signal data into pixel data, subtracting two adjacent groups of pixel data to obtain a sample image sequence, then annotating the sample image sequence, and then training the convolutional neural network based on the sample image sequence.

[0045] When collecting sample data, radar can be used to collect signal data of different scenes and different occupant categories, so that the trained convolutional neural network can be applied to various working conditions and accurately detect whether there are occupants in the cabin under various working conditions. Among them, different scenes, such as: scenes of vehicles in motion, scenes of vehicles parked in different areas such as garages, roadsides, and communities, scenes in the cabins of different models, etc.; different occupant categories, such as: no occupants in the cabin, only children and / or pets in the cabin, adults in the cabin, etc.

[0046] When labeling the sample image sequence, the labeling is performed according to the category of the required detection result. For example, the required detection result category includes no passengers in the cabin and passengers in the cabin, and the category of passengers in the cabin further includes only children in the cabin and adults in the cabin; thus, when labeling the sample image sequence, the label "0" can be used to represent the category of no passengers in the cabin, the label "1" can be used to represent the category of only children in the cabin, and the label "2" can be used to represent the category of adults in the cabin.

[0047] Taking the three categories of no passengers in the cabin, only children in the cabin, and adults in the cabin as examples, in the converted images: when there are no passengers in the cabin, the image appears smooth as a whole; when there are only children in the cabin, there are a large number of white columnar points in some areas of the image; when there are adults in the cabin, the image has obvious cracks. The image features of these three categories can be extracted and learned by convolutional neural networks, thereby achieving accurate classification.

[0048] In summary, the in-cabin occupant detection method of the present invention can effectively solve the problem that it is difficult to distinguish small breathing and heartbeats and high-speed movements of adults due to complex background noise, improve the accuracy of radar-based in-cabin occupant detection, and make the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0049] For example, in the application scenario of occupant retention response, when the vehicle is turned off and locked, the in-cabin occupant detection method of the present invention can start working, scan the space inside the cabin through radar, and combine the conversion of signal data and the analysis of convolutional neural network to accurately detect whether children or other living organisms are left in the cabin. When it is detected that children or other living organisms are left in the cabin, the corresponding response mechanism is triggered to achieve safety control.

[0050] The cabin occupant detection method of the present invention has been tested in actual vehicle applications, and the detection accuracy is over 95%. In addition, since the image features of different classifications are very obvious, a lightweight convolutional neural network model can be used. This lightweight convolutional neural network model can be inferred on a mainstream onboard chip platform, occupying less system resources, and the reasoning process can be completed with the help of the vehicle's central processing unit (CPU) without the need to use the vehicle's graphics processing unit (GPU); and the lightweight convolutional neural network model can complete reasoning in about 10ms, obtain accurate detection results, and has high timeliness.

[0051] In some embodiments, each group of signal data is converted into pixel data, including: converting each signal data in a group of signal data into amplitude data to obtain a group of amplitude data, wherein each signal data corresponds to a sampling point and an acquisition period; using the sampling point and the acquisition period as pixel coordinates, converting each amplitude data in the group of amplitude data into pixel data to obtain pixel data corresponding to the group of signal data.

[0052] Figure 2 It shows how to convert a set of signal data into pixel data, combined with Figure 2 As shown, the signal data corresponding to 35 sampling points and 200 acquisition cycles are taken as a group, and the signal data in the form of analog signals collected by the radar are converted into amplitude data in the form of digital signals, and a group of amplitude data represented by matrix A is obtained. Among them, the conversion of the signal data in the form of analog signals into amplitude data in the form of digital signals can be realized based on methods such as fast Fourier transform (FFT), and fast Fourier transform is used to convert the signal from the time domain to the frequency domain; specifically in this embodiment, the signal data of each sampling point collected by the radar in each acquisition cycle can be converted into an amplitude data by using fast Fourier transform, and then the signal data of each sampling point collected by the radar in each acquisition cycle can be converted into corresponding amplitude data to obtain matrix A. The rows and columns of matrix A correspond to the sampling points and acquisition cycles respectively, and each value "x" in matrix A corresponds to the amplitude at a specific sampling point and a specific acquisition cycle.

[0053] Furthermore, the sampling point and the acquisition period are used as the row and column coordinates of the pixel points, respectively, and the amplitude data corresponding to the matrix A is converted into pixel data to obtain image B. Image B contains 35×200=7000 pixels, where the pixel value "p" of each pixel point is converted from the corresponding amplitude data in matrix A.

[0054] In a specific example, the corresponding pixel data can be obtained by normalizing the amplitude data, which facilitates the comparison and analysis of subsequent data.

[0055] Subsequently, each pixel value of the subsequent image is subtracted from each pixel value of the previous image. If the pixel value is a negative number, the absolute value can be taken to obtain a target image.

[0056] In some embodiments, the radar used to collect signal data in the cockpit is an ultra-wideband (UWB) radar, which is arranged in front of the cockpit and has a scanning range covering the cockpit space. For example, the ultra-wideband radar can be placed near the front and rear mirrors and slightly tilted downward relative to the horizontal direction, for example, at an angle of 30° to the horizontal direction, so that the scanning range of the ultra-wideband radar covers the entire cockpit space.

[0057] Ultra-wideband radar is a new type of radar technology that emits ultra-wideband pulse signals and collects echoes after the pulse signals are reflected by obstacles, that is, signal data. By analyzing the echo disturbance, it is possible to determine whether there is an object to be detected within the scanning range of the ultra-wideband radar. Specifically, when there is an object around the ultra-wideband radar, a pulse peak will appear at the corresponding position of the echo; when the object moves, the pulse peak at the corresponding position will change due to the Doppler effect. Therefore, when the object to be detected is a living organism, the signal data corresponding to the limb movement, breathing, heartbeat and other behaviors of the living organism will be clearly reflected in the signal data collected by the ultra-wideband radar, thereby achieving accurate collection of signal data representing the life characteristics of the occupants.

[0058] In some embodiments, before processing the target image sequence through a convolutional neural network, it also includes: converting the dimension of each target image into a preset dimension through image scaling to facilitate subsequent processing by the convolutional neural network; and converting each target image into a three-channel (RGB three-channel) target image through channel replication, which also facilitates subsequent processing by the convolutional neural network.

[0059] After training, the convolutional neural network can output a detection result of whether there is an occupant in the cabin based on the input target image sequence as described above. In some embodiments, the detection result of whether there is an occupant in the cabin may include: no occupant in the cabin and the presence of a first-category occupant and / or a second-category occupant in the cabin. The first-category occupant is, for example, an adult, and the second-category occupant is, for example, a child or a pet, but is not limited thereto.

[0060] Furthermore, in some embodiments, the convolutional neural network can be deployed in the vehicle's central processing unit, and the functional module that implements the above-mentioned signal conversion can also be deployed in the vehicle's central processing unit, or the convolutional neural network and the functional module that implements the above-mentioned signal conversion can be communicatively connected to the vehicle's central processing unit to realize in-cabin occupant detection through the vehicle's central processing unit, without using the vehicle's graphics processor, saving system resource consumption and improving the reasoning detection speed.

[0061] Of course, the cabin occupant detection method of the present invention may also be implemented by other suitable processing devices of the vehicle, and is not limited to the above examples.

[0062] The above are only a number of specific implementations of the method for detecting an occupant in the cabin of the present invention. Each implementation can be implemented independently or in combination, and the present invention is not limited thereto. In addition, the flowchart of the present invention is only illustrative, and the execution order between the steps is not limited thereto. The splitting, merging, order exchange, and other synchronous or asynchronous execution methods of the steps are all within the protection scope of the present invention.

[0063] The present invention also provides a cabin occupant detection device, which can be used to implement the cabin occupant detection method described in any of the above embodiments. The features and principles of the cabin occupant detection method described in any of the above embodiments can be applied to the following cabin occupant detection device embodiments.

[0064] Figure 3 The main modules of the cabin occupant detection device are shown in figure. Figure 3 As shown, the in-cabin occupant detection device 300 provided by the embodiment of the present invention includes: a signal data acquisition module 310, configured to obtain signal data collected by a radar in the cabin; a signal data conversion module 320, configured to take signal data corresponding to multiple sampling points and multiple acquisition cycles as a group, and convert each group of signal data into pixel data; a target image acquisition module 330, configured to subtract the pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; a target image processing module 340, configured to process the target image sequence through a convolutional neural network to obtain an occupant detection result.

[0065] The in-cabin occupant detection device 300 may be integrated into a central processing unit of the vehicle, communicatively connected to the central processing unit of the vehicle, or integrated / communicatively connected to other suitable processing devices of the vehicle.

[0066] Furthermore, the cabin occupant detection device 300 may also include modules for implementing other process steps of the above-mentioned cabin occupant detection method embodiments. The specific principles of each module can refer to the description of the above-mentioned cabin occupant detection method embodiments, and will not be repeated here.

[0067] The in-cabin occupant detection device of the present invention can effectively solve the problem that it is difficult to distinguish small breathing and heartbeats and high-speed movements of adults due to complex background noise, improve the accuracy of in-cabin occupant detection based on radar, and make the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0068] An embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein executable instructions are stored in the memory, and when the executable instructions are executed by the processor, the method for detecting an occupant in the cabin described in any of the above embodiments is implemented.

[0069] The electronic device of the present invention may be integrated into a central processing unit of a vehicle, communicatively connected to a central processing unit of a vehicle, or integrated / communicatively connected to other suitable processing devices of the vehicle.

[0070] The electronic device of the present invention can effectively solve the problem that it is difficult to distinguish small breathing and heartbeats and high-speed movements of adults due to complex background noise, improve the accuracy of radar-based occupant detection in the cabin, and make the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0071] Figure 4 Diagram the structure of the electronic device, combined with Figure 4 As shown, the electronic device 400 is in the form of a general computing device. The components of the electronic device 400 include but are not limited to: at least one processor 410, at least one memory 420, a bus 430 connecting different platform components (including the memory 420 and the processor 410), etc.

[0072] The memory 420 stores program codes, which can be executed by the processor 410, so that the processor 410 executes the steps of the cabin occupant detection method described in any of the above embodiments. For example, the processor 410 can execute the following steps: Figure 1 Steps shown.

[0073] The memory 420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory, and may further include a read-only memory unit (ROM). The memory 420 may also include a program / utility having one or more program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0074] Bus 430 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0075] The electronic device 400 can also communicate with one or more external devices, and the external device can be one or more of the devices such as keyboard, pointing device, bluetooth device, etc. These external devices enable the user to communicate interactively with the electronic device 400. The electronic device 400 can also communicate with one or more other computing devices, and the computer device shown includes a router and a modem. This communication can be carried out through an input / output (I / O) interface. In addition, the electronic device 400 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter.

[0076] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, which, when executed, implements the method for detecting an occupant in a cabin described in any of the above embodiments.

[0077] The storage medium of the present invention can be executed by the central processing unit of the vehicle or other suitable processing devices. When the storage medium of the present invention is executed, it can effectively solve the problem that it is difficult to distinguish between small breathing and heartbeat and high-speed movement of adults due to complex background noise, improve the accuracy of cabin occupant detection based on radar, and make the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0078] A storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0079] The storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. The readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0080] Program code for performing operations of the present invention may be written in any combination of one or more programming languages ​​and may be executed entirely on the user's computing device, partially on the user's device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as through the Internet using an Internet service provider.

[0081] The embodiment of the present invention further provides a computer program product, including a computer program, which implements the cabin occupant detection method described in any of the above embodiments when executed by a processor. The processor may be a central processing unit of a vehicle or other suitable processing device.

[0082] When the computer program product of the present invention is executed, it can effectively solve the problem that it is difficult to distinguish small breathing and heartbeats and high-speed movements of adults due to complex background noise, improve the accuracy of radar-based occupant detection in the cabin, and make the occupant detection results accurate and reliable, so as to serve application scenarios such as occupant retention response.

[0083] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for detecting passengers in a cabin, characterized in that: include: Obtain signal data collected by the radar in the cockpit; The signal data corresponding to a plurality of sampling points and a plurality of acquisition cycles are grouped together, and each group of signal data is converted into pixel data; Subtract pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; The target image sequence is processed by a convolutional neural network to obtain a detection result of whether there is an occupant in the cabin.

2. The method for detecting a passenger in a cabin according to claim 1, wherein: The converting each set of signal data into pixel data comprises: Convert each signal data in a set of signal data into amplitude data to obtain a set of amplitude data, wherein each signal data corresponds to a sampling point and an acquisition cycle; Taking the sampling point and the acquisition period as pixel coordinates, each amplitude data in the group of amplitude data is converted into pixel data to obtain pixel data corresponding to the group of signal data.

3. The method for detecting a passenger in a cabin as claimed in claim 2, characterized in that: The converting each amplitude data in the set of amplitude data into pixel data comprises: Normalize each amplitude data to obtain the corresponding pixel data.

4. The method for detecting a passenger in a cabin according to any one of claims 1 to 3, characterized in that: The radar is an ultra-wideband radar, which is arranged in front of the cockpit and has a scanning range covering the cockpit space.

5. The method for detecting a passenger in a cabin according to claim 1, wherein: The subtracting pixel data corresponding to two adjacent groups of signal data includes: According to the time sequence of the acquisition cycle, pixel data corresponding to the next group of signal data is subtracted from pixel data corresponding to the previous group of signal data to obtain a target image.

6. The method for detecting a passenger in a cabin as claimed in claim 1, characterized in that: Before processing the target image sequence by a convolutional neural network, the method further includes: By image scaling, the dimension of each target image is converted into a preset dimension; Each target image is converted into a three-channel target image by channel replication.

7. The method for detecting a passenger in a cabin as claimed in claim 1, characterized in that: The detection result of whether there is an occupant in the cabin includes whether there is no occupant in the cabin and whether there are first-category occupants and / or second-category occupants in the cabin.

8. The method for detecting a passenger in a cabin as claimed in claim 1, characterized in that: The convolutional neural network is deployed on the central processing unit of the vehicle.

9. A cabin occupant detection device, characterized in that: For implementing the method for detecting an occupant in a cabin as claimed in any one of claims 1 to 8, the occupant detection device in the cabin comprises: A signal data acquisition module configured to obtain signal data collected by a radar in the cockpit; A signal data conversion module is configured to group signal data corresponding to a plurality of sampling points and a plurality of acquisition cycles into a group, and convert each group of signal data into pixel data; A target image acquisition module is configured to subtract pixel data corresponding to two adjacent groups of signal data to obtain a target image sequence; The target image processing module is configured to process the target image sequence through a convolutional neural network to obtain an occupant detection result.

10. An electronic device, characterized in that: include: processor; A memory, wherein executable instructions are stored in the memory; Wherein, when the executable instructions are executed by the processor, the method for detecting occupants in the cabin as described in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the method for detecting an occupant in a cabin as described in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting an occupant in a cabin as claimed in any one of claims 1 to 8 is implemented.

Images