Human perception signal interaction method, device and medium based on new energy vehicles

Abstract

This application provides a method, device, and medium for personnel sensing signal interaction based on new energy vehicles. The method includes: in response to the activation operation of a personnel monitoring system, acquiring images of personnel inside the new energy vehicle; recognizing the personnel images using a sensing recognition algorithm to obtain raw sensing signals; generating a signal structure based on the attributes and enumeration values ​​corresponding to the raw sensing signals, and transmitting the signal structure to an intelligent sensing monitoring program; converting the enumeration values ​​in the signal structure into signal values ​​according to a predetermined signal parsing list to obtain processed sensing signals, and encapsulating the processed sensing signals into an interaction signal list; in response to the registration operation of the interaction module with the personnel monitoring system, sending the interaction signal list corresponding to the personnel monitoring system to the interaction module after successful registration. This application achieves real-time, accurate, and efficient transmission of intelligent sensing signals, improving the accuracy and efficiency of sensing signal processing.

Description

Technical Field

[0001] This application relates to the field of new energy vehicle technology, and in particular to a method, device and medium for human perception signal interaction based on new energy vehicles. Background Technology

[0002] In modern intelligent transportation systems, Driver Monitoring Systems (DMS) and Occupant Monitoring Systems (OMS) are crucial components. They provide continuous monitoring and evaluation of driver behavior, while also helping to adjust and optimize vehicle behavior. In the Android environment, these two systems typically need to transmit complex interactive signals accurately and in real time to other applications, such as map navigation and music playback.

[0003] However, in existing technologies, signal transmission in DMS & OMS is often complex and difficult to manage. At the same time, DMS & OMS may need to process multiple signals from different sources, such as signals from the driver smoking or the driver's gaze. When these signals are transmitted to multiple interactive applications, not only must the accuracy of the signals be guaranteed, but they must also be updated in real time to meet the needs of real-time feedback. Under such high signal processing load and real-time requirements, how to accurately and efficiently manage and transmit these signals has become a major technical challenge.

[0004] Furthermore, currently, each interaction module often needs to provide different AIDL (Application Interface Definition Language) methods for data interaction, which greatly increases the difficulty of software development and maintenance, and also reduces the project's adaptability. Therefore, how to design a simple and unified data interaction interface is also a current technical problem. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method, device and medium for human perception signal interaction based on new energy vehicles, in order to solve the problem that human perception signals cannot be accurately and efficiently transmitted to interactive applications, which leads to increased difficulty in software development and maintenance of interactive applications.

[0006] A first aspect of this application provides a method for human perception signal interaction based on a new energy vehicle, comprising: in response to the activation operation of a preset human monitoring system, acquiring images of people inside the new energy vehicle to obtain human images; recognizing the human images using a preset perception recognition algorithm to obtain raw perception signals, the raw perception signals containing human behavior and state information; generating a signal structure based on the attributes and enumeration values ​​corresponding to the predetermined raw perception signals, and transmitting the signal structure to an intelligent perception monitoring program; using the intelligent perception monitoring program to convert the enumeration values ​​corresponding to each attribute in the signal structure into signal values ​​according to a predetermined signal parsing list to obtain processed perception signals, and encapsulating the processed perception signals into an interaction signal list; in response to the registration operation of the human monitoring system by the interaction module, when the interaction module successfully registers, sending the interaction signal list corresponding to the human monitoring system to the interaction module.

[0007] A second aspect of this application provides a personnel perception signal interaction device based on a new energy vehicle, comprising: a data acquisition module configured to acquire images of personnel inside the new energy vehicle in response to a pre-set personnel monitoring system activation operation, thereby obtaining personnel images; a recognition module configured to recognize the personnel images using a pre-set perception recognition algorithm, thereby obtaining raw perception signals, the raw perception signals containing personnel behavior and status information; a generation module configured to generate a signal structure based on predetermined attributes and enumeration values ​​corresponding to the raw perception signals, and transmit the signal structure to an intelligent perception monitoring program; a conversion module configured to convert the enumeration values ​​corresponding to each attribute in the signal structure into signal values ​​using a pre-set signal parsing list, thereby obtaining processed perception signals, and encapsulating the processed perception signals into an interaction signal list; and a sending module configured to send the interaction signal list corresponding to the personnel monitoring system to the interaction module in response to a registration operation of the interaction module with the personnel monitoring system, after successful registration of the interaction module.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0011] In response to the activation of a preset personnel monitoring system, images of people inside the new energy vehicle are acquired, resulting in personnel images. A preset perception and recognition algorithm is used to identify these images, generating raw perception signals containing personnel behavior and status information. A signal structure is generated based on the predetermined attributes and enumeration values ​​corresponding to the raw perception signals, and this signal structure is passed to the intelligent perception monitoring program. The intelligent perception monitoring program, using a predetermined signal parsing list, converts the enumeration values ​​corresponding to each attribute in the signal structure into signal values, obtaining processed perception signals. These processed perception signals are then encapsulated into an interactive signal list. In response to the registration operation of the personnel monitoring system by the interactive module, once the interactive module successfully registers, the corresponding interactive signal list is sent to the interactive module. This application enables the real-time, accurate, and efficient transmission of intelligent perception signals of in-vehicle personnel to the interactive module, thereby reducing the difficulty of software development and maintenance for interactive applications. It also makes application development and project adaptation more convenient, contributing to improved accuracy and efficiency in perception signal processing. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the system architecture involved in the practical application of the personnel perception signal interaction method based on new energy vehicles provided in the embodiments of this application;

[0014] Figure 2 This is a flowchart illustrating the human perception signal interaction method based on new energy vehicles provided in an embodiment of this application;

[0015] Figure 3 This is a schematic diagram of the structure of the personnel perception signal interaction device based on new energy vehicles provided in the embodiments of this application;

[0016] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0018] Signal processing and management issues arise within the Android system of new energy vehicles, particularly in the Driver Monitoring System (DMS) and Occupant Monitoring System (OMS). When monitoring the behavior and status of the driver and occupants in real time, DMS and OMS may simultaneously detect multiple signals, such as the driver smoking or the driver's gaze being fixed on a point. These signals need to be transmitted to multiple applications within the system for further processing and interaction. However, accurately transmitting and managing these signals to prevent information loss or confusion remains a problem that needs to be solved.

[0019] To address the aforementioned technical issues, this application proposes an interaction approach using module registration, unifying the interaction implementation between DMS and OMS and multiple modules. This means that different modules no longer need to provide different AIDL (Android Interface Definition Language) methods; instead, they are handled uniformly through registration, avoiding the problem of each module needing to provide different methods in the traditional approach. Furthermore, this application proposes parsing DMS and OMS signals using a "signal list parsing" method. The basic idea of ​​this parsing method is to put all signals into a list and then obtain and process these signals by parsing this list. This interaction approach and signal processing method make program development and project adaptation more convenient and also help improve the accuracy and efficiency of signal processing.

[0020] Figure 1 This is a schematic diagram of the system architecture involved in the practical application of the personnel perception signal interaction method based on new energy vehicles provided in the embodiments of this application; as follows: Figure 1As shown, the system architecture includes an intelligent perception interaction module, a DMS&OMS Service, and an interaction module. The main task of the intelligent perception interaction module is to process and analyze image data to identify the driver's behavior and state. The DMS&OMS Service, also known as the intelligent perception monitoring program or intelligent perception monitoring service, is a service running within the Android system. It receives intelligent perception signals from the intelligent perception interaction module in real time and converts these signals into signals that can be recognized by other interaction modules according to certain rules. The interaction module includes, but is not limited to, a scene engine interaction module, a map interaction module, and other interaction modules. The interaction module parses the values ​​in the interaction signal list to obtain perception signals (behavior and state information of people inside the new energy vehicle) and uses these perception signals to implement interaction logic and processing.

[0021] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0022] Figure 2 This is a flowchart illustrating the human perception signal interaction method based on new energy vehicles provided in this application embodiment. Figure 2 The human perception signal interaction method based on new energy vehicles can be executed by the SoC of the new energy vehicle. For example... Figure 2 As shown, the human perception signal interaction method based on new energy vehicles may specifically include:

[0023] S201, in response to the activation operation of the preset personnel monitoring system, performs image acquisition on the personnel inside the new energy vehicle to obtain personnel images;

[0024] S202, using a preset perception recognition algorithm to recognize the personnel image to obtain the original perception signal, which contains personnel behavior and status information;

[0025] S203, Generate a signal structure based on the predetermined attributes and enumeration values ​​corresponding to the original sensing signal, and pass the signal structure to the intelligent sensing and monitoring program.

[0026] S204, using the intelligent sensing and monitoring program to convert the enumeration value corresponding to each attribute in the signal structure into a signal value according to the predetermined signal parsing list, to obtain the processed sensing signal, and encapsulate the processed sensing signal into an interactive signal list.

[0027] S205, in response to the registration operation of the interaction module for the personnel monitoring system, after the interaction module successfully registers, it sends the list of interaction signals corresponding to the personnel monitoring system to the interaction module.

[0028] In this application's technical solution, DMS and OMS are two personnel monitoring systems. DMS refers to the Driver Monitoring System, while OMS refers to the Occupant Monitoring System. DMS & OMS signals refer to the behavioral and state data of the driver or occupants collected from the intelligent perception and interaction module, such as smoking, yawning, blinking, etc. These signals are used to monitor the driver's behavior and state to improve driving safety. Specifically, DMS focuses on the driver's behavior and state, such as whether they are fatigued, distracted, yawning, or smoking, as well as information like the driver's gaze direction. OMS, on the other hand, focuses on the behavior and state of all occupants in the vehicle, such as their position, whether they are wearing seatbelts, their age, gender, etc. Therefore, DMS & OMS signals refer to various behavioral and state information of the driver and occupants, which are identified and encoded into a series of signals by the intelligent perception and interaction module for subsequent processing and decision-making.

[0029] In some embodiments, in response to the activation operation of a preset personnel monitoring system, image acquisition of personnel inside a new energy vehicle is performed, including: when the new energy vehicle is detected to be powered on, activating the personnel monitoring system to enable the camera to turn on, and using the camera to continuously acquire images of personnel located inside the vehicle to obtain personnel images.

[0030] Specifically, for occupants inside new energy vehicles, in response to the activation of a pre-set occupant monitoring system (including a DMS system and an OMS system), such as when the vehicle is powered on, the system can capture images of the occupants. For example, when the new energy vehicle is detected to be powered on, the occupant monitoring system will automatically start. Subsequently, the system will control the camera to turn on and continuously capture images of the occupants, thereby obtaining occupant images.

[0031] Furthermore, after acquiring image data, the intelligent sensing interaction module begins operation. This module integrates intelligent sensing capabilities. It can provide a perception and recognition algorithm for each frame of the acquired image. For example, through this module, the system can identify behaviors such as a driver smoking, yawning, and blinking. This intelligent sensing interaction module integrates intelligent sensing capabilities and internally integrates an algorithm library (.so files and .h files). The algorithm library enables perception and recognition of each acquired frame of image, and then generates corresponding perception signals. The following detailed description of the image recognition process and principles of the intelligent sensing interaction module, using specific embodiments, will include the following:

[0032] First, the intelligent sensing and interaction module needs to collect image data of people. This data can come from in-vehicle cameras, and each frame of the image data needs to be processed and analyzed.

[0033] After acquiring the image data of the person, the algorithm library within the intelligent perception and interaction module begins to work. In practical applications, the perception and recognition algorithm can incorporate various image processing and machine learning techniques, such as Convolutional Neural Networks (CNNs) for object detection and recognition, or Recurrent Neural Networks (RNNs) for processing time-series data. Through these algorithms, the module can identify the driver's behavior in the image, such as whether they are smoking, yawning, or blinking.

[0034] Finally, each behavior and state identified by the algorithm is converted into a raw perceptual signal. These raw perceptual signals are recorded in a data structure, such as a list or dictionary, so that they can be easily looked up and passed around.

[0035] It should be noted that the signals generated by the intelligent sensing interaction module can be used not only for local processing but also transmitted to the DMS&OMS service (i.e., the intelligent sensing monitoring program). By interacting with the intelligent sensing monitoring program, the behavior and status data of occupants can be shared with other interaction modules and applications, thereby enabling more functions, such as reminding the driver to rest or reminding rear-seat passengers to fasten their seat belts.

[0036] In some embodiments, generating a signal structure based on the attributes and enumeration values ​​corresponding to the predetermined original sensing signal, and transmitting the signal structure to the intelligent sensing monitoring program, includes: storing the attributes corresponding to the original sensing signal and the enumeration values ​​corresponding to the attributes in a predetermined data structure, generating a signal structure using the data structure, and transmitting the signal structure to the intelligent sensing monitoring program via FDBus communication, wherein the data structure is a list or a dictionary.

[0037] Specifically, after generating the original intelligent sensing signals (such as the driver's fatigue level, inattention level, and emotional state), this application generates a signal structure. First, the intelligent sensing interaction module stores the attributes corresponding to the original sensing signals and their corresponding enumerated values ​​in a predetermined data structure. In practical applications, this data structure can be a list or a dictionary. Then, this data structure is used to generate the signal structure, which contains attributes such as seat ID, alarm type, and hazard level. For example, the ASD_OCCUPANT_INFO structure is such an example; these signal structures fully represent the driver's current state information.

[0038] Furthermore, the intelligent perception interaction module transmits the generated signal structure to the intelligent perception monitoring program via FDBus communication. FDBus is a communication protocol for data transmission that provides cross-domain communication and can also transmit data via JSON, making it more flexible than traditional socket communication. In practical applications, the signal source from the intelligent perception interaction module is passed to the DMS&OMS Service (intelligent perception monitoring program). This process is real-time to ensure that the driver's status information can be acquired and processed by the monitoring system in real time.

[0039] On the DMS&OMS server side, the received signal structure will be processed and parsed. By parsing the signal structure, the DMS&OMS Service can obtain each type of state information, classify and label this information. That is, the DMS&OMS Service converts the received intelligent sensing signals into signals that can be recognized by other interactive modules according to certain rules, and sends the processed sensing signals to the registered interactive modules, so that the interactive modules can generate corresponding warnings or take corresponding measures, such as reminding the driver to take a rest or warning the driver to stop dangerous behavior.

[0040] The format and content of the signal structure are described in detail below with reference to specific embodiments. In one example, the signal structure generated by the intelligent sensing interaction module may include the following:

[0041]

[0042] As shown above, each attribute corresponds to an enumeration value. For example, the enumeration type of setId is:

[0043]

[0044] The enumeration type of alarmType is:

[0045]

[0046] Each attribute in ASD_OCCUPANT_INFO corresponds to an enumeration value, and each enumeration value corresponds to an object value (i.e., 0, 1, 2, 3, etc.). Other enumeration values ​​will not be listed here, but are similar to the enumeration values ​​listed in the above embodiments.

[0047] In some embodiments, the intelligent sensing and monitoring program converts the enumeration value corresponding to each attribute in the signal structure into a signal value according to a predetermined signal parsing list. This includes: converting the enumeration value corresponding to each attribute in the signal structure into a corresponding signal value according to the mapping relationship between the enumeration value and the signal value in the signal parsing list, and using the signal value as the processed sensing signal.

[0048] Specifically, the DMS&OMS Service (Intelligent Sensing Monitoring Program) running within the Android system receives raw sensing signals from the Intelligent Sensing SDK. These signals are passed in as structures, each containing numerous attributes, and each attribute corresponds to an enumeration value. For example, the ASD_OCCUPANT_INFO structure might contain attributes such as seat ID, alarm type, and danger level.

[0049] Furthermore, upon receiving the sensing signals, the DMS&OMS service processes and encapsulates these signals to convert them into signals that can be recognized by other interactive modules. Since multiple signals may exist simultaneously (such as smoking, staring, etc.), the received sensing signals need to be processed and encapsulated before the DMS&OMS service's interactive modules can recognize them. This embodiment uses a "signal list parsing" method to ensure that the DMS&OMS service's interactive modules can correctly recognize the processed signals.

[0050] Furthermore, when parsing the enumeration values ​​using the signal parsing list, the DMS&OMS service parses these enumeration values ​​into corresponding int values ​​(i.e., signal values) based on the mapping relationship between enumeration values ​​and signal values. This process must follow predefined rules; for example, the int value corresponding to ASD_SEAT_ID_CO_DRIVER is 0x1000. After parsing, the DMS&OMS service encapsulates these int values ​​in a list (i.e., the interaction signal list) and then passes this list to the interaction module. In this way, the interaction module only needs to parse the interaction signal list to know which signals have been transmitted.

[0051] The following detailed explanation, using specific embodiments, illustrates the process of converting enumerated values ​​into corresponding int values ​​using a signal parsing list, which may include the following:

[0052] In one example, the signal parsing list provided in this application embodiment is as follows:

[0053]

[0054]

[0055]

[0056] Based on the above signal parsing list, ASD_SEAT_ID_CO_DRIVER = 0x1000 / / Co-pilot, which means that the int value corresponding to ASD_SEAT_ID_CO_DRIVER is 0x1000, that is, the int value corresponding to the enumeration value "ASD_SEAT_ID_CO_DRIVER" is "0x1000".

[0057] After converting each enumeration value in the signal structure into an int value according to the above signal parsing list, all the int values ​​are encapsulated in a list (i.e., the interaction signal list), and then the interaction signal list is sent to the interaction module. In this way, the interaction module only needs to parse the values ​​in the list to obtain the real-time transmitted sensing signals.

[0058] For example, if an interaction signal list contains [0x1001, 0x1002, 0x1003], it means that three sensing signals are being transmitted: 0x1001 indicates smoking, 0x1002 indicates distraction warning, and 0x1003 indicates anger.

[0059] Through the signal parsing method described in this application embodiment, the DMS&OMS service converts the signals of the intelligent sensing SDK into signals that can be recognized by other interactive modules, thereby enabling accurate interaction and communication between the modules. Simultaneously, by encapsulating the signal values ​​in a list, the signal transmission process is greatly simplified, making the interaction between modules more efficient and stable.

[0060] In some embodiments, the personnel monitoring system includes a DMS system and / or an OMS system; in response to the registration operation of the interaction module on the personnel monitoring system, after the interaction module successfully registers, a list of interaction signals corresponding to the personnel monitoring system is sent to the interaction module, including: establishing an inter-process communication connection between the interaction module and the intelligent sensing monitoring program; in response to the registration operation of the interaction module on the DMS system and / or OMS system, calling the registration method of the DMS system and / or OMS system; when calling the registration method, passing the parameters of the callback interface into the registration method; after the interaction module successfully registers, the registration method returns the list of interaction signals corresponding to the DMS system and / or OMS system to the interaction module through the callback interface, so that the interaction module can obtain the processed sensing signals.

[0061] Specifically, before acquiring the processed sensing signal, the interactive module needs to register itself with the personnel monitoring system. Therefore, this application embodiment also provides methods for the interactive module to register with the DMS system and / or OMS system, namely the registerDMS method and the registerOMS method. The following uses the registration principle of the registerDMS method as an example to describe in detail the process of the interactive module registering and acquiring the processed sensing signal, which may include the following:

[0062] `registerDMS` is a method provided in the DMS & OMS SDK that allows the interactive module to register and receive sensing signals from the DMS (Driver Monitoring System). This registration process involves Android's AIDL (Android Interface Definition Language) mechanism, which allows cross-process communication. The following is a detailed explanation of this process:

[0063] First, an AIDL connection is established. The interaction module needs to establish a connection with the DMS & OMS Service through AIDL. This is because the DMS & OMS Service and the interaction module may run in different processes, and AIDL is an IPC (Inter-Process Communication) mechanism provided by the Android system that allows objects in two processes to communicate.

[0064] After the AIDL connection is successfully established, the interaction module can selectively register with DMS and / or OMS. If you choose to register with DMS, you need to call the registerDMS method; if you choose to register with OMS, you need to call the registerOMS method.

[0065] Furthermore, when calling the `registerDMS` method, a `DMSCallback` parameter (a parameter of the callback interface) needs to be passed in. This is a callback interface through which the DMS&OMS Service will send back the DMS's sensing signals to the interaction module. In other words, after successful registration, if DMS signals exist, the DMS&OMS Service will send these signals back to the interaction module via `DMSCallback`, thus allowing the interaction module to obtain the DMS's sensing information.

[0066] It should be noted that the implementation process and principle of the registerOMS method are similar to those of the registerDMS method described above, so they will not be repeated here.

[0067] In some embodiments, the method further includes: integrating the software development kits of the DMS system and the OMS system into each interactive module, wherein the software development kits of the DMS system and the OMS system are used to provide a registration method, and each interactive module corresponds to an application.

[0068] Specifically, this application also relates to a key component, namely the DMS&OMS software development kit (DMS&OMSSDK). The DMS&OMS SDK is provided by the DMS&OMS service. In order to obtain the perception data of the DMS&OMS service, each interaction module needs to integrate the DMS&OMS SDK into its own system.

[0069] In one example, each interaction module can correspond to an application. For instance, suppose there is an interaction module A that corresponds to application A. Then, application A needs to integrate the DMS&OMS SDK. In this way, application A can obtain the awareness data from the DMS&OMS service.

[0070] In practical applications, the DMS & OMS SDK provides two registration methods: registerDMS and registerOMS. These two methods provide the interaction module with channels to obtain DMS-aware data and OMS-aware data, respectively. Of course, the interaction module can also selectively register with the DMS system and / or the OMS system according to actual business needs.

[0071] For example, if the interaction module needs to obtain the perception results from the DMS system, its corresponding application needs to call the `registerDMS` method provided by the DMS & OMS SDK. After calling this method, the interaction module can obtain the DMS perception results (driver behavior and status information). Similarly, if the interaction module needs to obtain the OMS perception results, its corresponding application needs to call the `registerOMS` method provided by the DMS & OMS SDK. After calling this method, the interaction module can obtain the OMS perception results (occupant behavior and status information).

[0072] In some embodiments, after sending the list of interactive signals corresponding to the personnel monitoring system to the interaction module, the method further includes: the interaction module parses the list of interactive signals to obtain the sensing signals in the list of interactive signals, makes real-time judgments on the behavior and status of the personnel in the new energy vehicle based on the parsed sensing signals, and performs business processing and response operations based on the judgment results.

[0073] Specifically, after receiving the list of interaction signals corresponding to the personnel monitoring system, the interaction module first parses it to obtain the various sensing signals in the list. The sensing signals contain information about the behavior and state of the people inside the new energy vehicle, such as whether the driver is smoking, yawning, or blinking.

[0074] Furthermore, after parsing the sensing signals, the interaction module will make real-time judgments on the behavior and state of the occupants inside the new energy vehicle based on these signals. This judgment process includes identifying and parsing the information in the sensing signals, and then interpreting this information appropriately to understand the real-time behavior and state of the occupants inside the vehicle. Based on the judgment results of the sensing signals, the interaction module will execute corresponding business processing and response operations, which may include, but are not limited to, the following:

[0075] If the system detects signs of driver fatigue (such as frequent blinking or yawning), the interactive module will issue a reminder suggesting that the driver stop and rest. If it detects that a passenger in the back seat is not wearing a seatbelt, the interactive module will issue a reminder asking the passenger to fasten their seatbelt.

[0076] According to the technical solution provided in the embodiments of this application, this application converts the signals of the intelligent perception SDK into signals that can be recognized by other interactive modules through DMS & OMS services, thereby realizing effective communication between various modules. Simultaneously, by encapsulating the signals in a list, the signal processing and transmission process is greatly simplified, making the interaction between modules more efficient and stable. This application provides real-time and accurate driver status monitoring by accurately generating and transmitting perception signals, and accurately processing and parsing signals, greatly improving the safety of new energy vehicles. This application achieves efficient acquisition and transmission of perception data from DMS & OMS services, effectively improving the working efficiency of interactive modules and providing more accurate monitoring of driver behavior. This application uses a signal list parsing method to parse the perception signals of OMS and DMS, which simplifies the complex signal parsing task and facilitates program development and project adaptation. The processing method of perception signals in this application can more effectively manage and transmit DMS and OMS signals, reducing complexity and facilitating program development and project adaptation.

[0077] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0078] Figure 3 This is a schematic diagram of the structure of a personnel perception signal interaction device based on new energy vehicles provided in an embodiment of this application. Figure 3As shown, the human perception signal interaction device based on new energy vehicles includes:

[0079] The acquisition module 301 is configured to acquire images of people inside the new energy vehicle in response to the start operation of the preset personnel monitoring system.

[0080] The recognition module 302 is configured to recognize the person image using a preset perception recognition algorithm to obtain the original perception signal, which contains the person's behavior and status information.

[0081] The generation module 303 is configured to generate a signal structure based on the predetermined attributes and enumeration values ​​corresponding to the original sensing signal, and then pass the signal structure to the intelligent sensing and monitoring program.

[0082] The conversion module 304 is configured to use the intelligent sensing monitoring program to convert the enumeration value corresponding to each attribute in the signal structure into a signal value according to a predetermined signal parsing list, thereby obtaining the processed sensing signal, and then encapsulating the processed sensing signal into an interactive signal list.

[0083] The sending module 305 is configured to respond to the registration operation of the interaction module on the personnel monitoring system. After the interaction module successfully registers, it sends the list of interaction signals corresponding to the personnel monitoring system to the interaction module.

[0084] In some embodiments, Figure 3 When the acquisition module 301 detects that the new energy vehicle is powered on, it starts the personnel monitoring system so that the personnel monitoring system controls the camera to turn on and uses the camera to continuously acquire images of the people inside the vehicle.

[0085] In some embodiments, Figure 3 The generation module 303 stores the attributes corresponding to the original sensing signal and the enumeration values ​​corresponding to the attributes in a predetermined data structure, generates a signal structure using the data structure, and transmits the signal structure to the intelligent sensing and monitoring program through the FDBus communication method. The data structure adopts a list or dictionary.

[0086] In some embodiments, Figure 3 The conversion module 304 converts the enumeration value corresponding to each attribute in the signal structure into the corresponding signal value according to the mapping relationship between the enumeration value and the signal value in the signal parsing list, and uses the signal value as the processed sensing signal.

[0087] In some embodiments, the personnel monitoring system includes a DMS system and / or an OMS system. Figure 3The sending module 305 establishes an inter-process communication connection between the interaction module and the intelligent sensing monitoring program. In response to the interaction module's registration operation of the DMS system and / or OMS system, it calls the registration method of the DMS system and / or OMS system. When calling the registration method, the parameters of the callback interface are passed into the registration method. After the interaction module successfully registers, the registration method returns the list of interaction signals corresponding to the DMS system and / or OMS system to the interaction module through the callback interface, so that the interaction module can obtain the processed sensing signals.

[0088] In some embodiments, Figure 3 The integration module 306 integrates the software development kits of the DMS system and the OMS system into each interactive module. The software development kits of the DMS system and the OMS system are used to provide registration methods, and each interactive module corresponds to an application.

[0089] In some embodiments, Figure 3 After the parsing module 307 sends the list of interactive signals corresponding to the personnel monitoring system to the interaction module, the interaction module parses the list of interactive signals to obtain the perception signals in the list. Based on the parsed perception signals, the interaction module makes real-time judgments on the behavior and status of the personnel in the new energy vehicle and performs business processing and response operations based on the judgment results.

[0090] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] Figure 4 This is a schematic diagram of the structure of the electronic device 4 provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 of this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0092] For example, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in electronic device 4.

[0093] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0094] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0095] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 402 can include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0097] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0103] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for human perception signal interaction based on new energy vehicles, characterized in that, include: In response to the activation of the preset personnel monitoring system, images of the people inside the new energy vehicle are captured to obtain personnel images; The personnel image is identified using a preset perception and recognition algorithm to obtain an original perception signal, which contains personnel behavior and state information; The attributes corresponding to the original sensing signal and the enumerated values ​​corresponding to the attributes are stored in a predetermined data structure. The data structure is used to generate a signal structure, and the signal structure is transmitted to the intelligent sensing and monitoring program through the FDBus communication method. The data structure is a list or a dictionary. The intelligent sensing and monitoring program uses a predetermined signal parsing list to convert the enumeration value corresponding to each attribute in the signal structure into a signal value to obtain the processed sensing signal, and then encapsulates the processed sensing signal into an interactive signal list. In response to the registration operation of the personnel monitoring system by the interaction module, after the interaction module successfully registers, it sends the list of interaction signals corresponding to the personnel monitoring system to the interaction module. The interaction module parses the list of interaction signals to obtain the perception signals in the list. Based on the parsed perception signals, it makes real-time judgments on the behavior and status of the people in the new energy vehicle, and performs business processing and response operations based on the judgment results.

2. The method according to claim 1, characterized in that, The method of responding to the activation operation of a preset personnel monitoring system and acquiring images of people inside the new energy vehicle includes: When the power is detected to be on in a new energy vehicle, the personnel monitoring system is activated so that the system controls the camera to turn on and uses the camera to continuously capture images of the people inside the vehicle.

3. The method according to claim 1, characterized in that, The step of using the intelligent sensing and monitoring program to convert the enumerated value corresponding to each attribute in the signal structure into a signal value according to a predetermined signal parsing list includes: Based on the mapping relationship between the enumeration values ​​and the signal values ​​in the signal parsing list, the enumeration value corresponding to each attribute in the signal structure is converted into the corresponding signal value, and the signal value is used as the processed sensing signal.

4. The method according to claim 1, characterized in that, The personnel monitoring system includes a DMS system and / or an OMS system; in response to the registration operation of the interaction module on the personnel monitoring system, after the interaction module successfully registers, the list of interaction signals corresponding to the personnel monitoring system is sent to the interaction module, including: Establish an inter-process communication connection between the interaction module and the intelligent sensing and monitoring program, and in response to the registration operation of the interaction module on the DMS system and / or OMS system, call the registration method of the DMS system and / or OMS system; When the registration method is called, the parameters of the callback interface are passed into the registration method. After the interaction module is successfully registered, the registration method returns the list of interaction signals corresponding to the DMS system and / or OMS system to the interaction module through the callback interface, so that the interaction module can obtain the processed perception signal.

5. The method according to claim 4, characterized in that, The method further includes: The software development kits of the DMS system and OMS system are integrated into each of the interactive modules, wherein the software development kits of the DMS system and OMS system are used to provide the registration method, and each interactive module corresponds to an application.

6. A human perception signal interaction device based on new energy vehicles, characterized in that, include: The data acquisition module is configured to acquire images of people inside the new energy vehicle in response to the activation operation of the preset personnel monitoring system. The recognition module is configured to recognize the person image using a preset perception recognition algorithm to obtain an original perception signal, the original perception signal containing person behavior and state information; The generation module is configured to store the attributes corresponding to the original sensing signal and the enumeration values ​​corresponding to the attributes in a predetermined data structure, generate a signal structure using the data structure, and transmit the signal structure to the intelligent sensing and monitoring program via FDBus communication. The data structure is a list or a dictionary. The conversion module is configured to use the intelligent sensing and monitoring program to convert the enumeration value corresponding to each attribute in the signal structure into a signal value according to a predetermined signal parsing list, thereby obtaining the processed sensing signal, and to encapsulate the processed sensing signal into an interactive signal list. The sending module is configured to respond to the registration operation of the personnel monitoring system by the interaction module. After the interaction module successfully registers, it sends the list of interaction signals corresponding to the personnel monitoring system to the interaction module. The interaction module parses the list of interaction signals to obtain the perception signals in the list. Based on the parsed perception signals, it makes real-time judgments on the behavior and status of the people in the new energy vehicle and performs business processing and response operations based on the judgment results.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images