Vehicle information processing method and apparatus, electronic device, and vehicle

Abstract

Provided are a vehicle information processing method and apparatus, an electronic device, and a vehicle. The vehicle information processing method comprises: acquiring device data and an in-vehicle image of a vehicle; on the basis of the device data and the in-vehicle image, obtaining in-vehicle space information comprising device position information of an in-vehicle device and passenger characteristic information; and on the basis control requirements of a control unit, sending to the control unit adjustment information comprising one or more of the in-vehicle space information. In this way, upon receiving the adjustment information, the control unit can adapt to a change in an in-vehicle environment on the basis of the adjustment information, and control and adjust a control object on the basis of the adjustment information, thereby improving the control effect on the control object, and improving vehicle use experience of a user.

Description

Vehicle information processing method, device, electronic device and vehicle

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 26, 2024, with application number 202410122840.9 and application name “Vehicle Information Processing Method, Device, Electronic Device and Vehicle”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of vehicle intelligent technology, and in particular to a vehicle information processing method, device, electronic equipment and vehicle. Background Art

[0003] With the development of vehicle technology, users have higher and higher expectations for driving comfort, convenience, and intelligent operation. To enhance the user experience, vehicles are equipped with more and more functions, such as music playback and intelligent device adjustment for user entertainment.

[0004] However, currently, functions such as music playback and intelligent device adjustment are difficult to adapt to changes in the interior space environment of the car, making it difficult to provide users with an adaptive personalized interactive experience, affecting the effect and resulting in a poor user experience. Summary of the Invention

[0005] The embodiments of the present application provide a vehicle information processing method, device, electronic device and vehicle. By sensing changes in passengers and equipment in the vehicle, the control unit can adapt to changes in the vehicle environment, improve the control effect of the control object, and enhance the user's vehicle experience.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a vehicle information processing method is provided, the method comprising: acquiring device data and an in-vehicle image of the vehicle; obtaining in-vehicle space information based on the device data and the in-vehicle image, the in-vehicle space information including device location information of in-vehicle devices and passenger characteristic information; sending first adjustment information to the first control unit based on a control requirement of the first control unit, the first adjustment information including device location information and / or passenger characteristic information of the in-vehicle devices; wherein the first adjustment information is used by the first control unit to control and adjust a first control object.

[0008] The solution provided in the first aspect acquires vehicle device data and interior images, and based on the device data and interior images, obtains interior space information, including device location information and passenger characteristics, and sends adjustment information, including one or more of the interior space information, to the control unit based on the control requirements of the control unit. Upon receiving the adjustment information, the control unit can adapt to changes in the interior environment based on the adjustment information and adjust the controlled objects accordingly, thereby improving the control effect of the controlled objects and enhancing the user's vehicle experience.

[0009] As a possible implementation, the method further includes: sending second adjustment information to the second control unit based on the control requirements of the second control unit, the second adjustment information including device location information of the in-vehicle device and / or passenger characteristic information; wherein the second adjustment information is the same as the first adjustment information, or the second adjustment information is different from the first adjustment information. In this way, in-vehicle space information is obtained through device data and in-vehicle images, and corresponding adjustment information is sent to the control unit based on the in-vehicle space information and the control requirements of the control unit. This can avoid repeated collection of required adjustment information by different control units when controlling and adjusting the control objects, thus saving computing resources and improving efficiency.

[0010] As a possible implementation, in-vehicle spatial information is obtained based on device data and in-vehicle images. This includes: obtaining in-vehicle device position change data based on the current and initial adjustment data of the in-vehicle device in the device data, as well as the in-vehicle device's motion model, where the motion model characterizes the motion patterns of the in-vehicle device under changes in the adjustment data; obtaining device position information based on the in-vehicle device position change data and the in-vehicle device's initial position information; and performing object detection on the in-vehicle image to obtain passenger feature information. In this way, by sensing the spatial positions of in-vehicle passengers and in-vehicle devices, the control unit can promptly adapt to changes in the in-vehicle environment, achieving adaptive adjustment of the in-vehicle device and improving the adjustment effect.

[0011] As a possible implementation, object detection is performed on the in-vehicle image to obtain passenger characteristic information, including: performing object detection on the in-vehicle image to obtain passenger body position information and passenger distribution information; obtaining passenger height information based on the passenger body position information and device location information; and obtaining passenger size information based on the passenger height information; the passenger characteristic information includes at least one of the following: passenger body position information, passenger distribution information, and passenger size information. In this way, by spatially locating the passenger and in-vehicle devices, passenger size information is estimated. Further, when control and adjustment are performed based on the passenger size information, passenger distribution information, passenger body position information, and device location information, personalized intelligent adjustment can be achieved for each user.

[0012] As a possible implementation, when the first control unit is an audio control unit, the first adjustment information includes at least one of the following: device position information of the playback device in the vehicle, device position information of the seat device, passenger distribution information, and passenger ear position information within the passenger body position information. By using the playback device position information, seat device position information, passenger ear position information, and passenger distribution information in the audio control unit, the audio control unit can adapt to changes in the vehicle's interior spatial environment based on the playback device position information, seat device position information, passenger ear position information, and passenger distribution information, enabling real-time control and adjustment of audio playback, achieving more intelligent audio adjustment, improving the audio adjustment effect, and providing passengers with a better audio experience.

[0013] In one possible implementation, when the first control unit is a seat adjustment control unit, the first adjustment information includes at least one of the following: seat equipment position information, passenger size information, and passenger eye position information within the passenger body part position information. Thus, by using the seat equipment position information, passenger eye position information, and passenger size information in the seat adjustment control unit, the seat adjustment control unit can implement personalized intelligent seat adjustment for each user based on the seat equipment position information, passenger eye position information, and passenger size information.

[0014] As one possible implementation, when the first control unit is a steering wheel adjustment control unit, the first adjustment information includes at least one of passenger eye position information and passenger size information. Thus, by incorporating the passenger eye position information and passenger size information into the steering wheel adjustment control unit, the steering wheel adjustment control unit can implement personalized intelligent steering wheel adjustment for each user based on the passenger eye position information and passenger size information.

[0015] As one possible implementation, when the first control unit is a rearview mirror adjustment control unit or a head-up display adjustment control unit, the first adjustment information includes at least one item of passenger eye position information within the passenger body position information. In this way, the sensed passenger eye position information is used by the rearview mirror adjustment control unit and the head-up display adjustment control unit, enabling personalized intelligent adjustment of the rearview mirror and head-up display for each user.

[0016] As a possible implementation, the method further includes: receiving a subscription request from the first control unit via a subscription interface; and obtaining the control requirements of the first control unit based on the subscription request. In this way, the control requirements of the control unit can be obtained through subscription, and the required adjustment information can be sent to the control unit in a timely manner, enabling the control unit to promptly adapt to changes in the vehicle interior environment and improve the control effect on the controlled object.

[0017] As one possible implementation, sending the first adjustment information to the first control unit includes: upon receiving a call request from the first control unit, sending the first adjustment information to the first control unit based on the control requirements in the adjustment request. In this way, the required adjustment information can be sent to the control unit in a timely manner, enabling the control unit to promptly adapt to changes in the vehicle interior environment and improve control effectiveness of the controlled object.

[0018] As a possible implementation, the method further includes: verifying the accuracy of the device location information and the passenger location information in the passenger profile information based on the relative position of the passenger's body part and the device. This can improve the accuracy of the information and enhance the control unit's control over the in-vehicle device.

[0019] In a second aspect, a vehicle information processing device is provided, which includes: an acquisition module for acquiring vehicle equipment data and in-vehicle images; an information perception module for obtaining in-vehicle space information based on the equipment data and the in-vehicle images, wherein the in-vehicle space information includes device location information of in-vehicle equipment and passenger characteristic information; an information sending module for sending first adjustment information to the first control unit according to the control requirements of the first control unit, wherein the first adjustment information includes device location information of in-vehicle equipment and / or passenger characteristic information; wherein the first adjustment information is used by the first control unit to control and adjust the first control object.

[0020] The solution provided in the second aspect acquires vehicle device data and interior images, and based on the device data and interior images, obtains interior space information, including device location information and passenger characteristics, and sends adjustment information, including one or more of the interior space information, to the control unit based on the control requirements of the control unit. Upon receiving the adjustment information, the control unit can adapt to changes in the interior environment based on the adjustment information and adjust the controlled objects accordingly, thereby improving the control effect of the controlled objects and enhancing the user's vehicle experience.

[0021] As a possible implementation, the information transmission module is further configured to: transmit second adjustment information to the second control unit based on the control requirements of the second control unit, the second adjustment information including device location information of the in-vehicle device and / or passenger characteristic information; wherein the second adjustment information may be the same as the first adjustment information or different from the first adjustment information. In this manner, by using device data and in-vehicle images to obtain in-vehicle spatial information, and based on the in-vehicle spatial information and the control requirements of the control unit, corresponding adjustment information is transmitted to the control unit. This avoids repeated acquisition of required adjustment information by different control units when controlling and adjusting the controlled objects, thus saving computing resources and improving efficiency.

[0022] As a possible implementation, the information perception module is configured to: derive position change data for in-vehicle devices based on their current and initial adjustment data, as well as their motion models, as contained in the device data. The motion models characterize the motion patterns of in-vehicle devices under varying adjustment data; derive device position information for the in-vehicle devices based on their position change data and initial position information; and perform object detection on in-vehicle images to obtain passenger feature information. By sensing the spatial positions of passengers and in-vehicle devices, the control unit can promptly adapt to changes in the in-vehicle environment, achieving adaptive adjustment of the in-vehicle devices and improving their effectiveness.

[0023] As a possible implementation, the information perception module is specifically configured to: perform target detection on in-vehicle images to obtain passenger body position information and passenger distribution information; obtain passenger height information based on the passenger body position information and device location information; and obtain passenger size information based on the passenger height information; wherein the passenger characteristic information includes at least one of the following: passenger body position information, passenger distribution information, and passenger size information. In this way, by spatially locating passengers and in-vehicle devices, passenger size information can be estimated. Further, when control and adjustment are made based on passenger size information, passenger distribution information, passenger body position information, and device location information, personalized intelligent adjustment can be achieved for each user.

[0024] As a possible implementation, when the first control unit is an audio control unit, the first adjustment information includes at least one of the following: device position information of the playback device in the vehicle, device position information of the seat device, passenger distribution information, and passenger ear position information within the passenger body position information. By using the playback device position information, seat device position information, passenger ear position information, and passenger distribution information in the audio control unit, the audio control unit can adapt to changes in the vehicle's interior spatial environment based on the playback device position information, seat device position information, passenger ear position information, and passenger distribution information, enabling real-time control and adjustment of audio playback, achieving more intelligent audio adjustment, improving the audio adjustment effect, and providing passengers with a better audio experience.

[0025] In one possible implementation, when the first control unit is a seat adjustment control unit, the first adjustment information includes at least one of the following: seat equipment position information, passenger size information, and passenger eye position information within the passenger body part position information. Thus, by using the seat equipment position information, passenger eye position information, and passenger size information in the seat adjustment control unit, the seat adjustment control unit can implement personalized intelligent seat adjustment for each user based on the seat equipment position information, passenger eye position information, and passenger size information.

[0026] As one possible implementation, when the first control unit is a steering wheel adjustment control unit, the first adjustment information includes at least one of passenger eye position information and passenger size information. Thus, by incorporating the passenger eye position information and passenger size information into the steering wheel adjustment control unit, the steering wheel adjustment control unit can implement personalized intelligent steering wheel adjustment for each user based on the passenger eye position information and passenger size information.

[0027] As one possible implementation, when the first control unit is a rearview mirror adjustment control unit or a head-up display adjustment control unit, the first adjustment information includes at least one item of passenger eye position information within the passenger body position information. In this way, the sensed passenger eye position information is used by the rearview mirror adjustment control unit and the head-up display adjustment control unit, enabling personalized intelligent adjustment of the rearview mirror and head-up display for each user.

[0028] As a possible implementation, the information sending module is further configured to receive a subscription request from the first control unit via a subscription interface and obtain the control requirements of the first control unit based on the subscription request. In this way, the control unit's control requirements can be obtained through subscription, and the required adjustment information can be sent to the control unit in a timely manner, enabling the control unit to promptly adapt to changes in the vehicle's in-vehicle environment and improve the control effect on the controlled object.

[0029] As a possible implementation, the information sending module is further configured to, upon receiving a call request from the first control unit, send the first adjustment information to the first control unit based on the control requirements in the adjustment request. This allows the required adjustment information to be sent to the control unit in a timely manner, enabling the control unit to promptly adapt to changes in the vehicle interior environment and improve the control effect on the controlled object.

[0030] As a possible implementation, the information sensing module is also used to verify the accuracy of the device location information and the passenger location information in the passenger profile information based on the relative position of the passenger's body part and the device. This can improve the accuracy of the information and enhance the control unit's control over the in-vehicle device.

[0031] According to a third aspect, an electronic device is provided, comprising: a memory for storing computer program instructions; and a processor for executing the computer program instructions to support the electronic device in implementing the method described in any possible embodiment of the first aspect.

[0032] In a fourth aspect, a vehicle is provided, comprising: a memory for storing computer program instructions; and a processor for executing the computer program instructions to support the vehicle in implementing the method as described in any possible embodiment of the first aspect.

[0033] In a fifth aspect, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processing circuit, the method described in any possible implementation manner in the first aspect is implemented.

[0034] In a sixth aspect, a computer program product comprising instructions is provided, which, when the computer program product is run on a computer, enables the computer to execute the method as described in any possible implementation of the first aspect.

[0035] In the seventh aspect, a chip system is provided, which includes a processing circuit and a storage medium, wherein the storage medium stores computer program instructions; when the computer program instructions are executed by the processing circuit, the method described in any possible implementation method of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1 is a schematic structural diagram of a vehicle provided in an embodiment of the present application;

[0037] FIG2 is a flow chart of a vehicle information processing method according to an embodiment of the present application;

[0038] FIG3 is a schematic diagram of a process for determining device location information according to an embodiment of the present application;

[0039] FIG4 is a schematic diagram of a process for determining passenger position information according to an embodiment of the present application;

[0040] FIG5 is a schematic diagram of a process for determining passenger size information according to an embodiment of the present application;

[0041] FIG6 is a schematic diagram of the interaction between the vehicle information processing device and the control unit provided in an embodiment of the present application;

[0042] FIG7 is a schematic diagram of the interaction between the vehicle information processing device and the audio control unit provided in an embodiment of the present application;

[0043] FIG8 is a schematic diagram of the interaction between the vehicle information processing device and the seat adjustment control unit provided in an embodiment of the present application;

[0044] FIG9 is a schematic diagram of the interaction between the vehicle information processing device and the steering wheel adjustment control unit provided in an embodiment of the present application;

[0045] FIG10 is a schematic diagram of the interaction between the vehicle information processing device and the rearview mirror adjustment control unit provided in an embodiment of the present application;

[0046] FIG11 is a schematic diagram of the interaction between the vehicle information processing device and the HUD adjustment control unit provided in an embodiment of the present application;

[0047] FIG12 is a second flow chart of the vehicle information processing method provided in an embodiment of the present application;

[0048] FIG13 is a schematic diagram showing a principle of coordinate system conversion provided by an embodiment of the present application;

[0049] FIG14 is a schematic diagram showing a principle of a vehicle information processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0051] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application complies with the provisions of relevant laws and regulations and does not violate public order and good morals. For example, in the technical solution of this application, the processing of user personal information is carried out with the user's authorization, which is explained here and will not be repeated below.

[0052] The terms "including," "having," and any variations thereof mentioned in the description of the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0053] In the following, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features.

[0054] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0055] In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more. "And / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0056] As described in the background, with the advancement of vehicle technology, users have increasingly higher expectations for driving comfort, convenience, and intelligent operation. To enhance the user experience, vehicles are being equipped with an increasing number of features, such as music playback and intelligent device adjustments for user entertainment.

[0057] However, currently, functions such as music playback and intelligent device adjustment are difficult to adapt to changes in the interior space environment of the car, making it difficult to provide users with an adaptive personalized interactive experience, affecting the effect and resulting in a poor user experience.

[0058] For example, taking the in-car audio used for music playback as an example, in-car audio usually needs to use technologies such as sound reproduction or noise reduction to provide users in the car with a better audio experience. However, when the acoustic environment in the car changes, such as the movement of the human ear, the position of the speaker, the position of the seat and / or the distribution of people, the in-car audio is difficult to adapt to the changes in the environment, resulting in poor audio effects, which affects the audio experience of users in the car.

[0059] For another example, for some hardware in the car, such as rearview mirrors, head-up display (HUD) devices, seats, steering wheels, etc., when the position of the hardware itself changes, the users in the car change, and / or the distribution of people changes, these hardware adjustments cannot be personalized and intelligently adjusted for each person in the car, and can only be fixed-position adjustments, resulting in poor usage effects and affecting the user's car experience.

[0060] Furthermore, currently, some functions are adjusted based on the same information. However, the corresponding information for each function is collected separately by the control unit / adjustment unit, resulting in repeated collection of the same information by different control units / adjustment units, wasting resources. For example, adjusting the rearview mirror and the head-up display requires passenger eye information. Consequently, both the rearview mirror adjustment unit and the head-up display adjustment unit need to collect passenger eye information during these adjustments, resulting in repeated collection of passenger eye information and wasting resources.

[0061] Based on the above research, an embodiment of the present application provides a vehicle information processing method, which obtains the device data and in-vehicle images of the vehicle, obtains the in-vehicle space information including the device location information of the in-vehicle devices and the passenger characteristic information based on the device data and the in-vehicle images, and sends one or more adjustment information including the in-vehicle space information to the control unit according to the control requirements of the control unit. In this way, after receiving the adjustment information, the control unit can adapt to the changes in the in-vehicle environment based on the adjustment information, and control and adjust the control object according to the adjustment information to improve the control effect of the control object. At the same time, the embodiment of the present application realizes the control and adjustment of the control object based on the passenger characteristic information and the device location information, so as to achieve a personalized intelligent adjustment experience for each user and improve the user's car experience.

[0062] In addition, the embodiment of the present application obtains the interior space information of the vehicle through device data and interior images, and sends corresponding adjustment information to the control unit based on the interior space information. This can avoid different control units from controlling and adjusting the control objects and repeatedly collecting the required adjustment information, thereby saving resources and improving efficiency.

[0063] The vehicle information processing method provided in the embodiments of the present application can be applied to various in-cabin interaction scenarios of a vehicle, including but not limited to audio control scenarios, seat intelligent adjustment, steering wheel intelligent adjustment, rearview mirror intelligent adjustment, and HUD intelligent adjustment scenarios.

[0064] As an example, the vehicle described in the embodiments of the present application can be a means of transportation (such as a car, bus, subway, high-speed rail, motorcycle, flying car, train, etc.), an industrial vehicle (such as a forklift, trailer, tractor, etc.), an engineering vehicle (such as an excavator, bulldozer, crane, etc.), agricultural equipment (such as a lawn mower, harvester, etc.), amusement equipment, a toy vehicle, a boat, an air cushion vehicle, a submarine, an airplane, a helicopter, etc. The embodiments of the present application do not limit the specific type, form, and function of the vehicle.

[0065] For ease of understanding, the following describes the structure of the vehicle. Please refer to Figure 1, which is a schematic diagram of the structure of a vehicle provided in an embodiment of the present application. As shown in Figure 1, the vehicle 100 provided in an embodiment of the present application includes various subsystems, such as a computer system 110, a sensor system 120, a control system 130, one or more peripheral devices 140, and a user interface 150.

[0066] It is understood that the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. For example, the vehicle 100 may also include a propulsion system, a power supply, etc., the specific details of which are not described in detail in the embodiments of this application. It is understood that each subsystem and component of the vehicle 100 may be interconnected via wired or wireless connections.

[0067] The sensor system 120 may include a plurality of sensors for sensing and measuring information about the environment surrounding the vehicle 100. For example, the sensor system 120 may include a positioning system (the positioning system may be a global positioning system (GPS), a BeiDou system, or other positioning systems), an inertial measurement unit (IMU), a radar, a laser rangefinder, and a camera.

[0068] Among other things, the positioning system can be used to estimate the geographic location of vehicle 100. The IMU is used to sense changes in the position and orientation of vehicle 100 based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. A laser rangefinder can utilize lasers to sense objects in the environment surrounding vehicle 100. In some examples, the laser rangefinder may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0069] In an embodiment of the present application, the sensor system 120 may include multiple cameras, which may be located within the vehicle and configured to capture images within the vehicle. For example, one or more cameras may be positioned near the rearview mirror at the front of the vehicle to capture images of passengers in all seats within the vehicle. Alternatively, one or more cameras may be positioned in the front row of the vehicle to capture images of passengers in the front row. One or more cameras may be positioned in the rear row of the vehicle to capture images of passengers in the rear row. In some examples, the cameras may also be configured to capture multiple images of the surrounding environment of the vehicle 100.

[0070] In some examples, the camera can be a still camera or a video camera. In some examples, the camera can capture monocular images or depth images.

[0071] In the embodiment of the present application, the radar may be a millimeter wave radar. Millimeter wave radar operates in the millimeter wave band and is mainly used to detect moving objects. Its operating frequency band is distributed in the 30GHz to 300GHz frequency domain (wavelength is 1mm to 10mm). When working, the millimeter wave radar continuously transmits (radiates) a specific form of wireless electromagnetic signal and receives the electromagnetic echo signal reflected by the object, and determines the spatial information of the object by comparing the difference between the transmitted signal and the received signal. Millimeter wave radar has the characteristics of small size and high spatial resolution; deployed in a vehicle, it can be used to detect the position of the human body (passenger) in the vehicle, physiological characteristics (such as breathing rate, heart rate, etc.) and human posture and other information.

[0072] In some examples, radar can also use radio signals to sense objects in the surrounding environment of vehicle 100, and can be specifically manifested as millimeter wave radar or laser radar. In some examples, in addition to sensing objects, radar can also be used to sense the speed and / or direction of the object.

[0073] It will be appreciated that the above is merely an example of the sensor system 120 in the embodiment of the present application and does not constitute a specific limitation on the sensor system 120. For example, the sensor system 120 may further include a pressure sensor or a human infrared sensor provided for each seat in the vehicle 100. The pressure sensor or human infrared sensor may be used to detect whether a passenger is occupying the seat.

[0074] In the embodiment of the present application, the control system 130 is used to control the vehicle 100 and its components to perform corresponding operations. As an example, the control system 130 may include an on-board domain controller.

[0075] In some examples, the domain controller can be divided into several domains (also called "functional domains") according to the functions of various parts of the car, such as the intelligent driving domain, cockpit domain, chassis domain, power domain, body domain, etc. Based on this, the on-board domain controller may include but is not limited to any one or more of the following: intelligent driving domain controller, cockpit domain controller, chassis domain controller, power domain controller, body domain controller. Among them, the intelligent driving domain controller is mainly used to provide autonomous driving perception, decision-making and other services, such as image information reception, image information processing and judgment, data processing and calculation, navigation and route planning, and rapid judgment and decision-making for real-time situations. The intelligent driving domain controller needs to process algorithms at the three levels of perception, decision-making, and control, and has the highest requirements for the domain controller's software and hardware. The cockpit domain controller is mainly used to control various electronic information system functions in the vehicle's intelligent cockpit, such as the central control system, in-vehicle infotainment system, head-up display, seat system, instrument system, rearview mirror system, driving behavior monitoring system, navigation system, etc. The chassis domain controller is primarily used to control the vehicle's driving behavior and posture. Its functions include, but are not limited to, brake system management, transmission system management, driving system management, steering system management, vehicle speed sensor management, body posture sensor management, air suspension system management, and airbag system management. The power domain controller is primarily used to control the vehicle's powertrain, optimize the vehicle's power performance, and ensure vehicle power safety. These functions include engine management, transmission management, battery management, power distribution management, emissions management, speed limit management, and fuel and power management. The body domain controller is primarily used to control various vehicle body functions, including, but not limited to, control of headlights, taillights, interior lights, door locks, windows, sunroof, wipers, electric trunk, smart key, air conditioning, antenna, and gateway communications.

[0076] It is understandable that the control system 130 may further include more or fewer components, and the above description of the control system 130 is not intended to limit the control system 130 .

[0077] In the embodiment of the present application, the vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripheral device 140. The peripheral device 140 may include a wireless communication system, a microphone, a speaker, and / or an onboard computer.

[0078] Among them, the wireless communication system can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, 4G cellular communication, 5G cellular communication, or wireless local area network (WLAN) communication, etc. In some examples, the wireless communication system can use infrared links, Bluetooth, or ZigBee to communicate directly with the device. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0079] In some examples, after the vehicle 100 establishes a connection with an electronic device (such as a mobile phone, a wearable device, a tablet computer, etc.) through a wireless communication system, the electronic device can project the screen through the central control screen of the vehicle 100. When the electronic device projects the screen through the central control screen of the vehicle 100, the content displayed on the central control screen of the vehicle 100 may not be exactly the same as the screen content of the electronic device. The content displayed on the central control screen may be an application that has been configured for the vehicle 100 and multiple applications in the terminal device. That is, when the vehicle 100 is not connected to the electronic device, it has its own user interface (UI). After the vehicle 100 is connected to the electronic device, there may be more icons for applications on the UI interface of the vehicle 100, or the functions of the vehicle 100 may be increased.

[0080] As an example, after the vehicle 100 and the electronic device are connected, they can also achieve mutual assistance at the hardware level. For example, the electronic device can use the global positioning system (GPS) configured in the vehicle 100 to improve the accuracy of the electronic device's navigation; for example, the electronic device can use the camera on the vehicle 100 to make video calls.

[0081] In embodiments of the present application, the peripheral device 140 may also provide a means for the user of the vehicle 100 to interact with the user interface 150. For example, the onboard computer may provide information to the user of the vehicle 100. The user interface 150 may also operate the onboard computer to receive user input. The onboard computer may be operated via a touch screen.

[0082] In an embodiment of the present application, the peripheral device 140 may also provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, a microphone may receive audio (e.g., voice commands or other audio input) from a user of the vehicle 100. Similarly, a speaker may output audio to the user of the vehicle 100.

[0083] In the embodiment of the present application, the speakers can be installed in the headrests of the seats of the vehicle 100 so that passengers can receive audio. It is understood that in some examples, the speakers can also be installed in the center console, four doors, ceiling, trunk, etc. of the vehicle 100. The specific installation can be based on actual needs and is not limited by the embodiment of the present application.

[0084] In an embodiment of the present application, some or all functions of the vehicle 100 are controlled by a computer system 110. The computer system 110 may include at least one processor and a memory. The processor may be a central processing unit (CPU) or other specific integrated circuit. The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In actual applications, the computer system 110 may also include multiple processors, and the processor may include one or more processor cores.

[0085] The memory may contain instructions (e.g., program logic) that are executable by the processor to perform various functions of the vehicle 100. The memory may also contain additional instructions, including instructions to send data to one or more of the sensor system 120, the control system 130, and the peripheral devices 140, instructions to receive data from one or more of the sensor system 120, the control system 130, and the peripheral devices 140, instructions to interact with one or more of the sensor system 120, the control system 130, and the peripheral devices 140, and instructions to control one or more of the sensor system 120, the control system 130, and the peripheral devices 140.

[0086] In addition to instructions, the memory may also store data such as road maps, route information, the vehicle's location, direction, speed, and other such vehicle data, as well as other information. This information may be used by the vehicle 100 and the computer system 110 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0087] It will be appreciated that the processor or memory may actually include multiple processors or memories that are not stored in the same physical housing. For example, the memory may be a hard drive or other storage medium located in a different housing than the computer system 110. Therefore, references to a processor or memory will be understood to include references to a collection of processors or memories that may or may not operate in parallel.

[0088] In some examples, different components may have their own processors that perform only calculations related to the functionality specific to the component.

[0089] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 100. For example, the memory may be partially or completely separate from the vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0090] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 1 should not be understood as a limitation to the embodiments of the present application.

[0091] As an example, the vehicle information processing method provided in the embodiment of the present application can be applied to a vehicle information processing device, which can be pre-loaded into the memory of the computer system 110 in the form of a computer program product, and the processor of the computer system 110 can execute the instructions in the memory to implement the steps of the vehicle information processing method provided in the embodiment of the present application.

[0092] As an example, the vehicle information processing device can also be set in the vehicle. In a specific example, the vehicle information processing device can be a vehicle-mounted terminal, a vehicle computer, an in-vehicle audio and video entertainment device, etc. set in the vehicle.

[0093] For ease of understanding, the vehicle information processing method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Please refer to Figure 2, which is a schematic flow diagram of a vehicle information processing method provided in the embodiments of the present application. The method shown in Figure 2 is applied to the vehicle shown in Figure 1 and executed by a vehicle information processing device. As shown in Figure 2, the vehicle information processing method provided in the embodiments of the present application includes steps S201 to S204.

[0094] S201, obtaining vehicle equipment data and in-vehicle images.

[0095] Vehicle device data refers to the device data of adjustable devices inside the vehicle (hereinafter referred to as "in-vehicle devices"). For example, in-vehicle devices include, but are not limited to, one or more devices such as a steering wheel, seats, rearview mirrors, HUD devices, and playback devices (such as speakers).

[0096] In the embodiment of the present application, each in-vehicle device is correspondingly configured with a driving device (such as a motor) and a control unit. The control unit can control and adjust the in-vehicle device by controlling the driving device.

[0097] It can be understood that the control unit corresponding to each in-vehicle device can be a separate controller or a control module on the controller, without specific limitation.

[0098] As an example, the driving device corresponding to each in-vehicle device may have multiple gears, and the control unit may control and adjust the in-vehicle device by adjusting the gears of the driving device.

[0099] In the embodiment of the present application, the vehicle's device data includes current adjustment data and initial adjustment data of each in-vehicle device.

[0100] As an example, the current adjustment data of the in-vehicle device may be the gear value of the driving device corresponding to the in-vehicle device. The initial adjustment data of the in-vehicle device may be the initial gear value of the driving device corresponding to the in-vehicle device, that is, the gear value that has not been adjusted.

[0101] As an example, the vehicle information processing device can obtain vehicle device data through the control unit corresponding to the in-vehicle device. For example, the control unit can collect the current gear value of the drive device corresponding to the in-vehicle device in real time and send the current gear value and initial gear value of the drive device corresponding to the in-vehicle device to the vehicle information processing device in real time. For another example, the vehicle information processing device can read the current gear value and initial gear value of the drive device corresponding to the in-vehicle device collected by the control unit in real time through the control unit.

[0102] In the embodiment of the present application, the in-vehicle image of the vehicle refers to the image of the interior of the vehicle, and the image of the interior of the vehicle can reflect the changes in the spatial environment inside the vehicle.

[0103] As an example, a vehicle information processing device may capture an image of the vehicle's interior using a camera or video camera installed in the vehicle. For example, the vehicle information processing device may control the camera to capture images of the vehicle's interior, thereby obtaining the vehicle interior image from the camera. In another example, the vehicle information processing device may extract frames from the video captured by the camera to obtain the vehicle interior image.

[0104] As an example, the acquired in-vehicle image may be a monocular image or a depth image.

[0105] As an example, the vehicle information processing device can acquire the vehicle's device data and interior images in real time. To save energy, in some examples, the vehicle information processing device can acquire the vehicle's device data and interior images in real time after detecting a user getting on the vehicle, and stop acquiring the vehicle's device data and interior images after detecting the user getting off the vehicle.

[0106] It is understandable that in some examples, the vehicle information processing device can also obtain the vehicle's equipment data and in-vehicle images according to a preset period or obtain the vehicle's equipment data and in-vehicle images according to other needs. This application does not impose specific restrictions on this and can be set according to actual needs.

[0107] S202 , obtaining in-vehicle space information based on the device data and the in-vehicle image, where the in-vehicle space information includes device location information of in-vehicle devices and passenger feature information inside the vehicle.

[0108] The device location information of the in-vehicle device refers to the current location information of the in-vehicle device.

[0109] In an embodiment of the present application, after obtaining the device data, for each in-vehicle device, the device location information of the in-vehicle device, that is, the current location information of the in-vehicle device, can be obtained based on the motion model of the in-vehicle device and the current adjustment data and initial adjustment data of the in-vehicle device in the device data.

[0110] The motion model can characterize the motion patterns of the in-vehicle device as the adjustment data changes. For example, the motion model characterizes the motion patterns of the in-vehicle device as the gear position of the corresponding drive device changes. The motion model can be used to determine the distance moved and / or angle rotated by the in-vehicle device per gear position.

[0111] As an example, the motion model of the in-vehicle device is obtained by the motion mode of the in-vehicle device. For example, the motion mode includes, but is not limited to, translation, two-dimensional rotation, and three-dimensional motion.

[0112] For translation and two-dimensional rotation, a motion model for the in-vehicle device can be established based on the device's mechanical motion structure. For example, for translation, the device's movement data at each gear position can be directly acquired to determine the motion pattern of the in-vehicle device under the corresponding drive device gear position changes. For two-dimensional rotation, the device's rotation angle data at each gear position can also be directly acquired to determine the motion pattern of the in-vehicle device under the corresponding drive device gear position changes.

[0113] For three-dimensional motion, the three-dimensional motion of in-vehicle devices can be estimated and fitted based on image calibration to derive the device's motion patterns. For example, for each in-vehicle device, the gear position of the corresponding drive device is adjusted, and for each gear position, an image of the device's position on a calibration plate is obtained. By analyzing the position images of the device on the calibration plate at different gear positions, the device's position information at each gear position can be obtained. By fitting the device's position information at different gear positions, the device's motion patterns can be derived as the gear position of the corresponding drive device changes.

[0114] Taking the example of an in-vehicle seat, seat adjustments include fore-aft distance adjustment, backrest angle adjustment, and height adjustment. When adjusting fore-aft distance, the seat's movement is translational, and the distance traveled by the seat in each gear position can be directly determined. Based on this distance, the motion pattern of the seat under the corresponding drive gear position during fore-aft distance adjustment can be determined. When adjusting height, the seat's movement is three-dimensional, so the three-dimensional motion of the seat can be fitted based on image calibration to determine the motion pattern of the seat under the corresponding drive gear position during height adjustment.

[0115] The movement of the backrest can be regarded as a rotation around the intersection of the backrest and the seat cushion. Therefore, when the backrest angle is adjusted, the movement of the seat device is a two-dimensional rotational motion, and the rotation angle of the backrest in each gear can be directly obtained. Based on the rotation angle of the backrest in each gear, the movement law of the seat device under the gear change of the corresponding drive device when the backrest angle is adjusted is obtained.

[0116] In an embodiment of the present application, for each in-vehicle device, the current adjustment data of the in-vehicle device represents the current gear value of the drive device corresponding to the in-vehicle device, and the initial adjustment data of the in-vehicle device represents the initial gear value of the drive device corresponding to the in-vehicle device. Obtaining the device position information of the in-vehicle device based on the motion model of the in-vehicle device and the current adjustment data and initial adjustment data of the in-vehicle device can be performed by obtaining the gear change information of the drive device corresponding to the in-vehicle device based on the current adjustment data and initial adjustment data of the in-vehicle device, obtaining the position change data of the in-vehicle device based on the gear change information and the motion model of the in-vehicle device, and obtaining the device position information of the in-vehicle device based on the initial position information of the in-vehicle device and the position change data of the in-vehicle device, i.e., the current position information of the in-vehicle device.

[0117] It can be understood that when the drive device corresponding to the in-vehicle device is in the initial gear position, the position of the in-vehicle device is the initial position information of the in-vehicle device. For example, based on the current adjustment data of the in-vehicle device, the gear position of the drive device corresponding to the in-vehicle device is obtained as gear A, and based on the initial adjustment data of the in-vehicle device, the initial gear position of the drive device corresponding to the in-vehicle device is obtained as gear B. The gear change information is AB. According to the motion model, the in-vehicle device moves X1 at each gear position of the drive device. The position change data of the in-vehicle device is (AB)*X1. Assuming the initial position information of the in-vehicle device is Y1, the current position information of the in-vehicle device is Y1+(AB)*X1.

[0118] As an example, each vehicle's memory stores a complete vehicle digital model. The vehicle information processing device can obtain the initial position information of each in-vehicle device from the complete vehicle digital model. The complete vehicle digital model mathematically describes the vehicle's geometry, material properties, kinematic characteristics, and dynamic characteristics. Therefore, the complete vehicle digital model can represent the vehicle's digital simulation model. The position information of the in-vehicle device in the complete vehicle digital model can represent the initial position information of the in-vehicle device in the vehicle.

[0119] Taking the in-vehicle seat as an example, as shown in Figure 3, the initial position information of the seat is obtained from the vehicle digital model. Based on the current and initial gear values ​​of the seat, the gear change information of the corresponding drive device of the seat is obtained. This includes gear change information of the drive device for adjusting the fore-aft distance, the drive device for adjusting the backrest angle, and the drive device for adjusting the height. The gear change information is input into the motion model of the seat, including a motion model based on a mechanical motion structure and image calibration. This yields position change data for the seat, including changes in the fore-aft distance, the angle of the seatback, and the height. The seat position change data is then applied to the initial position information of the seat, i.e., by adding the seat position change data to the initial position information, to obtain the device position information of the seat, i.e., the current position information of the seat.

[0120] It can be understood that the device location information of the in-vehicle device refers to the current location information of the in-vehicle device in the entire vehicle coordinate system.

[0121] In the embodiment of the present application, target detection can be performed on the acquired in-vehicle image to obtain passenger feature information. Passenger feature information includes passenger body position information, such as the position information of the passenger's eyes, ears, arm joints, and shoulders.

[0122] In order to further facilitate the control unit to control and adjust the control object, achieve personalized adjustment for each user, and improve the user's car-using experience, in an embodiment of the present application, the passenger characteristic information may also include passenger distribution information and passenger size information.

[0123] As an example, target detection is performed on the in-car image to obtain passenger feature information, including

[0124] (A) Perform object detection on the in-car image to obtain passenger location information and passenger distribution information.

[0125] (B) Obtain the passenger's height information based on the passenger's body position information and the position information of the seat and in-vehicle equipment.

[0126] (C) Obtain passenger size information based on passenger height information.

[0127] The following example illustrates the process of obtaining passenger body part location information using a monocular image. Referring to Figure 4 (a), for a monocular image, target detection is first performed to obtain the target face image and the target person image. Keypoint detection is then performed on the target face image to obtain facial key points, such as the eyes, ears, and mouth.

[0128] It is understandable that the position information of facial key points obtained based on the target facial image is the 2D coordinates of the facial key points in the camera coordinate system (referred to as "2D facial key points"). To improve accuracy, after detecting and obtaining facial key points, the facial key points can be mapped to standard 3D key points in a preset head template to obtain the 3D position coordinates of the facial key points in the camera coordinate system.

[0129] After obtaining the 3D position coordinates of the facial key points in the camera coordinate system, we can perform body estimation on the target person image to obtain a body model. The body model contains body key points as well as facial key points. Therefore, after obtaining the body model, we can map the 3D position coordinates of the facial key points in the camera coordinate system to the facial key points in the body model, thereby transforming the body model to the camera coordinate system and obtaining the 3D position coordinates of the body key points in the body model in the camera coordinate system. Body key points include shoulders, arm joints, etc.

[0130] As an example, target detection algorithms such as region-CNN (RCNN), SPP-Net (Spatial Pyramid Pooling), and YOLO (You Only Look Once) can be used to detect targets in vehicle interior images to obtain target face images and target person images. There are no specific restrictions and these algorithms can be configured based on actual needs.

[0131] As an example, a facial key point detection algorithm, such as an active shape model (ASM), an active appearance model (AAM), or a deep learning algorithm, can be used to detect key points in a target facial image to obtain facial key points. The specific algorithm is not limited and can be set based on actual needs.

[0132] As an example, a human keypoint detection algorithm, such as OpenPose or Mask R-CNN, can be used to estimate the human body in the target person image to obtain a human body model. There are no specific restrictions and this can be set based on actual needs.

[0133] Please refer to (b) in Figure 4 , using the in-car image as a depth image as an example to illustrate the process of obtaining passenger body position information. For the depth image, the depth image and the corresponding RGB image are used to estimate the human body, fit a human body model in the camera coordinate system, and then directly obtain the 3D position coordinates of the key points of the face and body from the human body model. In some examples, point cloud data can also be obtained based on the depth image and the corresponding RGB image, and the position information of the key points of the face and body can be obtained from the point cloud data. For details, please refer to conventional techniques and will not be described in detail here. In some examples, the RGB image can also be an IR image, which is not limited and can be set according to actual needs. The IR image is the infrared image output by the infrared image sensor, and the RGB image is a digital image format, which refers to the color space (red, green, and blue) used by the image format and also refers to the pixel values ​​of the three color channels of red, green, and blue contained in the image itself.

[0134] In the embodiment of the present application, the passenger part position information includes the position information of the key points of the face, that is, the 3D position coordinates of the key points of the face in the camera coordinate system, such as the eye P eye (x,y,z), human ear P ear (x,y,z), mouth P mouth (x, y, z), and the position information of the key points of the body parts, that is, the 3D position coordinates of the key points of the body parts in the camera coordinate system, for example, the shoulder P J (x,y,z), arm joint point P SG (x,y,z).

[0135] As an example, performing target detection on the in-vehicle image to obtain passenger distribution information can be performing target detection on the in-vehicle image to obtain passengers at different positions inside the vehicle, and numbering the passengers at different positions to obtain the passenger distribution information inside the vehicle.

[0136] It should be understood that the above is merely an example of how to obtain passenger distribution information in the embodiments of the present application and does not constitute a specific limitation on the method of obtaining passenger distribution information. In some examples, passenger distribution information can also be obtained using other methods. For example, millimeter-wave radar can be used to detect human bodies inside the vehicle to obtain passenger distribution information within the vehicle.

[0137] In an embodiment of the present application, after obtaining the passenger body position information and the device position information of the in-vehicle equipment, the passenger height information can be obtained based on the passenger body position information and the device position information of the in-vehicle equipment, and the passenger size information can be obtained based on the passenger height information.

[0138] For example, the passenger's height information can be obtained based on the passenger's eye position in the passenger's body position information and the device position information of the seat device in the in-vehicle device, and then the passenger's size information can be obtained based on the passenger's height information.

[0139] As shown in Figure 5, based on the passenger's eye position and the equipment position information of the seat equipment, the passenger's height information can be obtained by first obtaining the passenger's half body length based on the passenger's eye position information and the equipment position information of the seat cushion in the seat equipment, and then obtaining the passenger's height based on the passenger's half body length.

[0140] Among them, before determining the passenger's half-body length, the passenger's part position information and the equipment position information of the in-vehicle equipment can be converted into the same coordinate system. In the same coordinate system, the passenger's half-body length can be obtained based on the passenger's eye position information and the equipment position information of the seat cushion in the seat equipment.

[0141] The equipment position information of the seat equipment includes the equipment position information of the seat cushion. The position of the seat cushion can be regarded as the position of the contact point between the human body and the seat cushion, that is, the position of the human body's buttocks. Therefore, the passenger's half body length can be estimated based on the passenger's eye position and the equipment position information of the seat cushion.

[0142] As an example, the passenger's height can be obtained based on the passenger's half-thigh length. The passenger's height can be estimated based on the passenger's half-thigh length and the standard proportions of the human body. The standard proportions of the human body indicate the proportional relationship between half-thigh length and height. The passenger's height can be estimated based on this proportional relationship and the passenger's half-thigh length.

[0143] As an example, human body data can also be collected through data acquisition, and the relationship between half-length and height can be obtained based on the human body data fitting. Then, the passenger's height can be estimated based on the relationship between half-length and height and the passenger's half-length.

[0144] After obtaining the passenger's height, the passenger's size information can be obtained based on the standard proportions of the human body.

[0145] As an example, the passenger size information of the passenger includes the passenger's upper arm length, forearm length, thigh length, calf length, etc.

[0146] In an embodiment of the present application, after obtaining the device location information and passenger characteristic information of the in-vehicle device, it is possible to detect whether the position of the device has changed, whether the passenger has moved, etc. based on the device location information and passenger characteristic information of the in-vehicle device, so as to timely perceive the changes in the people and in-vehicle equipment in the vehicle, and timely perceive the changes in the spatial environment in the vehicle.

[0147] As an example, to facilitate control and adjustment of the corresponding control object, the control unit can determine the relative position of the passenger's body and the in-vehicle device based on the obtained passenger and device position information. Based on this relative position, the control unit can promptly detect changes in the in-vehicle spatial environment. Based on this, the in-vehicle spatial information can also include the phase position between the passenger and the in-vehicle device.

[0148] S203: Send first adjustment information to the first control unit according to the control requirements of the first control unit. The first adjustment information includes device location information of the in-vehicle device and / or passenger characteristic information. The first adjustment information is used by the first control unit to control and adjust the first control object.

[0149] S204, sending second adjustment information to the second control unit according to the control requirements of the second control unit, the second adjustment information including the device location information and / or passenger characteristic information of the in-vehicle device, the second adjustment information is the same as the first adjustment information, or the second adjustment information is different from the first adjustment information.

[0150] Among them, in order to improve the control effect of the control object and realize real-time control and adjustment of the control object, in an embodiment of the present application, after obtaining the in-vehicle equipment location information and passenger characteristic information, the first adjustment information can be sent to the first control unit according to the control requirements of the first control unit, so that the first control unit can control and adjust the first control object according to the first adjustment information, and the second adjustment information can be sent to the second control unit according to the control requirements of the second control unit, so that the second control unit can control and adjust the second control object according to the second adjustment information.

[0151] The first control unit refers to one or more control units corresponding to the multiple in-vehicle devices, and the second control unit refers to one or more control units corresponding to the multiple in-vehicle devices other than the first control unit. The first adjustment information of the first control unit may be the same as or different from the second adjustment information of the second control unit.

[0152] It is understandable that the first control unit and the second control unit are merely examples in the embodiments of the present application and are not intended to limit the control unit. In some examples, a third control unit, a fourth control unit, etc. may also be present, which are specifically configured according to actual needs.

[0153] In an embodiment of the present application, after obtaining the in-vehicle space information, the vehicle information processing device can send corresponding adjustment information to each control unit according to the control requirements of each control unit, so that each control unit can control and adjust the controlled object according to the adjustment information.

[0154] As an example, the vehicle information processing device can synchronously send corresponding adjustment information to each control unit according to the control requirements of each control unit, or it can send corresponding adjustment information to a certain control unit separately according to the control requirements of the control unit. There is no specific restriction and it can be set according to actual needs.

[0155] In the embodiment of the present application, the control object of each control unit refers to the in-vehicle device controlled by the control unit. For example, the control object corresponding to the audio control unit is the playback device (such as a speaker), and the control object corresponding to the seat control unit is the seat.

[0156] In the embodiments of the present application, the control requirements of each control unit indicate the adjustment information required by the control unit to control and adjust the controlled object. It is understood that the adjustment information required by the control unit is one or more pieces of in-vehicle spatial information, that is, the adjustment information includes device location information of in-vehicle devices and / or passenger characteristic information.

[0157] For example, the adjustment information required by the control unit can be obtained according to the control requirements of the control unit, and then the adjustment information can be sent to the control unit. For example, the control unit is an audio control unit. The control object corresponding to the audio control unit is the playback device. When controlling and adjusting the playback device, the human ear position information and the playback device position information are required. Therefore, according to the control requirements of the audio control unit, the adjustment information of the audio control unit can be obtained, including the human ear position information in the passenger characteristic information and the device position information of the playback device in the device position information of the in-vehicle device. Then the human ear position information and the device position information of the playback device are sent to the audio control unit.

[0158] As an example, the vehicle information processing device provides a subscription interface to each control unit, and each control unit can send the control requirements of the control unit to the vehicle information processing device through the subscription interface. Therefore, the vehicle information processing method provided in the embodiment of the application can also include the following steps:

[0159] The subscription request of the control unit is received through the subscription interface, and the control requirements of the control unit are obtained according to the subscription request.

[0160] Each control unit can send a subscription request to the vehicle information processing device via a subscription interface. The subscription request includes the adjustment information that the control unit needs to subscribe to. Therefore, after receiving the subscription request sent by the control unit via the subscription interface, the vehicle information processing device can obtain the adjustment information required by the control unit from the subscription request, thereby determining the control requirements of the control unit.

[0161] It can be understood that after obtaining the control requirements of the control unit through the subscription interface, the vehicle information processing device can send the required adjustment information to the control unit according to the control requirements of the control unit.

[0162] As an example, the vehicle information processing device may send the required adjustment information to the control unit through a reporting interface. The reporting interface may be an interface different from the subscription interface, or the reporting interface may be the same interface as the subscription interface.

[0163] As an example, after the vehicle information processing device obtains the device location information and passenger characteristic information of the in-vehicle equipment, if the adjustment information required by the control unit is changed based on the device location information and passenger characteristic information of the in-vehicle equipment, the latest adjustment information is sent to the control unit so that the control unit can perceive the changes in the spatial environment in a timely manner and control and adjust the control object based on the latest adjustment information.

[0164] For example, using the audio control unit as an example, the adjustment information required by the audio control unit includes human ear position information and playback device position information. After obtaining the device position information and passenger characteristic information of the in-vehicle device, if the playback device position information and human ear position information obtained based on the in-vehicle device position information and passenger characteristic information have changed compared to the information obtained last, the vehicle information processing device can send the latest obtained playback device position information and human ear position information to the audio control unit.

[0165] As an example, in order to enable the control unit to perceive changes in the spatial environment inside the vehicle in a timely manner, and to control and adjust the control object in a timely manner based on the changes in the spatial environment to improve the control effect, the vehicle information processing device can also send the required adjustment information to the control unit in real time according to the control requirements of the control unit.

[0166] As an example, the vehicle information processing device may further provide the control unit with an information receiving switch, which the control unit may use to determine whether to receive the adjustment information. For example, when the control unit turns off the information receiving switch, the vehicle information processing device does not need to send the required adjustment information to the control unit. When the control unit turns on the information receiving switch, the vehicle information processing device needs to send the required adjustment information to the control unit.

[0167] As an example, the control unit may also send a call request to the vehicle information processing device and, based on the call request, indicate the control unit's control requirements to the vehicle information processing device. For example, the control unit may send a call request to the vehicle information processing device, the call request including the control unit's control requirements, i.e., the adjustment information required by the control unit. After receiving the call request from the control unit, the vehicle information processing device can obtain the control unit's control requirements based on the call request and send the required adjustment information to the control unit based on the control unit's control requirements.

[0168] As an example, the control unit may periodically send a call request to the vehicle information processing device. After receiving the call request, the vehicle information processing device sends adjustment information to the control unit according to the control requirements in the call request.

[0169] As an example, in order to enable the control unit to perceive changes in the spatial environment inside the vehicle in a timely manner, and to control and adjust the control object in a timely manner based on the changes in the spatial environment to improve the control effect, the vehicle information processing device can send adjustment information to the control unit in real time according to the control requirements in the call request after receiving the call request from the control unit.

[0170] It is understandable that after receiving the adjustment information, the control unit can control and adjust the control object according to the adjustment information. For example, after receiving the adjustment information, the control unit can adjust the gear of the corresponding driving device according to the adjustment information, thereby controlling and adjusting the control object.

[0171] As shown in Figure 6, the vehicle information processing device sends corresponding adjustment information to the audio control unit, seat adjustment control unit, steering wheel adjustment control unit, rearview mirror adjustment control unit, and HUD adjustment control unit respectively. The audio control unit, seat adjustment control unit, steering wheel adjustment control unit, rearview mirror adjustment control unit, and HUD adjustment control unit adjust the gear position of the corresponding drive device according to the adjustment information, thereby achieving adjustment of in-vehicle equipment such as speakers, seats, steering wheels, rearview mirrors, and HUD.

[0172] In order to improve the control effect of each control unit on the control object, in an embodiment of the present application, each control unit can control and adjust the control object based on the information of people and / or equipment in the vehicle according to passenger characteristic information and / or device location information.

[0173] The following uses the audio control unit as an example. For this control unit, changes in the in-vehicle sound field, such as seat position and passenger distribution, can alter the in-vehicle sound field. Even changes in the relative position of the human ear and the playback device (e.g., speakers) can affect passengers' audio reception. Therefore, to improve audio control, it's necessary to control and adjust the audio parameters of the playback device based on seat position information, playback device location information, passenger distribution information, and passenger ear position information.

[0174] In an embodiment of the present application, as shown in FIG7 , the adjustment information for the audio control unit includes passenger distribution information and passenger ear position information from the passenger characteristic information, and device position information for playback devices and seat devices from the in-vehicle device position information. Therefore, after the vehicle information processing device obtains the passenger distribution information, passenger ear position information, playback device position information, and seat device position information based on the device data and in-vehicle images, it can transmit the passenger ear position information, passenger distribution information, playback device position information, and seat device position information to the audio control unit.

[0175] Among them, for the playback device, the gear change information of the driving device corresponding to the playback device can be obtained based on the current adjustment data and initial adjustment data of the playback device, and the position change data of the playback device can be obtained based on the gear change information and the motion model of the playback device. Based on the initial position information of the playback device and the position change data of the playback device, the device position information of the playback device can be obtained.

[0176] Correspondingly, for the seat equipment, the gear change information of the driving device corresponding to the seat equipment can be obtained based on the current adjustment data and initial adjustment data of the seat equipment, and the position change data of the seat equipment can be obtained based on the gear change information and the motion model of the seat equipment. Based on the initial position information of the seat equipment and the position change data of the seat equipment, the device position information of the seat equipment can be obtained.

[0177] Passenger distribution information and passenger ear position information can be obtained by performing target detection on the in-vehicle image. The specific process can be referred to step S202 and will not be described in detail here.

[0178] It can be understood that after obtaining the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information, the audio control unit can control and adjust the audio parameters of the playback device and / or the audio played by the playback device according to the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information to control the audio effect played by the playback device.

[0179] As an example, an audio control strategy is set in the audio control unit, which refers to a strategy for controlling and adjusting the audio parameters of the playback device and / or the audio played by the playback device based on the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information.

[0180] For example, let the position information of the passenger's ear be P ear (x,y,z), the device position information of the playback device is P loudspeaker (x, y, z), the device position information of the seat device is P seat (x, y, z), the passenger distribution information is A[1, 0, 0, 0, 0], where the passenger distribution information A[1, 0, 0, 0, 0] represents the code of different positions in the car, 1 means there is a passenger at the corresponding position, and 0 means there is no passenger at the corresponding position. ear (x,y,z),P loudspeaker (x,y,z),P seat After (x, y, z), A[1, 0, 0, 0, 0] is sent to the audio control unit, the audio control unit can then ear (x,y,z),P loudspeaker (x,y,z),P seat (x, y, z), A[1, 0, 0, 0, 0] and other information are used to control and adjust the playback device and / or audio parameters of the audio played by the playback device.

[0181] As an example, after receiving the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information, the audio control unit can adjust the position of the playback device according to the device position information of the playback device and the passenger ear position information so that the device position of the playback device matches the passenger ear position, and control and adjust the audio parameters of the audio played by the playback device according to the passenger distribution information and the device position information of the seat device so that the audio played by the playback device adapts to the sound field inside the car.

[0182] The embodiment of the present application collects the device data and images inside the vehicle in real time, and based on the device data and images inside the vehicle, perceives the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information in real time, and uses the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information for the audio control unit. The audio control unit can adapt to the changes in the spatial environment inside the vehicle according to the device position information of the playback device, the device position information of the seat device, the passenger ear position information and the passenger distribution information, and realize real-time control and adjustment of the audio playback, thereby achieving more intelligent audio adjustment, improving the audio adjustment effect, and providing passengers with a better audio experience.

[0183] For the seat adjustment control unit, the seat device can be adjusted through the device position information of the seat device. Taking into account that the size of the human body and the position of the seat will affect the passenger's use of the seat, in order to improve the seat adjustment effect, personalized adaptive adjustment can be achieved for each user. In an embodiment of the present application, for the seat adjustment control unit, the adjustment information includes the passenger size information and the passenger eye position information in the passenger characteristic information, and the device position information of the seat device in the device position information of the in-vehicle device. Therefore, the vehicle information processing device obtains the passenger size information, the passenger eye position information and the device position information of the seat device based on the device data and the in-vehicle image, and can send the passenger size information, the passenger eye position information and the device position information of the seat device to the seat adjustment control unit.

[0184] Among them, the process of the vehicle information processing device obtaining the passenger size information, the passenger eye position information and the device position information of the seat device based on the device data and the in-vehicle image can refer to step S202 and will not be repeated here.

[0185] As shown in Figure 8, after obtaining the passenger size information, the passenger eye position information, and the equipment position information of the seat equipment, the passenger size information, the passenger eye position information, and the equipment position information of the seat equipment can be sent to the seat adjustment control unit, and then the seat adjustment control unit can control and adjust the seat equipment according to the passenger size information, the passenger eye position information, and the equipment position information of the seat equipment, so as to realize personalized seat intelligent adjustment for each user, improve the seat adjustment effect, and enhance the user's car experience.

[0186] As an example, a seat adjustment control unit is provided with a seat adjustment strategy, which refers to a strategy for controlling and adjusting the seat device based on passenger size information, passenger eye position information, and device position information of the seat device.

[0187] As an example, after receiving passenger size information, passenger eye position information, and seat device position information, the seat adjustment control unit can obtain seat device height adjustment information based on the passenger eye position information and preset target eye position information, and obtain the target height position of the seat device based on the seat device height adjustment information and the seat device position information. The target horizontal position of the seat device is obtained based on the thigh length, calf length, and ergonomic data in the passenger size information. Based on the target horizontal position and target height position of the seat device, the target position P of the seat device can be obtained. H (x,y,z), (where P H The point is the intersection of the folded upper and lower body parts of the human body, which is approximately the target position of the seat equipment. (x, y) represents the target horizontal position of the seat equipment, and z represents the target height position of the seat equipment).

[0188] After obtaining the target position of the seat device, the seat adjustment control unit can obtain the gear value (f, h) corresponding to the target position of the seat device in combination with the initial position of the seat device in the vehicle digital model and the motion model of the seat device, where f represents the front and rear gear values ​​of the seat device, and h represents the up and down gear values ​​of the seat device.

[0189] It can be understood that the front and rear gear values ​​are used to adjust the front and rear distance of the seat device, thereby achieving the adjustment of the position of the seat device in the horizontal direction, and the upper and lower gear values ​​are used to adjust the height of the seat device, thereby achieving the adjustment of the position of the seat device in the height direction.

[0190] After obtaining the gear value corresponding to the target position of the seat equipment, the seat adjustment control unit can issue an adjustment instruction to the drive device of the seat equipment according to the gear value to control the drive device of the seat equipment to adjust the seat equipment based on the gear value, thereby realizing personalized intelligent adjustment of the seat equipment.

[0191] As an example, the initial position information of the seat device in the vehicle digital model may be sent by the vehicle information processing device to the seat adjustment control unit.

[0192] It is understandable that, in some examples, the initial position information of the seat device in the complete vehicle digital model may also be read by the seat adjustment control unit from the complete vehicle digital model stored in the vehicle.

[0193] In order to further improve the seat adjustment effect and achieve personalized adaptive adjustment for each user, as an example, for the seat adjustment control unit, the passenger characteristic information may include not only the passenger size information and the passenger eye position information, but also the passenger weight information.

[0194] As an example, each seat is provided with a weight sensor, and the vehicle information processing device can obtain passenger weight information through the weight sensor.

[0195] As an example, passenger weight information can also be obtained based on the in-vehicle image. For example, the passenger's body volume can be obtained based on the detection results of the in-vehicle image, and the passenger's weight information can be obtained based on the passenger's body volume. For example, if the in-vehicle image is a monocular image, the in-vehicle image can be detected to obtain the three-dimensional shape of the human body, including the positional information of human body parts. The human body volume can be estimated based on the three-dimensional shape of the human body. For another example, if the in-vehicle image is a depth image, human body estimation can be performed directly on the depth image to obtain the passenger's body volume information.

[0196] After obtaining the passenger's body volume, the passenger's weight information can be obtained based on the relationship between body volume and weight.

[0197] As an example, after obtaining the passenger size information, the passenger eye position information, the seat equipment position information and the passenger weight information, the passenger size information, the passenger eye position information, the seat equipment position information and the passenger weight information can be sent to the seat adjustment control unit so that the seat adjustment control unit can control and adjust the seat equipment.

[0198] For example, the initial target position of the seat device can be obtained based on the passenger size information, the passenger eye position information, and the device position information of the seat device. Then, based on the passenger weight information, it is detected whether the passenger weight exceeds the set weight threshold. If it exceeds the set weight threshold, the initial target position is adjusted to obtain the final target position of the seat device to leave enough seating space for the passenger. If it does not exceed the set weight threshold, the initial target position is determined as the final target position of the seat device.

[0199] The embodiment of the present application collects the device data and images inside the vehicle in real time, and based on the device data and images inside the vehicle, perceives the device position information of the seat device, the passenger's eye position information and the passenger size information in real time, and uses the device position information of the seat device, the passenger's eye position information and the passenger size information for the seat adjustment control unit, so that the seat adjustment control unit can realize personalized seat intelligent adjustment for each user according to the device position information of the seat device, the passenger's eye position information and the passenger size information.

[0200] Regarding the steering wheel adjustment control unit, since human body size and eye height can affect a passenger's use of the steering wheel, in order to improve the steering wheel adjustment effect and achieve personalized adaptive adjustment for each user, in the embodiment of the present application, the adjustment information for the steering wheel adjustment control unit includes passenger size information and passenger eye position information from the passenger characteristic information.

[0201] The acquisition of the passenger size information and the passenger eye position information may refer to step S202 and will not be described in detail here.

[0202] As shown in Figure 9, after obtaining the passenger size information and the passenger eye position information, the passenger size information and the passenger eye position information can be sent to the steering wheel adjustment control unit, and then the steering wheel adjustment control unit can control and adjust the steering wheel device according to the passenger size information and the passenger eye position information to achieve personalized steering wheel intelligent adjustment for each user, improve the steering wheel adjustment effect, and enhance the user's car experience.

[0203] As an example, a steering wheel adjustment control unit is provided with a steering wheel adjustment strategy, which refers to a strategy for controlling and adjusting the steering wheel device based on passenger size information and passenger eye position information.

[0204] As an example, after receiving the passenger size information and the passenger eye position information, the steering wheel adjustment control unit can estimate the target height position of the steering wheel device based on the passenger eye position information. Based on the upper arm length, lower arm length and ergonomic data in the passenger size information, the target horizontal position of the steering wheel device is obtained. Based on the target horizontal position and target height position of the steering wheel device, the target position P of the steering wheel device can be obtained. W (x,y,z), (where P W Point is the coordinate of the passenger's grip point on the steering wheel, (x, y) represents the target horizontal position of the steering wheel device, and z represents the target height position of the steering wheel device).

[0205] After obtaining the target position of the steering wheel device, the steering wheel adjustment control unit can obtain the gear value (k, l) corresponding to the target position of the steering wheel device in combination with the initial position of the steering wheel device in the vehicle digital model and the motion model of the steering wheel device, where k represents the front and rear gear values ​​of the steering wheel device, and l represents the up and down gear values ​​of the steering wheel device.

[0206] It can be understood that the front and rear gear values ​​are used to adjust the front and rear distance of the steering wheel in-vehicle device, thereby realizing the adjustment of the horizontal position of the steering wheel in-vehicle device, and the upper and lower gear values ​​are used to adjust the height of the steering wheel device, thereby realizing the adjustment of the height position of the steering wheel device.

[0207] After obtaining the gear value corresponding to the target position of the steering wheel device, the steering wheel adjustment control unit can issue an adjustment instruction to the driving device of the steering wheel device according to the gear value to control the driving device of the steering wheel device to adjust the steering wheel device based on the gear value, thereby realizing personalized and intelligent adjustment of the steering wheel device.

[0208] As an example, the initial position information of the steering wheel device in the vehicle digital model may be sent by the vehicle information processing device to the steering wheel adjustment control unit.

[0209] It can be understood that in some examples, the initial position information of the steering wheel device in the whole vehicle digital model can also be read by the steering wheel adjustment control unit from the whole vehicle digital model stored in the vehicle.

[0210] The embodiments of the present application collect in-vehicle device data and in-vehicle images in real time, perceive the passenger's eye position information and passenger size information based on the in-vehicle device data and in-vehicle images, and use the passenger's eye position information and passenger size information for the steering wheel adjustment control unit, so that the steering wheel adjustment control unit can realize personalized steering wheel intelligent adjustment for each user based on the passenger's eye position information and passenger size information.

[0211] Regarding the rearview mirror adjustment control unit and the HUD adjustment control unit, since the position of the human eye affects the passenger's use of the rearview mirror and HUD devices, in order to improve the adjustment effect of the rearview mirror and HUD devices and achieve personalized adaptive adjustment for each user, in the embodiment of the present application, the adjustment information of the rearview mirror adjustment control unit and the HUD adjustment control unit includes the passenger's human eye position information in the passenger characteristic information.

[0212] The process of obtaining the passenger's eye position information may refer to step S202 and will not be described in detail here.

[0213] It can be understood that after obtaining the passenger's eye position information, the passenger's eye position information can be sent to the rearview mirror adjustment control unit and the HUD adjustment control unit, so that the rearview mirror adjustment control unit can control and adjust the rearview mirror device according to the passenger's eye position information, and the HUD adjustment control unit can control and adjust the HUD device according to the passenger's eye position.

[0214] As an example, as shown in Figure 10, the vehicle information processing device also presets a mapping relationship between the eye position and the rearview mirror gear position, and the vehicle information processing device sends the passenger eye position information and the mapping relationship between the eye position and the rearview mirror gear position to the rearview mirror adjustment control unit.

[0215] After receiving the passenger's eye position information and the mapping relationship between the eye position and the rearview mirror gear position, the rearview mirror adjustment control unit can obtain the target gear position value of the rearview mirror device based on the passenger's eye position information and the mapping relationship between the eye position and the rearview mirror gear position. After obtaining the target gear position value of the rearview mirror device, the rearview mirror adjustment control unit can issue an adjustment instruction to the drive device of the rearview mirror device based on the target gear value, thereby controlling the drive device of the rearview mirror device to adjust the rearview mirror device based on the target gear value, thereby realizing personalized and intelligent adjustment of the rearview mirror device.

[0216] As an example, the mapping relationship between the human eye position and the rearview mirror gear position can also be stored in the vehicle's memory, and the rearview mirror adjustment control unit can also read the mapping relationship between the human eye position and the rearview mirror gear position from the vehicle.

[0217] As an example, the mapping relationship between the human eye position and the rearview mirror gear position can be obtained by fitting a large amount of human eye position information and rearview mirror gear position data.

[0218] The embodiment of the present application uses the perceived passenger eye position information and the mapping relationship between the eye position and the rearview mirror gear position in the rearview mirror adjustment control unit, thereby realizing personalized rearview mirror intelligent adjustment for each user.

[0219] As an example, as shown in Figure 11, the vehicle information processing device also presets a mapping relationship between the eye position and the HUD height level. The vehicle information processing device sends the passenger eye position information and the mapping relationship between the eye position and the HUD height level to the HUD adjustment control unit.

[0220] After receiving the passenger's eye position information and the mapping relationship between the eye position and the HUD height level, the HUD adjustment control unit can obtain the target height level value h of the HUD device based on the passenger's eye position information and the mapping relationship between the eye position and the HUD height level. After obtaining the target height level value h of the HUD device, the HUD adjustment control unit can issue an adjustment instruction to the HUD device's drive device based on the target height level value h, thereby controlling the HUD device's drive device to adjust the height of the HUD device based on the target height level value, thereby achieving personalized and intelligent adjustment of the HUD device.

[0221] As an example, the mapping relationship between the human eye position and the HUD height gear can also be stored in the vehicle's memory, and the HUD adjustment control unit can also read the mapping relationship between the human eye position and the HUD height gear from the vehicle.

[0222] As an example, the mapping relationship between the human eye position and the HUD height level can be obtained by fitting a large amount of human eye position information and HUD height level data.

[0223] The embodiment of the present application uses the perceived passenger eye position information and the mapping relationship between the eye position and the HUD height level in the HUD adjustment control unit, thereby realizing personalized HUD intelligent adjustment for each user.

[0224] The vehicle information processing method provided in the embodiment of the present application can obtain spatial information within the vehicle based on the detection data detected by the detection device within the vehicle, and send one or more pieces of spatial information within the vehicle to the control unit according to the control requirements of the control unit, so that the control unit can promptly perceive the environmental changes in the space within the vehicle, and control and adjust the control object as the environment within the vehicle changes, thereby improving the control effect of the control object.

[0225] Furthermore, the embodiments of the present application sense the in-vehicle spatial information and send corresponding adjustment information to the control unit based on this in-vehicle spatial information. This avoids repeated acquisition of the required adjustment information by different control units when adjusting the control objects, thus avoiding wasted computing power, saving resources, and improving efficiency. For example, the control and adjustment of the steering wheel device, rearview mirror device, and HUD device share the same eye position information and can be performed simultaneously. Therefore, repeated acquisition of eye position information can be avoided, saving computing power and improving processing efficiency.

[0226] To improve information accuracy and enhance the control unit's effectiveness in controlling in-vehicle devices, in this embodiment of the present application, after obtaining in-vehicle spatial information, the in-vehicle spatial information can also be verified for accuracy, thereby improving the accuracy of the in-vehicle spatial information. In this regard, please refer to Figure 12, which is a second flow chart of the vehicle information processing method provided in this embodiment of the present application. The method illustrated in Figure 12 is applied to the vehicle illustrated in Figure 1 and executed by the vehicle information processing device. As shown in Figure 12, in this embodiment of the present application, the vehicle information processing method can also include steps S301 to S305.

[0227] S301, obtaining vehicle equipment data and in-vehicle images.

[0228] S302 , obtaining in-vehicle space information based on the device data and the in-vehicle image, where the in-vehicle space information includes device location information of in-vehicle devices and passenger feature information inside the vehicle.

[0229] The process from step S301 to step S302 may refer to the process from step S201 to step S202 and will not be described in detail here.

[0230] S303: Verify the accuracy of the device location information and the passenger body part location information in the passenger characteristic information.

[0231] Among them, the device position information obtained based on the device data is the position information in the whole vehicle coordinate system, and the passenger part position information obtained based on the in-vehicle image is the position information in the camera coordinate system. In order to verify the accuracy of the device position information and the passenger part position information, and to facilitate the subsequent control unit to control and adjust the control object, in an embodiment of the present application, the device position information and the passenger part position information can be converted into a common coordinate system.

[0232] As an example, the common coordinate system can be a camera coordinate system or a vehicle coordinate system. In some examples, the common coordinate system can also be a coordinate system independent of the camera coordinate system and the vehicle coordinate system. The specific setting can be based on actual needs and is not limited by this application.

[0233] In order to facilitate the control unit to control and adjust the control object, in an embodiment of the present application, the device position information and the passenger part position information can be converted into the vehicle coordinate system, that is, the common coordinate system is the vehicle coordinate system.

[0234] Referring to Figure 13 , a calibration plate can be placed inside the vehicle and photographed with a camera inside the vehicle to obtain an image of the plate. Based on the pixel coordinates of the corner points on the calibration plate image and their coordinates in the vehicle coordinate system, the extrinsic parameters between the camera coordinate system and the vehicle coordinate system are obtained. For details, refer to conventional techniques and are not detailed here. Once the extrinsic parameters between the camera coordinate system and the vehicle coordinate system are obtained, they can be used to convert the passenger's position information in the camera coordinate system to the vehicle coordinate system, thereby obtaining the passenger's position information in the vehicle coordinate system.

[0235] It can be understood that the above is only an example of the embodiment of the present application for determining the external parameters between the camera coordinate system and the vehicle coordinate system, and does not serve as a limitation on the method of determining the external parameters between the camera coordinate system and the vehicle coordinate system. In some examples, the external parameters between the camera coordinate system and the vehicle coordinate system can also be obtained by other means, for example, they can also be read from the vehicle digital model.

[0236] After the passenger body position information and the in-vehicle device position information are converted into the same coordinate system, it is possible to detect whether the passenger body position information and the device position information are accurate in the same coordinate system.

[0237] For example, the accuracy of passenger body part position information and device position information can be checked based on the preset relative positional relationship between the passenger's body and the in-vehicle device. For example, a passenger's buttocks should be located above the seat cushion. If the detected buttocks position information is below the seat cushion position information, then the buttocks position information and / or the seat cushion position information are incorrect. For another example, the position information of a passenger's eyes, ears, etc. should be located in front of the seat headrest. If the detected position information of the passenger's eyes, ears, etc. is located behind the headrest position information, then the position information of the passenger's eyes, ears, etc. and / or the headrest position information are incorrect.

[0238] As an example, after converting the passenger body position information and the device position information to the same coordinate system, the relative position of the passenger's body and the in-vehicle device is obtained based on the passenger body position information and the device position information in the same coordinate system, and a test is performed to determine whether the relative position of the passenger's body and the in-vehicle device conforms to a preset relative position relationship, thereby detecting whether the passenger body position information and the device position information are accurate. If the relative position of the passenger's body and the device does not conform to the preset relative position relationship, then the passenger body position information and / or the device position information are incorrect. If the relative position of the passenger's body and the in-vehicle device conforms to the preset relative position relationship, then the passenger body position information and the device position information are accurate.

[0239] When it is detected that the passenger part location information and / or device location information is incorrect, the process can return to step S301 to step S303, i.e., reacquire the vehicle's device data and interior image, obtain the interior space information based on the reacquired device data and interior image, and perform accuracy verification on the reacquired device location information and passenger part location information.

[0240] When the detected passenger position information and device position information are accurate, step S304 and step S305 can be executed.

[0241] S304: Send first adjustment information to the first control unit according to the control requirements of the first control unit. The first adjustment information includes device location information of the in-vehicle device and / or passenger characteristic information. The first adjustment information is used by the first control unit to control and adjust the first control object.

[0242] S305, sending second adjustment information to the second control unit according to the control requirements of the second control unit, the second adjustment information including the device location information and / or passenger characteristic information of the in-vehicle device, the second adjustment information is the same as the first adjustment information, or the second adjustment information is different from the first adjustment information.

[0243] The processes of step S304 and step S305 may refer to step S203 and step S204, and are not described in detail here.

[0244] It is understood that when the first adjustment information includes the device location information and / or the passenger part location information in the passenger characteristic information, the device location information and / or the passenger part location information is verified for accuracy. Correspondingly, when the second adjustment information includes the device location information and / or the passenger part location information in the passenger characteristic information, the device location information and / or the passenger part location information is verified for accuracy.

[0245] The embodiment of the present application combines the relative position relationship between the passenger's body and the device to verify the accuracy of the device position information and the passenger's body position information, thereby improving the accuracy of the in-vehicle control information, and further ensuring the control effect of the control unit on the in-vehicle equipment, thereby improving the user's car experience.

[0246] The vehicle information processing method of the present embodiment proposes a spatial position sensing technology for in-vehicle interaction of people and devices. This technology includes spatial positioning of human body parts, estimation of passenger size information, spatial positioning of in-vehicle devices, and passenger position distribution. As shown in Figure 14, by acquiring an in-vehicle image and detecting it, the passenger part positions and passenger position distribution are obtained. By acquiring the adjustment data of the in-vehicle devices and their initial positions in the vehicle digital model, the device positions of the in-vehicle devices are obtained based on the motion model, adjustment data, and initial positions of the in-vehicle devices. By converting the device positions and passenger part positions of the in-vehicle devices into a common coordinate system, the passenger part positions and device positions in the common coordinate system are obtained. Passenger size information is estimated based on the passenger part positions and device positions in the common coordinate system to obtain passenger size information. By encapsulating the obtained information, such as passenger part positions, in-vehicle device positions, passenger size information, and passenger position distribution, into an interface, a control unit can subscribe to the required corresponding information through the subscription interface according to business needs. This can be flexibly used in different interactive application scenarios and control systems, providing a personalized intelligent adjustment experience for each user.

[0247] The embodiment of the present application can sense the spatial positions of passengers and equipment in the vehicle, so that the control unit can adapt to changes in the vehicle environment in a timely manner, achieve adaptive adjustment of the vehicle equipment, and improve the adjustment effect.

[0248] The embodiments of the present application can estimate passenger size information by spatially positioning passengers and in-vehicle equipment, and then achieve personalized intelligent adjustment for each user by controlling and adjusting based on passenger size information, passenger distribution information, passenger body position information, and equipment position information.

[0249] Based on the same inventive concept, an embodiment of the present application provides an electronic device, which includes a memory for storing computer program instructions; and a processor for executing computer program instructions to support the electronic device in implementing the functions or steps in the above-mentioned vehicle information processing method.

[0250] As an example, the processor may be a central processing unit (CPU) or other specific integrated circuit. The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In practical applications, electronic devices may also include multiple processors, and the processors may include one or more processor cores.

[0251] Memory is typically used to store executable program code for computer programs. Executable program code includes instructions, and the processor executes the instructions stored in memory to execute various functional applications. Memory includes a program storage area and a data storage area. The program storage area can store the operating system and at least one application required for a function, while the data storage area can store data generated during the use of the electronic device.

[0252] In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0253] It is understood that the structures listed above do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine or separate certain components, or arrange the components differently, and the components may be implemented in hardware, software, or a combination of software and hardware.

[0254] In addition, an embodiment of the present application also provides a vehicle, including a memory for storing computer program instructions; and a processor for executing computer program instructions to support the vehicle in implementing the functions or steps in the above-mentioned vehicle information processing method.

[0255] In addition, an embodiment of the present application also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by the processing circuit, the functions or steps in the above-mentioned vehicle information processing method are implemented.

[0256] In addition, an embodiment of the present application also provides a chip system, which includes a processing circuit and a storage medium, in which computer program instructions are stored; when the computer program instructions are executed by the processing circuit, the functions or steps in the above-mentioned vehicle information processing method are implemented.

[0257] In addition, an embodiment of the present application also provides a computer program product containing instructions. When the computer program product is run on a computer, it enables the computer to execute the functions or steps in the above-mentioned vehicle information processing method.

[0258] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working processes of the chip system, electronic device, computer-readable storage medium, computer program product containing instructions, and vehicle described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0259] It is understandable that the steps of the method or algorithm described in conjunction with the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory, a flash memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, a read-only optical disc, or any other form of storage medium. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). In addition, the ASIC can be located in an electronic device. Of course, the processor and the storage medium can also be present in an electronic device as discrete components.

[0260] In an optional manner, when software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is implemented in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disk (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)).

[0261] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A vehicle information processing method, characterized in that, The method includes: Obtaining device data of the vehicle and in-vehicle images; Based on the device data and the in-vehicle images, obtaining in-vehicle space information, where the in-vehicle space information includes device position information of in-vehicle devices and passenger characteristic information; According to the control requirements of the first control unit, sending first adjustment information to the first control unit, where the first adjustment information includes the device position information of the in-vehicle devices and / or the passenger characteristic information; Wherein, the first adjustment information is used for the first control unit to control and adjust a first control object.

2. The method according to claim 1, characterized in that, The method further includes: According to the control requirements of the second control unit, sending second adjustment information to the second control unit, where the second adjustment information includes the device position information of the in-vehicle devices and / or the passenger characteristic information; Wherein, the second adjustment information is the same as the first adjustment information, or the second adjustment information is different from the first adjustment information.

3. The method according to claim 1 or 2, characterized in that, The obtaining of the in-vehicle space information based on the device data and the in-vehicle images includes: Based on the current adjustment data, initial adjustment data of the in-vehicle devices in the device data and the motion model of the in-vehicle devices, obtaining position change data of the in-vehicle devices, where the motion model characterizes the motion law of the in-vehicle devices under the change of adjustment data; Based on the position change data of the in-vehicle devices and the initial position information of the in-vehicle devices, obtaining the device position information of the in-vehicle devices; Performing object detection on the in-vehicle images to obtain the passenger characteristic information.

4. The method according to claim 3, wherein The performing of object detection on the in-vehicle images to obtain the passenger characteristic information includes: Performing object detection on the in-vehicle images to obtain passenger part position information and passenger distribution information; Based on the passenger part position information and the device position information, obtaining passenger height information; Based on the passenger height information, obtaining passenger size information; The passenger characteristic information includes at least one of the passenger part position information, the passenger distribution information, and the passenger size information.

5. The method according to any one of claims 1 to 4, characterized in that When the first control unit is an audio control unit, the first adjustment information includes at least one of the device position information of the playback device in the in-vehicle devices, the device position information of the seat device, the passenger distribution information, and the ear position information of the passenger in the passenger part position information.

6. The method according to any one of claims 1-4, characterized in that When the first control unit is a seat adjustment control unit, the first adjustment information includes at least one of the device position information of the seat device in the in-vehicle devices, the passenger size information, and the eye position information of the passenger in the passenger part position information.

7. The method according to any one of claims 1 to 4, characterized in that, When the first control unit is a steering wheel adjustment control unit, the first adjustment information includes at least one of the passenger size information and the eye position information of the passenger in the passenger part position information.

8. The method according to any one of claims 1 to 4, characterized in that, When the first control unit is a rearview mirror adjustment control unit or a head-up display adjustment control unit, the first adjustment information includes at least one of the eye position information of the passenger in the passenger part position information.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: Receiving a subscription request from the first control unit through a subscription interface; Obtain the control requirements of the first control unit according to the subscription request.

10. The method according to any one of claims 1-8, characterized in that, The sending of the first adjustment information to the first control unit includes: When receiving a call request from the first control unit, send the first adjustment information to the first control unit according to the control requirements in the adjustment request.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: Perform accuracy verification on the passenger part position information in the device position information and the passenger feature information according to the relative position relationship between the passenger part and the device.

12. A vehicle information processing device, characterized in that, The vehicle information processing device includes: An acquisition module, configured to acquire device data of the vehicle and in-vehicle images; An information perception module, configured to obtain in-vehicle space information according to the device data and the in-vehicle images, where the in-vehicle space information includes device position information of in-vehicle devices and passenger feature information; An information sending module, configured to send first adjustment information to the first control unit according to the control requirements of the first control unit, where the first adjustment information includes the device position information of the in-vehicle devices and / or the passenger feature information; Wherein, the first adjustment information is used for the first control unit to perform control adjustment on a first control object.

13. The vehicle information processing device according to claim 12, wherein The information sending module is further configured to: Send second adjustment information to the second control unit according to the control requirements of the second control unit, where the second adjustment information includes the device position information of the in-vehicle devices and / or the passenger feature information; Wherein, the second adjustment information is the same as the first adjustment information, or the second adjustment information is different from the first adjustment information.

14. The vehicle information processing device according to claim 13, characterized in that, The information perception module is configured to: Obtain position change data of the in-vehicle device according to the current adjustment data, initial adjustment data of the in-vehicle device in the device data and the motion model of the in-vehicle device, where the motion model represents the motion law of the in-vehicle device under the change of adjustment data; Obtain the device position information of the in-vehicle device according to the position change data of the in-vehicle device and the initial position information of the in-vehicle device; Perform target detection on the in-vehicle image to obtain the passenger feature information.

15. The vehicle information processing device according to claim 14, wherein Specifically, the information perception module is configured to: Perform target detection on the in-vehicle image to obtain passenger part position information and passenger distribution information; Obtain passenger height information according to the passenger part position information and the device position information; ​ ​ 16. The vehicle information processing device according to any one of claims 12-15, characterized in that, ​ 17. The vehicle information processing device according to any one of claims 12-15, characterized in that, ​ 18. The vehicle information processing device according to any one of claims 12-15, characterized in that, When the first control unit is a steering wheel adjustment control unit, the first adjustment information includes at least one of passenger size information and passenger eye position information among the passenger part position information.

19. The vehicle information processing device according to any one of claims 12-15, characterized in that, When the first control unit is a rearview mirror adjustment control unit or a head-up display adjustment control unit, the first adjustment information includes at least one of passenger eye position information among the passenger part position information.

20. An electronic device, characterized in that, Comprising: A memory for storing computer program instructions; A processor for executing the computer program instructions to support the electronic device in implementing the method according to any one of claims 1-11.

21. A vehicle, characterized in that, Comprising: A memory for storing computer program instructions; A processor for executing the computer program instructions to support the vehicle in implementing the method according to any one of claims 1-11.

22. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processing circuit, the method according to any one of claims 1-11 is implemented.

23. A computer program product comprising instructions, characterized in that, When the computer program product runs on a computer, the computer is caused to execute the method according to any one of claims 1-11.

Images