Vehicle light control method and device, vehicle end control equipment and readable storage medium

Abstract

The invention relates to a vehicle light control method and device, vehicle end control equipment and a readable storage medium, and relates to the technical field of vehicles. The method comprises the following steps: acquiring a lip image of each user in a vehicle under the condition that light control voice for the vehicle is received; determining a user identifier initiating the light control voice based on the light control voice and each lip image; based on the user identifier, a light control strategy corresponding to the light control voice is determined, and light control of the vehicle is carried out through the light control strategy; therefore, personalized light control on the vehicle based on the identity of the speaker is realized.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle lighting control method, device, vehicle-side control equipment, and readable storage medium. Background Technology

[0002] In related technologies, voice control systems in vehicle smart cockpits generally suffer from a "user-indiscriminate" problem when controlling vehicle lights. Specifically, when different users use the same voice command (such as "turn on ambient lights"), the system only executes a uniform default operation, failing to recognize the speaker's identity or associate it with their personalized lighting preferences, operating permissions, or effective control range. This problem may lead to non-drivers using voice control commands for vehicle lights, affecting the driver's driving experience and even driving safety.

[0003] Therefore, there is an urgent need for a personalized lighting control solution that can be based on the speaker's identity. Summary of the Invention

[0004] Therefore, it is necessary to provide a vehicle lighting control method, device, vehicle-side control equipment, computer-readable storage medium, and computer program product that can achieve corresponding personalized lighting control based on different user identities to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a vehicle lighting control method, the method comprising:

[0006] Upon receiving a voice command to control the vehicle's lights, acquire lip images of each user inside the vehicle.

[0007] Based on the light control voice and each of the lip images, determine the user identifier that initiated the light control voice;

[0008] Based on the user identifier, the lighting control strategy corresponding to the lighting control voice is determined, and the vehicle's lighting is controlled through the lighting control strategy.

[0009] In one embodiment, determining the lighting control strategy corresponding to the lighting control voice based on the user identifier includes:

[0010] Based on the user identifier, at least one of the following is determined: pre-adapted lighting control preference information, lighting control permission information, and lighting control operation scope;

[0011] Based on at least one of the lighting control preference information, the lighting control permission information, and the lighting control operation range, the lighting control strategy corresponding to the lighting control voice is determined.

[0012] In one embodiment, determining the lighting control strategy corresponding to the lighting control voice based on the user identifier includes:

[0013] The system acquires the lighting area information included in the lighting control voice command and determines the target lighting area for the vehicle based on the lighting area information.

[0014] The target lighting component to be controlled is determined based on the target lighting area;

[0015] Based on the user identifier, a lighting control strategy is determined for the target lighting component.

[0016] In one embodiment, determining the target lighting area for the vehicle based on the lighting area information includes:

[0017] The information obtained regarding the lighting area includes the text of the lighting reference object and the text of the lighting location;

[0018] Based on the lighting reference text, the lighting location text, and the preset vehicle coordinate system, the target lighting area for the vehicle is determined.

[0019] In one embodiment, determining the target lighting area for the vehicle based on the lighting area information includes:

[0020] Based on the lighting area information and the preset lighting angle quantization strategy, the target lighting area for the vehicle is determined.

[0021] In one embodiment, the method further includes at least one of the following:

[0022] When the vehicle is in parking standby mode, activate the monitoring of voice commands for vehicle lighting control;

[0023] When the vehicle is in ignition-off mode, monitoring of the vehicle's lighting control voice commands is activated based on preset listening keywords.

[0024] In one embodiment, determining the user identifier that initiated the light control voice based on the light control voice and each of the lip images includes:

[0025] The valid lighting control voice is obtained from the lighting control voice, and feature extraction processing is performed on the valid lighting control voice to obtain the lighting control voice features;

[0026] Feature extraction processing is performed on each of the lip images to obtain the corresponding lip image features;

[0027] Based on the cross-validation processing of the light control voice features and each of the lip image features, the user identifier that initiated the light control voice is determined.

[0028] Secondly, this application also provides a vehicle lighting control device, the device comprising:

[0029] The data acquisition module is used to acquire lip images of each user inside the vehicle when a voice command for vehicle lighting control is received.

[0030] The user identification module is used to determine the user identifier who initiated the light control voice based on the light control voice and each of the lip images;

[0031] The lighting control module is used to determine the lighting control strategy corresponding to the lighting control voice based on the user identifier, and to control the vehicle's lighting through the lighting control strategy.

[0032] Thirdly, this application also provides a vehicle-side control device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the first aspect.

[0033] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the first aspect.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the first aspect.

[0035] The vehicle lighting control method, apparatus, vehicle-side control device, computer-readable storage medium, and computer program product provided in this application, wherein the vehicle lighting control method, upon receiving a voice command for vehicle lighting control, indicates that a user wishes to control at least one light on the vehicle, and further acquires lip images of each user inside the vehicle; based on the analysis and processing of the lighting control voice command and each lip image, the user identifier of the specific user initiating the lighting control voice command is determined; based on the user identifier, the lighting control strategy corresponding to the lighting control voice command is determined, so as to determine a lighting control strategy suitable for different user types, and then control the vehicle lighting through the determined lighting control strategy. In this way, real-time user identification through voice and lip images is achieved, and their exclusive lighting configuration strategy is dynamically loaded. While ensuring vehicle safety, this enables a personalized intelligent lighting interaction experience, which is beneficial to improving the level of cockpit intelligence and user experience. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a diagram illustrating the application environment of a vehicle lighting control method in one embodiment.

[0038] Figure 2 This is a flowchart illustrating a vehicle lighting control method in one embodiment;

[0039] Figure 3 This is a flowchart illustrating the process of determining a lighting control strategy in one embodiment;

[0040] Figure 4 This is a structural block diagram of a vehicle lighting control device in one embodiment;

[0041] Figure 5 This is an internal structural diagram of the vehicle-side control device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0044] The vehicle lighting control method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers. Terminal 102 can be a vehicle, or an architectural component such as a cockpit domain controller inside the vehicle. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0045] In one exemplary embodiment, such as Figure 2 As shown, a vehicle lighting control method is provided. Taking the application of this method to a cockpit domain controller in a vehicle as an example, the method includes the following steps 201 to 203. Wherein:

[0046] Step 201: Upon receiving a voice command to control the vehicle's lights, acquire lip images of each user inside the vehicle.

[0047] Among them, vehicles can be any passenger car, commercial vehicle, or special vehicle; passenger cars include sedans, SUVs, etc.; commercial vehicles include city buses, long-distance buses, trucks, etc.; special vehicles include fire trucks, ambulances, police cars, emergency rescue vehicles, etc.

[0048] Users inside the vehicle may include the driver and other passengers.

[0049] Among them, lighting control voice is the voice output by the user inside the vehicle through their mouth; lighting control voice can be voice containing explicit preset words for controlling the interior lights, or it can be voice containing the intention to control the interior lights. However, lighting control voice does not include audio emitted from electronic devices.

[0050] The user's lip images can be captured by visual sensors such as cameras inside the vehicle. Specifically, they are dynamic images or video sequences of the lip area of ​​a person who is speaking. These images not only include the static shape of the lips, but more importantly, they record the temporal changes in lip shape, opening and closing, and movement trajectory during the pronunciation process.

[0051] For example, the user's lip image can be extracted from the acquired user's facial image, or it can be obtained based on the directly acquired lip image.

[0052] For example, the cockpit domain controller continuously monitors voice commands inside the vehicle that control certain vehicle components. When it detects voice commands from users inside the vehicle to control the vehicle lights, it further acquires lip images of each user inside the vehicle during the time the voice command is sent. This allows for subsequent fusion analysis of the voice commands and lip images to improve the accuracy of the determined lighting control strategy.

[0053] Step 202: Based on the light control voice and each lip image, determine the user identifier that initiated the light control voice.

[0054] Among them, the user identifier of the vehicle occupant (user) can be used to identify the type of vehicle occupant; the type of vehicle occupant may include, for example, driver type, resident passenger type (e.g., family member), priority passenger type, ordinary passenger type, child type, infant type, etc.

[0055] The identification of user identifiers for each occupant inside the vehicle can be achieved through at least one of the following methods: facial recognition, voiceprint recognition, and iris recognition.

[0056] For example, when the cockpit domain controller acquires the lighting control voice inside the vehicle and the lip images of each vehicle occupant during the relevant time period, it can determine the initiator (speaker) of the lighting control voice by fusing the lighting control voice and each lip image, and further identify the user identifier of the user who initiated the lighting control voice.

[0057] In this method, the initiator of the light control voice is identified by using both the voice commands for controlling the lights inside the vehicle and the lip images of each user. This is equivalent to using at least facial recognition and voiceprint recognition technologies to identify the speaker. This multi-factor approach to identifying users helps improve the accuracy of identifying the voice initiator.

[0058] It should also be noted that the vehicle may detect that the voice commands for controlling the lights are being made by people outside the vehicle. By integrating the lip images of the users inside the vehicle, it is possible to filter out situations where the voice commands for controlling the lights are not made by the occupants of the vehicle, thus preventing non-occupants from controlling the vehicle lights.

[0059] Additionally, if the vehicle needs to respond to voice commands for lighting control from users outside the vehicle, an external camera can be used to capture images of the user's lips to confirm whether the user who sent the voice command has the authority to control the vehicle's lights.

[0060] Step 203: Based on the user identifier, determine the lighting control strategy corresponding to the lighting control voice, and control the vehicle's lighting through the lighting control strategy.

[0061] For each type of user identifier, a corresponding lighting control strategy can be pre-set for the same or similar lighting control voice commands. The lighting control strategy may involve differences in the number of lighting components that can be controlled by different user identifiers, differences in specific components, and differences in lighting parameters. The lighting parameters may involve the light output direction, light output brightness, light output color, light output effect, and light output duration of the lighting components.

[0062] For example, when the cockpit domain controller determines the user identifier of the occupant who initiates the lighting control voice, it can determine the lighting control strategy matching the lighting control voice based on the user identifier and the control content included in the lighting control voice. Then, it can control the illumination of relevant lighting components in the vehicle based on the determined lighting control strategy to illuminate the area of ​​the vehicle interior that the user wants to illuminate.

[0063] The vehicle lighting control method provided in this application, upon receiving a voice command requesting vehicle lighting control, indicates that a user wishes to control at least one light on the vehicle. It further acquires lip images of each user inside the vehicle. Based on the analysis and processing of the voice command and lip images, it determines the user identifier of the specific user initiating the voice command. Based on the user identifier, it determines the corresponding lighting control strategy for the voice command, thus enabling the determination of a suitable lighting control strategy for different user types. The vehicle lighting is then controlled according to the determined lighting control strategy. This achieves real-time user identification through voice and lip images, and dynamically loads a personalized lighting configuration strategy for each user. While ensuring vehicle safety, it provides a personalized intelligent lighting interaction experience, which is beneficial for improving the level of cabin intelligence and user experience.

[0064] In one exemplary embodiment, reference may be made to Figure 3 The above steps, based on the user identifier, determine the lighting control strategy corresponding to the lighting control voice, including steps 301 and 302, wherein:

[0065] Step 301: Determine at least one of the pre-adapted lighting control preference information, lighting control permission information, and lighting control operation scope based on the user identifier;

[0066] Step 302: Determine the lighting control strategy corresponding to the lighting control voice based on at least one of the lighting control preference information, lighting control permission information, and lighting control operation range.

[0067] The lighting control preference information can be pre-set for different user identifiers, meaning that different user identifiers can have corresponding pre-set lighting control preference information, or multiple user identifiers can have at least two corresponding pre-set lighting control preference information. For example, the lighting control preference information may be a preference for light output parameters such as the color temperature, brightness, and color of the light output component.

[0068] Correspondingly, the lighting control permission information can be pre-set for different user identifiers. That is, different user identifiers can have corresponding pre-set lighting control permission information, or multiple user identifiers can have at least two corresponding pre-set lighting control permission information. For example, the lighting control permission information may be settings related to whether certain lighting components can be controlled, or settings related to whether the lighting parameters of certain lighting components can be controlled, etc.

[0069] Correspondingly, the lighting control operation range can be pre-set for different user identifiers, that is, different user identifiers can have corresponding pre-set lighting control operation ranges, or multiple user identifiers can have corresponding at least two pre-set lighting control operation ranges. For example, the lighting control operation range is the range of adjustment for the light output parameters of a certain lighting component, such as the light output color temperature, light output brightness, and light output color.

[0070] As shown in Table 1 below, different user levels are equivalent to different user identifiers. Users with different user identifiers have at least partial differences in their control over the brightness condition range, angle control permissions, and special functions of the lighting components. Among them, the brightness condition range is included in the above-mentioned lighting control operation range, the angle control permissions are included in the above-mentioned lighting control permission information, and the special functions are included in the above-mentioned lighting control preference information.

[0071] Table 1

[0072]

[0073] The values ​​related to "calibration" in Table 1 may vary depending on the vehicle model, and each vehicle will be provided based on the calibration results before leaving the factory. The lighting control preference information (special functions) can be adjusted by the vehicle user (e.g., the owner) according to the needs of different vehicle occupants.

[0074] It should also be added that when a child's voiceprint (high frequency > 280Hz) is detected, the system can be pre-set to automatically restrict some control permissions of the in-vehicle lighting components, preventing excessive operation of the in-vehicle lights by children from affecting vehicle safety. For example, the upper limit for brightness adjustment of the in-vehicle lighting components by child occupants can be set to 60%, and angle adjustment can be prohibited.

[0075] For example, when the cockpit domain controller determines the user identifier of the vehicle occupant who initiated the lighting control voice, it can further obtain the pre-matched lighting control preference information, lighting control permission information, and lighting control operation range of the lighting components based on the user identifier. Then, based on at least one of the obtained lighting control preference information, lighting control permission information, and lighting control operation range, it determines the lighting control strategy corresponding to the relevant lighting control voice.

[0076] In this embodiment, user identifiers are used to acquire pre-configured lighting control preferences, permissions, and operation ranges related to the lighting components. Based on these parameters, a final lighting control strategy adapted to the lighting control voice is determined. This allows vehicle occupants with different user identifiers to output the same or similar lighting control voice commands, enabling adaptation to different lighting control strategies. It also ensures that the determined lighting control strategy closely matches the intent of the user issuing the relevant voice commands, thus improving the user experience of the in-vehicle lighting. Furthermore, real-time user identification via voice and lip-reading is achieved, dynamically loading their personalized lighting configuration strategies. This ensures vehicle safety while providing a personalized intelligent lighting interaction experience, enhancing the level of cockpit intelligence and user experience.

[0077] In an exemplary embodiment, the above steps of determining the lighting control strategy corresponding to the lighting control voice based on the user identifier include: obtaining lighting area information included in the lighting control voice, and determining the target lighting area for the vehicle based on the lighting area information; determining the target lighting component to be controlled based on the target lighting area; and determining the lighting control strategy for the target lighting component based on the user identifier.

[0078] Among them, the lighting control voice is a voice command used to control the vehicle lights. It is not just a simple command, but can contain multiple structured information elements. For example, the lighting control voice may include a lighting control wake-up word (keyword or phrase), information about the lighting area to be illuminated, etc.

[0079] For example, the wake-up word for lighting control is, for instance, the wake-up word for the vehicle's global voice assistant, such as "Xiao Mou"; the lighting area information is, for instance, words that can characterize a specific area inside the vehicle, such as "passenger seat" or "armrest box".

[0080] Different target lighting areas can be illuminated using different lighting components, thus different target lighting areas can be equipped with different target lighting components.

[0081] For example, in determining the lighting control strategy adapted to the lighting control voice, the cockpit domain controller first identifies and analyzes the monitored lighting control voice to determine the lighting area information included therein. Then, based on the lighting area information, it determines the corresponding target lighting area to be illuminated in the vehicle. Further, it can determine the target lighting component to be controlled based on the target lighting area. The determined target lighting component is used to illuminate the target lighting area. Finally, it can further combine the user identifier of the output lighting control voice to determine the pre-set lighting control strategy adapted to the target lighting component.

[0082] In this embodiment, the target lighting component used to illuminate the relevant target lighting area is first determined by the lighting area information included in the lighting control voice, thereby determining the range of the target lighting component to be controlled. Then, the lighting control strategy adapted to the target lighting component is determined based on the user identifier that issued the lighting control voice, which helps to ensure the accuracy of the determined lighting control strategy.

[0083] Furthermore, this application decouples voice recognition and hardware control logic through a step-by-step execution method of "first parsing the lighting area information in the voice, then determining the target lighting area based on the lighting area information, and then mapping the target lighting area to specific lighting components," which helps to improve the control accuracy of in-vehicle lighting components and avoid misoperation.

[0084] In an exemplary embodiment, the above steps of determining the target lighting area for a vehicle based on lighting area information include: obtaining lighting reference text and lighting position text included in the lighting area information; and determining the target lighting area for the vehicle based on the lighting reference text, the lighting position text, and a preset vehicle coordinate system.

[0085] Among them, the lighting reference text and the lighting position text can be determined based on text recognition of the lighting area information; for example, if the lighting area information in the light control voice includes the text "illuminate the ground to the left of the driver's seat", the lighting reference text is "driver's seat" and the lighting position text is "ground to the left".

[0086] Among them, the pre-set vehicle coordinate system is a set of three-dimensional reference coordinates that is predefined and fixed during the vehicle design or software system initialization stage to describe the spatial position inside the vehicle. It is the basis for the spatial positioning and control of systems such as intelligent cockpit, sensor fusion, and human-machine interaction.

[0087] For example, when the cockpit domain controller determines the lighting area information corresponding to the lighting control voice, it can further perform semantic analysis on the lighting area information to obtain the lighting reference text and lighting position text included in the lighting area information. Then, in combination with the vehicle's preset on-board coordinate system, it determines the position parameters in the on-board coordinate system corresponding to the lighting reference text and lighting position text, thereby determining the position parameters corresponding to the target lighting area inside the vehicle to be illuminated corresponding to the lighting control voice.

[0088] In this embodiment, the position coordinates (position parameters) corresponding to the lighting reference text and lighting position text included in the lighting area information are determined by a preset vehicle coordinate system. This allows the position coordinates of the target lighting area inside the vehicle to be illuminated to be determined in the vehicle coordinate system. This facilitates the subsequent determination of the target light component to be turned on, improves the accuracy of the target light component determination, and enhances the matching degree between the vehicle interior area illuminated by the target light component and the area to be illuminated by the user.

[0089] The lighting components that can be controlled by this application can be any lighting component inside the vehicle, or they can be specially designed lighting components (such as follow spots). For example, they can be lighting components specially designed for locations inside the vehicle, such as the roof area or the side area of ​​the vehicle body. These lighting components can be evenly distributed inside the vehicle, or they can be distributed according to requirements. Any light-emitting element in these lighting components can be adapted to a light emission direction control element. For example, for a passenger car with two rows of seats, four, five, or six lighting components can be evenly distributed in the roof area of ​​the first and second rows, or five lighting components can be set up in accordance with the seating arrangement. This application does not limit the specific number or location of the lighting components inside the vehicle that can be controlled by the vehicle lighting control method provided in this application.

[0090] One embodiment is that after receiving voice input for vehicle lighting control, the cockpit domain controller performs dynamic spatial relationship calculation. For example, if the user command is "illuminate the ground to the left of the driver's seat", the spatial vector of "ground to the left" needs to be identified to establish a three-dimensional vehicle coordinate system (such as taking the origin as the position of the follow spot); the follow spot is also the target lighting component.

[0091] The coordinate system definition may include: X-axis: front of vehicle → rear of vehicle; Y-axis: driver's side → passenger side; Z-axis: roof of vehicle → ground.

[0092] The transformation from semantics to coordinates may involve instruction parsing pseudocode;

[0093] Taking an instruction containing "left side" as an example, the relevant instruction parsing pseudocode is as follows:

[0094] If "left side" is in the instruction:

[0095] y_offset = -1.5 # Default lateral offset on the driver's side

[0096] If the instruction contains "back row", the relevant instruction parsing pseudocode is as follows:

[0097] If "back row" is in the instruction:

[0098] y_offset *= 0.7 # Back row space scaling factor

[0099] If the instruction contains "ground", the relevant instruction parsing pseudocode is as follows:

[0100] if "ground" in instruction:

[0101] z_target = 0 # Ground height

[0102] angle_vertical = arctan(light height / z_distance) # Automatically calculate the tilt angle

[0103] The content following "#" is a comment.

[0104] In an exemplary embodiment, when the cockpit domain controller performs semantic analysis on the received voice commands for vehicle lighting control, it may also involve adaptive quantization of fuzzy quantifiers. For example, if the user command is "turn the angle a little to the right", there may be a situation where the calibration step size does not meet the user's expectations. A dynamic quantization strategy is provided, as shown in Table 2, which involves the relationship between fuzzy quantifiers, base values, influencing factors and final value calculation.

[0105] Table 2

[0106]

[0107] Therefore, in an exemplary embodiment, the above steps of determining the target lighting area for the vehicle based on lighting area information may include: determining the target lighting area for the vehicle based on lighting area information and a preset lighting angle quantization strategy.

[0108] For example, when the cockpit domain controller analyzes and recognizes the voice commands for lighting control to obtain lighting area information, if the lighting area information includes fuzzy quantifiers, that is, words with unclear targeting, it can determine the target lighting area pointed to by the obtained fuzzy quantifiers based on the correlation between the pre-set fuzzy quantifiers and the lighting angle quantification strategy.

[0109] In this embodiment, by establishing the correlation between pre-set fuzzy quantifiers and lighting angle quantification strategies, the target lighting area pointed to by the fuzzy quantifiers included in the lighting area information is determined. This helps improve the vehicle's ability to analyze the received lighting control voice commands and enhances the user experience.

[0110] Another embodiment is as follows: for example, if a user commands "light here" (without a specific direction), multi-source localization fusion processing can be used to analyze the target area inside the vehicle that the user wants to illuminate. The data involved in multi-source localization fusion may include, for example, the user's gesture skeletal tracking information, gaze focus information, the center of the heat map distribution of occupants inside the vehicle, lip images, and other data.

[0111] In an exemplary embodiment, the lighting area information may simultaneously include lighting reference text, lighting location text, and fuzzy quantifiers. In this case, the target lighting area pointed to by the lighting area information can be determined based on the correlation between the lighting reference text, lighting location text, a preset vehicle coordinate system, and a preset fuzzy quantifier and lighting angle quantification strategy.

[0112] In an exemplary embodiment, the vehicle lighting control method provided in this application further includes: when the vehicle is in a parking standby mode, activating the monitoring of vehicle lighting control voice commands.

[0113] Among them, the parking standby mode refers to a vehicle usage state in which the vehicle is stationary and the driver has finished driving.

[0114] For example, the cockpit domain controller can be configured to only activate the monitoring of vehicle lighting control voice when the vehicle is detected to be in parking standby mode, so that the vehicle lighting control method provided in this application is only used in parking standby mode.

[0115] In this embodiment, by setting the vehicle lighting control method provided in this application to be used only in the vehicle parking standby mode, it can be ensured that the lighting control performed by the vehicle lighting control method provided in this application will not affect the vehicle's driving process, thus ensuring the safety of lighting use while the vehicle is in motion.

[0116] Specifically, the parking standby mode can be defined as: the vehicle is completely stopped + the power system is off (or in standby) + the driver has no intention of driving; in traditional fuel vehicles, this usually corresponds to "P gear + engine off"; in new energy / intelligent vehicles, it may be manifested as "P gear + release the brake pedal + no accelerator pedal pressed", even if the high voltage system is not completely de-energized, it is considered to be entering the parking mode.

[0117] In an exemplary embodiment, the vehicle lighting control method provided in this application further includes: when the vehicle is in an off mode, monitoring of the vehicle's lighting control voice based on preset listening keywords.

[0118] The preset monitoring keywords can be pre-set core words for light control, or they can include wake-up words and words corresponding to the information of the lighting area to be illuminated.

[0119] For example, the cockpit domain controller can be configured to automatically activate the directional microphone when the vehicle is unlocked or the door is slightly ajar (not fully open) to prepare to receive relevant voice commands for lighting control, thus avoiding accidental wake-up. Furthermore, after the vehicle is turned off, i.e. in the off-state mode, the voice module controlling the vehicle enters keyword monitoring mode, i.e., the monitoring function for vehicle lighting control voice commands is enabled. For example, in this mode, only preset core words such as "lighting" are responded to.

[0120] Furthermore, in the keyword monitoring mode, the full-function module can be woken up a second time by the door vibration sensor. When the full-function module is in the open state, the relevant technical solutions of the vehicle lighting control method provided in this application can be fully implemented.

[0121] In this embodiment, since the vehicle enters a low-power sleep state after the engine is turned off, continuous operation of high-computing-power full-time voice recognition would significantly increase the static current (potentially reaching hundreds of milliamperes), and long-term parking could easily lead to low-voltage battery depletion. Therefore, this application controls the monitoring of vehicle lighting control voice by preset listening keywords when the vehicle is in the off mode, without enabling the full-function module for vehicle lighting control, which helps to significantly reduce system power consumption and avoid battery depletion.

[0122] In one exemplary embodiment, based on the light control voice and each lip image, the user identifier that initiated the light control voice is determined, including:

[0123] The valid lighting control voice is obtained from the lighting control voice, and feature extraction processing is performed on the valid lighting control voice to obtain the lighting control voice features;

[0124] Feature extraction processing was performed on each lip image to obtain the corresponding lip image features;

[0125] Based on cross-validation of the lighting control voice features and various lip image features, the user identifier that initiated the lighting control voice is determined.

[0126] Among them, effective lighting control speech includes, for example, any of the above-mentioned lighting reference text, lighting location text, and fuzzy quantifiers; the effective lighting control speech does not involve other speech content, such as wake words, prepositions, or modal particles.

[0127] For example, when the cockpit domain controller receives a voice command for vehicle lighting control, it can first filter the voice command to remove voice content unrelated to lighting control and retain only valid voice commands. Then, the cockpit domain controller can perform feature extraction processing on the filtered valid voice commands to obtain the voice command features. At the same time, it can perform feature extraction processing on the acquired lip images of each user to obtain the corresponding lip image features. Afterwards, the user who initiated the voice command and the user identifier of that user can be determined by cross-validation processing of the voice command features and the lip image features.

[0128] In this embodiment, by extracting features from the effective light control voice and lip images in the light control voice, and then by cross-validating the light control voice features and the lip image features, the user who initiated the light control voice and the user's identifier are determined. This achieves accurate identification of the voice output person by synchronizing the time of lip movements with the voice signal, and also avoids accidental activation of light control by non-command voices such as background conversations, broadcasts, or children playing.

[0129] Furthermore, the vehicle lighting control method system of this application integrates voiceprint recognition technology to distinguish different users and automatically associate preset preferences, application permissions, and operating scope based on user identity. Specifically:

[0130] For example, three 5-second segments of valid speech from a user (e.g., "Turn on my lights") are collected, and the following features are extracted: MFCC (Mel-Frequency Cepstral Coefficients): used to analyze the spectral characteristics of the speech; fundamental frequency trajectory: used to capture the changing patterns of vocal cord vibration frequency; formant structure: used to identify vocal tract resonance characteristics; dynamic features: used to record the changes of the above features over time; simultaneously, the user's lip micro-movements (such as opening and closing amplitude, movement frequency) are monitored in real time using millimeter-wave radar, and the speech features and lip movement features are combined for cross-validation to effectively reject recording attacks;

[0131] Among them, the fundamental frequency trajectory is used to perform time-frequency analysis on each frame of speech signal and calculate the candidate value of the fundamental frequency;

[0132] Example, autocorrelation method for fundamental frequency detection:

[0133] autocorrelation = np.correlate(frame, frame, mode='full') # Calculate the autocorrelation of a frame.

[0134] peaks = find_peaks(autocorrelation[len(frame) / / 2:]) # Find the main period peaks from the latter half of the autocorrelation result

[0135] F0 = sample_rate / (peaks[0] + len(frame) / / 2) # Calculate the fundamental frequency, which is equal to the sampling rate divided by the period length, where the period length is the peak position found from the autocorrelation result plus half the frame length.

[0136] It may also involve median filtering to eliminate abnormal fluctuations and filtering to retain trend characteristics for smoothing.

[0137] Pseudocode example, dynamic feature extraction of fundamental frequency trajectory:

[0138] f0 = extract_f0(audio) # Extract the original fundamental frequency sequence

[0139] f0_smoothed = median_filter(f0, window=5) # Median filtering

[0140] delta_f0 = np.gradient(f0_smoothed) # First-order difference (rate of change)

[0141] delta2_f0 = np.gradient(delta_f0) # Second-order difference (changing acceleration)

[0142] Stability using first-order / second-order difference quantization (ΔF0 = rate of change, Δ²F0 = acceleration):

[0143] stability_score = 1 / (1 + np.abs(delta_f0) + 0.5*np.abs(delta2_f0))

[0144] Selecting the optimal base frequency based on comprehensive analysis:

[0145] best_f0_index = np.argmax(stability_score)final_f0 = valid_f0[best_f0_index]

[0146] Resonant structure: the acoustic resonance characteristics used to describe the shape of the vocal tract;

[0147] Furthermore, a Hamming window can be added to each frame of signal to reduce spectral leakage, and the spectrum can be obtained through Fourier transform, with the power spectrum obtained by squaring the spectrum. Taking the "open" vowel / a / segment as an example, LPC (Linear Predictive Coding) extraction is performed. The stable segment of the vowel / a / is extracted, with the Hamming window: w(n) = 0.54 - 0.46 * cos(2πn / (N-1)) (where N is the frame length). Autocorrelation calculation: Where N represents the total number of signal samples or the total length, and R(k) represents the similarity between signal x(m) and its own signal x(m+k) k time units later; the order p=12, simulating the audio channel; the LPC coefficients are output by solving the Yule-Walker equation through the Levinson-Durbin recursive method: {a1,a2,…a12}; the LPC spectrum is calculated. ω is the angular frequency, a k Here, e^{-jωk} represents the LPC coefficients, and e^{-jωk} represents the complex exponent on the unit circle. The peak values ​​of the spectral envelope are detected to locate the formant frequencies. The formant frequencies of continuous speech frames are time-aligned to form a dynamic trajectory.

[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0149] Based on the same inventive concept, this application also provides a vehicle lighting control device for implementing the vehicle lighting control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more vehicle lighting control device embodiments provided below can be found in the limitations of the vehicle lighting control method described above, and will not be repeated here.

[0150] In one exemplary embodiment, such as Figure 4As shown, a vehicle lighting control device 400 is provided, including: a data acquisition module 41, a user confirmation module 42, and a lighting control module 43, wherein:

[0151] The data acquisition module 41 is used to acquire the lip images of each user inside the vehicle when a voice command for controlling the vehicle's lights is received.

[0152] User identification module 42 is used to identify the user who initiated the light control voice based on the light control voice and each lip image;

[0153] The lighting control module 43 is used to determine the lighting control strategy corresponding to the lighting control voice based on the user identifier, and to control the vehicle's lighting through the lighting control strategy.

[0154] In an exemplary embodiment, the lighting control module 43 is used to determine the lighting control strategy corresponding to the lighting control voice based on the user identifier, specifically: determining at least one of the pre-adapted lighting control preference information, lighting control permission information, and lighting control operation range based on the user identifier; and determining the lighting control strategy corresponding to the lighting control voice based on at least one of the lighting control preference information, lighting control permission information, and lighting control operation range.

[0155] In an exemplary embodiment, the lighting control module 43 is used to determine the lighting control strategy corresponding to the lighting control voice based on the user identifier. Specifically, it is used to: obtain the lighting area information included in the lighting control voice, and determine the target lighting area for the vehicle based on the lighting area information; determine the target lighting component to be controlled based on the target lighting area; and determine the lighting control strategy for the target lighting component based on the user identifier.

[0156] In an exemplary embodiment, the lighting control module 43 is used to determine the target lighting area for the vehicle based on the lighting area information, specifically: acquiring the lighting reference text and lighting position text included in the lighting area information; and determining the target lighting area for the vehicle based on the lighting reference text, the lighting position text, and a preset vehicle coordinate system.

[0157] In an exemplary embodiment, the lighting control module 43 is used to determine the target lighting area for the vehicle based on the lighting area information, specifically: determining the target lighting area for the vehicle based on the lighting area information and a preset lighting angle quantization strategy.

[0158] In an exemplary embodiment, the data acquisition module 41 is further configured to: activate the monitoring of the vehicle's lighting control voice commands when the vehicle is in a parking standby mode.

[0159] In an exemplary embodiment, the data acquisition module 41 is further configured to: when the vehicle is in an off mode, activate the monitoring of the vehicle's lighting control voice based on preset listening keywords.

[0160] In an exemplary embodiment, the user identification module 42 is used to determine the user identifier who initiated the light control voice based on the light control voice and each lip image. Specifically, it is used to: obtain valid light control voice in the light control voice, and perform feature extraction processing on the valid light control voice to obtain light control voice features; perform feature extraction processing on each lip image to obtain corresponding lip image features; and determine the user identifier who initiated the light control voice based on the cross-validation processing of the light control voice features and each lip image features.

[0161] Each module in the aforementioned vehicle lighting control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device (specifically, a vehicle-side control device), or stored in the memory of the computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0162] In one exemplary embodiment, such as Figure 5 The illustration shows a vehicle-side control device including a processor and a memory. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium storing a computer program. When executed by the processor, the computer program implements a vehicle lighting control method.

[0163] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment (specifically, vehicle-side control equipment) on which the present application is applied. The specific computer equipment may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] In one exemplary embodiment, a vehicle-side control device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement steps related to a vehicle lighting control method.

[0165] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements steps related to a vehicle lighting control method.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements steps related to a vehicle lighting control method.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A vehicle lighting control method, characterized in that, The method includes: Upon receiving a voice command to control the vehicle's lights, acquire lip images of each user inside the vehicle. Based on the light control voice and each of the lip images, determine the user identifier that initiated the light control voice; Based on the user identifier, the lighting control strategy corresponding to the lighting control voice is determined, and the vehicle's lighting is controlled through the lighting control strategy.

2. The method according to claim 1, characterized in that, The step of determining the lighting control strategy corresponding to the lighting control voice based on the user identifier includes: Based on the user identifier, at least one of the following is determined: pre-adapted lighting control preference information, lighting control permission information, and lighting control operation scope; Based on at least one of the lighting control preference information, the lighting control permission information, and the lighting control operation range, the lighting control strategy corresponding to the lighting control voice is determined.

3. The method according to claim 1, characterized in that, The step of determining the lighting control strategy corresponding to the lighting control voice based on the user identifier includes: The system acquires the lighting area information included in the lighting control voice command and determines the target lighting area for the vehicle based on the lighting area information. The target lighting component to be controlled is determined based on the target lighting area; Based on the user identifier, a lighting control strategy is determined for the target lighting component.

4. The method according to claim 3, characterized in that, Determining the target lighting area for the vehicle based on the lighting area information includes: The information obtained regarding the lighting area includes the text of the lighting reference object and the text of the lighting location; Based on the lighting reference text, the lighting location text, and the preset vehicle coordinate system, the target lighting area for the vehicle is determined.

5. The method according to claim 3, characterized in that, Determining the target lighting area for the vehicle based on the lighting area information includes: Based on the lighting area information and the preset lighting angle quantization strategy, the target lighting area for the vehicle is determined.

6. The method according to claim 1, characterized in that, The method further includes at least one of the following: When the vehicle is in parking standby mode, activate the monitoring of voice commands for vehicle lighting control; When the vehicle is in ignition-off mode, monitoring of the vehicle's lighting control voice commands is activated based on preset listening keywords.

7. The method according to any one of claims 1-6, characterized in that, The step of determining the user identifier who initiated the light control voice based on the light control voice and each of the lip images includes: The valid lighting control voice is obtained from the lighting control voice, and feature extraction processing is performed on the valid lighting control voice to obtain the lighting control voice features; Feature extraction processing is performed on each of the lip images to obtain the corresponding lip image features; Based on the cross-validation processing of the light control voice features and each of the lip image features, the user identifier that initiated the light control voice is determined.

8. A vehicle lighting control device, characterized in that, The device includes: The data acquisition module is used to acquire lip images of each user inside the vehicle when a voice command for vehicle lighting control is received. The user identification module is used to determine the user identifier who initiated the light control voice based on the light control voice and each of the lip images; The lighting control module is used to determine the lighting control strategy corresponding to the lighting control voice based on the user identifier, and to control the vehicle's lighting through the lighting control strategy.

9. A vehicle-end control device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-7.

Images