Gaze-adaptive luminescence adjustment method for automobile interactive components, computer device, and storage medium

Abstract

The present invention discloses a gaze-adaptive illumination adjustment method for automobile interactive components, a computer device, and a storage medium. The present invention can achieve real-time tracking of a vehicle occupant's line of sight and control the illumination adjustment of a target automobile interactive component in response. This allows the occupant to trigger illumination adjustment of the target automobile interactive component by gazing at the target automobile interactive component. This ensures that the illumination parameters of the target automobile interactive component meet the occupant's visual requirements, allowing the occupant to observe and operate the target automobile interactive component in an appropriate lighting environment. This helps ensure that the occupant can accurately and efficiently control the automobile interactive component, reduces the risk of distraction caused by incorrect or repeated operation of the automobile interactive component, and ensures traffic safety. The present invention has broad application in the automotive technology field.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a gaze-adaptive luminescence adjustment method for automobile interactive components, a computer device, and a storage medium. Background Art

[0002] While driving a car, users often need to use interactive components to control functional features. Due to the airtight nature of the vehicle cabin, the lighting inside is often insufficient, making it difficult for passengers to identify and accurately access the interactive components. While passengers can manually adjust the lighting parameters of the interactive components or turn on interior lighting to improve the lighting conditions, this requires the passenger to distract themselves, making operation inconvenient and even dangerous. Summary of the Invention

[0003] In view of the technical problems in current automobile technology such as the difficulty in identifying and using automobile interactive components, the purpose of the present invention is to provide a gaze-adaptive automobile interactive component lighting adjustment method, computer device and storage medium.

[0004] In one aspect, an embodiment of the present invention includes a gaze-adaptive method for adjusting the luminescence of an automobile interactive component, the gaze-adaptive method comprising the following steps:

[0005] Acquire images of people in the vehicle;

[0006] Extracting features from the image of the person in the vehicle to obtain sight features;

[0007] determining a gaze area according to the sight line characteristics;

[0008] determining a target vehicle interactive component according to the gaze area;

[0009] The target automobile interactive component is illuminated and adjusted.

[0010] Furthermore, the feature extraction of the image of the person on the vehicle to obtain the sight line feature includes:

[0011] Run the first feature extraction network and the second feature extraction network;

[0012] Inputting the image of the person on the vehicle into the first feature extraction network for processing, and obtaining background-related features output by the first feature extraction network;

[0013] Inputting the image of the person on the vehicle into the second feature extraction network for processing, and obtaining the sight-related features output by the second feature extraction network;

[0014] The sight line feature is determined according to the background related feature and the sight line related feature.

[0015] Furthermore, determining the sight line feature according to the background-related feature and the sight line-related feature includes:

[0016] Eliminating the sight-line related features from the image of the person on the vehicle to obtain an image without sight-line related features;

[0017] Obtaining background features by intersecting the background-related features and the line-of-sight-removed feature image;

[0018] The background feature is eliminated from the image of the person on the vehicle to obtain the sight line feature.

[0019] Furthermore, the gaze-adaptive automobile interactive component luminescence adjustment method further includes:

[0020] Acquire a first training image and a second training image; the first training image corresponds to a marked sight feature, and the second training image corresponds to a marked background feature;

[0021] Alternatingly performing multiple rounds of first adversarial training processes and second adversarial training processes on the first feature extraction network and the second feature extraction network;

[0022] In the first adversarial training process, the first training image is input into the first feature extraction network for processing, a first feature output by the first feature extraction network is obtained, the first feature is eliminated from the first training image to obtain an image without the first feature, the image without the first feature is input into the second feature extraction network for processing, a first loss function value is determined based on an output result of the second feature extraction network and the marked line of sight feature, and parameters of the first feature extraction network and / or the second feature extraction network are adjusted based on the first loss function value;

[0023] In the second adversarial training process, the second training image is input into the second feature extraction network for processing, the second feature output by the second feature extraction network is obtained, the second feature is eliminated from the second training image, and the second feature-removed image is obtained. The second feature-removed image is input into the first feature extraction network for processing, and the second loss function value is determined based on the output result of the first feature extraction network and the marked background feature. The parameters of the first feature extraction network and / or the second feature extraction network are adjusted according to the second loss function value.

[0024] Furthermore, determining the gaze area according to the sight line feature includes:

[0025] Running the third feature extraction network and the fourth feature extraction network;

[0026] Inputting the sight line feature into the third feature extraction network for processing, and obtaining a left eye sight line vector output by the third feature extraction network;

[0027] Inputting the sight line feature into the fourth feature extraction network for processing, and obtaining a right eye sight line vector output by the fourth feature extraction network;

[0028] An intersection area is determined according to the left eye sight line vector and the right eye sight line vector, and the intersection area is used as the gaze area.

[0029] Furthermore, determining the gaze area according to the sight line feature includes:

[0030] Run the fifth feature extraction network;

[0031] Inputting the sight line feature into the fifth feature extraction network for processing, and obtaining a monocular sight line vector output by the fifth feature extraction network;

[0032] The spatial area through which the monocular sight line vector passes is used as the gaze area.

[0033] Furthermore, determining a target vehicle interactive component according to the gaze area includes:

[0034] Determining candidate vehicle interactive components according to the gaze area; the candidate vehicle interactive components are vehicle interactive components located within the gaze area;

[0035] When the maintaining existence time of the gaze area is greater than a time threshold, the candidate vehicle interactive component is determined as the target vehicle interactive component.

[0036] Furthermore, the step of adjusting the luminescence of the target automobile interactive component includes:

[0037] When the target vehicle interactive component is a self-luminous component, controlling the target vehicle interactive component to adjust its brightness;

[0038] When the target vehicle interactive component is a non-self-luminous component, a lighting component is determined according to the target vehicle interactive component, and the lighting component is controlled to adjust the brightness; wherein the target vehicle interactive component is located within the lighting range of the lighting component.

[0039] On the other hand, an embodiment of the present invention also includes a computer device including a memory and a processor, the memory being used to store at least one program, and the processor being used to load at least one program to execute the gaze-adaptive automotive interactive component luminescence adjustment method of the embodiment.

[0040] On the other hand, an embodiment of the present invention further includes a computer-readable storage medium storing a program executable by a processor. When the program is executed by the processor, it is used to execute the gaze-adaptive automobile interactive component lighting adjustment method in the embodiment.

[0041] The beneficial effects of the present invention are as follows: through the gaze-adaptive automobile interactive component lighting adjustment method in the embodiment, real-time tracking of the line of sight of the people on the vehicle can be achieved, and the target automobile interactive component can be controlled to adjust the lighting in response, so that the people on the vehicle can trigger the lighting adjustment of the target automobile interactive component by looking at the target automobile interactive component, so that the lighting parameters of the target automobile interactive component can meet the visual requirements of the people on the vehicle, and the people on the vehicle can observe and operate the target automobile interactive component in a suitable lighting environment, which is conducive to ensuring that the people on the vehicle can accurately and efficiently control the automobile interactive components, reduce the risks faced by distraction caused by incorrect operation of the automobile interactive components or repeated operation of the automobile interactive components, and ensure traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the structure of a vehicle-mounted system to which the gaze-adaptive method for adjusting the light emission of an automobile interactive component can be applied in an embodiment;

[0043] Figure 2 Schematic diagram of the steps of the gaze-adaptive automobile interactive component luminescence adjustment method in an embodiment;

[0044] Figure 3 Schematic diagram of the structure of the first feature extraction network and the second feature extraction network in the embodiment;

[0045] Figure 4 Schematic diagram of the principles of the first adversarial training process and the second adversarial training process in the embodiment;

[0046] Figure 5 Schematic diagram of the principle of the first feature extraction network and the second feature extraction network in the embodiment;

[0047] Figure 6 Schematic diagram of the structure of the third feature extraction network and the fourth feature extraction network in the embodiment. DETAILED DESCRIPTION

[0048] In this embodiment, the gaze-adaptive luminescence adjustment method for automobile interactive components can be applied to Figure 1 In the vehicle system shown.

[0049] Reference Figure 1The vehicle-mounted system to which the gaze-adaptive illumination adjustment method for vehicle interactive components can be applied includes a control module, a camera, vehicle interactive components, and lighting components. The control module is a component with data acquisition, data processing, data output, and control functions, and can specifically be an electronic control unit (ECU). The camera captures the interior of the cabin, thereby capturing information such as the facial expressions of the occupants. The vehicle interactive components are components in the vehicle with interactive functions, which can be two-way human-machine interaction or one-way command or information transmission from human to machine or from machine to human. Specifically, they can include a central control screen that implements an intelligent two-way interactive interface, a head-up display (HUD) system that electronically displays information, instrumentation or ambient lighting, touch buttons that electronically receive control commands, mechanical knobs or gear levers that receive control commands, and mechanical handles or pedals that transmit control actions.

[0050] Reference Figure 1 , the car interaction components may be self-luminous or non-self-luminous. For example, the car interaction components such as the central control screen, head-up display system, instrument or ambient light are luminous when working, and therefore belong to the self-luminous car interaction components in this embodiment; the car interaction components such as touch buttons, knobs, gear levers, handles or pedals are not luminous when working, and therefore belong to the non-self-luminous car interaction components in this embodiment. The luminous parameters (including brightness, color, flashing frequency, etc.) of the self-luminous car interaction components are adjustable, and the control module can send control instructions to the self-luminous car interaction components to adjust the luminous parameters of the self-luminous car interaction components; the non-self-luminous car interaction components themselves do not emit light, refer to Figure 1 , non-self-luminous automobile interactive components can be equipped with lighting components to illuminate the non-self-luminous automobile interactive components. The lighting components can specifically be indicator lights or light strips installed on the automobile interactive components, or components such as lighting lamps installed on the automobile roof and other locations. The lighting components installed on the automobile interactive components can directly illuminate the automobile interactive components when they are illuminated, and the lighting lamps have a certain lighting range, and the corresponding automobile interactive components are located within the lighting range; the control module can send control instructions to the lighting components equipped with the non-self-luminous automobile interactive components, thereby adjusting the lighting parameters of the lighting components to achieve adjustment of the lighting parameters of the non-self-luminous automobile interactive components.

[0051] In this embodiment, refer to Figure 2 The method for adjusting the luminous intensity of an automobile interactive component based on gaze adaptation includes the following steps:

[0052] S1. Obtain an image of the person in the vehicle;

[0053] S2. Extract features from the image of the person in the vehicle to obtain sight features;

[0054] S3. Determine the gaze area based on the sight line characteristics;

[0055] S4. Determine the target vehicle interactive component based on the gaze area;

[0056] S5. Adjust the luminescence of the target vehicle interactive component.

[0057] In this embodiment, each step of the gaze-adaptive vehicle interactive component illumination adjustment method, including steps S1-S5, can be executed by the control module. When the control module needs to acquire certain data or perform certain operations while executing the gaze-adaptive vehicle interactive component illumination adjustment method, the control module can send instructions to the corresponding vehicle component, thereby invoking the vehicle component to collect data or perform the operation.

[0058] In step S1, the control module uses a camera to capture an image of a person on board. The image can be a head shot of a person on board, such as the driver or a passenger, captured by the camera. The image can be one or more still images or a dynamic video. The camera typically captures a person on board who has high control over the vehicle, such as the driver. The camera then transmits the captured image to the control module.

[0059] In this embodiment, a still image captured of the driver is used as an example of the image of people in the vehicle for description.

[0060] In step S2, the control module can run an artificial intelligence model to extract features from the occupant image to obtain sightline features. Sightline features represent characteristic information contained in the occupant image that can reflect the direction of the occupant's (driver's) eyes.

[0061] In step S3, since the sight line feature obtained in step S2 can determine the driver's sight line direction, the spatial position where the driver's eyes are looking when the image of the person in the car is being taken, that is, the gaze area, can be determined based on the sight line feature.

[0062] In step S4, the spatial coordinates of each vehicle interactive component on the vehicle can be recorded in advance through calibration experiments or other methods. Based on the spatial coordinates, the positional relationship between each vehicle interactive component and the gaze area determined in step S3 can be determined. In this embodiment, if the spatial coordinates of a vehicle interactive component are within the gaze area, then when executing step S4, this vehicle interactive component is determined as the target vehicle interactive component, that is, the vehicle interactive component to be illuminated.

[0063] In step S5, the target vehicle interactive component determined in step S4 is illuminated so that the illumination of the target vehicle interactive component can meet the visual needs of the occupant (driver) of the vehicle and respond to the gaze action of the occupant (driver).

[0064] In this embodiment, steps S1-S5 are executed in a dynamic loop, that is, each time after steps S1-S5 are completed, the next step S1-S5 is executed. If the performance of the control module is sufficient, it takes a short time to execute steps S1-S5. Therefore, executing steps S1-S5 can achieve real-time tracking of the line of sight of the occupants and control the target vehicle interactive component to adjust the light in response, so that the occupants can trigger the light adjustment of the target vehicle interactive component by looking at the action of the target vehicle interactive component, so that the light parameters of the target vehicle interactive component can meet the visual requirements of the occupants, so that the occupants can observe and operate the target vehicle interactive component in a suitable lighting environment, thereby ensuring that the occupants can accurately and efficiently control the vehicle interactive component, reduce the dangers faced by the occupants due to incorrect operation of the vehicle interactive component or repeated operation of the vehicle interactive component causing distraction, and ensure traffic safety.

[0065] In this embodiment, when the control module executes step S2, it can run the first feature extraction network and the second feature extraction network to extract features from the image of the person on the vehicle to obtain the sight line features. Specifically, convolutional neural networks can be used as the first feature extraction network and the second feature extraction network. The first feature extraction network and the second feature extraction network both have Figure 3 The structure shown.

[0066] In this embodiment, the control module can locate the positions of the eyes in the image of the person on the vehicle, take the center of the line connecting the positions of the eyes as the position of the eyebrow center, take the position of the eyebrow center as the origin, take the line connecting the positions of the eyes as the first coordinate axis, take a straight line located in the plane where the image of the person on the vehicle is located, passing through the origin and perpendicular to the first coordinate axis as the second coordinate axis, and take a straight line perpendicular to the plane where the image of the person on the vehicle is located and passing through the origin as the third coordinate axis, thereby establishing a unified coordinate system, in which the coordinates of feature points in the image or spatial positions in the car cabin are represented.

[0067] In this embodiment, the following steps may be performed to train the first feature extraction network and the second feature extraction network:

[0068] P1. Obtain a first training image and a second training image;

[0069] P2. Alternately perform multiple rounds of the first adversarial training process and the second adversarial training process on the first feature extraction network and the second feature extraction network.

[0070] The first training image corresponds to a labeled sight feature, and the second training image corresponds to a labeled background feature;

[0071] In step P1, multiple training images can be obtained. The content of each training image is a facial image of a person in a car cabin, including the person's eye image information. Among them, some training images are first training images, each of which has a corresponding marked sight line feature. The marked sight line feature can be obtained by manual marking or other methods. The marked sight line feature represents sight line features such as the sight line direction corresponding to the eye image information of the person in the first training image; another part of the training images are second training images, each of which has a corresponding marked background feature. The marked background feature can be obtained by manual marking or other methods. The marked background feature represents features in the second training image other than the sight line feature. Specifically, it can include features such as background objects in the car cabin, shooting light, skin texture of the person, and facial expression of the person.

[0072] In this embodiment, when executing step P1, multiple training images can be obtained, each training image has a corresponding marked line of sight feature and a marked background feature, and each training image can be used as both a first training image and a second training image; when executing step P2, the training image used to mark the line of sight feature is the first training image, and the training image used to mark the background feature is the second training image.

[0073] In step P2, multiple rounds of the first adversarial training process and the second adversarial training process are alternately performed on the first feature extraction network and the second feature extraction network. For example, one round of the first adversarial training process is first performed, followed by one round of the second adversarial training process, then another round of the first adversarial training process, then another round of the second adversarial training process, and so on, until a stopping condition such as the total number of rounds reaching a threshold is met. The total number of rounds of the first adversarial training process and the total number of rounds of the second adversarial training process can be equal or different.

[0074] In this embodiment, a first round of adversarial training process and a subsequent second round of adversarial training process in step P2 are taken as an example for description.

[0075] The principle of the first adversarial training process is as follows Figure 4 As shown in part (a) of . During step P2, a first training image and its corresponding labeled gaze feature are obtained. The first training image is input into a first feature extraction network for processing, and a first feature is obtained as output by the first feature extraction network. The first feature is a feature extracted from the first training image that is related to the background portion of the first training image.

[0076] Reference Figure 4In part (a), after obtaining the first feature, the first feature is eliminated from the first training image to obtain a first-feature-removed image. Specifically, the first feature can be subtracted from the first training image to obtain the first-feature-removed image, i.e., the first-feature-removed image lacks the first feature relative to the first training image.

[0077] Reference Figure 4 In part (a), after obtaining the first feature-removed image, the first feature-removed image is fed into the second feature extraction network for processing, obtaining the output of the second feature extraction network. The output of the second feature extraction network is the features extracted from the first feature-removed image that are related to the line of sight of the first training image.

[0078] Reference Figure 4 In part (a), a first loss function value is calculated based on the output of the second feature extraction network and the marked line of sight features using a loss function such as L2. Then, based on the first loss function value, parameters of at least one of the first and second feature extraction networks are adjusted using a backpropagation algorithm, thereby completing the parameter update of the first and / or second feature extraction networks in this round of first adversarial training.

[0079] The principle of the second adversarial training process is as follows Figure 4 As shown in part (b) of the figure, during step P2, a second training image and its corresponding labeled background features are obtained. The second training image is input into a second feature extraction network for processing, and a second feature is obtained as output by the second feature extraction network. The second feature is a feature extracted from the second training image and is related to the line of sight of the second training image.

[0080] Reference Figure 4 In part (b), after obtaining the second feature, the second feature is eliminated from the second training image to obtain a second feature-removed image. Specifically, the second feature can be subtracted from the second training image to obtain the second feature-removed image, i.e., the second feature-removed image lacks the second feature relative to the second training image.

[0081] Reference Figure 4 In part (b), after obtaining the image without the second feature, the image without the second feature is input into the first feature extraction network for processing to obtain the output of the first feature extraction network. The output of the first feature extraction network is the features extracted from the image without the second feature that are related to the background portion of the second training image.

[0082] Reference Figure 4In part (b), a second loss function value is calculated based on the output of the first feature extraction network and the labeled background features using a loss function such as L2. Then, based on the second loss function value, parameters of at least one of the first and second feature extraction networks are adjusted using a backpropagation algorithm, thereby completing the parameter update of the first and / or second feature extraction networks in this round of first adversarial training.

[0083] After the second adversarial training process is completed, if the stopping condition is not met, the next round of the first adversarial training process can be performed, using a new pair of first training images and their corresponding labeled gaze features for training; after the next round of the first adversarial training process is completed, if the stopping condition is not met, the next round of the second adversarial training process can be performed, using a new pair of second training images and their corresponding labeled background features for training...

[0084] In this embodiment, the principle of executing steps P1-P2 is that: during the first adversarial training process, the first feature extraction network is used to process the first training image to obtain a first feature-removed image, and the performance of the first feature extraction network is used to eliminate at least a portion of the features related to the background part, and then the second feature extraction network is used to extract features related to the line of sight part from the first feature-removed image, that is, during the first adversarial training process, the first feature extraction network is equivalent to a generator, which can generate a first feature-removed image for the second feature extraction network to train the line of sight feature extraction performance, and the second feature extraction network is equivalent to a discriminator, which can extract features related to the line of sight part from the first feature-removed image; during the second adversarial training process, the second feature extraction network is used to process the second training image to obtain a second feature-removed image. For example, the performance of the second feature extraction network is used to eliminate at least a part of the features related to the line of sight part, and then the first feature extraction network is used to extract features related to the background part from the second feature-removed image, that is, in the second adversarial training process, the second feature extraction network is equivalent to a generator, which can generate a first feature-removed image for the second feature extraction network to train the line of sight feature extraction performance, and the first feature extraction network is equivalent to a discriminator, which can extract features related to the background part from the second feature-removed image; through the alternating execution of the first adversarial training process and the second adversarial training process, the generative adversarial training of the first feature extraction network and the second feature extraction network can be realized, thereby training the first feature extraction network with background feature extraction performance and the second feature extraction network with line of sight feature extraction performance.

[0085] In this embodiment, steps P1-P2 may be performed before step S2. After performing steps P1-P2 to train and obtain the first feature extraction network and the second feature extraction network, step S2 may be performed.

[0086] In this embodiment, when the control module executes step S2, that is, extracting features from the image of the person on the vehicle to obtain the sight line features, the control module may specifically execute the following steps:

[0087] S201. Run the first feature extraction network and the second feature extraction network;

[0088] S202. Inputting the image of the person on the vehicle into the first feature extraction network for processing, obtaining the background-related features output by the first feature extraction network;

[0089] S203. Input the image of the person on the vehicle into the second feature extraction network for processing, and obtain the sight-related features output by the second feature extraction network;

[0090] S204. Determine sight line features based on background related features and sight line related features.

[0091] The principles of steps S201-S204 are as follows: Figure 5 shown.

[0092] In step S201, the control module may run the first feature extraction network and the second feature extraction network trained in steps P1-P2.

[0093] In step S202, the control module takes the image of the person on the vehicle obtained in step S1 and Input to the first feature extraction network for processing, and use the background feature extraction performance of the first feature extraction network to extract the image of the person in the car Background-related features in .

[0094] In step S203, the control module takes the image of the person on the vehicle obtained in step S1 and The input is sent to the second feature extraction network for processing, and the line of sight feature extraction performance of the second feature extraction network is used to extract the image of the person in the car. Gaze-related features in .

[0095] In step S204, the control module uses the background related features obtained in step S202 to and the sight-related features obtained in step S203 , determine the sight line characteristics .

[0096] In this embodiment, when the control module executes step S204, that is, the step of determining the sight line feature based on the background-related features and the sight line-related features, the control module may specifically execute the following steps:

[0097] S20401. Eliminate sight-related features from the vehicle image to obtain a sight-related feature-free image;

[0098] S20402. Obtain background features by intersecting the background-related features and the image without sight-related features;

[0099] S20403. Eliminate background features from the image of the person in the vehicle and obtain sight features.

[0100] In step S20401, the image of the people in the car All features contained in as the full set, from the image of the person in the car Eliminate sight-related features After that, the obtained de-lineation related feature image is equivalent to .

[0101] In step S20402, background related features and the image without sight-related features Perform intersection to obtain background features ,Right now =

[0102] In step S20403, the image of the people on the vehicle is used All features contained in as the full set, from the image of the person in the car Eliminate background features Then, the sight line features are obtained ,Right now .

[0103] In this embodiment, the principle of executing steps S20401-S20403 is: the image of the person on the vehicle Can be regarded as background features and sight characteristics The first feature extraction network has the background feature extraction performance, but due to the errors or deviations in training, the background related features directly extracted by the first feature extraction network are May contain only background features Most of them, including sight features Similarly, the second feature extraction network directly extracts the sight-related features May contain only line-of-sight features Most of the, also including background features A small part of the; According to the set principle, we can know the line of sight characteristics satisfy , can be calculated by executing steps S20401-S20403 , thereby obtaining the line of sight features Therefore, executing steps S20401-S20403 can reduce the influence of actual factors such as training errors or deviations, so that the final result obtained is close to or equal to the line of sight feature .

[0104] In this embodiment, the sight line feature Specifically, it can be represented as an image of people in the car The coordinates of multiple key points in the human eye.

[0105] In this embodiment, when the control module executes step S3, that is, the step of determining the gaze area according to the sight line characteristics, the control module may specifically execute the following steps:

[0106] S301A runs the third feature extraction network and the fourth feature extraction network;

[0107] S302A. Input the sight feature into the third feature extraction network for processing, and obtain the left eye sight vector output by the third feature extraction network;

[0108] S303A. Input the sight feature into the fourth feature extraction network for processing, and obtain the right eye sight vector output by the fourth feature extraction network;

[0109] S304A. Determine an intersection area based on the left eye sight line vector and the right eye sight line vector, and use the intersection area as the gaze area.

[0110] Steps S301A-S304A are the first execution mode of step S3.

[0111] In this embodiment, when the control module executes step S301A, a convolutional neural network can be used as the third feature extraction network and the fourth feature extraction network. The third feature extraction network and the fourth feature extraction network both have Figure 6 The structure shown.

[0112] In this embodiment, the third feature extraction network and the fourth feature extraction network are both trained networks, wherein the third feature extraction network has the performance of extracting the left eye sight vector based on the sight feature (all or part of the left eye), and the fourth feature extraction network has the performance of extracting the right eye sight vector based on the sight feature (all or part of the right eye).

[0113] Specifically, in steps S302A-S303A, the left eye sight vector represents the sight feature The corresponding left eye sight direction of the person in the car, and the right eye sight vector represents the sight feature The corresponding direction of sight of the right eye of the person in the car.

[0114] In step S304A, the control module uses a vector algorithm to calculate the intersection point (extended line) based on the left eye sight vector and the right eye sight vector, and determines a space with a certain radius extending from the intersection point as the center as the intersection area. This intersection area is the gaze area of ​​the person in the car.

[0115] By executing steps S301A-S304A, it is possible to The determined binocular sight line vector is used to calculate the area that the person in the car is looking at.

[0116] In this embodiment, when the control module executes step S3, that is, the step of determining the gaze area according to the sight line characteristics, the control module may specifically execute the following steps:

[0117] S301B runs the fifth feature extraction network;

[0118] S302B inputs the sight feature into the fifth feature extraction network for processing, obtaining the monocular sight vector output by the fifth feature extraction network;

[0119] S303B. The spatial area passed by the monocular sight line vector is regarded as the gaze area.

[0120] Steps S301B-S304B are a second execution method of step S3.

[0121] In step S301B, the fifth feature extraction network used can be one of the third feature extraction network and the fourth feature extraction network. The three features are input into the third feature extraction network and the fourth feature extraction network respectively. The third feature extraction network cannot extract the line of sight features. Processing, and the fourth feature extraction network can be used to extract the line of sight features For processing, the fourth feature extraction network is used as the fifth feature extraction network in step S301B, and the right eye sight vector output by the fourth feature extraction network is used as the monocular sight vector in step S302B.

[0122] In step S303B, a spatial area is determined with a certain radius extending from the point on the monocular sight line vector obtained in step S302B as the center. This spatial area is the gaze area of ​​the occupants of the vehicle.

[0123] By executing steps S301B-S303B, when a person on board has monocular vision impairment or both eyes in the image of the person on board are unclear, the gaze area can be determined based on the recognition of the monocular eye, thereby expanding the scope of application of the gaze-adaptive automobile interactive component lighting adjustment method.

[0124] In this embodiment, when the control module executes step S4, that is, the step of determining the target vehicle interactive component according to the gaze area, the control module may specifically execute the following steps:

[0125] S401. Determine candidate vehicle interaction components based on the gaze area;

[0126] S402: When the maintenance time of the gaze area is greater than the time threshold, the candidate vehicle interactive component is determined as the target vehicle interactive component.

[0127] In step S401 , the control module may detect, in a unified coordinate system, which vehicle interaction components have position coordinates within the gaze area obtained in step S3 , and determine those vehicle interaction components within the gaze area as candidate vehicle interaction components.

[0128] Because steps S1-S5 are executed in a dynamic loop, the duration of the gaze area can be calculated. Specifically, the duration of the gaze area's coordinates remains unchanged (or changes by less than a threshold). If the duration of the gaze area is less than the threshold, step S401 is executed again. If the duration of the gaze area is greater than the threshold, step S402 is executed, and the candidate vehicle interactive component obtained in step S401 is identified as the target vehicle interactive component for illumination adjustment in step S5.

[0129] In this embodiment, by executing steps S401-S402, the vehicle interactive components that are located in the gaze area of ​​the vehicle occupants and are gazed at by the vehicle occupants for a long time can be detected and determined as target vehicle interactive components, so that further light adjustment can be performed, which can effectively respond to the gaze actions of the vehicle occupants and reduce the possibility of false triggering.

[0130] In this embodiment, when the control module executes step S5, that is, the step of adjusting the light emission of the target vehicle interactive component, it can specifically perform the following steps:

[0131] S501. When the target vehicle interactive component is a self-luminous component, the target vehicle interactive component is controlled to adjust the brightness;

[0132] S502. When the target vehicle interactive component is a non-self-luminous component, determine the lighting component according to the target vehicle interactive component, and control the lighting component to adjust the brightness.

[0133] If the target vehicle interactive component determined by executing step S4 is a self-luminous component such as a central control screen or an ambient light, the control module sends a brightness adjustment instruction to the target vehicle interactive component, thereby controlling the target vehicle interactive component itself to adjust its brightness; if the target vehicle interactive component determined by executing step S4 is a non-self-luminous component such as a knob or a handle, the control module sends a brightness adjustment instruction to the lighting component that matches the non-target vehicle interactive component, thereby controlling the lighting component to adjust its brightness, and the brightness adjustment effect is projected onto the target vehicle interactive component.

[0134] Specifically, when executing steps S501-S502 and controlling the target vehicle interactive component or lighting component to adjust the brightness, the brightness can be uniformly increased. For example, when performing brightness adjustment, the target vehicle interactive component or lighting component can be controlled to increase the brightness by 30% based on the original brightness. If the brightness increase of 30% exceeds the brightness upper limit, the brightness can be adjusted to the brightness upper limit. Since steps S1-S5 are dynamically executed multiple times, if the gaze area detected by the last execution of steps S1-S5 changes relative to the gaze area detected by the previous execution of steps S1-S5, the brightness adjustment performed by the last execution of steps S1-S5 can be canceled, for example, by reducing the brightness of the target vehicle interactive component or lighting component, thereby restoring the luminous brightness of the target vehicle interactive component or lighting component to its original level. In this way, when steps S1-S5 are dynamically executed multiple times, it can be achieved that when the people on the vehicle are looking at the target vehicle interactive component, the target vehicle interactive component itself is either illuminated and has a higher luminous brightness, and when the people on the vehicle are not looking at the target vehicle interactive component, the luminous brightness of the target vehicle interactive component is reduced to the original level or does not emit light. This effect is thereby achieved, thereby intelligently detecting and responding to the gaze operations of the people on the vehicle, reducing the types and number of operations that the people on the vehicle need to perform, providing good lighting conditions for the people on the vehicle to use the target vehicle interactive component, improving the ease of use of the target vehicle interactive component, and ensuring traffic safety.

[0135] A computer program that executes the gaze-adaptive luminescence adjustment method for automobile interactive components in this embodiment can be written and written into a computer device or storage medium. When the computer program is read out and run, the gaze-adaptive luminescence adjustment method for automobile interactive components in this embodiment is executed, thereby achieving the same technical effect as the gaze-adaptive luminescence adjustment method for automobile interactive components in the embodiment.

[0136] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in this disclosure are only relative to the relative positional relationships of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "" and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by those skilled in the art. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.

[0137] It should be understood that, although the terms first, second, third, etc. may be used to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as", etc.) provided in the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, will not impose limitations on the scope of the present invention.

[0138] It should be appreciated that embodiments of the present invention can be implemented or practiced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and figures described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed application-specific integrated circuit for this purpose.

[0139] Furthermore, the operations of the processes described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The processes described in this embodiment (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. A computer program includes multiple instructions that can be executed by one or more processors.

[0140] Furthermore, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0141] The computer program can be applied to input data to perform the functions of the present embodiment, thereby converting the input data to generate output data that is stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0142] The above are merely preferred embodiments of the present invention. The present invention is not limited to the aforementioned embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A gaze-adaptive luminescence adjustment method for automobile interactive components, characterized in that: The gaze-adaptive automobile interactive component luminescence adjustment method comprises: Acquire images of people in the vehicle; Extracting features from the image of the person in the vehicle to obtain sight features; determining a gaze area according to the sight line characteristics; determining a target vehicle interactive component according to the gaze area; performing luminescence adjustment on the target automobile interactive component; The extracting features of the image of the person on the vehicle to obtain sight features includes: Run the first feature extraction network and the second feature extraction network; Inputting the image of the person on the vehicle into the first feature extraction network for processing, and obtaining background-related features output by the first feature extraction network; Inputting the image of the person on the vehicle into the second feature extraction network for processing, and obtaining the sight-related features output by the second feature extraction network; determining the sight line feature according to the background-related feature and the sight line-related feature; The determining the sight line feature according to the background-related feature and the sight line-related feature includes: Eliminating the sight-line related features from the image of the person on the vehicle to obtain an image without sight-line related features; Obtaining background features by intersecting the background-related features and the line-of-sight-removed feature image; The background feature is eliminated from the image of the person on the vehicle to obtain the sight line feature.

2. The gaze-adaptive luminescence adjustment method for automobile interactive components according to claim 1, characterized in that: The gaze-adaptive automobile interactive component luminescence adjustment method further includes: Acquire a first training image and a second training image; the first training image corresponds to a marked sight feature, and the second training image corresponds to a marked background feature; Alternatingly performing multiple rounds of first adversarial training processes and second adversarial training processes on the first feature extraction network and the second feature extraction network; In the first adversarial training process, the first training image is input into the first feature extraction network for processing, a first feature output by the first feature extraction network is obtained, the first feature is eliminated from the first training image to obtain an image without the first feature, the image without the first feature is input into the second feature extraction network for processing, a first loss function value is determined based on an output result of the second feature extraction network and the marked line of sight feature, and parameters of the first feature extraction network and / or the second feature extraction network are adjusted based on the first loss function value; In the second adversarial training process, the second training image is input into the second feature extraction network for processing, the second feature output by the second feature extraction network is obtained, the second feature is eliminated from the second training image, and the second feature-removed image is obtained. The second feature-removed image is input into the first feature extraction network for processing, and the second loss function value is determined based on the output result of the first feature extraction network and the marked background feature. The parameters of the first feature extraction network and / or the second feature extraction network are adjusted according to the second loss function value.

3. The gaze-adaptive luminescence adjustment method for automobile interactive components according to claim 1, characterized in that: The determining of the gaze area according to the sight line feature includes: Running the third feature extraction network and the fourth feature extraction network; Inputting the sight line feature into the third feature extraction network for processing, and obtaining a left eye sight line vector output by the third feature extraction network; Inputting the sight line feature into the fourth feature extraction network for processing, and obtaining a right eye sight line vector output by the fourth feature extraction network; An intersection area is determined according to the left eye sight line vector and the right eye sight line vector, and the intersection area is used as the gaze area.

4. The gaze-adaptive automotive interactive component luminescence adjustment method according to claim 1, characterized in that: The determining of the gaze area according to the sight line feature includes: Run the fifth feature extraction network; Inputting the sight line feature into the fifth feature extraction network for processing, and obtaining a monocular sight line vector output by the fifth feature extraction network; The spatial area through which the monocular sight line vector passes is used as the gaze area.

5. The gaze-adaptive automotive interactive component luminescence adjustment method according to claim 1, characterized in that: The step of determining a target automobile interactive component according to the gaze area includes: Determining candidate vehicle interactive components according to the gaze area; the candidate vehicle interactive components are vehicle interactive components located within the gaze area; When the maintaining existence time of the gaze area is greater than a time threshold, the candidate vehicle interactive component is determined as the target vehicle interactive component.

6. The gaze-adaptive automotive interactive component luminescence adjustment method according to claim 1, characterized in that: The step of adjusting the light emission of the target automobile interactive component includes: When the target vehicle interactive component is a self-luminous component, controlling the target vehicle interactive component to adjust its brightness; When the target vehicle interactive component is a non-self-luminous component, a lighting component is determined according to the target vehicle interactive component, and the lighting component is controlled to adjust the brightness; wherein the target vehicle interactive component is located within the lighting range of the lighting component.

7. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load at least one program to execute the gaze-adaptive automobile interactive component luminescence adjustment method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute the gaze-adaptive automobile interactive component lighting adjustment method described in any one of claims 1-6 when executed by the processor.

Images