Control method and device for in-vehicle electronic equipment and electronic equipment

Abstract

The present invention provides a method, device, and electronic device for controlling in-vehicle electronic devices, the method comprising: obtaining image information of a user in the vehicle; determining the user's target position based on the image information, the target position including the relative position of the user's target body part in the vehicle; and controlling the in-vehicle electronic device to output a trigger event corresponding to the target trigger point when the target position meets the trigger condition of the target trigger point, the trigger condition of the target trigger point including that the distance between the target position and the target trigger point is less than a preset distance threshold. In an embodiment of the present invention, the user only needs to move the target body part, and the position of the target body part in the vehicle can be used to control the in-vehicle electronic device to output a trigger event. Compared to voice command recognition, position recognition is undoubtedly more accurate, thereby being able to more accurately identify the user's intention and control the in-vehicle electronic device to output a trigger event to meet the user's needs.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a control method and device for in-vehicle electronic equipment, and the electronic equipment. Background Art

[0002] With the development of social economy, cars have begun to enter thousands of households. While making people's travel more convenient, the development of more functions for cars has also begun.

[0003] In addition to practical functions like map navigation and fuel gauge displays, today's cars also include entertainment features like audio and lighting effects. In recent years, voice recognition technology and in-car electronics have enabled intelligent human-vehicle interaction. For example, a passenger can say "turn on the music" in the car, and the in-car electronics will activate the audio function based on the recognized voice.

[0004] However, during the recognition process of voice recognition technology, language or accent problems may cause the voice recognition results to be unclear, that is, the recognized user intention is not accurate enough, resulting in the vehicle being unable to accurately turn on the in-vehicle electronic devices that the user really needs to turn on. Summary of the Invention

[0005] In view of this, the present invention aims to provide a control method for an in-vehicle electronic device to solve the problem of inaccurate recognition of user intention when recognizing user intention through voice.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A method for controlling an in-vehicle electronic device, the method comprising:

[0008] Obtain image information of the user in the car;

[0009] determining a target position of the user according to the image information, wherein the target position includes a relative position of a target body part of the user in the vehicle;

[0010] When the target position meets the triggering condition of the target trigger point, the electronic device in the vehicle is controlled to output the triggering event corresponding to the target trigger point, wherein the triggering condition of the target trigger point includes that the distance between the target position and the target trigger point is less than a preset distance threshold.

[0011] Furthermore, the triggering condition of the target trigger point also includes: the triggering moment is within a preset time period, wherein the triggering moment includes the moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

[0012] Furthermore, the image information includes image information of the user in the vehicle continuously collected at different moments, and the target position includes: the relative position of the target body part of the user in the vehicle in the image information at each moment at the different moments, and when the target position meets the trigger condition of the target trigger point, controlling the electronic device in the vehicle to output the trigger event corresponding to the target trigger point includes:

[0013] When the relative position of the user's target body part in the car in the image information at the target moment and the distance between the target trigger point is less than a preset distance threshold, and the target moment is within the preset time period, the electronic equipment in the car is controlled to output the trigger event corresponding to the target trigger point.

[0014] Furthermore, before obtaining the image information of the user in the car, the method further includes:

[0015] Obtain a three-dimensional model of the vehicle interior environment;

[0016] receiving a first input from a user regarding the three-dimensional model;

[0017] In response to the first input, positions inside the vehicle corresponding to a target number of point cloud positions in the three-dimensional model are determined as candidate trigger points, wherein the target trigger point is any one of the candidate trigger points.

[0018] Furthermore, after obtaining the three-dimensional model of the vehicle interior environment, the method further includes:

[0019] receiving a second input from a user regarding the three-dimensional model;

[0020] In response to the second input, performing three-dimensional point cloud completion on the three-dimensional model to obtain a target three-dimensional model;

[0021] The target three-dimensional model is magnified and displayed.

[0022] Furthermore, completing the three-dimensional point cloud of the three-dimensional model to obtain a target three-dimensional model includes:

[0023] Extracting red, green, and blue (RGB) features and depth image features from the three-dimensional model respectively;

[0024] Downsampling is performed based on the RGB feature to obtain a first sampling result;

[0025] Perform upsampling based on the depth image feature to obtain a second sampling result;

[0026] The first sampling result and the second sampling result are subjected to feature fusion to obtain the target three-dimensional model.

[0027] Furthermore, after determining, in response to the first input, positions within the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, the method further includes:

[0028] receiving a third input from the user on a target option control in a selection interface, wherein the selection interface displays a plurality of option controls, each of the option controls corresponds to a trigger event, and the target option control is any one of the plurality of option controls;

[0029] In response to the third input, the trigger event corresponding to the target option control is determined as the trigger event corresponding to the candidate trigger point.

[0030] Furthermore, the triggering event includes playing an audio file preset by the user and / or starting a lighting device in the car.

[0031] Compared with the prior art, the control method of the in-vehicle electronic device of the present invention has the following advantages:

[0032] Based on the user's in-car image, the relative position of the user's target body part within the vehicle can be determined. If the user's target body part meets the trigger conditions of a target trigger point set within the vehicle, the in-car electronic device is controlled to output a trigger event corresponding to the target trigger point. The user simply moves the target body part, and the position of the target body part within the vehicle controls the in-car electronic device to output the trigger event. Compared to voice command recognition, position recognition is undoubtedly more accurate, allowing for more precise identification of the user's intent and controlling the in-car electronic device to output trigger events to meet the user's needs.

[0033] Another object of the present invention is to provide a control device for an in-vehicle electronic device to solve the problem of inaccurate recognition of user intention when recognizing user intention through voice.

[0034] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0035] A control device for an in-vehicle electronic device, the device comprising:

[0036] An acquisition module is used to obtain image information of the user in the car;

[0037] a position module, configured to determine a target position of the user based on the image information, wherein the target position includes a relative position of a target body part of the user in the vehicle;

[0038] A control module is used to control the electronic device in the vehicle to output a trigger event corresponding to the target trigger point when the target position meets the trigger condition of the target trigger point, wherein the trigger condition of the target trigger point includes that the distance between the target position and the target trigger point is less than a preset distance threshold.

[0039] Furthermore, the triggering condition of the target trigger point also includes: the triggering moment is within a preset time period, wherein the triggering moment includes the moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

[0040] Furthermore, the image information includes image information of the user in the vehicle continuously collected at different moments, and the target position includes: the relative position of the target body part of the user in the vehicle in the image information at each moment at the different moments;

[0041] The control module is specifically used to control the electronic device in the vehicle to output the trigger event corresponding to the target trigger point when the relative position of the user's target body part in the image information at the target moment and the distance between the target trigger point is less than a preset distance threshold, and the target moment is within the preset time period.

[0042] Furthermore, the device further includes:

[0043] 3D module, used to obtain a 3D model of the vehicle interior environment;

[0044] A first receiving module, configured to receive a first input of the user on the three-dimensional model;

[0045] The first response module is used to determine, in response to the first input, positions in the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, wherein the target trigger point is any one of the candidate trigger points.

[0046] Furthermore, the device further comprises:

[0047] A second receiving module, configured to receive a second input of the user on the three-dimensional model;

[0048] a second response module, configured to perform 3D point cloud completion on the 3D model in response to the second input to obtain a target 3D model;

[0049] The magnification module is used to magnify and display the target three-dimensional model.

[0050] Furthermore, the second response module includes:

[0051] An extraction unit, configured to extract red, green, and blue (RGB) features and depth image features from the three-dimensional model respectively;

[0052] a downsampling unit, configured to perform downsampling based on the RGB feature to obtain a first sampling result;

[0053] an upsampling unit, configured to perform upsampling based on the depth image feature to obtain a second sampling result;

[0054] A fusion unit is used to perform feature fusion on the first sampling result and the second sampling result to obtain the target three-dimensional model.

[0055] Furthermore, the device further comprises:

[0056] a third receiving module, configured to receive a third input from a user on a target option control in a selection interface, wherein the selection interface displays a plurality of option controls, each of the option controls corresponds to a trigger event, and the target option control is any one of the plurality of option controls;

[0057] The third response module is configured to determine, in response to the third input, the trigger event corresponding to the target option control as the trigger event corresponding to the candidate trigger point.

[0058] Furthermore, the triggering event includes playing an audio file preset by the user and / or starting a lighting device in the car.

[0059] The control device for in-vehicle electronic equipment and the control method for in-vehicle electronic equipment have the same advantages over the prior art, which will not be described in detail here.

[0060] Another object of the present invention is to provide an electronic device to solve the problem of inaccurate recognition of user intention when recognizing user intention through voice.

[0061] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0062] An electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned control method for in-vehicle electronic devices when executing the program.

[0063] The advantages of the electronic device and the control method of the above-mentioned in-vehicle electronic device over the prior art are the same, and will not be repeated here.

[0064] Another object of the present invention is to provide a readable storage medium to solve the problem of inaccurate recognition of user intention when recognizing user intention through voice.

[0065] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0066] A readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned control method for an in-vehicle electronic device.

[0067] The advantages of the readable storage medium and the above-mentioned control method of the in-vehicle electronic device over the prior art are the same, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 This is a flowchart of the steps of the control method of the in-vehicle electronic device according to an embodiment of the present invention;

[0070] Figure 2 A schematic diagram of the application architecture of the control method for an in-vehicle electronic device according to an embodiment of the present invention;

[0071] Figure 3 This is a structural block diagram of a control device for an in-vehicle electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0072] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0073] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0074] See also Figure 1 The present invention provides a method for controlling an in-vehicle electronic device, the method comprising:

[0075] Step 101: Obtain image information of the user in the car.

[0076] It should be noted that an image acquisition device can be installed in the vehicle to capture image information within the vehicle. When a user enters the vehicle, the captured image information includes the user's image information and image information of the vehicle's interior environment. For example, when a user enters the vehicle and sits in the driver's seat, the captured image information will include an image of the user sitting in the driver's seat. It should be noted that when acquiring image information of the user in the vehicle, the image information can be acquired using electronic devices such as cameras and infrared testers, such as, but not limited to, time of fight cameras.

[0077] Step 102: Determine the user's target location based on the image information.

[0078] It should be noted that when the image information contains an image of the user, the relative positions of various parts of the user within the vehicle can be determined based on the image information. The target position includes the relative position of a target body part of the user within the vehicle. The target body part can be any one or more body parts of the user. For example, the target body part can be a hand, elbow, head, foot, knee, etc., but is not limited thereto.

[0079] It is understood that when a part of the user's body contacts the seat armrest, the image of that part of the body in the image information will be displayed together with the image of the seat armrest in the vehicle, so that the relative position of the part of the body in the vehicle can be easily determined based on the image in the image information. Preferably, a deep learning model can be used to determine the user's target position, but the present invention is not limited thereto.

[0080] Step 103: When the target position meets the triggering condition of the target triggering point, control the in-vehicle electronic device to output a triggering event corresponding to the target triggering point.

[0081] It should be noted that if the target location does not meet the trigger conditions of the target trigger point, the in-vehicle electronic device is prohibited from outputting the trigger event corresponding to the target trigger point. The trigger conditions of the target trigger point include the distance between the target location and the target trigger point being less than a preset distance threshold. Here, the preset distance threshold is typically small, so a situation where the distance between the target location and the target trigger point is less than the preset distance threshold can be considered as the user's target body part contacting the target trigger point. The target trigger point can be a predetermined location within the vehicle. When the user's target body part contacts this location, the user's intention can be recognized, i.e., the user wants to trigger the trigger event corresponding to the target trigger point, thereby controlling the in-vehicle electronic device to output the trigger event corresponding to the target trigger point. The target trigger point can be any location within the vehicle, and the trigger event can be any event that the in-vehicle electronic device can output, without limitation. It is understood that the trigger event output by the in-vehicle electronic device will not affect driving safety. Of course, if the trigger event output by the in-vehicle electronic device does affect driving safety, the vehicle's state can be detected when outputting the trigger event. If the vehicle is in motion, the trigger event output is prohibited; if the vehicle is parked, the trigger event output is permitted.

[0082] In this embodiment of the present invention, the relative position of a target body part within the vehicle can be determined based on image information of the user within the vehicle. Furthermore, if the target body part meets the trigger conditions of a target trigger point set within the vehicle, the in-vehicle electronic device is controlled to output a trigger event corresponding to the target trigger point. The user simply moves the target body part, and the position of the target body part within the vehicle controls the in-vehicle electronic device to output the trigger event. Compared to voice command recognition, position recognition is undoubtedly more accurate, allowing for more precise identification of the user's intent and controlling the in-vehicle electronic device to output the trigger event to meet the user's needs.

[0083] Optionally, the triggering condition of the target trigger point further includes: the triggering moment is within a preset time period, wherein the triggering moment includes the moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

[0084] It should be noted that the preset time period is any time period pre-set by the user. It can be one or more fixed time periods every day, or one or more time periods on a specific date. For example, it can be from 9:00 a.m. to 10:00 a.m. every day, or from 9:00 a.m. to 11:00 a.m. on February 20, 2022. Specifically, the user can set the time period through the multimedia device in the car. Since any user entering the car can set the time period, a user who can enter the car can create surprises for other users who can also enter the car. For example, the family of the vehicle driver can enter the car to set it up, and when the driver enters the car, it will trigger the electronic equipment in the car to output a trigger event, thereby creating a surprise. Preferably, the preset time period can be some festivals, holidays, days that are of special significance to the user, etc.

[0085] In this embodiment of the present invention, users can freely set a preset time period, so that in-vehicle electronic devices can only output trigger events during this preset time period. This not only improves user operability but also avoids the potential user disruption caused by outputting a trigger event every time the distance between the target location and the target trigger point falls below a preset distance threshold. Furthermore, this can also meet special user needs, such as creating a surprise for other users in the vehicle on a designated day.

[0086] Optionally, the image information includes image information of the user in the vehicle continuously collected at different times, and the target position includes: the relative position of the target body part of the user in the vehicle in the image information at each time. When the target position meets the trigger condition of the target trigger point, controlling the electronic device in the vehicle to output a trigger event corresponding to the target trigger point includes:

[0087] When the relative position of the user's target body part in the car in the image information at the target moment and the distance between the target trigger point is less than a preset distance threshold, and the target moment is within a preset time period, the electronic equipment in the car is controlled to output a trigger event corresponding to the target trigger point.

[0088] It should be noted that image information from within the vehicle can be continuously collected after a user enters the vehicle or after the vehicle is started. This allows the relative position of the target body part within the vehicle to be determined based on the image information at each moment. By monitoring the target body part's relative positions within the vehicle over a series of periods, it is possible to promptly determine whether the target position meets the trigger conditions of the target trigger point. The trigger conditions here include two different conditions, both of which must be met for a trigger event to be output.

[0089] In an embodiment of the present invention, by continuously collecting image information of the user in the car at different times, it is possible to promptly determine whether the target position meets the trigger conditions of the target trigger point. When the target position meets the trigger conditions of the target trigger point, the electronic equipment in the car is promptly controlled to output the trigger event corresponding to the target trigger point.

[0090] Optionally, before step 101: obtaining image information of the user in the car, the method further includes:

[0091] Obtain a 3D model of the vehicle interior environment.

[0092] In this step, the three-dimensional model is a three-dimensional model pre-built based on the in-car environment. The three-dimensional model can be obtained using 3D reconstruction technology. Among them, 3D reconstruction refers to the process of capturing depth data, pre-processing, point cloud registration and fusion, and generating surfaces to depict the real scene into a mathematical model that conforms to computer logic expression. In the embodiment of the present invention, the in-car environment 3D reconstruction module can be used to obtain the depth information of the in-car environment, and then the three-dimensional model can be obtained based on the depth information. Of course, the three-dimensional model here can be a three-dimensional model transmitted from other electronic devices. It is understandable that the in-car environment of each car model is the same, so the same three-dimensional model can be used. When the car manufacturer or the owner of the same car model has built a three-dimensional model of the in-car environment, the three-dimensional model of the in-car environment can be directly obtained from the car manufacturer or the owner of the same car model.

[0093] A first input of the user to the three-dimensional model is received.

[0094] In this step, the three-dimensional model of the in-car environment is displayed to the user through the multimedia device in the car, and a first input can be performed on the currently displayed three-dimensional model, where the first input can be an operation such as clicking, sliding, and long pressing, but is not limited to these.

[0095] In response to the first input, positions within the vehicle corresponding to a target number of point cloud positions in the three-dimensional model are determined as candidate trigger points.

[0096] In this step, based on the user's first input to the three-dimensional model, at least one candidate trigger point is determined, wherein each candidate trigger point can respond to the user's first input, and the candidate trigger point that currently responds to the user's first input is the target trigger point. Therefore, the target trigger point is any one of the candidate trigger points. Preferably, when there are multiple candidate trigger points, a certain candidate trigger point can be randomly determined as the target trigger point, and the target trigger point can respond to the user's first input, while the other remaining candidate trigger points cannot respond to the user's first input. Furthermore, after a certain candidate trigger point responds to the user's first input as the target trigger point, another candidate trigger point can be randomly determined as the target trigger point again, and the re-determined target trigger point can respond to the user's first input, while the other remaining candidate trigger points cannot respond to the user's first input, and so on, so that the position of the target trigger point can be random each time.

[0097] Preferably, a number of pre-selected point cloud locations can be displayed in the 3D model, and the locations within the vehicle corresponding to these point cloud locations can be identified as candidate trigger points. These locations within the vehicle corresponding to these point cloud locations are easily accessible to the user. If the user determines that these point cloud locations do not meet their needs, they can delete them or add new point cloud locations, thereby enabling the deletion and addition of subsequent trigger points.

[0098] It is understood that the number of candidate trigger points can be one or more. If there are multiple candidate trigger points, there can also be multiple target trigger points. If there are multiple target trigger points at the same time, the in-vehicle electronic device can be controlled to randomly output a trigger event corresponding to a target trigger point. Of course, if the trigger events corresponding to multiple target trigger points do not conflict with each other, the in-vehicle electronic device can also be controlled to output trigger events corresponding to multiple target trigger points simultaneously.

[0099] In the embodiment of the present invention, a three-dimensional model of the in-car environment is provided to the user to help the user independently select candidate trigger points, which further improves the user's operability and can meet more user needs.

[0100] Optionally, after obtaining the three-dimensional model of the vehicle interior environment, the method further includes:

[0101] receiving a second input of the user on the three-dimensional model;

[0102] In response to the second input, performing 3D point cloud completion on the 3D model to obtain a target 3D model;

[0103] Zoom in and display the target 3D model.

[0104] It should be noted that the second input can be an operation to enlarge the three-dimensional model. Here, the display ratio of the three-dimensional model may not be the display ratio that the user wants to see. In this case, the user will need to enlarge the three-dimensional model. It is understandable that when the three-dimensional model is enlarged, if the three-dimensional point cloud is not completed, the enlarged three-dimensional model will be very sparse and cannot meet the user's viewing or operation needs. Therefore, it is necessary to perform three-dimensional point cloud completion on the three-dimensional model before enlarging and displaying the three-dimensional model, and enlarge and display the three-dimensional model after the three-dimensional point cloud is completed, that is, the target three-dimensional model. Here, after enlarging and displaying the target three-dimensional model, the user can perform the first input again according to their needs and re-determine the candidate trigger points.

[0105] In an embodiment of the present invention, the three-dimensional model can be completed with a three-dimensional point cloud, and the three-dimensional model after the three-dimensional point cloud completion can be enlarged and displayed, which not only meets the user's viewing or operation needs, but also avoids the enlarged three-dimensional model from being too sparse.

[0106] Optionally, performing 3D point cloud completion on the 3D model to obtain a target 3D model includes:

[0107] Extract RGB (red, green, and blue) features and depth image features from the three-dimensional model respectively;

[0108] Downsampling is performed based on the RGB feature to obtain a first sampling result;

[0109] Upsampling is performed based on the depth image feature to obtain a second sampling result;

[0110] The first sampling result and the second sampling result are subjected to feature fusion to obtain a target three-dimensional model.

[0111] It should be noted that the RGB image mode is the most commonly used color mode. An RGB image consists of three color channels: red, green, and blue. Each channel uses 8 bits of color information, with each color ranging from 0 to 255. This color information constitutes the red, green, and blue (RGB) features of the 3D model. Using deep learning methods, 3D point cloud completion can be achieved by using RGB data to guide upsampling of depth data. The network structure used in deep learning can be similar to, but not limited to, MS-Net. Of course, traditional visual methods can also be used to replace the process of upsampling depth data. Specifically, an RGBD (red, green, blue, and depth) aligned RGB image is obtained. When upsampling the depth data, the RGB image provides color similarity weights, which are combined with spatial similarity weights to obtain new upsampling weights. This will not be discussed in detail here.

[0112] In the embodiment of the present invention, based on the upsampling of the depth data, the three-dimensional point cloud can be quickly completed, ensuring that the three-dimensional model after the three-dimensional point cloud completion has a good display effect.

[0113] Optionally, after determining, in response to the first input, positions within the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, the method further includes:

[0114] receiving a third input from the user on a target option control in the selection interface, wherein the selection interface displays a plurality of option controls, each option control corresponds to a trigger event, and the target option control is any one of the plurality of option controls;

[0115] In response to the third input, the trigger event corresponding to the target option control is determined as the trigger event corresponding to the candidate trigger point.

[0116] It should be noted that different option controls correspond to different trigger events. Here, the multiple trigger events corresponding to multiple option controls can be regarded as a trigger event pool. The user can select a trigger event from the trigger event pool for each candidate trigger point to bind or correspond, so that the trigger events corresponding to different candidate trigger points are different or all the same.

[0117] In an embodiment of the present invention, multiple option controls corresponding to different trigger events are provided to the user to help the user independently select a trigger event for each candidate trigger point, thereby improving the diversity of trigger events corresponding to the candidate trigger points and meeting more user needs.

[0118] Optionally, the triggering event includes playing an audio file preset by the user and / or starting a lighting device in the vehicle.

[0119] It should be noted that the audio file can be a music file or recording file pre-stored in the electronic device in the car. Preferably, when the trigger condition of the target trigger point also includes the trigger moment being within a preset time period, the trigger event corresponding to the target trigger point is a recording file associated with the preset time period. The trigger moment includes the moment when the distance between the target position and the target trigger point is less than a preset distance threshold. Specifically, the preset time period is one or more days corresponding to a certain festival, and the trigger event corresponding to the target trigger point is a recording associated with the festival. For example, the preset time period is the user's birthday, and the trigger event corresponding to the target trigger point is a recording file recorded by the user's family or friends wishing the user a happy birthday. It is understandable that the festival can be any national statutory holiday, holiday, or festival that has special significance for the user.

[0120] Of course, trigger events can also include other events that can be controlled by in-vehicle electronic devices, such as opening the sunroof or windows, turning on the air conditioning, adjusting the seats, and retracting or deploying the hood. It's understood that different trigger events can provide different user experiences, thereby meeting different user needs. Different trigger events can be pre-set based on different user needs. For example, by adjusting a seat, by adjusting the angle between the seatback and the seat cushion to the maximum angle supported by the seat, the user can lie back as much as possible, allowing for better rest or relaxation. Therefore, based on the user's rest or relaxation needs, a trigger event can be set, including adjusting the angle between the seatback and the seat cushion to a target angle. The target angle is relatively large, but not limited to the maximum angle supported by the seat. Preferably, the target angle can be pre-set by the user.

[0121] In the embodiment of the present invention, under the control of the in-vehicle electronic device, playing an audio file and / or starting the lighting device in the vehicle are used as trigger events, which can meet the needs of users in various scenarios.

[0122] Optionally, the trigger condition of the target trigger point also includes: the user's facial image in the image information successfully matches a preset facial image. Here, a successful match of two facial images means that the similarity of the two facial images is higher than a certain threshold, and the two facial images can be regarded as facial images of the same person.

[0123] It should be noted that before obtaining image information from the user's vehicle interior, facial images of different users can be pre-scanned and created, and each facial image can be stored in a facial database. Thus, when determining whether the target location meets the trigger conditions of the target trigger point, it is also determined whether the user in the image information corresponds to the user whose facial image is in the facial database. If not, the in-vehicle electronic device is prohibited from outputting the trigger event corresponding to the target trigger point. Preferably, when determining whether the user in the image information corresponds to the user whose facial image is in the facial database, it is determined whether the user in the image information corresponds to the user whose facial image is pre-specified by the user in the facial database. For example, the user can pre-specify the user corresponding to a facial image in the facial database as the target person. Only if the facial image in the obtained in-vehicle image information is that of the target person, can the in-vehicle electronic device be controlled to output the trigger event corresponding to the target trigger point and continue to determine other trigger conditions of the target trigger point. If the facial image in the obtained in-vehicle image information is not that of the target person, the in-vehicle electronic device is prohibited from outputting the trigger event corresponding to the target trigger point.

[0124] In an embodiment of the present invention, by pre-storing the facial image of a designated user, it can be ensured that only when the user in the car is the designated user, the electronic device in the car is controlled to output the trigger event corresponding to the target trigger point, thereby avoiding interference from other users in the car.

[0125] like Figure 2 As shown, it is a schematic diagram of the application architecture of the control method of the in-vehicle electronic device provided by an embodiment of the present invention. The application architecture includes two parts: an online part and an offline part. Among them, the offline part includes: a three-dimensional reconstruction module of the in-vehicle environment, a trigger condition setting module, and a surprise form setting module; the online part includes: a body part detection module, a trigger judgment module, and a surprise output module.

[0126] Among them, the in-car environment 3D reconstruction module is used to obtain the in-car environment through the TOF camera to construct a 3D model. Specifically, the depth information of the in-car environment is obtained through the TOF camera, and the 3D model of the in-car environment is created based on the acquired depth information. In addition, since the depth information resolution of the TOF camera is not high, and after the point cloud of the 3D model is magnified and stretched, if no special processing is performed, the magnified and stretched 3D model will be very sparse and have low accuracy. Therefore, when magnifying and stretching, the depth data can be upsampled, that is, the 3D point cloud completion processing, which will not be repeated here.

[0127] The trigger condition setting module is used to set trigger conditions. There are two trigger conditions: one is the trigger point setting, which is determined by selecting multiple fixed point cloud locations based on the 3D model of the vehicle's interior environment. The user can select the number of trigger points, and a candidate trigger point library is generated based on the user's selected trigger points. The other is the trigger time, which allows the user to enter a trigger time in advance, such as a family holiday or statutory holiday.

[0128] The surprise form setting module allows users to set trigger events in advance, where trigger events include playing music and user-defined recordings. Here, users can set the corresponding trigger event for each trigger point, so that different trigger points correspond to different trigger events.

[0129] The body part detection module uses a deep learning model to detect and locate target body parts in the depth map (image information of the user in the car). It should be noted that since the depth map is visualized as a 3D point cloud and grayscale image, if the depth map is converted to grayscale, it can be detected using a 2D detection network structure, which can adopt a deep learning model similar to YOLOX, but is not limited to this. If the depth map is converted to a 3D point cloud, it needs to be detected using a 3D detection network structure, which can adopt a deep learning model similar to pointNet, but is not limited to this.

[0130] The trigger determination module is used to determine whether the trigger condition has been met. Specifically, if the distance between the user's target location and the three-dimensional coordinates of the target trigger point is less than the target distance threshold, and the current time is within the trigger time, the trigger condition is considered met; otherwise, the trigger condition is considered not met. The target trigger point can be a randomly selected trigger point from the trigger point candidate library, and a new target trigger point is selected after each trigger event is output.

[0131] The surprise output module is used to output a surprise when the trigger condition is met. Specifically, when the trigger condition is met, it controls the electronic devices in the vehicle to output the trigger event corresponding to the target trigger point, so that the user can hear music or pre-recorded audio.

[0132] In this embodiment of the present invention, user interaction and interaction with in-vehicle electronic devices can be achieved through body movements, thereby creating surprises for the user. Furthermore, since the target trigger point can be reselected after each trigger event, the touch position becomes random when the user repeatedly triggers the in-vehicle electronic device to output trigger events, increasing the sense of surprise and entertainment.

[0133] like Figure 3 As shown, an embodiment of the present invention further provides a control device for an in-vehicle electronic device, the device comprising:

[0134] An acquisition module 31 is used to acquire image information of the user in the car;

[0135] a position module 32 for determining a target position of the user based on the image information, wherein the target position includes a relative position of a target body part of the user within the vehicle;

[0136] The control module 33 is used to control the electronic device in the vehicle to output a trigger event corresponding to the target trigger point when the target position meets the trigger condition of the target trigger point, wherein the trigger condition of the target trigger point includes that the distance between the target position and the target trigger point is less than a preset distance threshold.

[0137] Optionally, the triggering condition of the target trigger point further includes: the triggering moment is within a preset time period, wherein the triggering moment includes the moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

[0138] Optionally, the image information includes image information of the user in the car continuously collected at different moments, and the target position includes: the relative position of the target body part of the user in the car in the image information at each moment at different moments;

[0139] The control module 33 is specifically used to control the electronic device in the vehicle to output a trigger event corresponding to the target trigger point when the distance between the relative position of the user's target body part in the vehicle and the target trigger point in the image information at the target moment is less than a preset distance threshold and the target moment is within a preset time period.

[0140] Optionally, the device further comprises:

[0141] 3D module, used to obtain a 3D model of the vehicle interior environment;

[0142] A first receiving module, configured to receive a first input of a user on a three-dimensional model;

[0143] The first response module is used to determine, in response to the first input, positions in the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, wherein the target trigger point is any one of the candidate trigger points.

[0144] Optionally, the device further comprises:

[0145] A second receiving module, configured to receive a second input of the user on the three-dimensional model;

[0146] A second response module is configured to complete the three-dimensional point cloud of the three-dimensional model in response to the second input to obtain a target three-dimensional model;

[0147] The zoom module is used to zoom in and display the target three-dimensional model.

[0148] Optionally, the second response module includes:

[0149] An extraction unit, used to extract RGB features and depth image features from the three-dimensional model respectively;

[0150] A downsampling unit, configured to perform downsampling based on the RGB feature to obtain a first sampling result;

[0151] an upsampling unit, configured to perform upsampling based on the depth image feature to obtain a second sampling result;

[0152] The fusion unit is used to perform feature fusion on the first sampling result and the second sampling result to obtain a target three-dimensional model.

[0153] Optionally, the device further comprises:

[0154] a third receiving module, configured to receive a third input from the user on a target option control in the selection interface, wherein the selection interface displays a plurality of option controls, each option control corresponds to a trigger event, and the target option control is any one of the plurality of option controls;

[0155] The third response module is configured to, in response to the third input, determine the trigger event corresponding to the target option control as the trigger event corresponding to the candidate trigger point.

[0156] Optionally, the triggering event includes playing an audio file preset by the user and / or starting a lighting device in the vehicle.

[0157] The control device for in-vehicle electronic equipment provided by the embodiment of the present invention can achieve Figures 1 to 2 To avoid repetition, the various processes implemented by the method for controlling the in-vehicle electronic device in the method embodiment will not be described here.

[0158] In this embodiment of the present invention, the relative position of a target body part within the vehicle can be determined based on image information of the user within the vehicle. Furthermore, if the target body part meets the trigger conditions of a target trigger point set within the vehicle, the in-vehicle electronic device is controlled to output a trigger event corresponding to the target trigger point. The user simply moves the target body part, and the position of the target body part within the vehicle controls the in-vehicle electronic device to output the trigger event. Compared to voice command recognition, position recognition is undoubtedly more accurate, allowing for more precise identification of the user's intent and controlling the in-vehicle electronic device to output the trigger event to meet the user's needs.

[0159] On the other hand, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the control method of the in-vehicle electronic device provided by the above-mentioned embodiments of the application is implemented.

[0160] On the other hand, an embodiment of the present application further provides a readable storage medium. When the instructions in the readable storage medium are executed by the processor of the electronic device, the electronic device can execute the control method of the in-vehicle electronic device provided in the above embodiments of the application.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for controlling an in-vehicle electronic device, characterized in that: The method comprises: Obtain image information of the user in the car; determining a target position of the user according to the image information, wherein the target position includes a relative position of a target body part of the user in the vehicle; When the target position satisfies a trigger condition of a target trigger point, controlling the in-vehicle electronic device to output a trigger event corresponding to the target trigger point, wherein the trigger condition of the target trigger point includes that the distance between the target position and the target trigger point is less than a preset distance threshold; Before obtaining the image information of the user in the vehicle, the method further includes: Obtain a three-dimensional model of the vehicle interior environment; receiving a first input from a user regarding the three-dimensional model; In response to the first input, determining positions within the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, wherein the target trigger point is any one of the candidate trigger points; Wherein, when there are multiple candidate trigger points, a certain candidate trigger point is randomly determined as the target trigger point, and after the target trigger point responds to the first input, another certain candidate trigger point is randomly determined as the target trigger point; After obtaining the three-dimensional model of the vehicle interior environment, the method further includes: receiving a second input from a user regarding the three-dimensional model; In response to the second input, performing three-dimensional point cloud completion on the three-dimensional model to obtain a target three-dimensional model; The target three-dimensional model is magnified and displayed.

2. The method according to claim 1, characterized in that The triggering condition of the target trigger point further includes: a triggering moment is within a preset time period, wherein the triggering moment includes a moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

3. The method according to claim 2, characterized in that The image information includes image information of the user in the vehicle continuously collected at different moments, and the target position includes: the relative position of the target body part of the user in the vehicle in the image information at each moment at the different moments. When the target position meets the trigger condition of the target trigger point, controlling the electronic device in the vehicle to output a trigger event corresponding to the target trigger point includes: When the relative position of the user's target body part in the car in the image information at the target moment and the distance between the target trigger point is less than a preset distance threshold, and the target moment is within the preset time period, the electronic equipment in the car is controlled to output the trigger event corresponding to the target trigger point.

4. The method according to claim 1, wherein The performing three-dimensional point cloud completion on the three-dimensional model to obtain a target three-dimensional model includes: Extracting red, green, and blue (RGB) features and depth image features from the three-dimensional model respectively; Downsampling is performed based on the RGB feature to obtain a first sampling result; Perform upsampling based on the depth image feature to obtain a second sampling result; The first sampling result and the second sampling result are subjected to feature fusion to obtain the target three-dimensional model.

5. The method according to claim 1, wherein After determining, in response to the first input, positions within the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, the method further includes: receiving a third input from the user on a target option control in a selection interface, wherein the selection interface displays a plurality of option controls, each of the option controls corresponds to a trigger event, and the target option control is any one of the plurality of option controls; In response to the third input, the trigger event corresponding to the target option control is determined as the trigger event corresponding to the candidate trigger point.

6. The method according to claim 1 or 5, characterized in that The triggering event includes playing an audio file preset by the user and / or starting a lighting device in the car.

7. A control device for an in-vehicle electronic device, characterized in that: The device comprises: An acquisition module is used to obtain image information of the user in the car; a position module, configured to determine a target position of the user based on the image information, wherein the target position includes a relative position of a target body part of the user in the vehicle; a control module, configured to control an in-vehicle electronic device to output a trigger event corresponding to a target trigger point when the target position satisfies a trigger condition of a target trigger point, wherein the trigger condition of the target trigger point includes that a distance between the target position and the target trigger point is less than a preset distance threshold; 3D module, used to obtain a 3D model of the vehicle interior environment; A first receiving module, configured to receive a first input of a user on a three-dimensional model; a first response module, configured to, in response to a first input, determine positions in the vehicle corresponding to a target number of point cloud positions in the three-dimensional model as candidate trigger points, wherein the target trigger point is any one of the candidate trigger points; a module for randomly determining a candidate trigger point as a target trigger point, for randomly determining a candidate trigger point as the target trigger point when there are multiple candidate trigger points, and for randomly determining another candidate trigger point as the target trigger point after the target trigger point responds to the first input; A second receiving module, configured to receive a second input of the user on the three-dimensional model; a second response module, configured to perform 3D point cloud completion on the 3D model in response to the second input to obtain a target 3D model; The magnification module is used to magnify and display the target three-dimensional model.

8. The device according to claim 7, characterized in that The triggering condition of the target trigger point further includes: a triggering moment is within a preset time period, wherein the triggering moment includes a moment when the distance between the target position and the target trigger point is less than a preset distance threshold.

9. The device according to claim 8, characterized in that The image information includes image information of the user in the car continuously collected at different moments, and the target position includes: the relative position of the target body part of the user in the car in the image information at each moment at the different moments; The control module is specifically used to control the electronic device in the vehicle to output the trigger event corresponding to the target trigger point when the relative position of the user's target body part in the image information at the target moment and the distance between the target trigger point is less than a preset distance threshold, and the target moment is within the preset time period.

10. The device according to claim 7, characterized in that The second response module includes: An extraction unit, configured to extract red, green, and blue (RGB) features and depth image features from the three-dimensional model respectively; a downsampling unit, configured to perform downsampling based on the RGB feature to obtain a first sampling result; an upsampling unit, configured to perform upsampling based on the depth image feature to obtain a second sampling result; A fusion unit is used to perform feature fusion on the first sampling result and the second sampling result to obtain the target three-dimensional model.

11. The device according to claim 7, characterized in that The device further comprises: a third receiving module, configured to receive a third input from a user on a target option control in a selection interface, wherein the selection interface displays a plurality of option controls, each of the option controls corresponds to a trigger event, and the target option control is any one of the plurality of option controls; The third response module is configured to determine, in response to the third input, the trigger event corresponding to the target option control as the trigger event corresponding to the candidate trigger point.

12. The device according to claim 7 or 11, characterized in that The triggering event includes playing an audio file preset by the user and / or starting a lighting device in the car.

13. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for controlling an in-vehicle electronic device according to one or more of claims 1 to 6 is implemented.

14. A readable storage medium, characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for controlling an in-vehicle electronic device as claimed in one or more of claims 1 to 6.

Images