Data processing method and device based on vehicle, vehicle and related equipment

Abstract

The invention relates to a data processing method and device based on a vehicle, the vehicle and related equipment, the vehicle comprises a control device, the control device and a computer readable storage medium, and a computer program product is used for executing the steps in the data processing method based on the vehicle, so that the target attitude data is obtained based on the target attitude data and the equipment binding relation. According to the method, the electronic equipment bound with the target object is determined, the reminding information associated with the electronic equipment is output, and reminding of the leaving condition of the electronic equipment is more accurate by considering the target posture data and the equipment binding relation.

Description

Technical Field

[0001] The present application relates to the technical field of vehicles, and in particular, to a data processing method, device, vehicle and related equipment based on a vehicle. Background Art

[0002] Currently, with the upgrade of the electronic and electrical architecture and the development of technology, many functions and components that can provide convenience for occupants have been installed in the vehicle cockpit. For example, many vehicles have configured wireless charging seats in the front row. When the mobile phone of the driver or the passenger needs to be charged and has a wireless charging function, the mobile phone only needs to be placed on the charging seat to be charged.

[0003] However, the situation where the mobile phone is left in the vehicle often occurs, but the reminder for the situation where the mobile phone is left is often inaccurate. Summary of the Invention

[0004] Embodiments of the present application provide a data processing method, device, vehicle and related equipment based on a vehicle, aiming to make the reminder of the situation where the electronic device is left more accurate.

[0005] To achieve the above object, according to the first aspect of the present application, a data processing method based on a vehicle is provided, including: Obtaining target attitude data of a target object in the vehicle and a device binding relationship; Determining an electronic device bound to the target object based on the target attitude data and the device binding relationship; Outputting a reminder message associated with the electronic device.

[0006] According to the second aspect of the present application, a control device is provided, including a processor, a memory, and a computer program or instruction, wherein the computer program or instruction is stored in the memory and is executed by the processor to implement any one of the above-mentioned data processing methods based on a vehicle.

[0007] According to the third aspect of the present application, a computer-readable storage medium is provided, on which a computer program or instruction is stored, and the computer program or instruction is executed by a processor to implement any one of the above-mentioned data processing methods based on a vehicle.

[0008] According to the fourth aspect of the present application, a computer program product is provided, including a computer program or instruction, and the computer program or instruction is executed by a processor to implement any one of the above-mentioned data processing methods based on a vehicle.

[0009] According to the fifth aspect of the present application, a vehicle is provided, including any one of the above-mentioned control devices.

[0010] In summary, the vehicle according to the embodiment of the present application includes a control device. The control device, computer-readable storage medium, and computer program product are used to execute the steps in the data processing method based on the vehicle, so as to determine the electronic device bound to the target object based on the target attitude data and device binding relationship, and output a reminder message associated with the electronic device. By considering the target attitude data and device binding relationship, the reminder of the left-behind situation of the electronic device is made more accurate.

[0011] Other features and advantages of the present application will be described in detail in the subsequent specific implementation part. Brief Description of the Drawings

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0013] In order to more completely understand the present application and its beneficial effects, the following will be described in conjunction with the drawings, where the same reference numerals represent the same parts in the following description.

[0014] Figure 1 is a schematic flowchart of an embodiment of the data processing method based on the vehicle provided in the exemplary embodiment of the present application; Figure 2 is a partial logic schematic diagram of the data processing method based on the vehicle provided in the exemplary embodiment of the present application; Figure 3 is another partial logic schematic diagram of the data processing method based on the vehicle provided in the exemplary embodiment of the present application; Figure 4 is still another partial logic schematic diagram of the data processing method based on the vehicle provided in the exemplary embodiment of the present application; Figure 5 is yet another partial logic schematic diagram of the data processing method based on the vehicle provided in the exemplary embodiment of the present application; Figure 6 is a schematic structural diagram of a control device provided in the exemplary embodiment of the present application; Figure 7 is another schematic structural diagram of a control device provided in the exemplary embodiment of the present application. Detailed Description of the Embodiments

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.

[0016] Based on the problems mentioned in the foregoing background art, in the related art, when using a vehicle to charge an electronic device such as a mobile phone, it often happens that the mobile phone is left in the vehicle. For example, the mobile phone is still placed on the charging stand after the driver or passenger gets out of the car.

[0017] Moreover, when there are people in both the driver's and passenger's seats, the reminder scheme for a charging mobile phone left in the vehicle generally does not distinguish between the driver's and passenger's mobile phones, which causes certain inconvenience. For example, when the driver needs to get out of the car temporarily, even if only the passenger's mobile phone is charging at this time, the vehicle system will issue a reminder of leaving behind, bringing a relatively bad experience to the passenger, that is, the reminder is accidentally triggered. Especially, most current vehicles are equipped with one or two wireless charging stands in the front row, and the configuration with only one wireless charging stand is more common. In this case, the probability of accidental triggering of the mobile phone leaving reminder is higher.

[0018] In addition, the reminder scheme for a charging mobile phone left in the vehicle generally issues a reminder only after the car door is opened. However, when the vehicle is parked by the roadside, the passenger will receive the reminder issued by the vehicle system only after opening the car door. At this time, the passenger needs to return to the vehicle to retrieve the mobile phone, but the car door will block the passage of other vehicles or pedestrians on the road. Therefore, the passenger needs to close the car door, get the mobile phone and then open the car door again, which will bring great inconvenience. Even worse, if the passenger Figure 1 temporarily keeps the car door open for convenience, the car door may be scratched or collided, causing substantial damage to the vehicle.

[0019] In response to this, the present application provides a data processing method, device, vehicle and related equipment based on a vehicle to at least partially solve the above problems.

[0020] According to a first aspect of the present application, a data processing method based on a vehicle is provided. Please refer to Figure 1 , the data processing method based on a vehicle includes the following steps: Step 101, obtaining target attitude data and device binding relationships of a target object in the vehicle.

[0021] Step 102, determining an electronic device bound by the target object based on the target attitude data and the device binding relationship.

[0022] Step 103, outputting reminder information associated with the electronic device.

[0023] In this embodiment, the target posture data of the target object in the vehicle refers to the data of the posture of the target object in the vehicle. The target object can be, for example, the driver, passenger, etc. in the vehicle.

[0024] The device binding relationship of the target object refers to the binding relationship between the target object and the electronic device, indicating that the electronic device belongs to the target object. The reminder information associated with the electronic device refers to the information used to prompt that the electronic device is left behind. This information can be set based on actual needs and is not limited here.

[0025] Among them, the electronic device can be a movable terminal device such as a smart phone, smart watch, tablet computer, laptop computer, etc.

[0026] It can be seen that in this embodiment, by considering the target posture data and the device binding relationship, the electronic device bound by the target object is determined to output the reminder information associated with the electronic device. When the electronic device is left in the vehicle, the output of the reminder information associated with the electronic device can be more accurate and timely.

[0027] In some embodiments, the above-mentioned electronic device is placed in the vehicle, that is, the above-mentioned reminder information is output for the electronic device placed in the vehicle, so as to give an accurate reminder when the electronic device is left in the vehicle.

[0028] In some embodiments, determining the electronic device bound by the target object based on the target posture data and the device binding relationship may include: detecting whether the target posture data meets a preset first posture condition, where the first posture condition represents the posture when the target object has the intention to open the door, for example, the posture when the target object is about to open the door; if the target posture data meets the first posture condition, in the device binding relationship, determine the electronic device bound by the target object.

[0029] It can be seen that if the target posture data meets the first posture condition, it indicates that the target object has the intention to open the door at this time and is about to open the door, and the electronic device bound by the target object is left in the vehicle. Therefore, the reminder information associated with the electronic device can be output to remind of the situation that the electronic device is left in the vehicle, making the reminder of the situation of the left electronic device more accurate and timely.

[0030] In some embodiments, before outputting the reminder information associated with the electronic device, it may further include: detecting that an object approaches the target door in the area where the target object is located. In this way, the output of the reminder information associated with the electronic device also needs to detect that an object approaches the target door before it can be executed, making the output timing of the reminder information more accurate and avoiding the mis-output of the reminder information.

[0031] Among them, the target door in the area where the target object is located can be, for example, the door closest to the target object. Taking the area where the target object is located as the area where the driver's seat is located as an example, the driver's door is the closest to the target object, so the target door can be the driver's door. Taking the area where the target object is located as the area where the passenger seat is located as an example, the passenger door is the closest to the target object, so the target door can be the passenger door.

[0032] In some embodiments, a proximity sensor is provided on the target door. The proximity sensor is used to detect whether there is an object approaching the target door in a way that does not require contact with the detection object. In this way, when the hand of the target object approaches the target door and is about to open the target door, it can be detected by the proximity sensor, making the detection of whether there is an object approaching the target door more convenient. Among them, the proximity sensor can include, for example, a capacitive proximity sensor, an inductive proximity sensor, a laser ranging proximity sensor, an ultrasonic proximity sensor, an infrared ranging proximity sensor, etc.

[0033] In some embodiments, the detection of whether there is an object approaching the target door is further described. Specifically, the data processing method based on the vehicle may further include: using the proximity sensor to determine whether there is an object within a preset distance range of the proximity sensor; if there is an object within the preset distance range of the proximity sensor, it is determined that there is an object approaching the target door; if there is no object within the preset distance range of the proximity sensor, it is determined that there is no object approaching the target door, thereby realizing the detection of whether there is an object approaching the target door.

[0034] Among them, the preset distance range can be set according to actual needs. For example, it can be set to any value within the range of 1 to 3 centimeters, so as to prevent false triggering as much as possible while ensuring the detection sensitivity.

[0035] In some other embodiments, the detection of whether there is an object approaching the target door is further described. Specifically, the data processing method based on the vehicle may further include: using the proximity sensor to determine the distance between the object approaching the proximity sensor and the proximity sensor; if the distance is less than the preset distance threshold, it is determined that there is an object approaching the target door; if the distance is greater than or equal to the distance threshold, it is determined that there is no object approaching the target door, thereby realizing the detection of whether there is an object approaching the target door.

[0036] Among them, the distance threshold can be set according to actual needs. For example, it can be set to any value between 1 and 3 centimeters, so as to prevent false triggering as much as possible while ensuring the detection sensitivity.

[0037] In some embodiments, the prerequisite for outputting reminder information associated with an electronic device may further include more requirements. For example, before outputting reminder information associated with an electronic device, it may further include: determining that the vehicle is in the parking gear (e.g., P gear), and / or determining that the seat belt in the area where the target object is located is in the unfastened state. In this way, the output timing of the reminder information will be more accurate to avoid misoutput of the reminder information.

[0038] In some embodiments, the output manner of the reminder information is described. Specifically, outputting reminder information associated with an electronic device includes at least one of the following steps: displaying an image corresponding to the reminder information through a display device of the vehicle, where the display device of the vehicle may be, for example, the central control display screen of the vehicle, and the image corresponding to the reminder information may include, for example, the text content, pattern content, etc. of the reminder information; playing a voice corresponding to the reminder information through an audio device of the vehicle, where the audio device of the vehicle may be, for example, the audio device of the vehicle, and the voice corresponding to the reminder information may include, for example, the voice content of the reminder information. Reminding the leaving of the electronic device through the image and / or voice can make the leaving reminder more obvious and timely.

[0039] In some embodiments, the electronic device is a device for charging the vehicle, so as to remind of the situation where the device for charging the vehicle is left in the vehicle. For example, when using the vehicle to charge the electronic device, it is very likely that the mobile phone is left in the vehicle, so timely reminder can be made through the output of the reminder information associated with the electronic device.

[0040] In some embodiments, the generation process of the device binding relationship can be implemented based on the charging step of the vehicle for the electronic device. Specifically, the device binding relationship can be generated through the following steps: detecting whether the target attitude data meets the preset second attitude condition and whether the number of electronic devices charged by the vehicle increases, where the second attitude condition represents the attitude when the target object uses the vehicle to charge the electronic device; if the target attitude data meets the second attitude condition and the number of electronic devices charged by the vehicle increases, it indicates that the target object is using the vehicle to charge the increased electronic device, so it can be determined that the increased electronic device is the electronic device of the target object, and a device binding relationship between the increased electronic device and the target object is generated to record this subordinate situation between the increased electronic device and the target object, thereby providing a basic condition for the intelligence of charging-related application scenarios and facilitating the improvement of the intelligence level of charging-related application scenarios and the convenient use of vehicle-related functions.

[0041] In some embodiments, another generation method of the device binding relationship is provided. Specifically, taking a vehicle including a first seat and a second seat, and the target pose data is obtained by an image acquisition device facing the first seat as an example, the data processing method based on the vehicle may further include: detecting whether the target pose data meets a preset second pose condition and whether the number of electronic devices charging the vehicle increases, where the second pose condition represents the pose when the target object charges the electronic device using the vehicle; if the target pose data does not meet the second pose condition and the number of electronic devices charging the vehicle increases, it indicates that the increased electronic device is not the electronic device of the target object. At this time, the increased electronic device can be used as the electronic device of the object in the area where the second seat is located, and the device binding relationship between the increased electronic device and the object in the area where the second seat is located is generated. In this way, when an image acquisition device is provided facing one of the first seat and the second seat, the binding between the object in the area where the other of the first seat and the second seat is located and the electronic device can also be realized.

[0042] In some embodiments, the function of the vehicle for charging electronic devices is described. Specifically, the vehicle includes a preset charging seat for charging electronic devices. For example, the electronic device can be charged by plugging the electronic device into the charging seat or placing it on the charging seat. Correspondingly, the data processing method based on the vehicle may further include: when detecting the charging connection establishment information based on the charging seat, determining that the number of electronic devices charging the vehicle increases. The charging connection establishment information can be obtained through the charging protocol of the charging seat, thus avoiding the need to additionally add a device to detect the increase in the number of electronic devices charging the vehicle.

[0043] In some embodiments, a deletion logic for the device binding relationship is provided. After obtaining the target pose data and the device binding relationship of the target object in the vehicle, it may further include: detecting whether the number of electronic devices charging the vehicle decreases; if the number of electronic devices charging the vehicle does not decrease, it indicates that the electronic device charging the vehicle has not been taken away, which does not meet the deletion condition of the device binding relationship. Therefore, the step of determining the electronic device bound by the target object based on the target pose data and the device binding relationship can be executed to determine whether it is necessary to output a reminder message associated with the electronic device to remind of the remaining situation of the electronic device.

[0044] In some embodiments, after detecting whether the number of electronic devices charging in the vehicle has decreased, it may further include: if the number of electronic devices charging in the vehicle has decreased and the target attitude data meets a preset third attitude condition, it indicates that the target object has stopped using the vehicle to charge the decreased electronic devices. Therefore, it can be determined that the decreased electronic devices are the electronic devices of the target object, and the device binding relationship between the decreased electronic devices and the target object is deleted to accurately delete the device binding relationship. Among them, the third attitude condition characterizes the attitude of the target object when it stops using the vehicle to charge the electronic devices.

[0045] In some embodiments, another deletion logic for the device binding relationship is provided. Specifically, taking the vehicle including a first seat and a second seat, and the target attitude data being obtained by an image acquisition device facing the first seat as an example, after detecting whether the number of electronic devices charging in the vehicle has decreased, it may further include: if the number of electronic devices charging in the vehicle has decreased and the target attitude data does not meet the preset third attitude condition, it indicates that the decreased electronic devices are not the electronic devices of the target object. At this time, the decreased electronic devices can be regarded as the electronic devices of the object in the area where the second seat is located, and the device binding relationship between the decreased electronic devices and the object in the area where the second seat is located is deleted. In this way, when an image acquisition device is provided facing one of the first seat and the second seat, it is also possible to delete the device binding relationship between the object in the area corresponding to the other of the first seat and the second seat and the electronic devices.

[0046] In some embodiments, since the function of the vehicle for charging electronic devices can be set in the area where the front-row seats in the vehicle are located, that is, the area where the driver's seat or the passenger seat in the vehicle is located, the first seat can be one of the driver's seat and the passenger seat in the vehicle, and the second seat can be the other of the driver's seat and the passenger seat in the vehicle. In this way, the target attitude image can be obtained for the driver of the driver's seat and the passenger of the passenger seat, making the obtained target attitude data more effective. Of course, in some other embodiments, at least one of the first seat and the second seat can also be a rear seat in the vehicle, which is not limited herein.

[0047] In some embodiments, a function of charging an electronic device possessed by a vehicle is described. Specifically, the vehicle includes a preset charging seat for charging the electronic device. For example, the electronic device can be charged by plugging the electronic device into the charging seat or placing it on the charging seat. Accordingly, the data processing method based on the vehicle may further include: when detecting the charging connection disconnection information based on the charging seat, determining that the number of electronic devices charged by the vehicle decreases. The charging connection disconnection information can be obtained through the charging protocol of the charging seat, thus avoiding the need to additionally add a device to detect the decrease in the number of electronic devices charged by the vehicle.

[0048] In some embodiments, the charging seat may specifically include a wireless charging seat for wirelessly charging the electronic device. For example, the electronic device can be wirelessly charged by placing it on the wireless charging seat, making the charging of the electronic device more convenient.

[0049] In some embodiments, the specific setting position of the charging seat may include the front seat area of the vehicle. Of course, in other embodiments, the specific setting position of the charging seat may also include the rear seat area of the vehicle, which is not limited herein.

[0050] In some embodiments, the target pose data can be determined from the pose image of the target object collected by the image acquisition device disposed in the vehicle. The pose image may include, for example, an RGB (Red, Green, Blue) image, an infrared image, etc. The image acquisition device can face the first seat and / or the second seat in the vehicle to collect the pose image of the target object.

[0051] In some embodiments, the determination method of the target pose data is described. Specifically, obtaining the target pose data of the target object in the vehicle may include: obtaining the pose image of the target object; identifying the limb key points in the pose image. The limb key points may include, for example, points such as the elbows, shoulders, and wrists of the human body; using the limb key points to determine the target pose data to make the target pose data more accurate.

[0052] In some embodiments, using the limb key points to determine the target pose data may include: using the coordinates of the limb key points in the pose image to determine the connection lines between at least two limb key points, such as the connection line between the shoulder and the elbow, the connection line between the elbow and the wrist, etc.; determining the target pose data according to the included angle information between the connection line and the preset direction. The preset direction may be, for example, the horizontal direction. The target pose data may include, for example, the included angle information between the connection line and the preset direction. Through the included angle information between the connection line and the preset direction, it can be made easier and more accurate to determine whether the target pose data meets at least one of the first pose condition, the second pose condition, and the third pose condition.

[0053] In some embodiments, the limb key points in the posture image are identified by a preset human posture estimation model, which may include a MediaPipe Pose model, for example. The MediaPipe Pose model is a model for real-time human posture estimation, which can identify and mark multiple limb key points of the human body in the posture image, and the multiple limb key points cover the main joints of the human body. Therefore, the limb key points in the posture image can be accurately identified by using the human posture estimation model.

[0054] In some embodiments, a vehicle-based data processing method is described by way of example. Specifically, in the related art, the reminder scheme for charging mobile phones in the car cannot distinguish between the driver and the co-driver, and can only send a reminder message after detecting that the door is opened, causing great inconvenience. However, the vehicle-based data processing method of this embodiment can sense the user's intention to open the door and get off the car in advance, and send a reminder message in time before the door is opened, correctly reminding the corresponding person to retrieve the phone.

[0055] Taking the target object as the front seat occupant of a vehicle, the charging base as a wireless charging base, and the electronic device as a mobile phone as an example, the vehicle-based data processing method may include: After the front-seat passenger gets on the bus and sits down, artificial intelligence (AI) and the camera inside the vehicle detect the front-seat passenger's posture in real time and identify their actions, that is, the action of placing the phone on the wireless charging stand. After identifying the action, it is further confirmed whether the phone and the wireless charging stand have established a connection, that is, whether the wireless charging stand has changed from no phone charging to phone charging. When this state change occurs, it can be determined that the person is the holder of the charging phone, and the relationship between the two is bound, that is, the information about the charging phone including the binding relationship between the two devices is obtained.

[0056] After that, until the front-seat occupant is about to get off the car, the front-seat occupant's posture and action must still be detected in real time, that is, the action of taking the phone off the wireless charging stand and the action of placing the phone on the wireless charging stand. First, if someone takes the phone away, the original device binding relationship between the phone and the front-seat occupant is released, and the information about the charged phone is reset. Then, consistent with the above process, the new information about the charged phone is obtained by using the action recognition of placing the phone and the state change of the wireless charging stand.

[0057] Finally, when the front-row passenger is about to get off the vehicle, a series of steps will be taken to determine whether a reminder for a left-behind charging mobile phone should be issued. First, when the vehicle is in the P gear, the in-vehicle system will continuously detect whether the seat belts of the front-row passengers are unfastened. If a person's seat belt is unfastened and the charging mobile phone belongs to that person, the in-vehicle camera and the AI model will be used to identify whether the person has the action of opening the door, that is, the posture when having the intention of opening the door. When both conditions are met, the in-vehicle system determines that the passenger's mobile phone will be left in the vehicle. Further, if a proximity sensor is installed on the door switch, when the hand of the person who unfastens the seat belt is close enough to the door switch, the sensor can sense that an object is approaching, and this sensed information is used as one of the additional judgment conditions. When all the above-mentioned conditions are met, the in-vehicle system determines that the person's mobile phone will be left in the vehicle and issues a reminder for the left-behind mobile phone.

[0058] Based on the above detection steps and judgment criteria, it will finally be decided whether a reminder for a left-behind mobile phone needs to be issued, including but not limited to displaying obvious reminder text on the vehicle's central control screen and emitting corresponding voice through the in-vehicle audio system.

[0059] Through the above process, it is possible to clearly identify the owner of the charging mobile phone, greatly reduce the occurrence frequency of false reminders, and clarify the intention of the person in the vehicle to open the door and get off before the door is opened through various sensing information. When the person's mobile phone is being charged through the wireless charging stand, reminders are issued in various ways such as using the vehicle's central control and the in-vehicle audio system, thus avoiding possible inconveniences and safety hazards.

[0060] In summary, the key points of the data processing method based on the vehicle are as follows: Using the in-vehicle camera and the trained AI model, obtain the posture detection information of the front-row passengers, judge whether the front-row passengers have the action of placing the mobile phone on the wireless charging stand, determine the person who places the mobile phone, and then judge whether the mobile phone starts charging through the change in the charging state of the wireless charging stand. Bind the charging mobile phone to its owner through these two pieces of information to obtain the information of the owner of the charging mobile phone. In addition, the information of the owner of the charging mobile phone can also be updated by judging whether the front-row passengers have the actions of taking and placing the mobile phone on the wireless charging stand and the change information of the charging state of the wireless charging stand. Thus, based on the precise relationship between the charging mobile phone and its owner, the occurrence frequency of false triggering of left-behind mobile phone reminders caused by not distinguishing between the driver's and passenger's seats can be greatly reduced.

[0061] Based on the in-vehicle camera and the trained AI model, it is determined whether the person has an action of opening the car door, and optionally, based on the sensing information of the proximity sensor on the door switch, it is determined whether the person's hand is close to the door switch. Combining the vehicle gear state information, the seat belt fastening / unfastening state information, and the information of the charging mobile phone, it is determined whether a reminder for leaving the mobile phone should be issued. This method can accurately sense in advance the intention of the person in the vehicle to open the door and get out of the car, and can further reduce the false trigger frequency of the reminder while issuing a reminder before the door is opened, providing great convenience.

[0062] In the vehicle-based data processing method, first, it is necessary to update the information of the charging mobile phone in real time through the AI model and the in-vehicle camera. Specifically, based on the posture recognition of the front-row person's action of taking or placing the mobile phone on the wireless charging stand, the driver or co-driver can be locked. In addition, based on the change of the wireless charging stand between the state of no mobile phone charging and the state of having a mobile phone charging, the charging mobile phone can be locked. Combining the recognition of the front-row person's action posture and the state change of the wireless charging stand, the front-row person can be bound to the charging mobile phone. On this basis, the information of the owner of the charging mobile phone is updated, including adding a set of device binding relationships for the charging mobile phone when the front-row person places the charging mobile phone, and deleting a set of device binding relationships for the charging mobile phone when the front-row person takes away the charging mobile phone.

[0063] Finally, based on the information of the owner of the charging mobile phone, when the front-row person is about to get out of the car, a series of steps will be taken to determine whether a reminder for leaving the charging mobile phone should be issued. First, as a prerequisite for unlocking the door, the vehicle should be in the P gear. After that, the vehicle system will continuously detect whether the seat belt of the front-row person is unfastened. If a person's seat belt is unfastened, it is necessary to determine whether the charging mobile phone belongs to this person according to the information of the owner of the charging mobile phone. If the above conditions are all met, the posture of this person will be further recognized. Specifically, through the in-vehicle camera and the AI model, it can be recognized whether this person has an action of opening the car door, that is, to determine whether this person will open the door and get out of the car without carrying the mobile phone. Further, for vehicles equipped with proximity sensors on the door switch, when this person's hand is close enough to the door switch, the sensor can sense the proximity of the hand, and this sensing information can be used as an additional judgment condition for whether this person will open the door and get out of the car without carrying the mobile phone. When the above conditions are all met, the vehicle system determines that this person's mobile phone will be left in the car, and issues a reminder for leaving the mobile phone in the form of text, pattern, voice, etc. through the central control screen and the speaker.

[0064] It should be noted that the in-vehicle devices and associated electrical structures related to the vehicle-based data processing method are set to be able to continue operating for a period of time after the vehicle is powered off.

[0065] Refer to Figure 2, the vehicle-based data processing method includes: Step 201: Detect the posture of the front-row person placing or removing the mobile phone on the wireless charging stand, and update the information of the mobile phone being charged according to the wireless charging stand status change information; Step 202: When the front-row person is about to get off the vehicle, obtain the gear information of the vehicle and the fastening / unfastening status information of the front-row seat belt, and combine the information of the person to whom the mobile phone being charged belongs to determine whether there is a possibility that a person opens the door and gets off without taking the mobile phone being charged; Step 203: If someone is about to get off and there is a possibility of leaving the mobile phone behind, detect the action of the person opening the door and the sensing information of the proximity sensor on the door switch (if there is a sensor), and comprehensively determine whether it is necessary to issue a reminder for leaving the mobile phone being charged; Step 204: If it is determined that a reminder for leaving the mobile phone being charged needs to be issued, output the reminder information through the vehicle's central control text, voice, etc.

[0066] In a specific implementation, step 201 first detects the posture of the front-row person placing or removing the mobile phone on the wireless charging stand. By using the in-vehicle camera of the vehicle, such as the cameras in the vehicle's Driver Monitoring System (DMS) and Occupant Monitoring System (OMS), the posture image of the front-row person can be obtained, and their posture can be detected in real time. As one of the achievable methods, the Pose module of the MediaPipe framework can be used for posture estimation and recognition. MediaPipe Pose can estimate the limb key points such as elbows, shoulders, and wrists in the human body in real time, and estimate the human body posture by connecting these limb key points. After completing the estimation of the human body posture, information such as the serial number and corresponding coordinates of the key points can be obtained, and actions can be recognized based on this information. For example, when the front-row person places or removes the mobile phone on the wireless charging stand, there are generally actions of turning the hand and head. At this time, the actions of placing or removing the mobile phone can be recognized through information such as the limb key point coordinates and the angle between the limb key point connection line and the horizontal direction. In some embodiments, the postures of placing and removing the mobile phone are basically the same, so the two actions can be regarded as one posture (hereinafter referred to as the target posture). Among them, the posture of the front-row person placing the mobile phone on the wireless charging stand, that is, the posture when the vehicle charges the mobile phone, and the posture of the front-row person removing the mobile phone from the wireless charging stand, that is, the posture when the vehicle stops charging the mobile phone.

[0067] In addition, it is necessary to obtain the status change information of the wireless charging stand, that is, to obtain the connection status between the mobile phone and the charging stand. For example, the wireless charging stand supports protocols such as Qi. When the mobile phone establishes a connection with the wireless charging stand, the vehicle can obtain the charging connection establishment information based on the wireless charging protocol to confirm whether a mobile phone starts charging on a certain wireless charging stand, that is, the wireless charging stand enters the state of having a mobile phone charging. Similarly, when the mobile phone disconnects from the wireless charging stand, the charging connection disconnection information can also be obtained based on the wireless charging protocol. Finally, it can be known the status change of the wireless charging stand between having no mobile phone charging and having a mobile phone charging, that is, the status change information of the wireless charging stand is obtained.

[0068] Therefore, based on the detection of the target posture of the front-row personnel and the status change information of the wireless charging stand, the mobile phone and the person to whom the mobile phone belongs can be further bound to obtain the device binding relationship. For example, if the target posture of the driver's seat personnel is recognized and any wireless charging stand establishes a charging connection with the mobile phone, it can be considered that the charging mobile phone is placed on the wireless charging stand by the driver's seat personnel and belongs to the driver's seat personnel, and then the device binding relationship between the two is added to the information of the charging mobile phone. In addition, if the target posture of the passenger seat personnel is recognized and the charging connection of any wireless charging stand with the mobile phone is interrupted, it can be considered that the charging mobile phone is taken away by the passenger seat personnel and belongs to the passenger seat personnel, and then the device binding relationship between the two is deleted from the information of the charging mobile phone.

[0069] It should be noted that this method is also applicable to the case where the vehicle only has DMS. When it is detected that the driver's seat personnel have an action of placing the mobile phone on the wireless charging stand and a new charging mobile phone is connected to the wireless charging stand, the charging mobile phone is bound to the driver's seat personnel as in the previous steps; if it is not detected that the driver's seat personnel have an action of placing the mobile phone on the wireless charging stand, but it is detected that a new charging mobile phone is connected to the wireless charging stand, it can be considered that the passenger seat personnel place the mobile phone on the wireless charging stand and the charging mobile phone is bound to the passenger seat personnel. In addition, the situation of taking away the mobile phone can be inferred by analogy. In summary, the information of the charging mobile phone can be updated in real time by using the in-vehicle camera and the MediaPipe Pose model.

[0070] Step 202 is mainly to initially detect the presence of a charging mobile phone left behind. First, obtain the gear information of the vehicle to confirm whether the vehicle is in the P gear. If it is in the P gear, continue to detect whether the front-row seat belt has been unfastened. If the seat belt is unfastened at this time, it can be considered that the front-row personnel in the corresponding position may get out of the vehicle. Then, according to the information of the charging mobile phone in step 201, confirm whether this person has a mobile phone that is charging. If so, enter step 203.

[0071] In step 203, it has been learned that there is a person who may get off the vehicle and the person's mobile phone is charging. It will be further confirmed whether the person has the intention to open the door and get off. Similar to step 201, the posture of the person is detected through the cameras of DMS and OMS and the MediaPipePose model to determine whether the person is in the posture of opening the door, that is, the posture when determining whether the person has the intention to open the door. If the person is in the posture of opening the door and there is a proximity sensor on the door switch, further judgment is required. The door switch here includes but is not limited to: physical door handles and push switches. As a feasible approach, the proximity sensing information can come from the capacitive proximity sensor mounted on the door switch. Its working principle is that when an object approaches the electrode plate, the change in capacitance can be detected by the sensor, and thus the approach of the object can be sensed. The capacitive proximity sensor has the advantages of adjustable sensitivity, a wide range of detected object types, and non-contact, so it is suitable for detecting whether the hand of a person in the vehicle is approaching.

[0072] Specifically, according to the person who is preliminarily confirmed to get off the vehicle in step 202, the capacitive proximity sensor information of the door switch at the corresponding position of the person is selectively obtained. In addition, the maximum detection distance of the capacitive proximity sensor for the human hand should be reasonably set. For example, it can be set to 1 - 3 cm to prevent false triggering as much as possible while ensuring detection sensitivity. When the proximity sensor detects that an object is approaching, it means that the person will open the door through the door switch. Finally, based on the recognition of the door-opening posture and the selectively obtained door proximity sensor information, it can be judged whether a reminder for the left-behind charging mobile phone needs to be issued. If a reminder for the left-behind charging mobile phone needs to be issued, step 204 is entered.

[0073] In step 204, reminder information will be sent in the form of classical Chinese or the like. For example, obvious framed text prompts will be displayed on the car's central control: "Don't forget to take away the mobile phone that is charging", and at the same time, the same voice reminder will be issued through the in-vehicle audio.

[0074] In summary, by using the common DMS and OMS cameras on the vehicle to detect the posture of the people in the vehicle and performing corresponding action recognition through the MediaPipe Pose model, the detection accuracy of the left-behind charging mobile phone can be improved without adding additional hardware configuration. First, by identifying the posture of placing or taking away the mobile phone and the change information of the wireless charging stand status, the owner of the charging mobile phone is updated in real time, and the driver's seat and the passenger seat are distinguished, greatly reducing the false reminder frequency caused by incorrect judgment of the mobile phone owner. Further, by identifying the posture of opening the car door, the information of the owner of the charging mobile phone, and the optional door switch proximity sensor information to determine whether to issue a left-behind reminder, a reminder can be issued in time before the corresponding person opens the car door. Thus, this method provides great convenience, while reducing the false reminder frequency and potential door collision hazards.

[0075] Referring to Figure 3 , the data processing method based on the vehicle may further include: Step 301, obtain the current information of the owner of the charging mobile phone, and execute Step 302; Step 302, obtain the vehicle gear information, and execute Step 303; Step 303, determine whether the vehicle is in the P gear. If so, execute Step 304, otherwise execute Step 311; Step 304, obtain the information of the owner of the charging mobile phone and the seat belt fastening / unfastening status information, and execute Step 305; Step 305, determine whether the mobile phone of the person who unfastens the seat belt is charging. If so, execute Step 306, otherwise execute Step 311; Step 306, obtain the posture detection information of the person, and execute Step 307; Step 307, determine whether the person makes an action of opening the car door. If so, enter Step 308, otherwise execute Step 311; Step 308, whether there is a proximity sensor for the car door switch. If so, execute Step 309, otherwise execute Step 310; Step 309, whether the proximity sensor detects that an object is approaching. If so, execute Step 310, otherwise execute Step 311; Step 310, display reminder text on the vehicle's central control screen and play reminder voice through the in-vehicle audio; Step 311, end the current detection and reminder, and wait for the next cycle.

[0076] Referring to Figure 4 , for vehicles with both DMS and OMS, the update method of the information of the owner of the charging mobile phone involved in the embodiment specifically includes: Step 401: Obtain the pose images of the driver and the front passenger, and execute Step 402; Step 402: Determine whether any of the front-row passengers has a gesture of picking up / taking a mobile phone. If so, execute Step 403; otherwise, execute Step 409; Step 403: Identify the corresponding person as the mobile phone holder, and execute Step 404; Step 404: Obtain the status change information of the wireless charging dock, and execute Step 405; Step 405: Determine whether the number of charging mobile phones has increased. If so, execute Step 407; otherwise, execute Step 406; Step 406: Determine whether the number of charging mobile phones has decreased. If so, execute Step 408; otherwise, execute Step 409; Step 407: Bind the mobile phone holder to the newly added charging mobile phone, and add this device binding relationship to the information of the charging mobile phone; Step 408: Unbind the mobile phone holder from the reduced charging mobile phone, and delete this device binding relationship from the information of the charging mobile phone; Step 409: End this detection cycle.

[0077] Refer to Figure 5 , for a vehicle with only DMS but no OMS, the specific method for updating the information of the charging mobile phone involved in the embodiment includes: Step 501: Only obtain the pose image of the driver, and execute Step 502; Step 502: Determine whether the driver has a gesture of picking up / taking a mobile phone. If so, execute Step 503; otherwise, execute Step 504; Step 503: Identify the driver as the mobile phone holder, and execute Step 505; Step 504: Identify the front passenger as the mobile phone holder, and execute Step 505; Step 505: Obtain the status change information of the wireless charging dock, and execute Step 506; Step 506: Determine whether the number of charging mobile phones has increased. If so, execute Step 508; otherwise, execute Step 507; Step 507: Determine whether the number of charging mobile phones has decreased. If so, execute Step 509; otherwise, execute Step 510; Step 508: Bind the mobile phone holder to the newly added charging mobile phone, and add this device binding relationship to the information of the charging mobile phone; Step 509: Unbind the mobile phone holder from the reduced charging mobile phone, and delete this device binding relationship from the information of the charging mobile phone; Step 510: End this detection cycle.

[0078] According to a second aspect of the present application, there is provided a control device, which includes a processor, a memory, and computer programs or instructions. The computer programs or instructions are stored in the memory and executed by the processor to implement any one of the vehicle-based data processing methods described above. The control device has all the beneficial effects of the above-mentioned vehicle-based data processing method, which will not be elaborated herein again.

[0079] In some embodiments, as Figure 6 shown, it shows a schematic structural diagram of the control device involved in the embodiments of the present application. Specifically, it includes: an image acquisition module 601, an action and posture recognition module 602, a wireless charging module 603, a charging mobile phone ownership information storage and update module 604, a gear position information module 605, a seat belt unfastening detection module 606, a door switch proximity sensing module 607, and a mobile phone left reminder module 608.

[0080] In the update of the charging mobile phone ownership information, first, the posture image of the front-row person is obtained through the image acquisition module 601, and then the action and posture recognition module 602 is used to recognize the posture image to determine whether the front-row person has an action of taking / picking up the mobile phone. If so, the person is identified as the mobile phone holder. Combining the detection of establishing and interrupting the charging connection of the mobile phone in the wireless charging module 603, the binding relationship between the mobile phone holder and the charging mobile phone device in the charging mobile phone ownership information can be added or deleted in the charging mobile phone ownership information storage and update module 604.

[0081] In the detection and reminder of the mobile phone left in the vehicle, first, the front-row person needs to be locked through the gear position information module 605 and the seat belt unfastening detection module 606, and the mobile phone left reminder module 608 determines whether the person who unfastens the seat belt in the P gear of the vehicle is one of the mobile phone holders stored in the charging mobile phone ownership information storage and update module 604. If so, the action and posture recognition module 602 is further used to recognize the posture image to determine whether the front-row person has an action of opening the door. If there is a door switch proximity sensor on the vehicle, the door switch proximity sensing module 607 is further used to determine whether the hand of the front-row person is close to the door switch, and finally the mobile phone left reminder module 608 determines whether a mobile phone left reminder should be issued.

[0082] In some embodiments, as Figure 7 shown, it shows another schematic structural diagram of the control device involved in the embodiments of the present application. Specifically: The control device may include a processor 701 with one or more processing cores, a memory 702 of one or more computer-readable storage media, a power supply 703, an input unit 704, and other components. Those skilled in the art can understand,Figure 7 The control device structure shown in Processor 701 is the control center of the control device, connecting various parts of the entire control device through various interfaces and lines. By running or executing software programs and / or modules stored in memory 702, and calling data stored in memory 702, it executes various functions of the control device and processes data, thereby monitoring the control device as a whole. Optionally, processor 701 may include one or more processing cores; preferably, processor 701 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into processor 701 either.

[0083] Memory 702 can be used to store software programs and modules. Processor 701 executes various functional applications and data processing by running the software programs and modules stored in memory 702. Memory 702 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.); the data storage area can store data created according to the use of the control device. In addition, memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, memory 702 may also include a memory control device to provide processor 701 with access to memory 702.

[0084] The control device also includes a power supply 703 for powering each component. Preferably, power supply 703 can be logically connected to processor 701 through a power management system, thereby realizing functions such as management of charging, discharging, and power consumption management through the power management system. Power supply 703 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0085] The control device may further include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0086] Although not shown, the control device may further include a display unit and the like, which will not be elaborated here. Specifically, in this embodiment, the processor 701 in the control device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702 to implement various functions, such as implementing the steps in the vehicle-based data processing method described in any one of the above.

[0087] According to a third aspect of the present application, there is provided a computer-readable storage medium, which may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk or an optical disc, etc. A computer program or instruction is stored thereon, and the computer program or instruction is executed by a processor to implement the vehicle-based data processing method described in any one of the above.

[0088] According to a fourth aspect of the present application, there is provided a computer program product, including a computer program or instruction, and the computer program or instruction is executed by a processor to implement the vehicle-based data processing method described in any one of the above.

[0089] According to a fifth aspect of the present application, there is provided a vehicle, including the control device described in any one of the above. The vehicle has all the beneficial effects of the above control device, which will not be elaborated here in the present application.

[0090] In some embodiments, the vehicle may be a fuel vehicle, a plug-in hybrid vehicle, a new energy vehicle, etc., and the present application does not make specific limitations thereto.

[0091] In the description of the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "a plurality" is two or more, unless otherwise specifically defined.

[0092] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not elaborated in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0093] Among the embodiments, implementation manners and related technical features of the present application, they can be combined and replaced with each other without conflict.

[0094] The above are only the preferred embodiments of the present application and do not impose any formal restrictions on the present application. However, any simple modifications, equivalent changes, and decorations made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still fall within the scope of the technical solution of the present application.

Claims

1. A data processing method based on a vehicle, characterized in that, Including: Obtain the target pose data and device binding relationship of the target object in the vehicle; Based on the target pose data and the device binding relationship, determine the electronic device bound to the target object; Output a reminder message associated with the electronic device.

2. The data processing method based on a vehicle according to claim 1, wherein The electronic device is placed in the vehicle.

3. The data processing method based on a vehicle according to claim 1, wherein The determining the electronic device bound to the target object based on the target pose data and the device binding relationship includes: Detect whether the target pose data meets a preset first pose condition, where the first pose condition characterizes the pose when the target object has the intention to open the door; If the target pose data meets the first pose condition, determine the electronic device bound to the target object in the device binding relationship.

4. The data processing method based on a vehicle according to claim 1, wherein Before outputting the reminder message associated with the electronic device, it further includes: Detect that an object approaches the target door in the area where the target object is located.

5. The data processing method based on a vehicle according to claim 4, wherein The target door is provided with a proximity sensor for detecting whether an object approaches the target door.

6. The data processing method based on a vehicle according to claim 5, wherein The data processing method based on the vehicle further includes: Use the proximity sensor to determine whether an object is within a preset distance range of the proximity sensor; If an object is within the preset distance range of the proximity sensor, determine that an object approaches the target door.

7. The data processing method based on a vehicle according to claim 1, characterized in that Before outputting the reminder message associated with the electronic device, it further includes: Determine that the vehicle is in the parking gear, and / or determine that the seat belt in the area where the target object is located is in an unlocked state.

8. The data processing method based on a vehicle according to claim 1, wherein Outputting the reminder message associated with the electronic device includes at least one of the following steps: Display an image corresponding to the reminder message through the display device of the vehicle; Play a voice corresponding to the reminder message through the audio device of the vehicle.

9. The data processing method based on a vehicle according to any one of claims 1 to 8, characterized in that The electronic device is a device for charging the vehicle.

10. The data processing method based on a vehicle according to claim 9, characterized in that The device binding relationship is generated through the following steps: Detect whether the target pose data meets a preset second pose condition and whether the number of electronic devices for charging the vehicle increases, where the second pose condition characterizes the pose when the target object uses the vehicle to charge an electronic device; If the target pose data meets the second pose condition and the number of electronic devices for charging the vehicle increases, generate a device binding relationship between the increased electronic device and the target object.

11. The data processing method based on a vehicle according to claim 9, wherein, The vehicle includes a first seat and a second seat, and the target pose data is obtained through an image acquisition device facing the first seat. The data processing method based on the vehicle further includes: Detect whether the target pose data meets a preset second pose condition and whether the number of electronic devices for charging the vehicle increases, where the second pose condition characterizes the pose when the target object uses the vehicle to charge an electronic device; If the target pose data does not meet the second pose condition and the number of electronic devices for charging the vehicle increases, generate a device binding relationship between the increased electronic device and the object in the area of the second seat.

12. The vehicle-based data processing method according to claim 10 or 11, characterized in that The vehicle includes a preset charging dock for charging an electronic device, and the vehicle-based data processing method further includes at least one of the following: When detecting the charging connection establishment information based on the charging dock, determining that the number of electronic devices charged using the vehicle increases.

13. The data processing method based on a vehicle according to claim 12, wherein The charging dock includes a wireless charging dock.

14. The data processing method based on a vehicle according to claim 9, wherein After obtaining the target pose data of the target object in the vehicle and the device binding relationship, it further includes: Detecting whether the number of electronic devices charged by the vehicle decreases; If the number of electronic devices charged by the vehicle does not decrease, performing the step of determining the electronic device bound to the target object based on the target pose data and the device binding relationship.

15. The vehicle-based data processing method according to claim 14, wherein After detecting whether the number of electronic devices charged by the vehicle decreases, it further includes: If the number of electronic devices charged by the vehicle decreases and the target pose data meets the preset third pose condition, deleting the device binding relationship between the reduced electronic device and the target object; Wherein, the third pose condition represents the pose when the target object stops charging the electronic device using the vehicle.

16. The vehicle-based data processing method according to claim 14, wherein The vehicle includes a first seat and a second seat, and the target pose data is obtained by an image acquisition device facing the first seat. After detecting whether the number of electronic devices charged by the vehicle decreases, it further includes: If the number of electronic devices charged by the vehicle decreases and the target pose data does not meet the preset third pose condition, deleting the device binding relationship between the reduced electronic device and the object in the area where the second seat is located; Wherein, the third pose condition represents the pose when the target object stops charging the electronic device using the vehicle.

17. The vehicle-based data processing method according to claim 11 or 16, characterized in that, The first seat is one of the driver's seat and the passenger seat in the vehicle, and the second seat is the other of the driver's seat and the passenger seat in the vehicle.

18. The vehicle-based data processing method according to any one of claims 14 to 16, characterized in that The vehicle includes a preset charging dock for charging an electronic device, and the vehicle-based data processing method further includes at least one of the following: When detecting the charging connection disconnection information based on the charging dock, determining that the number of electronic devices charged using the vehicle decreases.

19. The data processing method based on a vehicle according to claim 18, wherein The charging dock includes a wireless charging dock.

20. The data processing method based on a vehicle according to claim 1, wherein Obtaining the target pose data of the target object in the vehicle includes: Obtaining the pose image of the target object; Identifying the limb key points in the pose image; Using the limb key points to determine the target pose data.

21. The data processing method based on a vehicle according to claim 20, wherein The using the limb key points to determine the target pose data includes: Using the coordinates of the limb key points in the pose image to determine the connection lines between at least two of the limb key points; Determining the target pose data according to the included angle information between the connection lines and the preset direction.

22. The data processing method based on a vehicle according to claim 20, wherein The limb key points in the pose image are identified by a preset human pose estimation model.

23. A control device, characterized in that, It includes a processor, a memory, and a computer program or instructions, where the computer program or instructions are stored in the memory and executed by the processor to implement the vehicle-based data processing method according to any one of claims 1 to 22.

24. A computer-readable storage medium, characterized in that, A computer program or instructions are stored thereon, and the computer program or instructions are executed by a processor to implement the vehicle-based data processing method according to any one of claims 1 to 22.

25. A computer program product, characterized in that, It includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the vehicle-based data processing method according to any one of claims 1 to 22.

26. A vehicle, characterized in that, It includes the control device according to claim 23.

Images