Vehicle-based data processing method, device, vehicle and related equipment

Abstract

The present application relates to a vehicle-based data processing method, device, vehicle and related equipment. The vehicle includes a control device, a control device, a computer-readable storage medium, and a computer program product for executing the steps in the vehicle-based data processing method, thereby determining the electronic device bound to the target object based on the target posture data and the device binding relationship, and outputting reminder information associated with the electronic device. By considering the target posture data and the device binding relationship, the reminder of the legacy situation of the electronic device is made more accurate.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle-based data processing method, device, vehicle, and related equipment. Background Art

[0002] With the advancement of electronic and electrical architectures and technological advancements, car cabins are now equipped with numerous features and components that provide convenience to passengers. For example, many cars now feature wireless charging pads in the front seats. When the driver or passenger's phone needs charging and has wireless charging capability, they simply place it on the pad to charge.

[0003] However, it often happens that a mobile phone is left in a car, but the reminder of the situation of the mobile phone being left behind is often not accurate enough. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle-based data processing method, apparatus, vehicle, and related equipment, aiming to make reminders of legacy situations of electronic devices more accurate.

[0005] To achieve the above objectives, according to a first aspect of the present application, a vehicle-based data processing method is provided, comprising:

[0006] Obtain target posture data and device binding relationship of target objects in the vehicle;

[0007] Determining the electronic device bound to the target object based on the target posture data and the device binding relationship;

[0008] Outputting reminder information associated with the electronic device.

[0009] According to a second aspect of the present application, a control device is provided, comprising a processor, a memory, and a computer program or instructions, wherein the computer program or instructions are stored in the memory and executed by the processor to implement any one of the vehicle-based data processing methods described.

[0010] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program or instructions are stored. The computer program or instructions are executed by a processor to implement any one of the vehicle-based data processing methods described above.

[0011] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement any one of the vehicle-based data processing methods.

[0012] According to a fifth aspect of the present application, a vehicle is provided, comprising a control device as described in any one of the items.

[0013] To summarize, the vehicle of the embodiment of the present application includes a control device, a computer-readable storage medium, and a computer program product for executing the steps in the vehicle-based data processing method, thereby determining the electronic device bound to the target object based on the target posture data and the device binding relationship, and outputting reminder information associated with the electronic device. By considering the target posture data and the device binding relationship, the reminder of the legacy situation of the electronic device is made more accurate.

[0014] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0016] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0017] Figure 1 is a flow chart of an example of a vehicle-based data processing method provided in an exemplary embodiment of the present application;

[0018] Figure 2 It is a logic diagram of a portion of a vehicle-based data processing method provided in an exemplary embodiment of the present application;

[0019] Figure 3 is another partial logic diagram of the vehicle-based data processing method provided in an exemplary embodiment of the present application;

[0020] Figure 4 It is a further logic diagram of a vehicle-based data processing method provided in an exemplary embodiment of the present application;

[0021] Figure 5 It is another logic diagram of a vehicle-based data processing method provided in an exemplary embodiment of the present application;

[0022] Figure 6 is a structural schematic diagram of a control device provided in an exemplary embodiment of the present application;

[0023] Figure 7 Schematic diagram of another structure of the control device provided in the exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0025] Based on the problems mentioned in the above background technology, in the related technology, when using a vehicle to charge electronic devices such as mobile phones, it often happens that the mobile phone is left in the car, for example, the mobile phone is still placed on the charging stand after the driver or passenger gets out of the car.

[0026] Furthermore, when both the driver and passenger seats are occupied, the reminder for phones left charging inside the car rarely differentiates between the driver's and passenger's phones, causing some inconvenience. For example, if the driver needs to leave the car for an emergency, even if only the passenger's phone is charging, the car system will issue a reminder, creating a poor user experience for the passenger and potentially causing a false triggering of the reminder. This is especially true given that most current vehicles have one or two wireless charging stations in the front row, with the majority having only one. This increases the probability of false triggering of the reminder.

[0027] In addition, the reminder scheme for leaving a charging phone in the car basically only issues a reminder after the car door is opened. However, when the vehicle is parked on the roadside, the passenger will not receive a reminder from the car system until the car door is opened. At this time, it is necessary to return to the car to retrieve the phone, but the car door will block the passage of other vehicles or pedestrians on the road. Therefore, the passenger will need to close the car door, get the phone, and then reopen the door, which will cause great inconvenience. What's worse, if the passenger is greedy, Figure 1 If you keep the door open for a long time, it may be scratched or collided, causing substantial damage to the vehicle.

[0028] To this end, the present application provides a vehicle-based data processing method, apparatus, vehicle, and related equipment to at least partially solve the above-mentioned problems.

[0029] According to the first aspect of the present application, a vehicle-based data processing method is provided. Figure 1 , the vehicle-based data processing method includes the following steps:

[0030] Step 101: Acquire target posture data and device binding relationship of a target object in a vehicle.

[0031] Step 102: Determine the electronic device bound to the target object based on the target posture data and the device binding relationship.

[0032] Step 103: output reminder information associated with the electronic device.

[0033] In this embodiment, the target posture data of the target object in the vehicle refers to data of the posture of the target object in the vehicle. The target object may be, for example, a driver, a passenger, etc. in the vehicle.

[0034] The target object's device binding relationship refers to the binding relationship between the target object and the electronic device, indicating that the electronic device belongs to the target object. The reminder information associated with the electronic device refers to the information used to indicate that the electronic device has been left behind. This information can be set based on actual needs and is not limited here.

[0035] Among them, the electronic device can be a mobile terminal device such as a smart phone, a smart watch, a tablet computer, a laptop computer, etc.

[0036] It can be seen that this embodiment determines the electronic device bound to the target object by considering the target posture data and the device binding relationship, so as to output reminder information associated with the electronic device. When faced with a situation where the electronic device is left in the car, the output of the reminder information associated with the electronic device can be more accurate and timely.

[0037] In some embodiments, the electronic device is placed in a vehicle, that is, the reminder information is output for the electronic device placed in the vehicle, so as to provide an accurate reminder when the electronic device is left in the vehicle.

[0038] In some embodiments, determining the electronic device bound to the target object based on the target posture data and the device binding relationship may include: detecting whether the target posture data meets a preset first posture condition, wherein the first posture condition represents the posture of the target object when it intends to open the car door, for example, it may be the posture of the target object when it is about to open the car door; if the target posture data meets the first posture condition, determining the electronic device bound to the target object in the device binding relationship.

[0039] It can be seen that if the target posture data meets the first posture condition, it indicates that the target object has the intention to open the car door and is about to open the car door, and the electronic device bound to the target object is left in the vehicle. Therefore, reminder information associated with the electronic device can be output to remind that the electronic device is left in the vehicle, making the reminder of the electronic device left behind more accurate and timely.

[0040] In some embodiments, before outputting the reminder information associated with the electronic device, it may also include: detecting that an object is approaching the target door in the area where the target object is located. In this way, the output of the reminder information associated with the electronic device also requires detecting that an object is approaching the target door before it can be executed, making the output timing of the reminder information more accurate and avoiding the erroneous output of the reminder information.

[0041] The target door in the target object's area can be, for example, the door closest to the target object. For example, if the target object is located in the driver's seat area, the driver's door is closest to the target object, and thus the target door can be the driver's door. For example, if the target object is located in the passenger seat area, the passenger's door is closest to the target object, and thus the target door can be the passenger's door.

[0042] In some embodiments, the target door is equipped with a proximity sensor that detects the presence of an object approaching the target door without contacting the target object. This allows the proximity sensor to detect when the target object's hand approaches the target door and is about to open it, making it easier to detect the presence of an object. Examples of proximity sensors include capacitive proximity sensors, inductive proximity sensors, laser ranging proximity sensors, ultrasonic proximity sensors, and infrared ranging proximity sensors.

[0043] In some embodiments, the detection of whether an object is approaching a target door is further described. Specifically, the vehicle-based data processing method may further include: utilizing a proximity sensor to determine whether an object is approaching within a preset distance range of the proximity sensor; if an object is approaching within the preset distance range of the proximity sensor, determining that an object is approaching the target door; if no object is approaching within the preset distance range of the proximity sensor, determining that no object is approaching the target door, thereby achieving detection of whether an object is approaching the target door.

[0044] Among them, the preset distance range can be set based on actual needs, for example, it can be set to a distance range with any value between 1 and 3 centimeters, so as to prevent false triggering as much as possible while ensuring detection sensitivity.

[0045] In other embodiments, the detection of whether an object is approaching a target door is further described. Specifically, the vehicle-based data processing method may further include: using a proximity sensor to determine the distance between the object approaching the proximity sensor and the proximity sensor; if the distance is less than a preset distance threshold, determining that an object is approaching the target door; if the distance is greater than or equal to the distance threshold, determining that no object is approaching the target door, thereby achieving detection of whether an object is approaching the target door.

[0046] The distance threshold can be set based on actual needs, for example, it can be set to any value between 1 and 3 centimeters, so as to prevent false triggering as much as possible while ensuring detection sensitivity.

[0047] In some embodiments, the prerequisites for outputting reminder information associated with an electronic device may include additional requirements. For example, before outputting the reminder information associated with the electronic device, the prerequisites may include: determining that the vehicle is in park (e.g., P gear) and / or determining that the seat belts in the area where the target object is located are unfastened. This allows for more accurate output of the reminder information, preventing erroneous output of the reminder information.

[0048] In some embodiments, the output method of the reminder information is described. Specifically, outputting the reminder information associated with the electronic device includes at least one of the following steps: displaying an image corresponding to the reminder information through the vehicle's display device, wherein the vehicle's display device can be, for example, the vehicle's central control display screen, and the image corresponding to the reminder information can include, for example, the text content, pattern content, etc. of the reminder information; playing the voice corresponding to the reminder information through the vehicle's audio device, wherein the vehicle's audio device can be, for example, the vehicle's sound device, and the voice corresponding to the reminder information can include, for example, the voice content of the reminder information. Reminding the user of the legacy of the electronic device through images and / or voice can make the legacy reminder more obvious and timely.

[0049] In some embodiments, the electronic device is a vehicle charging device, and thus a reminder is provided in the event that the vehicle charging device is left in the vehicle. For example, when a vehicle is used to charge an electronic device, a mobile phone is likely to be left in the vehicle, so a timely reminder can be provided by outputting a reminder information associated with the electronic device.

[0050] In some embodiments, the process of generating a device binding relationship can be implemented based on the steps of charging the electronic device by the vehicle. Specifically, the device binding relationship can be generated by the following steps: detecting whether the target posture data meets the preset second posture condition and whether the number of electronic devices charged by the vehicle has increased, wherein the second posture condition represents the posture of the target object when using the vehicle to charge the electronic device; if the target posture data meets the second posture condition and the number of electronic devices charged by the vehicle has increased, it indicates that the target object is using the vehicle to charge the increased electronic devices, so it can be determined that the increased electronic device is the electronic device of the target object, and a device binding relationship between the increased electronic device and the target object is generated to record this subordinate situation between the increased electronic device and the target object, thereby providing basic conditions for the intelligence of charging-related application scenarios, facilitating the improvement of the intelligence level of charging-related application scenarios and the convenient use of vehicle-related functions.

[0051] In some embodiments, another method for generating a device binding relationship is provided. Specifically, taking the case where a vehicle includes a first seat and a second seat, and the target posture data is obtained by an image acquisition device facing the first seat as an example, the vehicle-based data processing method may also include: detecting whether the target posture data meets a preset second posture condition and whether the number of electronic devices being charged by the vehicle has increased, wherein the second posture condition represents the posture of the target object when using the vehicle to charge the electronic device; if the target posture data does not meet the second posture condition and the number of electronic devices being charged by the vehicle has increased, it indicates that the increased electronic device is not the electronic device of the target object. At this time, the increased electronic device can be regarded as the electronic device of the object in the area where the second seat is located, and a device binding relationship is generated between the increased electronic device and the object in the area where the second seat is located. In this way, when an image acquisition device is provided facing one of the first seat and the second seat, binding between the object and the electronic device in the area where the other of the first seat and the second seat is located can also be achieved.

[0052] In some embodiments, the function of a vehicle for charging electronic devices is described. Specifically, the vehicle includes a preset charging base, which is used to charge electronic devices. For example, the electronic device can be charged by plugging it into the charging base or placing it on the charging base. Accordingly, the vehicle-based data processing method may further include: when detecting charging connection establishment information based on the charging base, determining that the number of electronic devices charged by the vehicle has increased. The charging connection establishment information can be obtained through the charging protocol of the charging base, thereby avoiding the need for additional devices to detect the increase in the number of electronic devices charged by the vehicle.

[0053] In some embodiments, device binding relationship deletion logic is provided. After obtaining the target posture data and device binding relationship of the target object in the vehicle, the following may also be included: detecting whether the number of electronic devices being charged by the vehicle has decreased; if the number of electronic devices being charged by the vehicle has not decreased, it indicates that the electronic devices being charged by the vehicle have not been removed and do not meet the device binding relationship deletion conditions. Therefore, the step of determining the electronic devices bound to the target object based on the target posture data and the device binding relationship may be executed to determine whether it is necessary to output reminder information associated with the electronic devices to remind the electronic devices that have been left behind.

[0054] In some embodiments, after detecting whether the number of electronic devices being charged by the vehicle has decreased, the method may further include: if the number of electronic devices being charged by the vehicle has decreased and the target posture data satisfies a preset third posture condition, indicating that the target object has stopped using the vehicle to charge the decreased electronic devices, the decreased electronic devices may be determined to be the target object's electronic devices, and the device binding relationship between the decreased electronic devices and the target object may be deleted to achieve accurate deletion of the device binding relationship. The third posture condition represents the posture of the target object when it stops using the vehicle to charge the electronic devices.

[0055] In some embodiments, another deletion logic of the device binding relationship is provided. Specifically, taking the case where the vehicle includes a first seat and a second seat, and the target posture data is obtained by an image acquisition device facing the first seat as an example, after detecting whether the number of electronic devices being charged by the vehicle is reduced, it can also include: if the number of electronic devices being charged by the vehicle is reduced, and the target posture data does not meet the preset third posture condition, indicating that the reduced electronic devices are not the electronic devices of the target object, the reduced electronic devices can be regarded as the electronic devices of the objects in the area where the second seat is located, and the device binding relationship between the reduced electronic devices and the objects in the area where the second seat is located is deleted. In this way, when an image acquisition device is provided facing one of the first seat and the second seat, the device binding relationship between the objects in the area where the other of the first seat and the second seat is located and the electronic devices can also be deleted.

[0056] In some embodiments, since the vehicle's electronic device charging function can be located in the area of ​​the vehicle's front seats, i.e., the area where the driver's seat or the passenger seat is located, the first seat can be one of the driver's seat and the passenger seat, and the second seat can be the other of the two. This allows target posture images to be acquired for the driver in the driver's seat and the passenger in the passenger seat, making the acquired target posture data more effective. Of course, in other embodiments, at least one of the first and second seats can also be a rear seat in the vehicle, without limitation.

[0057] In some embodiments, the function of a vehicle for charging electronic devices is described. Specifically, the vehicle includes a preset charging base, which is used to charge electronic devices. For example, the electronic device can be charged by plugging it into the charging base or placing it on the charging base. Accordingly, the vehicle-based data processing method may further include: when detecting charging connection disconnection information based on the charging base, determining that the number of electronic devices charged by the vehicle has decreased. The charging connection disconnection information can be obtained through the charging protocol of the charging base, thereby avoiding the need for additional devices to detect the reduction in the number of electronic devices charged by the vehicle.

[0058] In some embodiments, the charging stand may specifically include a wireless charging stand, which is used to wirelessly charge the electronic device. For example, the electronic device can be wirelessly charged by placing it on the wireless charging stand, making charging of the electronic device more convenient.

[0059] In some embodiments, the specific location of the charging station may include the front seat area of ​​the vehicle. Of course, in other embodiments, the specific location of the charging station may also include the rear seat area of ​​the vehicle, which is not limited here.

[0060] In some embodiments, the target pose data can be determined by capturing a pose image of the target object using an image capture device located within the vehicle. The pose image may include, for example, an RGB (Red, Green, Blue) image or an infrared image. The image capture device may capture the pose image of the target object while facing a first seat and / or a second seat within the vehicle.

[0061] In some embodiments, a method for determining target posture data is described. Specifically, obtaining target posture data of a target object within a vehicle may include: obtaining a posture image of the target object; identifying limb key points in the posture image, such as the elbow, shoulder, and wrist of a human body; and determining target posture data using the limb key points to make the target posture data more accurate.

[0062] In some embodiments, determining target posture data using limb key points may include: determining a connecting line between at least two limb key points, such as a connecting line between a shoulder and an elbow, or a connecting line between an elbow and a wrist, using the coordinates of the limb key points in the posture image; and determining the target posture data based on angle information between the connecting line and a preset direction. The preset direction may be, for example, a horizontal direction, and the target posture data may include angle information between the connecting line and the preset direction. Using the angle information between the connecting line and the preset direction, determining whether the target posture data satisfies at least one of the first posture condition, the second posture condition, and the third posture condition can be simplified and more accurate.

[0063] In some embodiments, limb key points in pose images are identified using a preset human pose estimation model, such as the MediaPipe Pose model. The MediaPipe Pose model is a real-time human pose estimation model that can identify and label multiple limb key points in pose images. These key points cover all major joints of the human body. Therefore, the human pose estimation model can accurately identify limb key points in pose images.

[0064] In some embodiments, a vehicle-based data processing method is described as an example. Specifically, in related art, legacy reminder solutions for charging mobile phones in vehicles fail to distinguish between the driver and the front passenger, and only issue a reminder after detecting that the vehicle door is open, causing significant inconvenience. However, the vehicle-based data processing method of this embodiment can proactively detect the user's intention to open the door and exit the vehicle, and promptly issue a reminder before the door is opened, accurately reminding the appropriate person to retrieve the phone.

[0065] Taking the target object as the front-seat passenger of a vehicle, the charging base as a wireless charging base, and the electronic device as a mobile phone as an example, the vehicle-based data processing method may include:

[0066] After the front-seat passenger enters the vehicle and takes their seat, artificial intelligence (AI) and the vehicle's internal cameras detect their posture in real time and identify their movements, specifically the placement of their phone on the wireless charging pad. After identifying this movement, the system further verifies whether a connection has been established between the phone and the wireless charging pad, specifically whether the wireless charging pad has changed from charging a phone to charging a phone. Upon this change in state, the user is identified as the owner of the charging phone, and the two are linked, obtaining information about the charging phone, including the binding relationship between the two devices.

[0067] After that, until the front-seat passenger is about to exit the vehicle, the system must still detect their posture and identify their movements in real time, namely, the action of removing the phone from the wireless charging pad and the action of placing the phone on the wireless charging pad. First, if someone removes the phone, the original device binding between the phone and the front-seat passenger is released, and the charging phone's ownership information is reset. Then, consistent with the previous process, the new charging phone's ownership information is obtained by using the action recognition of placing the phone and the state change of the wireless charging pad.

[0068] Finally, when the front passenger is about to get off the vehicle, a series of steps will be taken to determine whether a reminder should be issued for the charging phone. First, when the vehicle is in P gear, the car system will continuously detect whether the front passenger's seat belt is unbuckled. If someone's seat belt is unbuckled and the charging phone belongs to that person, the in-car camera and AI model will be used to identify whether the person has opened the door. The action of opening the door is the posture when the intention is to open the door. When both conditions are met, the car system determines that the passenger's phone will be forgotten in the car. Furthermore, if the door switch is equipped with a proximity sensor, when the hand of the person unbuckled the seat belt is close enough to the door switch, the sensor can sense the approach of an object, and use this perception information as one of the additional judgment conditions. When all the aforementioned conditions are met, the car system determines that the person's phone will be forgotten in the car and issues a reminder for the phone to be left behind.

[0069] Based on the aforementioned detection steps and judgment basis, it will ultimately be decided whether a mobile phone left-behind reminder needs to be issued, including but not limited to displaying obvious reminder text on the vehicle's central control screen and issuing corresponding voice through the vehicle's audio system.

[0070] Through the above process, the person who is charging the mobile phone can be identified, which greatly reduces the frequency of false reminders. In addition, through various sensing information, the intention of the person in the car to open the door and get out can be identified before the door is opened. When the person's mobile phone is charging through the wireless charging seat, reminders can be issued through various methods such as the car's central control and car audio, thereby avoiding possible inconvenience and safety hazards.

[0071] In summary, the key points of vehicle-based data processing methods are:

[0072] Using the in-car camera and the trained AI model, the posture detection information of the front-seat occupant is obtained to determine whether the front-seat occupant has placed the phone on the wireless charging stand, determine the person who placed the phone, and then determine whether the phone has started charging based on the change in the charging status of the wireless charging stand. The charging phone is bound to the person to whom it belongs through these two pieces of information, and the information about the charging phone is obtained. In addition, the information about the charging phone can also be updated by determining whether the front-seat occupant has taken and placed the phone on the wireless charging stand, as well as the charging status change information of the wireless charging stand. Therefore, based on the precise relationship between the charging phone and the person to whom it belongs, the frequency of false triggering of the phone left behind reminder due to not distinguishing between the main driver and the co-driver can be greatly reduced.

[0073] Using the in-car camera and a trained AI model, the system determines whether the person intends to open the door. It also optionally uses the proximity sensor on the door switch to determine whether the person's hand is near the door switch. Combined with information about the vehicle's gear position, seatbelt fastening and unfastening status, and the location of the charged phone, it determines whether a phone left behind reminder should be issued. This approach accurately detects the person's intention to open the door and exit the vehicle, issues a reminder before the door is opened, and further reduces the frequency of false triggering of reminders, providing significant convenience.

[0074] In the vehicle-based data processing method, it is first necessary to update the information about the charging mobile phone in real time through the AI ​​model and the camera inside the vehicle. Specifically, based on the posture recognition of the front passenger taking or putting the mobile phone on the wireless charging seat, the main driver or co-driver can be locked. In addition, the charging mobile phone can be locked according to the change of the wireless charging seat between the state of no mobile phone charging and the state of mobile phone charging. Combined with the recognition of the posture of the front passenger and the state change of the wireless charging seat, the front passenger can be bound to the charging mobile phone, and the ownership information of the charging mobile phone can be updated on this basis, including adding a group of charging mobile phone device binding relationships when the front passenger places the charging mobile phone, and deleting a group of charging mobile phone device binding relationships when the front passenger takes the charging mobile phone.

[0075] Finally, based on the information about the charging phone's ownership, a series of steps are used to determine whether a left-behind reminder should be issued when the front-seat occupant is about to exit the vehicle. First, as a prerequisite for unlocking the doors, the vehicle must be in P gear. Thereafter, the system continuously monitors whether the front-seat occupant's seatbelt is unbuckled. If a seatbelt is unbuckled, the system determines that the phone belongs to that person based on the information about the charging phone. If all of the above conditions are met, the system then further identifies the person's posture. Specifically, using in-vehicle cameras and AI models, the system can detect whether the person has attempted to open the door, thereby determining whether they will open the door and exit the vehicle without their phone. Furthermore, for vehicles equipped with proximity sensors on the door switches, the system can detect the presence of a hand when the person's hand is sufficiently close to the door switch, and use this information as an additional criterion for determining whether the person will open the door and exit the vehicle without their phone. If all of the above conditions are met, the system determines that the person's phone is likely to be left behind in the vehicle and issues a left-behind reminder via text, images, or voice commands on the central control screen and audio system.

[0076] It should be noted that the on-board equipment and associated electrical structures related to the vehicle-based data processing method are configured to be able to continue operating for a period of time after the vehicle is turned off.

[0077] Reference Figure 2, vehicle-based data processing methods include:

[0078] Step 201: Detecting the posture of the front-seat passenger placing or removing a mobile phone on the wireless charging stand, and updating the information of the charging mobile phone according to the status change information of the wireless charging stand;

[0079] Step 202: When the front-seat occupant is about to exit the vehicle, the vehicle's gear position information and the front seatbelt fastening and unfastening status information are obtained, and combined with the information of the person whose phone is being charged, it is determined whether there is a possibility that the person has opened the door and exited the vehicle without taking the phone.

[0080] Step 203: If someone is about to get off the vehicle and may have left their phone behind, the person's door-opening action and the proximity sensor on the door switch (if equipped) are detected to comprehensively determine whether a reminder of the left-behind charging phone is required.

[0081] Step 204: If it is determined that a reminder of the left-behind charging phone is required, the reminder information is output via the vehicle's central control text, voice, etc.

[0082] In a specific implementation, step 201 first detects the posture of the front-seat passenger when placing or removing a phone from the wireless charging pad. Using the vehicle's in-vehicle cameras, such as those in the driver monitoring system (DMS) and occupant monitoring system (OMS), the front-seat passenger's posture image can be captured and detected in real time. One possible method is to use the Pose module of the MediaPipe framework for posture estimation and recognition. MediaPipe Pose estimates key limb points, such as the elbow, shoulder, and wrist, in real time and estimates human posture by connecting these limb key points. After posture estimation, information such as the key point sequence number and corresponding coordinates can be obtained, and action recognition can be performed based on this information. For example, when the front-seat passenger places or removes a phone from the wireless charging pad, they typically rotate their hands and head. In this case, the placement or removal action can be identified based on information such as the coordinates of the limb key points and the angle between the connecting line and the horizontal. In some embodiments, the postures for placing and removing a phone are essentially the same, so the two actions can be considered as one posture (hereinafter referred to as the target posture). Specifically, the posture of the front-seat passenger placing the phone on the wireless charging pad is the posture when the phone is being charged by the vehicle, and the posture of the front-seat passenger removing the phone from the wireless charging pad is the posture when the phone stops being charged by the vehicle.

[0083] In addition, it is necessary to obtain the status change information of the wireless charging stand, that is, to obtain the connection status of the mobile phone and the charging stand. For example, the wireless charging stand can support protocols such as Qi. When the mobile phone establishes a connection with the wireless charging stand, the vehicle can obtain the charging connection establishment information based on the wireless charging protocol to confirm whether a mobile phone has started to charge on a certain wireless charging stand, that is, the wireless charging stand enters the state where a mobile phone is charging. Similarly, when the mobile phone is disconnected from the wireless charging stand, the charging connection disconnection information can also be obtained based on the wireless charging protocol. Ultimately, it can be known that the state of the wireless charging stand changes from no mobile phone charging to charging with a mobile phone, that is, the state change information of the wireless charging stand is obtained.

[0084] Therefore, based on the detection of the target posture of the front-seat occupant and the status change information of the wireless charging seat, the mobile phone can be further bound to the person to whom the mobile phone belongs, and a device binding relationship can be obtained. For example, if the target posture of the person in the main driver's seat is identified, and any wireless charging seat establishes a charging connection with the mobile phone, it can be considered that the charging mobile phone is placed on the wireless charging seat by the person in the main driver's seat and belongs to the person in the main driver's seat, and the device binding relationship between the two is added to the charging mobile phone ownership information. In addition, if the target posture of the person in the passenger seat is identified, and the charging connection between any wireless charging seat and the mobile phone is interrupted, it can be considered that the charging mobile phone is taken away by the person in the passenger seat and belongs to the person in the passenger seat, and the device binding relationship between the two is deleted from the charging mobile phone ownership information.

[0085] It is worth noting that this method is also applicable to the case where the vehicle only has a DMS. When it is detected that the person in the main driver's seat has placed a mobile phone on the wireless charging seat, and a new charging mobile phone is connected to the wireless charging seat, the charging mobile phone is bound to the person in the main driver's seat as in the previous steps; if the person in the main driver's seat is not detected to have placed a mobile phone on the wireless charging seat, but a new charging mobile phone is detected to be connected to the wireless charging seat, it can be considered that the passenger seat has placed the mobile phone on the wireless charging seat, and the charging mobile phone is bound to the passenger seat. In addition, the situation of taking away the mobile phone can be deduced by analogy. In summary, the information of the charging mobile phone can be updated in real time by using the in-car camera and the MediaPipe Pose model.

[0086] Step 202 primarily performs a preliminary check for left-behind charging phones. First, the vehicle's gear position information is obtained to determine whether the vehicle is in P gear. If so, the system then checks to see if the front seatbelts are unfastened. If so, it indicates that the front seat occupant in that position may have exited the vehicle. Based on the charging phone information from step 201, the system then determines whether the occupant has a charging phone. If so, the system proceeds to step 203.

[0087] In step 203, it is known that there is a person who may get off the vehicle and that the person's mobile phone is charging. The person's intention to open the door and get off the vehicle will be further confirmed. As in step 201, the person's posture is detected by the cameras of the DMS and OMS and the MediaPipePose model to determine whether the person is in a posture to open the vehicle door, that is, to determine whether the person has the intention to open the vehicle door. If the person is in a posture to open the vehicle door and there is a proximity sensor on the vehicle door switch, further judgment is required. The vehicle door switch here includes but is not limited to: physical door handles and push-type switches. As a feasible approach, proximity sensing information can be derived from a capacitive proximity sensor installed on the vehicle door switch. Its working principle is that when an object approaches the electrode plate, the change in capacitance can be detected by the sensor, and the proximity of the object can be sensed. Capacitive proximity sensors have the advantages of adjustable sensitivity, a wide range of detection objects, and non-contact operation. Therefore, they are suitable for detecting whether the hand of a person in the vehicle is approaching.

[0088] Specifically, based on the person initially identified as getting off the vehicle in step 202, the capacitive proximity sensor information for the door switch at the corresponding location of that person is selectively obtained. Furthermore, the maximum detection distance of the capacitive proximity sensor for human hands should be appropriately set, for example, to 1-3 cm, to minimize false triggering while ensuring detection sensitivity. When the proximity sensor detects an approaching object, it indicates that the person is about to open the door using the door switch. Finally, based on the identification of the door opening gesture and the optionally obtained door proximity sensor information, it is determined whether a reminder to charge the phone should be issued. If a reminder to charge the phone should be issued, the process proceeds to step 204.

[0089] In step 204, a reminder message is issued in a form of classical Chinese, for example, a prominent framed text reminder is displayed on the car's central console: "Don't forget to take your phone while it's charging," and a voice reminder is issued through the car's audio system.

[0090] In summary, by using the common DMS and OMS cameras on the vehicle to detect the posture of the people in the car, and performing corresponding action recognition through the MediaPipe Pose model, the accuracy of detecting the left-over charging phone can be improved without adding additional hardware configuration. First, by identifying the posture of placing or taking away the phone and the status change information of the wireless charging stand, the person to whom the charging phone belongs is updated in real time, and the main driver and the co-driver are distinguished, which greatly reduces the frequency of false reminders caused by incorrect judgment of the phone's ownership. Furthermore, by identifying the posture of opening the car door, the ownership information of the charging phone, and the optionally obtained door switch proximity sensor information, it is determined whether to issue a left-over reminder, and a reminder can be issued in time before the corresponding person opens the door. Therefore, this method provides greater convenience while reducing the frequency of false reminders and potential door collision hazards.

[0091] Reference Figure 3 , the vehicle-based data processing method may further include:

[0092] Step 301, obtain the information of the currently charging mobile phone, and execute step 302;

[0093] Step 302, obtain vehicle gear information, and execute step 303;

[0094] Step 303: Determine whether the vehicle is in P gear. If so, proceed to step 304; otherwise, proceed to step 311.

[0095] Step 304: Obtain the information of the charging mobile phone and the seat belt fastening or unfastening status information, and then proceed to step 305;

[0096] Step 305: Determine whether the mobile phone of the person who unbuckled his seat belt is charging. If so, proceed to step 306; otherwise, proceed to step 311.

[0097] Step 306: Acquire the person's posture detection information and execute step 307;

[0098] Step 307: Determine whether the person has opened the door. If yes, proceed to step 308; otherwise, proceed to step 311.

[0099] Step 308: Check if the door switch has a proximity sensor. If so, go to step 309; otherwise, go to step 310.

[0100] Step 309: Check whether the proximity sensor detects an object approaching. If so, proceed to step 310; otherwise, proceed to step 311.

[0101] Step 310: Display a reminder text on the vehicle's central control screen and play a reminder voice through the vehicle's audio system;

[0102] Step 311, end this detection and reminder, and wait for the next cycle.

[0103] Reference Figure 4 For vehicles equipped with both DMS and OMS, the method for updating the charging mobile phone information involved in the embodiment specifically includes:

[0104] Step 401: Acquire posture images of the driver and the co-driver, and proceed to step 402;

[0105] Step 402: Is any front row passenger in the position of taking / holding a mobile phone? If yes, proceed to step 403; otherwise, proceed to step 409;

[0106] Step 403: Determine the corresponding person as the mobile phone holder, and execute step 404;

[0107] Step 404: Obtain status change information of the wireless charging stand, and execute step 405;

[0108] Step 405: Determine whether the number of charging mobile phones has increased. If so, proceed to step 407; otherwise, proceed to step 406.

[0109] Step 406: Determine whether the number of charging mobile phones has decreased. If so, proceed to step 408; otherwise, proceed to step 409.

[0110] Step 407: Bind the mobile phone owner to the newly added charging mobile phone, and add the device binding relationship to the charging mobile phone's information;

[0111] Step 408: Unbinding the mobile phone holder from the reduced charging mobile phone and deleting the device binding relationship from the charging mobile phone's information;

[0112] Step 409: End this detection cycle.

[0113] Reference Figure 5 For a vehicle that only has a DMS but no OMS, the method for updating the charging mobile phone information involved in the embodiment specifically includes:

[0114] Step 501: only obtain the posture image of the main driver, and then proceed to step 502;

[0115] Step 502: Determine whether the driver is in the position of taking / holding the phone. If so, proceed to step 503; otherwise, proceed to step 504.

[0116] Step 503: Determine that the driver is the mobile phone holder, and proceed to step 505;

[0117] Step 504: Determine that the passenger is the mobile phone holder, and proceed to step 505;

[0118] Step 505: Obtain status change information of the wireless charging stand, and execute step 506;

[0119] Step 506: Determine whether the number of charging mobile phones has increased. If so, proceed to step 508; otherwise, proceed to step 507.

[0120] Step 507, determine whether the number of charging mobile phones has decreased. If so, execute step 509, otherwise execute step 510;

[0121] Step 508: Bind the mobile phone owner to the newly added charging mobile phone, and add the device binding relationship to the charging mobile phone's information;

[0122] Step 509: Unbinding the mobile phone holder from the reduced charging mobile phone and deleting the device binding relationship from the charging mobile phone's information;

[0123] Step 510: End this detection cycle.

[0124] According to a second aspect of the present application, a control device is provided, comprising a processor, a memory, and a computer program or instructions. The computer program or instructions are stored in the memory and executed by the processor to implement any of the vehicle-based data processing methods described above. The control device has all the beneficial effects of the aforementioned vehicle-based data processing methods, which are not further described in this application.

[0125] In some embodiments, as Figure 6 As shown, it shows a structural schematic diagram of the control device involved in the embodiment of the present application, specifically, it includes: an image acquisition module 601, a motion posture recognition module 602, a wireless charging module 603, a charging mobile phone information storage and update module 604, a gear information module 605, a seat belt unfastening detection module 606, a door switch proximity sensing module 607, and a mobile phone left behind reminder module 608.

[0126] To update the charging phone's information, the image acquisition module 601 first obtains a posture image of the front-seat occupant. The gesture recognition module 602 then identifies the posture image to determine if the front-seat occupant is reaching for the phone. If so, they are identified as the phone's owner. Combined with the wireless charging module 603's detection of charging connection establishment and termination, the charging phone information storage and update module 604 can add or delete device bindings between the phone's owner and the charging phone in the charging phone's information.

[0127] To detect and remind a person who's been left behind, the gear information module 605 and the seatbelt unfastening detection module 606 first identify the front-seat occupant. The phone-left reminder module 608 then determines whether the person unfastening their seatbelt while in P gear is one of the phone owners stored in the charging phone information storage and update module 604. If so, the gesture recognition module 602 then identifies the gesture image to determine if the front-seat occupant has opened the door. If the vehicle has a door switch proximity sensor, the door switch proximity sensing module 607 further determines whether the front-seat occupant's hand is near the door switch. Finally, the phone-left reminder module 608 determines whether a phone-left reminder should be issued.

[0128] In some embodiments, as Figure 7 , which shows another structural schematic diagram of the control device involved in the embodiment of the present application, specifically:

[0129] The control device may include one or more processing core processors 701, one or more computer readable storage media memories 702, a power supply 703, an input unit 704 and other components. Those skilled in the art will understand that Figure 7 The control device structure shown in the figure is not intended to limit the control device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0130] Processor 701 is the control center of the control device. It utilizes various interfaces and circuits to connect the various components of the entire control device. By running or executing software programs and / or modules stored in memory 702 and accessing data stored in memory 702, it performs various functions of the control device and processes data, thereby providing overall monitoring of the control device. Optionally, processor 701 may include one or more processing cores; preferably, processor 701 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 701.

[0131] Memory 702 can be used to store software programs and modules. Processor 701 executes various functional applications and data processing by running the software programs and modules stored in memory 702. Memory 702 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback); the data storage area may store data generated based on the use of the control device. Furthermore, memory 702 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 702 may also include a memory control device to provide processor 701 with access to memory 702.

[0132] The control device also includes a power supply 703 for supplying power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 703 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0133] The control device may further include an input unit 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0134] Although not shown, the control device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the control device loads the executable files corresponding to one or more application processes into the memory 702 according to the following instructions. The processor 701 then runs the application stored in the memory 702 to implement various functions, such as implementing any of the steps in the vehicle-based data processing method described above.

[0135] According to a third aspect of the present application, a computer-readable storage medium is provided. The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. A computer program or instructions are stored on the computer-readable storage medium, and the computer program or instructions are executed by a processor to implement any of the vehicle-based data processing methods described above.

[0136] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement any one of the vehicle-based data processing methods.

[0137] According to a fifth aspect of the present application, a vehicle is provided, comprising any one of the control devices described above. The vehicle has all the beneficial effects of the above control devices, which will not be described in detail in this application.

[0138] In some embodiments, the vehicle may be a fuel vehicle, a plug-in hybrid vehicle, or a new energy vehicle, etc., and this application does not make any specific limitations on this.

[0139] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0140] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0142] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A vehicle-based data processing method, characterized in that: include: Obtain target posture data and device binding relationship of target objects in the vehicle; Determining an electronic device bound to the target object based on the target posture data and the device binding relationship, wherein the electronic device is a device for charging the vehicle; outputting reminder information associated with the electronic device; The device binding relationship is generated through the following steps: detecting whether the target posture data satisfies a preset second posture condition and whether the number of electronic devices being charged by the vehicle increases, wherein the second posture condition represents the posture of the target object when charging the electronic devices using the vehicle; If the target posture data satisfies the second posture condition and the number of electronic devices being charged by the vehicle increases, a device binding relationship is generated between the increased electronic devices and the target object.

2. The vehicle-based data processing method according to claim 1, characterized in that: The electronic device is placed in the vehicle.

3. The vehicle-based data processing method according to claim 1, characterized in that: The determining, based on the target posture data and the device binding relationship, the electronic device bound to the target object includes: detecting whether the target posture data satisfies a preset first posture condition, wherein the first posture condition represents a posture of the target object when the target object intends to open the door; If the target posture data satisfies the first posture condition, the electronic device bound to the target object is determined in the device binding relationship.

4. The vehicle-based data processing method according to claim 1, characterized in that: Before outputting the reminder information associated with the electronic device, the method further includes: An object is detected approaching a target door in the area where the target object is located.

5. The vehicle-based data processing method according to claim 4, characterized in that: The target vehicle door is provided with a proximity sensor, and the proximity sensor is used to detect whether there is an object approaching the target vehicle door.

6. The vehicle-based data processing method according to claim 5, characterized in that: The vehicle-based data processing method further includes: Using the proximity sensor, determining whether an object is within a preset distance range of the proximity sensor; If an object approaches within a preset distance range of the proximity sensor, it is determined that the object is approaching the target door.

7. The vehicle-based data processing method according to claim 1, characterized in that: Before outputting the reminder information associated with the electronic device, the method further includes: It is determined that the vehicle is in a parking gear, and / or it is determined that the seat belts in the area where the target object is located are in an unfastened state.

8. The vehicle-based data processing method according to claim 1, characterized in that: Outputting the reminder information associated with the electronic device includes at least one of the following steps: displaying, via a display device of the vehicle, an image corresponding to the reminder information; Play the voice corresponding to the reminder information through the audio device of the vehicle.

9. The vehicle-based data processing method according to claim 1, characterized in that: The vehicle includes a first seat and a second seat, and the target posture data is obtained by an image acquisition device facing the first seat. The vehicle-based data processing method further includes: detecting whether the target posture data satisfies a preset second posture condition and whether the number of electronic devices being charged by the vehicle increases, wherein the second posture condition represents the posture of the target object when charging the electronic devices using the vehicle; If the target posture data does not satisfy the second posture condition and the number of electronic devices being charged by the vehicle increases, a device binding relationship is generated between the increased electronic devices and objects in the area where the second seat is located.

10. The vehicle-based data processing method according to claim 9, characterized in that: The vehicle includes a preset charging base, which is used to charge the electronic device. The vehicle-based data processing method further includes at least one of the following: When the charging connection establishment information based on the charging stand is detected, it is determined that the number of electronic devices charged by the vehicle has increased.

11. The vehicle-based data processing method according to claim 10, characterized in that: The charging base includes a wireless charging base.

12. The vehicle-based data processing method according to claim 1, characterized in that: After obtaining the target posture data and the device binding relationship of the target object in the vehicle, the method further includes: detecting whether the number of electronic devices being charged by the vehicle has decreased; If the number of electronic devices being charged by the vehicle has not decreased, the step of determining the electronic devices bound to the target object based on the target posture data and the device binding relationship is performed.

13. The vehicle-based data processing method according to claim 12, characterized in that: After detecting whether the number of electronic devices being charged by the vehicle is reduced, the method further includes: If the number of electronic devices being charged by the vehicle decreases, and the target posture data satisfies a preset third posture condition, deleting the device binding relationship between the reduced electronic devices and the target object; The third posture condition represents the posture of the target object when it stops using the vehicle to charge the electronic device.

14. The vehicle-based data processing method according to claim 12, characterized in that: The vehicle includes a first seat and a second seat, and the target posture data is obtained by an image acquisition device facing the first seat. After detecting whether the number of electronic devices being charged by the vehicle is reduced, the method further includes: If the number of electronic devices being charged in the vehicle decreases and the target posture data does not meet the preset third posture condition, deleting the device binding relationship between the reduced electronic devices and the objects in the area where the second seat is located; The third posture condition represents the posture of the target object when it stops using the vehicle to charge the electronic device.

15. The vehicle-based data processing method according to claim 9 or 14, characterized in that: The first seat is one of a driver's seat and a passenger seat in the vehicle, and the second seat is the other of the driver's seat and the passenger seat in the vehicle.

16. The vehicle-based data processing method according to any one of claims 12 to 14, characterized in that: The vehicle includes a preset charging base, which is used to charge the electronic device. The vehicle-based data processing method further includes at least one of the following: When the charging connection disconnection information based on the charging stand is detected, it is determined that the number of electronic devices charged by the vehicle has decreased.

17. The vehicle-based data processing method according to claim 16, characterized in that: The charging base includes a wireless charging base.

18. The vehicle-based data processing method according to claim 1, characterized in that: The acquiring target posture data of the target object in the vehicle includes: Acquire a posture image of the target object; Identifying limb key points in the posture image; The target posture data is determined using the limb key points.

19. The vehicle-based data processing method according to claim 18, characterized in that: Determining the target posture data by using the limb key points includes: Determine a connecting line between at least two of the limb key points using the coordinates of the limb key points in the posture image; The target posture data is determined according to the angle information between the connecting line and the preset direction.

20. The vehicle-based data processing method according to claim 19, characterized in that: The limb key points in the posture image are identified by using a preset human posture estimation model.

21. A control device, characterized in that: The device comprises a processor, a memory, and a computer program or instruction, wherein the computer program or instruction is stored in the memory and executed by the processor to implement the vehicle-based data processing method according to any one of claims 1 to 20.

22. A computer-readable storage medium, characterized in that A computer program or instruction is stored thereon, and the computer program or instruction is executed by a processor to implement the vehicle-based data processing method according to any one of claims 1 to 20.

23. A computer program product, characterized in that The method comprises a computer program or instructions, which are executed by a processor to implement the vehicle-based data processing method according to any one of claims 1 to 20.

24. A vehicle, characterized in that: Comprising the control device of claim 21.

Images