Passenger information processing method and device

Abstract

The present application provides a method and device for processing passenger information. First, it is determined that the passenger is in the cabin of a vehicle based on multiple first motion data collected by the smart watch; first image data of the space where the passenger is currently located is collected, and based on the first image, it is determined that the passenger is in the back row area of ​​the cabin; a reference motion state of the passenger relative to the back row area is determined based on multiple second motion data collected by the smart watch; the passenger's motion state misalignment perception in the current space is determined based on the reference motion state; when it is detected that the motion state misalignment perception is greater than a preset motion state misalignment perception threshold, a preset operation for guiding the passenger to adjust his or her posture is executed to reduce the passenger's motion state misalignment perception.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a method and device for processing passenger information. Background Art

[0002] In urban driving environments, due to the low speed and many traffic lights, many passengers do not wear seat belts, and those passengers who do not wear seat belts sometimes move inside the car, which can easily lead to a large difference between their own motion state and the motion state of the car body. In addition, in urban road environments, vehicles are often in a state of motion with large acceleration, such as starting and braking, which makes passengers more susceptible to motion sickness.

[0003] At present, it is generally possible to identify and remind passengers based on whether they are wearing seat belts and their riding status. However, due to the different probabilities of motion sickness caused by different degrees of abnormal riding behavior, a single reminder method is not flexible and accurate enough. The reminded users cannot perceive the degree of unreasonableness of their own behavior. How to determine an appropriate reminder strategy for each passenger's abnormal riding behavior has become a difficult problem. Summary of the invention

[0004] In view of this, the present application provides a method and device for processing passenger information, which can focus on monitoring passengers with abnormal riding behavior and provide adaptive reminders to prevent passengers from motion sickness, thereby improving passengers' riding safety and riding experience.

[0005] In a first aspect, an embodiment of the present application provides a method for processing passenger information.

[0006] Applied to a smart watch, the smart watch is worn by a passenger; the method comprises:

[0007] Determining that the passenger is in the cabin of the vehicle according to the plurality of first motion data collected by the smart watch; collecting first image data of the space where the passenger is currently located, and determining that the passenger is in the rear area of ​​the cabin according to the first image;

[0008] Determine a reference motion state of the passenger relative to the rear area according to the plurality of second motion data collected by the smart watch; determine a motion state misalignment perception of the passenger in the current space according to the reference motion state;

[0009] It is detected that the motion state misalignment perception is greater than a preset motion state misalignment perception threshold, and a preset operation for guiding the passenger to adjust his / her posture is performed to reduce the motion state misalignment perception of the passenger.

[0010] In a second aspect, an embodiment of the present application provides a device for processing passenger information.

[0011] A processing module applied to an information processing system, wherein the information processing system further comprises a smart watch, and the smart watch is worn by a passenger; the device comprises:

[0012] A first determination unit is configured to determine that the passenger is in the cabin of the vehicle according to the plurality of first motion data collected by the smart watch; collect first image data of the space where the passenger is currently located, and determine that the passenger is in the rear row area of ​​the cabin according to the first image;

[0013] A second determination unit is configured to determine a reference motion state of the passenger relative to the rear area according to the plurality of second motion data collected by the smart watch; and determine a motion state misalignment perception of the passenger in the current space according to the reference motion state;

[0014] The guiding unit is used to detect that the motion state misalignment perception is greater than a preset motion state misalignment perception threshold, and execute a preset operation for guiding the passenger to adjust his posture to reduce the passenger's motion state misalignment perception.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method in the first aspect of the embodiment of the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0018] It can be seen that through the above-mentioned passenger information processing method and device, first, according to the multiple first motion data collected by the smart watch, it is determined that the passenger is in the cabin of the vehicle; the first image data of the space where the passenger is currently located is collected, and according to the first image, it is determined that the passenger is in the back row area of ​​the cabin; according to the multiple second motion data collected by the smart watch, the reference motion state of the passenger relative to the back row area is determined; according to the reference motion state, the motion state misalignment perception of the passenger in the current space is determined; when it is detected that the motion state misalignment perception is greater than the preset motion state misalignment perception threshold, a preset operation for guiding the passenger to adjust his own posture is performed to reduce the motion state misalignment perception of the passenger. Passengers who are likely to have irregular riding behavior on the vehicle can be accurately identified, and an adapted reminder strategy can be determined for reminder, thereby distracting the target passenger's attention and stopping irregular riding behavior, greatly improving the passenger's riding experience and riding safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A system architecture diagram of a method for processing passenger information provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of a method for processing passenger information provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of another method for processing passenger information provided in an embodiment of the present application;

[0024] Figure 5 A block diagram of the functional units of a passenger information processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0027] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship. The "plurality" appearing in the embodiments of the present application refers to two or more.

[0028] In the embodiments of the present application, "at least one item" or similar expressions refer to any combination of these items, including any combination of single items or plural items, and refer to one or more, and multiple refers to two or more. For example, at least one item of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0029] The "connection" that appears in the embodiments of the present application refers to various connection methods such as direct connection or indirect connection to achieve communication between devices, and the embodiments of the present application do not impose any limitations on this.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] The following is an explanation of the relevant contents, concepts, meanings, technical issues, technical solutions, beneficial effects, etc. involved in the embodiments of the present application.

[0032] At present, the monitoring and reminders of passengers are generally limited to whether they are wearing seat belts. For some special passengers, such as children, mentally ill people, elderly people with cognitive impairments, etc., who have no accurate understanding of abnormal riding behaviors during the ride, it is impossible to promptly discover and stop such passengers when they exhibit abnormal riding behaviors, which makes it easy for such passengers to suffer from symptoms such as motion sickness during the driving of the vehicle, and there are certain safety hazards.

[0033] The above-mentioned problem has not been solved. The embodiments of the present application provide a method for processing passenger information and a related device, which can accurately identify passengers who are likely to engage in abnormal riding behavior for monitoring, avoid wasting computing power, and specify personalized reminder strategies for target passengers, thereby greatly improving the riding experience and riding safety of the target passengers.

[0034] First, combine Figure 1 The system architecture of a method for processing passenger information in an embodiment of the present application is described. Figure 1 A system architecture diagram of a method for processing passenger information provided in an embodiment of the present application includes a vehicle 110, a processing module 120 and a smart watch 130, wherein the processing module 120 is communicatively connected to the vehicle 110 and mounted on the smart watch 130.

[0035] Among them, vehicle 110 includes a pressure sensor and a camera module arranged on each seat. The pressure sensor can be used to determine whether there is a passenger in each seat and the pressure change data when there is a passenger in the seat. The camera module can be used to determine whether the passenger is the target passenger and the target passenger's sitting position change data and sitting posture change data.

[0036] Among them, the smart watch 130 equipped with the processing module 120 can determine that the passenger is in the vehicle cabin based on multiple first motion data collected by the smart watch; collect first image data of the space where the passenger is currently located, and determine that the passenger is in the back row area of ​​the cabin based on the first image; determine the reference motion state of the passenger relative to the back row area based on multiple second motion data collected by the smart watch; determine the passenger's motion state misalignment perception in the current space based on the reference motion state; when it is detected that the motion state misalignment perception is greater than a preset motion state misalignment perception threshold, execute a preset operation for guiding the passenger to adjust his or her posture to reduce the passenger's motion state misalignment perception.

[0037] In a possible embodiment, the processing module 120 can also analyze the pressure data from the vehicle 110 to determine whether there is a passenger in the seat and analyze the pressure change data to determine whether the passenger's movement conforms to inertia. It can also analyze the image data from the vehicle 110 to determine whether there is a target passenger and the target passenger's sitting position change data and sitting posture change data, and determine the target reminder strategy for the target passenger, and then control the smart watch 130 to execute the target reminder strategy.

[0038] The smart watch 130 is a device worn by the target passenger, which may be a watch, a bracelet, glasses, headphones, etc., which are not specifically limited here. The smart watch 130 may have voice broadcast function, vibration function and electric shock function to respond to the instruction of the processing module 120 to execute the target reminder strategy.

[0039] It can be seen that through the above system architecture, the target passengers can be accurately identified, and when the target passengers exist, the corresponding target reminder strategy can be generated and executed for the target passengers, so that the target passengers will no longer perform abnormal riding behaviors, greatly improving the riding experience and riding safety of the target passengers.

[0040] Combine the following Figure 2 The electronic device in the embodiment of the present application is described. Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the electronic device 20 includes one or more application processors 220, a memory 230, a communication module 240 and one or more programs 231. The application processor 220 is communicatively connected with the memory 230 and the communication module 240 via an internal communication bus.

[0041] The one or more programs 231 are stored in the memory 230 and configured to be executed by the application processor 220 . The one or more programs 231 include instructions for executing any step in the method embodiment.

[0042] Among them, the application processor 220 can be, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, units and circuits described in conjunction with the disclosure of this application. The application processor 220 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The communication unit can be a communication module 240, a transceiver, a transceiver circuit, etc., and the storage unit can be a memory 230.

[0043] The memory 230 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0044] It is understandable that the electronic device 20 may include more or fewer structural elements than those in the above structural block diagram, for example, including a power module, physical buttons, a Wi-Fi module, a speaker, a Bluetooth module, a sensor, a display module, an environment acquisition module, an image acquisition module, etc., which are not limited here. The electronic device may be the above-mentioned smart watch.

[0045] After understanding the software and hardware architecture of the embodiment of the present application, Figure 3 A method for processing passenger information in an embodiment of the present application is described. Figure 3 A flow chart of a method for processing passenger information provided in an embodiment of the present application is applied to a processing module in an information processing system, wherein the information processing system further includes a smart watch, and the smart watch is worn by a passenger; the method specifically includes the following steps:

[0046] Step 301: determine that the passenger is in the cabin of the vehicle based on multiple first motion data collected by the smart watch.

[0047] Among them, the motion direction characteristics of the multiple first motion data can be analyzed to determine at least one reference direction where there is continuous component motion data; and the vehicle driving characteristics correlation analysis can be performed on the continuous component motion data of each reference direction in the at least one reference direction to determine that there is a target direction that meets the vehicle driving characteristics in the at least one reference direction; finally, the passenger's spatial state is marked as the cabin occupancy state. For example, there are acceleration directions and acceleration magnitudes in vehicle driving. If the reference direction of a certain continuous component motion data of the smart watch is the same as the acceleration direction of the vehicle, it can be determined that there is a target direction in the reference direction, the smart watch is moving with the vehicle, and the passenger himself does not have other movements relative to the vehicle.

[0048] In a possible embodiment, whether a passenger is in the vehicle cabin can be determined by a pressure sensor at each seat except the driver's seat. When the pressure collected by the pressure sensor of any seat is greater than a certain pressure threshold, it can be considered that a passenger exists, and the first image data of the passenger in that seat needs to be acquired. The pressure threshold can be set according to the normal weight range of humans over 3 years old. This is because too little pressure can occur in one case when objects are placed on the seat and in another case when the passenger is too young and needs care to prevent abnormal riding behavior. The passengers in the embodiments of the present application can be understood as passengers who can wear seat belts and ride.

[0049] It can be seen that by determining that the passenger is in the vehicle cabin based on the multiple first motion data collected by the smart watch, the smart watch can be kept in a dormant state when the passenger is not in the vehicle cabin, thereby saving power consumption.

[0050] Step 302: collect first image data of the space where the passenger is currently located, and determine that the passenger is in the back row area of ​​the cabin based on the first image.

[0051] Among them, feature recognition can be performed on the first image data to obtain object feature recognition results and relative position relationship feature recognition results, the object feature recognition results include identifying objects at the boundary of the framing object image space and objects within the framing object image space, and the relative position relationship feature recognition results are used to characterize the relative position relationship between objects at the boundary of the framing object image space and objects within the framing object image space; then, it is determined that the passenger is in the back row area of ​​the cabin based on the object feature recognition results and the relative position relationship feature recognition results.

[0052] Specifically, objects at the boundary of the framing object image space may be a front windshield, etc., and objects within the framing object image space may be front seats, a clutch, etc. It can be determined that the relative position relationship feature recognition result is that the front seats are located behind the front windshield, and at this time it can be determined that the passengers are in the back area of ​​the cabin.

[0053] In a possible embodiment, the facial feature data of each passenger can also be determined based on the first image data of each passenger. The specific facial feature extraction algorithm can refer to the existing facial feature extraction algorithm, including preprocessing, feature point extraction and other steps, which are not described in detail here. Then, if there is target feature data matching any facial feature data in the preset target passenger database, it is determined that the confidence level of abnormal riding behavior of the passenger corresponding to the facial feature data matching the target feature data is higher than the preset confidence threshold, and the passenger corresponding to the facial feature data matching the target feature data is determined as the target passenger. It can be understood that for elderly passengers with cognitive impairment or mentally ill passengers, their facial images can be taken before they get on the bus and stored in the preset target passenger database. Whenever they get on the bus, facial matching can be directly performed to determine whether there is a target passenger. The accuracy and efficiency of identifying target passengers can be greatly improved.

[0054] In a possible embodiment, if the target feature data matching any facial feature data does not exist in the preset target passenger database, the age data of each passenger is determined based on the facial feature data of each passenger, and then, the confidence level of abnormal riding behavior of passengers whose age data is within the preset age range is determined to be higher than the preset confidence threshold, and the passengers whose age data is within the preset age range are determined as the target passengers. Among them, passengers aged 3 to 14 can be determined as target passengers, which is not specifically limited here.

[0055] In a possible embodiment, a target recognition model may be pre-constructed, and the target recognition model may determine whether a passenger is a target passenger based on the input first image data, which will not be elaborated herein.

[0056] In a possible embodiment, if the target passenger is determined to exist according to the first image data, the second image data of the target passenger is obtained, and at least two accurate images of the target passenger can be obtained to determine whether the subsequent riding behavior is abnormal, thereby improving the accuracy of the determination of the abnormal riding behavior of the target passenger, and performing image acquisition for the target passenger to avoid unnecessary waste of computing power. The second image data may include at least two images within a period of time, and the resolution of the second image data may be higher than that of the first image data.

[0057] Step 303: Determine a reference motion state of the passenger relative to the rear row area based on the multiple second motion data collected by the smart watch.

[0058] Among them, the driving data of the vehicle can be obtained; the correlation analysis of the vehicle driving characteristics is performed on each second motion data and the driving data, and the target motion data that does not conform to the vehicle driving characteristics is determined among the multiple second motion data; the reference motion state of the passenger relative to the rear area is determined according to the target motion data, and the reference motion state includes a reference motion trajectory and a reference motion speed.

[0059] Step 304: determining the passenger's motion state misalignment perception in the current space according to the reference motion state.

[0060] Among them, the motion trajectory difference data can be determined according to the reference motion trajectory and the driving trajectory in the driving data; the motion speed difference data can be determined according to the reference motion speed and the driving speed in the driving data; the motion state misalignment perception degree can be determined according to the motion trajectory difference data and the motion speed difference data, and the motion state misalignment perception degree is positively correlated with the motion trajectory difference data and the motion speed difference data.

[0061] Step 305: When it is detected that the motion state misalignment perception degree is greater than a preset motion state misalignment perception degree threshold, a preset operation for guiding the passenger to adjust his / her posture is executed.

[0062] Among them, the guidance can reduce the passenger's perception of misalignment in motion state.

[0063] Among them, the target reminder strategy can be determined according to the sitting position change data, and the target reminder strategy includes a first voice strategy and a first vibration strategy, the first voice strategy includes a voice prompting to stop moving, and the first vibration strategy includes a first vibration frequency and a first vibration intensity, and the first vibration frequency and the first vibration intensity are positively correlated with the sitting position change data.

[0064] In a possible embodiment, the road state data can be determined according to the position of the vehicle, the road state data including the road curvature, road slope and road roughness; a seat adjustment instruction is sent to the vehicle terminal according to the road curvature, road slope and road roughness to adjust at least one of the seat position, seat height and seat angle of the target passenger, so that the target user will not have passive changes in the sitting position and passive changes in the sitting posture due to inertia. The road state data can be determined according to the position of the vehicle, the road state data including the road curvature, road slope and road roughness; a seat adjustment instruction is sent to the vehicle terminal according to the road curvature, road slope and road roughness to adjust at least one of the seat position, seat height and seat angle of the target passenger, so that the target user will not have passive changes in the sitting position and passive changes in the sitting posture due to inertia.

[0065] It can be seen that through the above-mentioned passenger information processing method, first, according to the multiple first motion data collected by the smart watch, it is determined that the passenger is in the cabin of the vehicle; the first image data of the space where the passenger is currently located is collected, and according to the first image, it is determined that the passenger is in the back row area of ​​the cabin; according to the multiple second motion data collected by the smart watch, the reference motion state of the passenger relative to the back row area is determined; according to the reference motion state, the motion state misalignment perception of the passenger in the current space is determined; when it is detected that the motion state misalignment perception is greater than the preset motion state misalignment perception threshold, a preset operation for guiding the passenger to adjust his posture is performed to reduce the motion state misalignment perception of the passenger. Passengers who are likely to have irregular riding behavior in the vehicle can be accurately identified, and an adaptive reminder strategy can be determined to remind them, thereby distracting the target passenger's attention and stopping irregular riding behavior, greatly improving the passenger's riding experience and riding safety.

[0066] Combine the following Figure 4 Another method for processing passenger information in an embodiment of the present application is described. Figure 4 A flowchart of another method for processing passenger information provided in an embodiment of the present application is applied to a processing module in an information processing system, wherein the information processing system further includes a smart watch, and the smart watch is worn by a passenger; specifically, the following steps are included:

[0067] Step 401, obtaining first image data of each passenger.

[0068] Step 402: If it is determined according to the first image data that a target passenger exists, then second image data of the target passenger is acquired.

[0069] Step 403: Determine riding behavior change data of the target passenger according to the second image data.

[0070] In a possible embodiment, the second image data may include an image at a first moment and an image at a second moment, and the riding behavior change data may include riding position change data and riding posture change data. The riding position change data may reflect the displacement of the target passenger in the seat, such as moving from the back of the seat to the front of the seat, and the riding posture change data may reflect the change in the target passenger's movements in the seat, such as changing from placing his hands at the sides of the body to raising his hands above the head, etc., which will not be elaborated here.

[0071] In a possible embodiment, the first sitting position and the first sitting posture of the target passenger may be determined according to the first moment image, and the second sitting position and the second sitting posture of the target passenger may be determined according to the second moment image. Then, if the distance difference between the first sitting position and the second sitting position exceeds a preset distance threshold, the sitting position change data is determined according to the first sitting position and the second sitting position; if the matching degree between the first sitting posture and the second sitting posture is lower than a preset matching threshold, the sitting posture change data is determined according to the first sitting posture and the second sitting posture.

[0072] In a possible embodiment, the inertia direction and inertia magnitude can be determined according to the current vehicle speed, vehicle acceleration, and vehicle turning state, and then the third sitting position can be determined. The third sitting position can be the position of the target passenger after the estimated position has reasonably changed under the current inertia direction and inertia magnitude. It can be understood that the distance between the third sitting position and the first sitting position is positively correlated with the inertia magnitude. For example, when braking, the third sitting position will be closer to the front of the seat than the first sitting position, and when accelerating, the third sitting position will be closer to the back of the seat than the first sitting position. Then, the distance difference between the third sitting position and the second sitting position is used to determine the sitting position change data. It can be understood that this can eliminate the interference of inertia and can more accurately identify whether the target passenger has abnormal riding behavior.

[0073] In a possible embodiment, the inertia direction and inertia magnitude can be determined according to the current vehicle speed, vehicle acceleration, and vehicle turning state, and then the third sitting posture can be determined. The third sitting posture can be the estimated reasonable posture change of the target passenger under the inertia direction and inertia magnitude. It can be understood that the change amplitude of the third sitting posture compared with the first sitting posture is positively correlated with the inertia magnitude, and the third sitting posture can be divided into body leaning forward, body leaning backward, body leaning left, body leaning right, etc. For example, when accelerating, the third sitting posture can be body leaning backward, when braking, the third sitting posture can be body leaning forward, when turning left, the third sitting posture can be body leaning right, and when turning right, the third sitting posture can be body leaning left. The above "tilts" are all compared with the first sitting posture, and will not be elaborated here. Then, the matching degree of the third sitting posture and the second sitting posture is used to determine the sitting posture change data. It can be understood that in this way, the interference of inertia can be eliminated, and whether the target passenger has abnormal riding behavior can be more accurately identified.

[0074] It can be seen that by determining the riding behavior change data of the target passenger based on the second image data, the riding behavior change data of the target passenger can be accurately identified, thereby improving the accuracy of subsequent reminders.

[0075] Step 404: determine a target reminder strategy based on the riding behavior change data.

[0076] The target reminder strategy may be in the form of at least one of a voice reminder, a vibration reminder and an electric shock reminder.

[0077] In a possible embodiment, when there is only seating position change data, the target reminder strategy can be determined based on the seating position change data, the target reminder strategy includes a first voice strategy and a first vibration strategy, the first voice strategy includes a voice prompting to stop moving, the first vibration strategy includes a first vibration frequency and a first vibration intensity, and the first vibration frequency and the first vibration intensity are positively correlated with the seating position change data. It can be understood that since only the position change is performed, the probability of motion sickness of the target passenger is low, and there is basically no impact on driving safety, so only voice reminders and slight vibration reminders can be performed.

[0078] In a possible embodiment, when only sitting posture change data exists, the target reminder strategy can be determined according to the sitting posture change data, and the target reminder strategy includes a second voice strategy, a second vibration strategy and a first electric shock strategy, the second voice strategy includes a voice to stop posture change, the second vibration strategy includes a second vibration frequency and a second vibration intensity, the first electric shock strategy includes a first electric shock frequency and a first electric shock intensity, the second vibration frequency and the second vibration intensity are positively correlated with the sitting posture change data, and the first electric shock frequency and the first electric shock intensity are positively correlated with the sitting posture change data. It can be understood that since the target passenger does not change his position in the seat but has a disorderly behavior, the probability of motion sickness is moderate, and there is also an impact on driving safety, so in addition to the voice reminder and the moderate vibration reminder, a mild electric shock reminder can be added to facilitate guiding the target passenger to stop moving around.

[0079] In a possible embodiment, when there are both sitting posture change data and sitting position change data, the target reminder strategy can be determined according to the sitting position change data and the sitting posture change data, the target reminder strategy includes a third voice strategy, a third vibration strategy and a second electric shock strategy, the third voice strategy includes voices of stopping posture change and stopping movement, the third vibration strategy includes a third vibration frequency and a third vibration intensity, the second electric shock strategy includes a second electric shock frequency and a second electric shock intensity, the third vibration frequency and the third vibration intensity are positively correlated with the sitting posture change data and the position change data, the second electric shock frequency and the second electric shock intensity are positively correlated with the sitting posture change data and the position change data, the third vibration frequency is greater than the second vibration frequency, the third vibration intensity is greater than the second vibration intensity, the second electric shock frequency is greater than the first electric shock frequency, and the second electric shock intensity is greater than the first electric shock intensity. It can be understood that at this time, the probability of motion sickness of the target passenger is very high, and the impact on driving safety is also large. In addition to the voice reminder and the large vibration reminder, a strong electric shock reminder can be added to facilitate guiding the target passenger to stop abnormal riding behavior. It should be noted that the above electric shock frequency and electric shock intensity will not cause physical injury to the passenger.

[0080] It can be understood that if the target passenger's riding behavior change data indicates that the target passenger does not have any abnormal riding behavior, continuous monitoring can be carried out. Moreover, after the above-mentioned reminder strategy is executed, continuous monitoring can be carried out, and the reminder shock frequency and shock intensity can be increased according to the number of abnormal riding behaviors of the target passenger. Details will not be elaborated here.

[0081] Step 405: Control the target smart watch to execute the target reminder strategy.

[0082] Among them, the target smart watch is a device worn by the target passenger.

[0083] It can be seen that through the above-mentioned passenger information processing method, first, the first image data of each passenger is obtained; then, if it is determined that there is a target passenger according to the first image data, the second image data of the target passenger is obtained, and the target passenger is a passenger whose riding behavior abnormality confidence is higher than the preset confidence threshold, and the riding behavior abnormality confidence is used to indicate the probability of the passenger's riding behavior being abnormal on the vehicle; then, the riding behavior change data of the target passenger is determined according to the second image data; then, the target reminder strategy is determined according to the riding behavior change data, and the form of the target reminder strategy includes at least one of voice reminder, vibration reminder and electric shock reminder; finally, the target smart watch is controlled to execute the target reminder strategy, and the target smart watch is a device worn by the target passenger. Passengers who are likely to have irregular riding behavior on the vehicle can be accurately identified, and an adapted reminder strategy can be determined for reminder, thereby distracting the target passenger's attention and stopping irregular riding behavior, greatly improving the passenger's riding experience and riding safety.

[0084] Step 406, determining road status data according to the position of the vehicle.

[0085] The position of the vehicle can be determined by a positioning system, and then the road condition data of the position can be obtained. The road image captured by the vehicle's camera module can also be analyzed to determine the road condition data, which includes road curvature, road slope, road roughness, etc.

[0086] Step 407: adjust at least one of the seat position, seat height, and seat angle of the target passenger according to the road curvature, the road slope, and the road roughness.

[0087] In this way, the target user will not experience passive changes in sitting position and sitting posture due to inertia.

[0088] It can be seen that through the above steps, the interference caused by turns and bumps can be eliminated, the riding experience of the target passengers can be improved, and passive riding position changes and passive riding posture changes can be prevented, so that the accuracy of subsequent identification of whether the target passenger has abnormal riding behavior is greatly improved.

[0089] It can be seen that through the above-mentioned passenger information processing method, first, the first image data of each passenger is obtained; then, if it is determined that there is a target passenger according to the first image data, the second image data of the target passenger is obtained, and the target passenger is a passenger whose riding behavior abnormality confidence is higher than the preset confidence threshold, and the riding behavior abnormality confidence is used to indicate the probability of the passenger's riding behavior being abnormal on the vehicle; then, the riding behavior change data of the target passenger is determined according to the second image data; then, the target reminder strategy is determined according to the riding behavior change data, and the form of the target reminder strategy includes at least one of voice reminder, vibration reminder and electric shock reminder; finally, the target smart watch is controlled to execute the target reminder strategy, and the target smart watch is a device worn by the target passenger. Passengers who are likely to have irregular riding behavior on the vehicle can be accurately identified, and an adapted reminder strategy can be determined for reminder, thereby distracting the target passenger's attention and stopping irregular riding behavior, greatly improving the passenger's riding experience and riding safety.

[0090] The steps not described in detail above can be found in Figure 3 The description of the steps of the method will not be repeated here.

[0091] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0092] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0093] In the case of dividing each functional module into corresponding functional modules, Figure 5A functional unit composition block diagram of a passenger information processing device provided in an embodiment of the present application is applied to a processing module in an information processing system, wherein the information processing system further includes a smart watch, and the smart watch is worn by a passenger; the passenger information processing device 500 includes:

[0094] The first determination unit 510 is used to determine that the passenger is in the cabin of the vehicle according to the multiple first motion data collected by the smart watch; collect first image data of the space where the passenger is currently located, and determine that the passenger is in the back row area of ​​the cabin according to the first image;

[0095] The second determining unit 520 is used to determine a reference motion state of the passenger relative to the rear area according to the plurality of second motion data collected by the smart watch; and determine a motion state misalignment perception of the passenger in the current space according to the reference motion state;

[0096] The guiding unit 530 is used to detect that the motion state misalignment perception is greater than a preset motion state misalignment perception threshold, and execute a preset operation for guiding the passenger to adjust his posture to reduce the passenger's motion state misalignment perception.

[0097] It should be noted that the specific implementation of each operation can adopt the corresponding description of the method embodiment shown above, and the passenger information processing device 500 can be used to execute the above method embodiment of the present application, which will not be described in detail.

[0098] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiment, and the above computer includes an electronic device.

[0099] The embodiment of the present application also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0100] It should be noted that, for the above-mentioned various embodiments, for the sake of simple description, they are all expressed as a series of action combinations. Those skilled in the art should be aware that the present application is not limited by the described order of actions, because some steps in the embodiments of the present application can be performed in other orders or simultaneously. In addition, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions, steps, modules or units involved are not necessarily required by the embodiments of the present application.

[0101] In the above embodiments, the embodiments of the present application have different focuses on the description of each embodiment. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0102] The steps of the method or algorithm described in the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, read-only compact disks (CD-ROMs) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a terminal device or a management device. Of course, the processor and the storage medium can also be present in a terminal device or a management device as discrete components.

[0103] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0104] The modules / units included in the devices and products described in the above embodiments may be software modules / units or hardware modules / units, or may be partially software modules / units and partially hardware modules / units. For example, for the devices and products applied to or integrated in the chip, the modules / units included therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for the devices and products applied to or integrated in the chip module, the modules / units included therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or in different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs. The software programs run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits. It is implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in hardware such as circuits; for various devices and products applied to or integrated in the terminal equipment, the various modules / units contained therein can be implemented in hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal equipment, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the terminal equipment, and the remaining (if any) modules / units can be implemented in hardware such as circuits.

[0105] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above description is only the specific implementation method of the embodiments of the present application and is not intended to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A method for processing passenger information, characterized in that: Applied to a smart watch, the smart watch is worn by a passenger; the method comprises: Determining that the passenger is in the cabin of the vehicle according to the plurality of first motion data collected by the smart watch; collecting first image data of the space where the passenger is currently located, and determining that the passenger is in the rear area of ​​the cabin according to the first image; Determine a reference motion state of the passenger relative to the rear area according to the plurality of second motion data collected by the smart watch; determine a motion state misalignment perception of the passenger in the current space according to the reference motion state; detecting that the motion state misalignment perception degree is greater than a preset motion state misalignment perception degree threshold, executing a preset operation for guiding the passenger to adjust his / her posture to reduce the motion state misalignment perception degree of the passenger; The method comprises: obtaining driving data of the vehicle; Performing a vehicle driving characteristic correlation analysis on each second motion data and the driving data, and determining target motion data that does not conform to the vehicle driving characteristic among the plurality of second motion data; Determine a reference motion state of the passenger relative to the rear area according to the target motion data, wherein the reference motion state includes a reference motion trajectory and a reference motion speed; Determine motion trajectory difference data according to the reference motion trajectory and the driving trajectory in the driving data; Determine movement speed difference data according to the reference movement speed and the travel speed in the travel data; The motion state misalignment perception degree is determined according to the motion trajectory difference data and the motion speed difference data, and the motion state misalignment perception degree is positively correlated with the motion trajectory difference data and the motion speed difference data.

2. The method according to claim 1, characterized in that The determining that the passenger is in the cabin of the vehicle according to the plurality of first motion data collected by the smart watch includes: Performing motion direction characteristic analysis on the plurality of first motion data to determine at least one reference direction in which continuous component motion data exists; Performing a vehicle driving characteristic correlation analysis on the continuous component motion data of each reference direction of the at least one reference direction, and determining that there is a target direction that meets the vehicle driving characteristics in the at least one reference direction; The passenger's space status is marked as a cabin occupancy status.

3. The method according to claim 1, characterized in that The step of determining, according to the first image, that the passenger is in a rear row area of ​​the cabin includes: Performing feature recognition on the first image data to obtain an object feature recognition result and a relative position relationship feature recognition result, wherein the object feature recognition result includes identifying objects at the boundary of the framing object image space and objects within the framing object image space, and the relative position relationship feature recognition result is used to characterize the relative position relationship between the objects at the boundary of the framing object image space and the objects within the framing object image space; It is determined that the passenger is in the rear row area of ​​the cabin based on the object feature recognition result and the relative position relationship feature recognition result.

4. A passenger information processing device, characterized in that: Applied to a smart watch, the smart watch is worn by a passenger; the device comprises: A first determination unit is configured to determine that the passenger is in the cabin of the vehicle according to the plurality of first motion data collected by the smart watch; collect first image data of the space where the passenger is currently located, and determine that the passenger is in the rear row area of ​​the cabin according to the first image; A second determination unit is configured to determine a reference motion state of the passenger relative to the rear area according to the plurality of second motion data collected by the smart watch; and determine a motion state misalignment perception of the passenger in the current space according to the reference motion state; a guiding unit, configured to detect that the motion state misalignment perception degree is greater than a preset motion state misalignment perception degree threshold, and execute a preset operation for guiding the passenger to adjust his / her posture to reduce the motion state misalignment perception degree of the passenger; The second determination unit is specifically used to: obtain the driving data of the vehicle; perform a correlation analysis of the vehicle driving characteristics between each second motion data and the driving data, and determine the target motion data that does not conform to the vehicle driving characteristics among the plurality of second motion data; determine the reference motion state of the passenger relative to the rear area according to the target motion data, wherein the reference motion state includes a reference motion trajectory and a reference motion speed; The guidance unit is specifically used to: determine motion trajectory difference data according to the reference motion trajectory and the driving trajectory in the driving data; determine motion speed difference data according to the reference motion speed and the driving speed in the driving data; determine the motion state misalignment perception degree according to the motion trajectory difference data and the motion speed difference data, and the motion state misalignment perception degree is positively correlated with the motion trajectory difference data and the motion speed difference data.

5. An electronic device, characterized in that: include: A processor, a memory, and one or more programs; the one or more programs are stored in the memory and are configured to be executed by the processor, the programs comprising instructions for executing the steps in the method according to any one of claims 1 to 3.

6. A computer storage medium, characterized in that: The computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 3.

Images