Vehicle state data processing method and system of recording and navigation all-in-one machine

Abstract

The invention provides a vehicle state data processing method and system of a recording and navigation all-in-one machine, belongs to the technical field of data processing under the Internet industry or the technical field of vehicle parts in the new energy industry, and is applied to a vehicle machine system of a vehicle. The method comprises the steps of detecting that a parking guard mode of a vehicle is started, and detecting that a first person approaches the vehicle; identifying a first direction of the position of the first person relative to the vehicle; taking the first direction as a query identifier, querying a preset prediction model set, and obtaining a first behavior intention prediction model corresponding to the first direction; calling a first behavior intention prediction model to perform behavior intention prediction on the first user to obtain a first behavior intention prediction result; and controlling a first interaction part of the vehicle to execute first interaction reminding operation according to the first behavior intention prediction result. According to the invention, the intention of a person approaching the vehicle can be finely identified, and the interactive reminding function is carried out based on the intention, so that the parking safety of the vehicle is improved.

Description

Technical Field

[0001] This application belongs to the technical field of data processing under the Internet industry or the technical field of vehicle components in the new energy industry, and specifically relates to a method and system for processing vehicle state data of a recording navigation integrated machine. Background Art

[0002] The in-vehicle recording navigation integrated vehicle machine has developed rapidly. Currently, common integrated vehicle machines generally have functions such as reverse imaging, full-vehicle 360 imaging, and tire pressure monitoring. Moreover, with the development of new energy electric vehicle technology, it has become feasible to continuously supply power to the vehicle machine by the vehicle's energy system in the parked and off-vehicle state to achieve uninterrupted monitoring of the vehicle state.

[0003] Currently, many vehicle models have developed an uninterrupted status monitoring function such as the sentry mode based on the integrated vehicle machine system. Users can choose to enable this function. The vehicle machine system can identify predefined events such as abnormal tire pressure, abnormal vibration such as collision, and the approach of personnel around the vehicle, and can synchronize the identified abnormal status to the mobile phone of the registered user through the cloud.

[0004] In actual application scenarios, R & D personnel found that the current recognition and warning mechanism of the parking monitoring and guarding function based on the vehicle machine system is difficult to meet user needs in some scenario branches. For example, in the scenario branch where there are certain potential risk actions after personnel approach the vehicle, the vehicle machine system can only identify the approach of personnel, record videos, and support users to query afterwards. This makes some physical events on the vehicle that could have been avoided through active recognition and reminder occur unnecessarily. Summary of the Invention

[0005] This application provides a method and system for processing vehicle state data of a recording navigation integrated machine to achieve the function of finely identifying the intentions of personnel approaching the vehicle and performing interactive reminders based on the intentions, improving the parking safety of the vehicle, facilitating reducing or even avoiding user losses, and enhancing the user experience.

[0006] In a first aspect, an embodiment of this application provides a method for processing vehicle state data, which is applied to the vehicle machine system of a vehicle. The method includes: Detect that the parking guard mode of the vehicle is turned on and detect that a first person approaches the vehicle; Identify the first direction of the position of the first person relative to the vehicle; Using the first direction as a query identifier, query a preset set of prediction models to obtain a first behavior intention prediction model corresponding to the first direction. The prediction model set contains the correspondence between the direction and the behavior intention prediction model. The behavior intention prediction model is a prediction model created based on the training strategy for analyzing physical events of vehicle components. The training strategy for analyzing physical events of vehicle components includes the mapping relationship between the direction and vehicle components. In the mapping relationship, the vehicle components are used to construct the training data of the behavior intention prediction model; Invoke the first behavior intention prediction model to predict the behavior intention of the first user, and obtain a first behavior intention prediction result; Control the first interaction component of the vehicle to perform a first interaction reminder operation according to the first behavior intention prediction result.

[0007] In a second aspect, the present application provides a vehicle state data processing system, including a vehicle-mounted system, The vehicle-mounted system is configured to detect that the parking guard mode of the vehicle is turned on and detect that a first person approaches the vehicle; and identify a first direction of the position of the first person relative to the vehicle; and use the first direction as a query identifier to query a preset set of prediction models to obtain a first behavior intention prediction model corresponding to the first direction. The prediction model set contains the correspondence between the direction and the behavior intention prediction model. The behavior intention prediction model is a prediction model created based on the training strategy for analyzing physical events of vehicle components. The training strategy for analyzing physical events of vehicle components includes the mapping relationship between the direction and vehicle components. In the mapping relationship, the vehicle components are used to construct the training data of the behavior intention prediction model; and invoke the first behavior intention prediction model to predict the behavior intention of the first user, and obtain a first behavior intention prediction result; and control the first interaction component of the vehicle to perform a first interaction reminder operation according to the first behavior intention prediction result.

[0008] It can be seen that in this embodiment, when the in-vehicle system detects that the parking guard mode of the vehicle is turned on and detects that a first person approaches the vehicle, it first identifies the first direction of the position of the first person relative to the vehicle, and then uses the first direction as a query identifier to query a preset set of prediction models, obtains the first behavior intention prediction model corresponding to the first direction. The set of prediction models contains the correspondence between directions and behavior intention prediction models. The behavior intention prediction model is a prediction model created based on the training strategy for physical event analysis of vehicle components. The training strategy for physical event analysis of vehicle components includes the mapping relationship between the direction and vehicle components. The vehicle components in the mapping relationship are used to construct the training data of the behavior intention prediction model. Then, it calls the first behavior intention prediction model to predict the behavior intention of the first user, obtains the first behavior intention prediction result, and finally controls the first interaction component of the vehicle to perform a first interaction reminder operation according to the first behavior intention prediction result.

[0009] Compared with the current situation that the in-vehicle system in the parking guard mode cannot identify the intention of a person approaching the vehicle and cannot control the vehicle to perform obvious interaction reminders in real time based on the intention recognition result, this application can achieve the function of finely identifying the intention of a person approaching the vehicle and performing interaction reminders based on the intention, improving the parking safety of the vehicle, being beneficial to reducing or even avoiding user losses, and improving the use experience. And since decoupled prediction models are set for each direction and can be independently awakened and called, the model complexity is reduced to balance the computing power consumption and power consumption, and the algorithm efficiency is improved to reduce the power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a schematic diagram of the composition of a vehicle state data processing system provided by an embodiment of the present application; Figure 2 It is a schematic flowchart of a vehicle state data processing method provided by an embodiment of the present application; Figure 3 It is a schematic diagram of a prompt interface provided by an embodiment of the present application; Figure 4 It is a schematic diagram of an in-vehicle system recording file viewing interface provided by an embodiment of the present application; Figure 5It is one of the schematic diagrams of the vehicle component viewing interface of the in-vehicle system; Figure 6 It is the second schematic diagram of the vehicle component viewing interface of the in-vehicle system; Figure 7 The third schematic diagram of the vehicle component viewing interface of the in-vehicle system; Figure 8 It is the ROI image area corresponding to the left direction provided by the embodiment of the present application; Figure 9 It is the ROI image area corresponding to the tail direction provided by the embodiment of the present application. Detailed implementation manners

[0012] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be accurately and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0013] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other steps or modules inherent to these processes, methods, products or devices.

[0014] Referring to "embodiment" herein means that a specific feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0015] Currently, in the parking guard mode, the in-vehicle system cannot recognize the intentions of people in the scenario of people approaching the vehicle, nor can it perform obvious interactive reminders based on the intention recognition results to control the vehicle in real time. For example, in the scenario branch of people approaching the vehicle, the in-vehicle system can only recognize that there are people approaching, record videos, and support users to query afterwards, which makes some physical events on the vehicle end that could have been avoided through active recognition reminders occur unnecessarily.

[0016] In view of the above problems, the embodiments of the present application provide a method and system for processing vehicle status data of a recording navigation integrated machine. The embodiments of the present application will be introduced in detail below with reference to the accompanying drawings.

[0017] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the composition of a vehicle status data processing system provided by the embodiments of the present application. The vehicle status data processing system 10 includes a vehicle-mounted system 101 and a sensing device 102 communicatively connected to the vehicle-mounted system. The sensing device 102 can be a camera, millimeter-wave radar, ultrasonic radar, lidar, etc. installed on the body of different sides of the vehicle, as well as vibration sensors installed on other parts of the vehicle. The vehicle-mounted system 101 is located in the vehicle. In particular, the vehicle-mounted system 101 can be a recording navigation integrated machine. In a specific implementation, the vehicle-mounted system 101 can transmit the vehicle status to the user's mobile terminal 30 in real time through a server 20, and the user can view the vehicle status and the situation around the vehicle through the mobile terminal 30. The vehicle-mounted system 101 can also communicate with the mobile terminal through a Bluetooth module.

[0018] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a vehicle status data processing method provided by the embodiments of the present application. The vehicle status data processing method is applied to the vehicle-mounted system of the vehicle and specifically includes the following steps.

[0019] S201, it is detected that the parking guard mode of the vehicle is turned on and a first person is detected approaching the vehicle.

[0020] Among them, the user information of the vehicle owner can be obtained first. After the person approaching the vehicle is obtained, the approaching person is identified based on the user information. If the approaching person is not the vehicle owner, the approaching person is considered as the first person. In particular, when the vehicle owner leaves the vehicle, the identity information of other people who leave the vehicle together with the vehicle owner can also be obtained synchronously, and when the approaching person is not other people, it is determined that the approaching person is the first person. The identity information includes facial feature information and / or body feature information, clothing information.

[0021] S202, identify the first direction of the position of the first person relative to the vehicle.

[0022] Among them, the first direction includes the rear direction of the vehicle, the left side direction of the vehicle, the right side direction of the vehicle, and the front direction of the vehicle, etc.

[0023] S203, using the first direction as a query identifier, query the preset prediction model set, and obtain the first behavior intention prediction model corresponding to the first direction.

[0024] Among them, the prediction model set includes the correspondence between the direction and the behavior intention prediction model. The behavior intention prediction model is a prediction model created based on the training strategy for analyzing physical events of vehicle components. The training strategy for analyzing physical events of vehicle components includes the mapping relationship between the direction and the vehicle components. In the mapping relationship, the vehicle components are used to construct the training data of the behavior intention prediction model.

[0025] S204. Invoke the first behavior intention prediction model to predict the behavior intention of the first user, and obtain the first behavior intention prediction result.

[0026] Among them, the intention prediction result can be divided into a normal behavior intention and an abnormal behavior intention. The abnormal behavior intention includes the intention of damaging different vehicle components.

[0027] S205. Control the first interaction component of the vehicle to perform the first interaction reminder operation according to the first behavior intention prediction result.

[0028] Among them, the first interaction component may include interaction components such as the horn, headlights, and windshield wipers of the vehicle. In a specific implementation, for example Figure 3 shown Figure 3 is a schematic diagram of a prompt interface provided by an embodiment of the present application. When performing the first interaction reminder operation, a reminder message and real-time images can also be synchronously sent to the user's mobile terminal for the user to confirm the situation. The mobile terminal is communicatively connected to the in-vehicle system, and after the user goes out and leaves the vehicle, the vehicle situation can also be obtained in real time. As Figure 3 shown, when the in-vehicle system predicts the intention of the first person and the intention is an abnormal intention, a prompt message is sent to the mobile terminal to remind the user that "there is an intention to damage the rear headlights of the vehicle". At this time, the user can operate on the mobile terminal not to perform the interaction reminder operation, then the in-vehicle system does not perform this operation. If the user does not confirm not to prompt within a preset time period, the prompt operation is performed. After the prompt operation is completed, a prompt message is sent to the user again to prompt the user that it has been executed, for example Figure 3 shown, "The interaction reminder operation has been executed".

[0029] In a specific implementation, after the in-vehicle system controls the interaction component of the vehicle to perform the interaction reminder operation, the behavior intention image of the first person can also be displayed on the page. For example Figure 4 shown Figure 4It is a schematic diagram of an interface for viewing recorded files of a vehicle-mounted system provided by an embodiment of the present application. It can be seen that the user can query the historical situation of the execution of the interactive reminder operation on this viewing interface. After the vehicle-mounted system determines the first behavior intention, it can trace back the first time node corresponding to the execution of the interactive reminder operation to the second time node of the video frame with clear personnel images as the starting point, and the third time node when the personnel leave the vehicle as the end point, so as to obtain the video recording corresponding to this event. And in this video recording, the second time node can be marked, and the user can directly jump to the video frame corresponding to the second time node for display through the query identifier. And the viewing interface of the recorded files of the vehicle-mounted system can provide interactive reminder event entries, and each entry corresponds to a video segment that can quickly locate the complete process of the personnel from approaching the vehicle to leaving the vehicle. As Figure 4 shown, there are currently two interactive reminder events, corresponding to a "front headlight damage intention" and a "right rear wheel damage intention" respectively. When the user clicks on the first mutual reminder event, the corresponding video segment can be queried.

[0030] It can be seen that in this embodiment, compared with the current situation where the vehicle-mounted system in the parking guard mode cannot recognize the intention of personnel in the scene of personnel approaching the vehicle and cannot perform obvious interactive reminders based on the intention recognition result to control the vehicle in real time, the present application can realize the refined recognition of the intention of personnel approaching the vehicle and the function of interactive reminder based on the intention, improve the parking safety of the vehicle, is conducive to reducing or even avoiding user losses, and improves the use experience. And because decoupled prediction models are set in each direction, they can be independently awakened and called, reducing the model complexity to balance the computing power consumption and power consumption, and improving the algorithm efficiency and reducing the power consumption.

[0031] In a possible embodiment, the calling the first behavior intention prediction model to predict the behavior intention of the first user to obtain the first behavior intention prediction result includes: obtaining the first perception data group of the first perception device group in the first direction of the vehicle, where the first perception device group includes perception devices of at least one of the following device types: a shooting device, a radar device, and a vibration detection device; performing a data preprocessing operation on the first perception data group to obtain a second perception data group, where the data preprocessing operation includes a screening operation based on the regional perception data of the first vehicle component corresponding to the first direction, and the mapping relationship includes the corresponding relationship between the first direction and the first vehicle component; calling the first behavior intention prediction model to process the second perception data group to obtain the first behavior intention prediction result.

[0032] Among them, the perception device groups corresponding to different directions may not be exactly the same, that is, for example, the imaging devices corresponding to the first direction and the second direction are different. For example, the first direction is the vehicle head direction, i.e., the front side direction, and the first perception data group includes but is not limited to: radar system perception data (such as sensing data of millimeter wave radar, ultrasonic radar, lidar), and vision system perception data (such as recorded video data of the front view camera on the front windshield and the fisheye camera on the roof).

[0033] In a possible embodiment, if the first direction is the vehicle head direction of the vehicle, the first vehicle component includes any one of the front vehicle logo, front vehicle lamp, engine hood, and front bumper, and the first behavior intention prediction result includes one or more of normal behavior, behavior of damaging the front vehicle logo, behavior of damaging the vehicle lamp, behavior of damaging the engine hood, and behavior of damaging the front bumper; If the first direction is the vehicle tail direction of the vehicle, the first vehicle component includes any one of the rear vehicle logo, rear vehicle lamp, rear trunk, and rear bumper, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the rear vehicle logo, behavior of damaging the rear vehicle lamp, behavior of damaging the rear trunk, and behavior of damaging the rear bumper; If the first direction is the left side direction of the vehicle, the first vehicle component includes any one of the left front wheel, left front door, left rear wheel, left rear door, and left fender, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the left front wheel, behavior of damaging the left front door, behavior of damaging the left rear wheel, behavior of damaging the left rear door, and behavior of damaging the left fender; If the first direction is the right side direction of the vehicle, the first vehicle component includes any one of the right front wheel, right front door, right rear wheel, right rear door, and right fender, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the right front wheel, behavior of damaging the right front door, behavior of damaging the right rear wheel, behavior of damaging the right rear door, and behavior of damaging the right fender.

[0034] In specific implementation, when performing data preprocessing, screening can be performed based on whether the obtained first perception data group is related to the corresponding vehicle component. For example, when the first direction is the vehicle tail direction, the data in the first perception data group is screened, and only the data related to the rear vehicle logo, rear vehicle lamp, rear trunk, and rear bumper is screened, such as the imaging data that can display the real-time images of the rear vehicle logo, rear vehicle lamp, rear trunk, and rear bumper.

[0035] In particular, since the external environment is different, the perception data in the first perception data group may not be exactly the same. For example, when the current external environment is at night, if the first direction is the front of the vehicle and the first vehicle component is the front vehicle logo, the perception data in the corresponding first perception data group may include on-vehicle radar data, feedback data of the lighting system, and / or feedback data of the vibration sensor, etc. At night, the vibration sensor can monitor the vibration data in the logo area to determine whether the logo has been externally impacted or affected by strong wind. The vehicle logo is usually relatively flat or has certain reflective characteristics. If the logo is damaged or blocked, the signal returned by the on-vehicle radar may change. If the logo area is damaged, its position is shifted, or the logo is blocked by a person, the light emitted by the vehicle lamp may not evenly illuminate the logo. Such irregular illumination or deviation of the light may indicate an abnormality in the logo. For example, the reflection angle of the vehicle lamp may change, resulting in partial occlusion or incomplete illumination of the logo. If there is an abnormality in the logo, such as someone approaching, shifting, or dropping, the illumination angle, reflection pattern, etc. of the vehicle lamp may be inconsistent with the normal situation. At night, by combining the data of the on-vehicle radar and the on-vehicle lighting system to determine whether there is an abnormality at the logo, and then combining the vibration data to determine whether there is a risk of damage by a person. For example, when the radar detects abnormal reflection in a certain area, the vehicle lamp can focus on that area to confirm whether there is someone approaching at the logo and whether the logo is blocked. Then, determine whether there is an abnormality in the vibration data at the logo, so as to comprehensively judge whether there is an abnormality in the vehicle logo in the front direction of the vehicle at night.

[0036] During the day, even if the first direction is still the front of the vehicle and the first vehicle component is still the vehicle logo, the perception data in the corresponding first perception data group may be different from that at night, for example, including video data of the front-view camera, on-vehicle radar data, and vibration data.

[0037] In specific implementation, as Figure 5 shown, Figure 5 is a schematic diagram of the vehicle component viewing interface of the in-vehicle system. After enabling the parking guard mode, the user can also view the conditions of different vehicle components corresponding to different directions in real time through the display interface of the in-vehicle system. For example, the current radar data, video data, etc. at the front vehicle logo in the front direction of the vehicle. If there is an abnormal condition in a certain vehicle component, the display bar corresponding to that component can be highlighted, so that the user can know the current part where an abnormality may occur and check it in time. In this way, when someone approaches, the user can view the conditions of specific vehicle components without getting out of the vehicle, which can improve the safety of the user. As Figure 6 shown, the user can also select to view the real-time image information of each individual component within the corresponding direction. And the user can choose to view 2D images or 3D images. For example Figure 6If you select to view the 2D image of the right rear wheel in the lower right direction on the right, the corresponding image information will be displayed in the image display area. In particular, the image data recorded in the parking guard mode can also be viewed on the in-vehicle system interface. For example Figure 7 As shown, the historical images recorded under the parking guard model are displayed on the image viewing interface of the in-vehicle system. The user can select the corresponding direction to view the image information of specific components, or directly click on the comprehensive image files in each direction displayed on the display interface to view the vehicle conditions in the corresponding direction.

[0038] It can be seen that in this embodiment, preprocessing the acquired data based on the vehicle components corresponding to the direction can reduce the amount of data processing and improve the intention prediction efficiency.

[0039] In a possible embodiment, the first perception device group includes the photographing device; the creation process of the first training set of the first behavior intention prediction model includes the following steps: obtaining a basic picture containing the image information of the first vehicle component and the user; dividing the basic picture into a first basic picture with abnormal behavior and a second basic picture without abnormal behavior according to whether there is abnormal behavior; performing a first ROI image region division on the first basic picture to obtain a third basic picture containing the image information of the first vehicle component and the user; creating a corresponding damage behavior type label for the third basic picture to obtain a first training sample; performing a second ROI image region division on the second basic picture to obtain a fourth basic picture containing the image information of the first vehicle component and the user; creating a corresponding normal behavior label for the fourth basic picture to obtain a second training sample; constructing the first training set according to the first training sample and the second training sample.

[0040] Among them, when performing the Region of Interest (ROI) image region division, the image can be subjected to object detection through a trained model (such as Mask R-CNN), and the bounding boxes and segmentation masks of the image information of the first vehicle component and the user in the basic picture are respectively output based on the model, so as to extract the region of the first vehicle component and the user region in each basic picture. The abnormal behavior here is a coarse-grained description of abnormal behavior, such as damaging the vehicle. In specific implementation, a large number of pictures can be searched on the Internet, and the description information in the picture source may include the description information of damaging the vehicle, etc.

[0041] In particular, during training, the pictures included in the first training set can also be classified according to the vehicle direction, such as the front direction, left direction, rear direction, and right direction of the vehicle, etc. Then, the intention training model is trained respectively based on the classified training subsets to obtain behavior prediction models corresponding to different directions. When dividing the ROI image region, the images in the training subsets corresponding to different vehicle directions can be divided respectively. For example Figure 8 as shown Figure 8 is the ROI image region corresponding to the left direction provided in the embodiment of the present application. When dividing the ROI image region, it includes an image region of the user represented by a red frame, and two image regions of vehicle components represented by yellow frames, and the corresponding vehicle components are the left rear wheel and the left rear door respectively. For example Figure 9 as shown Figure 9 is the ROI image region corresponding to the rear direction provided in the embodiment of the present application. When dividing the ROI image region, it includes an image region of the user represented by a red frame, and three image regions of vehicle components represented by yellow frames, and the corresponding vehicle components are the rear lights, the rear logo, and the rear bumper respectively.

[0042] In specific implementation, when training the behavior intention prediction model, the features of the pictures in the first training set can be extracted first through a feature extraction network to capture static features, such as vehicle components, the postures of personnel, tools held by personnel, the spatial relationship between personnel and vehicle components, and the spatial relationship between the tool and vehicle components when holding the tool, etc. The feature extraction network can include multiple 3×3 convolutional layers and a pooling layer. In particular, video data can also be obtained, and then an action recognition model (such as I3D, C3D, or LSTM) is used to extract temporal features to obtain dynamic features, such as the number of movements and the movement trajectory of the human hand of the first person within a specific monitoring period, and this movement trajectory can be used to indicate a preset destruction behavior. Taking the example that person A approaches the left rear wheel area of vehicle B, person A holds a tool, and after approaching and stopping at the left rear wheel area, the static features can include the contact state feature between the left hand of person A and the C-pillar area of vehicle B, and the non-camera effective viewing space area where the tool held by the right hand of this person penetrates between the plane of the left rear wheel arch and the plane of the wheel. The dynamic feature can be the number of movements and the movement trajectory of the right hand of person A during the standing still state period, and this movement trajectory corresponds to the preset exclusive action trajectory feature of puncturing or scratching the tire.

[0043] Then, the static feature and the dynamic feature form a fusion feature according to their respective weights, and the model prediction result is obtained based on the fusion feature participating in the subsequent processing of the sub-network of the model. The fusion feature can be a feature vector containing spatial and temporal information. Among them, the video data can be generated based on multiple pictures with temporal continuity included in the above-mentioned first training set.

[0044] In a specific implementation, since approaching from the front or the rear depends more on relative speed and direction, while approaching from the side depends more on the viewing angle and relative distance, determining the weights of static features and dynamic features may include: determining the vehicle direction corresponding to the current training data. If the vehicle direction is the front direction or the rear direction, it is determined that the weight of the dynamic features is higher than that of the static features; if the vehicle direction is the left direction or the right direction, it is determined that the static weight is higher than the dynamic weight.

[0045] This can make the model more focused on the behavioral characteristics in a specific direction, thereby improving the accuracy. Moreover, the data in different directions may involve different behavioral patterns. Training these data separately can effectively reduce the data noise caused by direction changes. The model can avoid confusion in directions, thereby reducing the situation of misjudgment. And by training the data in different directions separately, the model will learn more targeted features for each direction, which helps to improve the model's recognition ability for different directions in the real scenario and enhance the generalization ability of the model.

[0046] In a possible embodiment, the controlling the first interaction component of the vehicle to perform the first interaction reminder operation according to the first behavior intention prediction result includes: querying a preset set of interaction reminder policies according to the first behavior intention prediction result to obtain a first interaction reminder policy. The set of interaction reminder policies contains the mapping relationship between the behavior intention prediction result and the interaction reminder policy. The interaction reminder policy includes the interaction component to be enabled and the information output content based on the interaction component; parsing the first interaction component and the first interaction information content in the first interaction reminder policy; controlling the first interaction component of the vehicle to output the first interaction information content.

[0047] Among them, the interaction components corresponding to different intention prediction results may be different, or the vehicle components or vehicle directions corresponding to the intention prediction results are different, and the corresponding interaction components may also be different. For example, if the current intention prediction result corresponds to an abnormal intention in front of the vehicle, the corresponding interaction components may include the front headlights, windshield wipers, and vehicle horn. If the current intention prediction result corresponds to an abnormal intention at the rear of the vehicle, the corresponding interaction components may include the rear headlights and vehicle horn. Or if the current intention prediction result is an abnormal behavior directed at the vehicle logo, the corresponding interaction component may be the vehicle horn, and if it is an abnormal behavior directed at the front headlights, the corresponding interaction component may be the front headlights. The interaction information can be associated with the interaction component. For example, if the interaction component is the vehicle horn, the interaction information may include the vehicle horn sounding briefly multiple times, or the vehicle horn sounding continuously for a period of time, which can be determined based on the interaction policies for different intention prediction results.

[0048] For example, when the vehicle is parked by the roadside and the parking guard mode is turned on, if a first person approaches the rear of the vehicle at this time, and then based on the behavior intention prediction model, it is predicted that the first person has the intention to damage the left rear wheel of the vehicle. At this time, based on the interaction reminder strategy, the interaction components are determined to be the vehicle horn and the vehicle rear lights, and the interaction information is that the vehicle horn sounds continuously and the rear lights flash.

[0049] It can be seen that in this embodiment, based on the rain prediction result, the interaction reminder strategy is obtained, then based on the interaction reminder strategy, the interaction components are determined, and then the interaction information is output. The function of interactive reminder based on intention can be realized, the parking safety of the vehicle can be improved, which is beneficial to reducing or even avoiding user losses and improving the use experience.

[0050] The present application also provides a vehicle state data processing system, including a vehicle-mounted system. The vehicle-mounted system is used to detect that the parking guard mode of the vehicle is turned on and detect that a first person approaches the vehicle; and identify the first direction of the position of the first person relative to the vehicle; and use the first direction as a query identifier to query a preset prediction model set to obtain a first behavior intention prediction model corresponding to the first direction. The prediction model set includes the correspondence between the direction and the behavior intention prediction model. The behavior intention prediction model is a prediction model created based on the physical event analysis training strategy of vehicle components being damaged. The physical event analysis training strategy of vehicle components being damaged includes the mapping relationship between the direction and the vehicle components. The vehicle components in the mapping relationship are used to construct the training data of the behavior intention prediction model; and call the first behavior intention prediction model to predict the behavior intention of the first user to obtain a first behavior intention prediction result; and control the first interaction component of the vehicle to perform a first interaction reminder operation according to the first behavior intention prediction result.

[0051] In a possible embodiment, in terms of calling the first behavior intention prediction model to predict the behavior intention of the first user to obtain a first behavior intention prediction result, the vehicle-mounted system is specifically used to: obtain a first perception data set of a first perception device group in the first direction of the vehicle. The first perception device group includes perception devices of at least one of the following device types: a photographing device, a radar device, and a vibration detection device; and perform data preprocessing operations on the first perception data set to obtain a second perception data set. The data preprocessing operations include a screening operation based on the regional perception data of the first vehicle component corresponding to the first direction. The mapping relationship includes the correspondence between the first direction and the first vehicle component; and call the first behavior intention prediction model to process the second perception data set to obtain the first behavior intention prediction result.

[0052] In a possible embodiment, if the first direction is the front direction of the vehicle, the first vehicle component includes any one of a front vehicle logo, front vehicle lights, an engine hood, and a front bumper, and the first behavior intention prediction result includes one or more of normal behavior, behavior of damaging the front vehicle logo, behavior of damaging the vehicle lights, behavior of damaging the engine hood, and behavior of damaging the front bumper; if the first direction is the rear direction of the vehicle, the first vehicle component includes any one of a rear vehicle logo, rear vehicle lights, a rear trunk, and a rear bumper, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the rear vehicle logo, behavior of damaging the rear vehicle lights, behavior of damaging the rear trunk, and behavior of damaging the rear bumper; if the first direction is the left side direction of the vehicle, the first vehicle component includes any one of a left front wheel, a left front door, a left rear wheel, a left rear door, and a left fender, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the left front wheel, behavior of damaging the left front door, behavior of damaging the left rear wheel, behavior of damaging the left rear door, and behavior of damaging the left fender; if the first direction is the right side direction of the vehicle, the first vehicle component includes any one of a right front wheel, a right front door, a right rear wheel, a right rear door, and a right fender, and the first behavior intention prediction result includes one or more of the normal behavior, behavior of damaging the right front wheel, behavior of damaging the right front door, behavior of damaging the right rear wheel, behavior of damaging the right rear door, and behavior of damaging the right fender.

[0053] In a possible embodiment, the first perception device group includes the photographing device; the creation process of the first training set of the first behavior intention prediction model includes the following steps: obtaining a basic picture containing image information of the first vehicle component and the user; dividing the basic picture into a first basic picture with abnormal behavior and a second basic picture without abnormal behavior according to whether there is abnormal behavior; performing a first ROI image region division on the first basic picture to obtain a third basic picture containing image information of the first vehicle component and the user; creating a corresponding damage behavior type label for the third basic picture to obtain a first training sample; performing a second ROI image region division on the second basic picture to obtain a fourth basic picture containing image information of the first vehicle component and the user; creating a corresponding normal behavior label for the fourth basic picture to obtain a second training sample; and constructing the first training set according to the first training sample and the second training sample.

[0054] In a possible embodiment, in terms of the first interaction component that controls the vehicle according to the predicted result of the first behavior intention performing a first interaction reminder operation, the in-vehicle system is specifically configured to: query a preset set of interaction reminder policies according to the predicted result of the first behavior intention to obtain a first interaction reminder policy, where the set of interaction reminder policies includes the mapping relationship between the predicted result of the behavior intention and the interaction reminder policy, and the interaction reminder policy includes the interaction components to be enabled and the information output content based on the interaction components; and, parse the first interaction component and the first interaction information content in the first interaction reminder policy; and, control the first interaction component of the vehicle to output the first interaction information content.

[0055] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other arbitrary combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless manner. The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0056] The embodiments of the present application further provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute some or all of the steps of any of the methods described in the above method embodiments, and the above computer includes an electronic device.

[0057] The embodiments of the present application further provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The computer program product can be a software installation package, and the above computer includes an electronic device.

[0058] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0059] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0060] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the above division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical or other forms.

[0061] The units described as separate components above may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0063] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in various embodiments of this application. The aforementioned memory includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), external hard drives, magnetic disks, or optical discs that can store program codes.

[0064] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory. The memory can include: flash drives, read-only memories (English: Read-Only Memory, abbreviated: ROM), random access memories (English: Random Access Memory, abbreviated: RAM), magnetic disks, or optical discs, etc.

[0065] The above has introduced the embodiments of this application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A vehicle status data processing method, characterized in that: The method is applied to a vehicle system of a vehicle, and comprises: Detecting that the parking guard mode of the vehicle is turned on and detecting that a first person is approaching the vehicle; identifying a first direction of a position of the first person relative to the vehicle; Taking the first direction as a query identifier, querying a preset prediction model set to obtain a first behavior intention prediction model corresponding to the first direction, wherein the prediction model set includes a correspondence between a direction and a behavior intention prediction model, wherein the behavior intention prediction model is a prediction model created based on a passive physical event analysis training strategy for vehicle components, wherein the passive physical event analysis training strategy for vehicle components includes a mapping relationship between the direction and vehicle components, wherein the vehicle components in the mapping relationship are used to construct training data for the behavior intention prediction model; Calling the first behavior intention prediction model to predict the behavior intention of the first user to obtain a first behavior intention prediction result; The first interactive component of the vehicle is controlled to perform a first interactive reminder operation according to the first behavior intention prediction result.

2. The method according to claim 1, characterized in that The calling the first behavior intention prediction model to predict the behavior intention of the first user to obtain a first behavior intention prediction result includes: Acquire a first perception data group of a first perception device group in the first direction of the vehicle, wherein the first perception device group includes a perception device of at least one of the following device types: a camera, a radar device, and a vibration detection device; Performing a data preprocessing operation on the first perception data group to obtain a second perception data group, wherein the data preprocessing operation includes a screening operation based on regional perception data of a first vehicle component corresponding to the first direction, and the mapping relationship includes a corresponding relationship between the first direction and the first vehicle component; The first behavior intention prediction model is called to process the second perception data group to obtain the first behavior intention prediction result.

3. The method according to claim 2, characterized in that If the first direction is the front direction of the vehicle, the first vehicle component includes any one of a front logo, a front headlight, a hood, and a front bumper, and the first behavior intention prediction result includes one or more of a normal behavior, a behavior of damaging a front logo, a behavior of damaging a headlight, a behavior of damaging a hood, and a behavior of damaging a front bumper; If the first direction is the rear direction of the vehicle, the first vehicle component includes any one of a rear vehicle logo, a rear light, a rear trunk, and a rear bumper, and the first behavior intention prediction result includes one or more of the normal behavior, the behavior of damaging the rear vehicle logo, the behavior of damaging the rear light, the behavior of damaging the rear trunk, and the behavior of damaging the rear bumper; If the first direction is the left direction of the vehicle, the first vehicle component includes any one of a left front wheel, a left front door, a left rear wheel, a left rear door, and a left fender, and the first behavior intention prediction result includes one or more of the normal behavior, damaging the left front wheel, damaging the left front door, damaging the left rear wheel, damaging the left rear door, and damaging the left fender; If the first direction is the right direction of the vehicle, the first vehicle component includes any one of the right front wheel, the right front door, the right rear wheel, the right rear door, and the right fender, and the first behavior intention prediction result includes one or more of the normal behavior, damaging the right front wheel, damaging the right front door, damaging the right rear wheel, damaging the right rear door, and damaging the right fender.

4. The method according to claim 2 or 3, characterized in that: The first sensing device group includes the photographing device; the process of creating the first training set of the first behavior intention prediction model includes the following steps: Acquire a basic picture including image information of the first vehicle component and the user; According to whether there is abnormal behavior, the basic picture is divided into a first basic picture with abnormal behavior and a second basic picture without abnormal behavior; Performing a first ROI image region division on the first basic image to obtain a third basic image including image information of the first vehicle component and the user; creating a corresponding destructive behavior type label for the third basic image to obtain a first training sample; Performing a second ROI image region division on the second basic image to obtain a fourth basic image containing image information of the first vehicle component and the user; creating a corresponding normal behavior label for the fourth basic image to obtain a second training sample; The first training set is constructed according to the first training sample and the second training sample.

5. The method according to any one of claims 1 to 3, characterized in that: The controlling the first interactive component of the vehicle to perform a first interactive reminder operation according to the first behavior intention prediction result includes: According to the first behavior intention prediction result, a preset interaction reminder strategy set is queried to obtain a first interaction reminder strategy, wherein the interaction reminder strategy set includes a mapping relationship between the behavior intention prediction result and the interaction reminder strategy, and the interaction reminder strategy includes an interaction component to be enabled and information output content based on the interaction component; Parsing the first interactive component and the first interactive information content in the first interactive reminder strategy; The first interactive component of the vehicle is controlled to output the first interactive information content.

6. A vehicle status data processing system, characterized in that: Including car system, The vehicle system is used to detect that the parking guard mode of the vehicle is turned on and that a first person is approaching the vehicle; and, identifying a first direction of the first person's position relative to the vehicle; and, using the first direction as a query identifier, querying a preset prediction model set to obtain a first behavior intention prediction model corresponding to the first direction, the prediction model set including a correspondence between directions and behavior intention prediction models, the behavior intention prediction model being a prediction model created based on a vehicle component passive physical event analysis training strategy, the vehicle component passive physical event analysis training strategy including a mapping relationship between the directions and vehicle components, the vehicle components in the mapping relationship being used to construct training data for the behavior intention prediction model; and, calling the first behavior intention prediction model to predict the behavior intention of the first user to obtain a first behavior intention prediction result; And, according to the first behavior intention prediction result, a first interactive component of the vehicle is controlled to perform a first interactive reminder operation.

7. The system according to claim 6, characterized in that In the aspect of calling the first behavior intention prediction model to predict the behavior intention of the first user and obtaining the first behavior intention prediction result, the vehicle system is specifically used to: Acquire a first perception data group of a first perception device group in the first direction of the vehicle, the first perception device group including perception devices of at least one of the following device types: a camera, a radar device, and a vibration detection device; and perform a data preprocessing operation on the first perception data group to obtain a second perception data group, the data preprocessing operation including a screening operation based on regional perception data of a first vehicle component corresponding to the first direction, the mapping relationship including a correspondence between the first direction and the first vehicle component; And, calling the first behavior intention prediction model to process the second perception data group to obtain the first behavior intention prediction result.

8. The system according to claim 7, characterized in that If the first direction is the front direction of the vehicle, the first vehicle component includes any one of a front logo, a front headlight, a hood, and a front bumper, and the first behavior intention prediction result includes one or more of a normal behavior, a behavior of damaging a front logo, a behavior of damaging a headlight, a behavior of damaging a hood, and a behavior of damaging a front bumper; If the first direction is the rear direction of the vehicle, the first vehicle component includes any one of a rear vehicle logo, a rear light, a rear trunk, and a rear bumper, and the first behavior intention prediction result includes one or more of the normal behavior, the behavior of damaging the rear vehicle logo, the behavior of damaging the rear light, the behavior of damaging the rear trunk, and the behavior of damaging the rear bumper; If the first direction is the left direction of the vehicle, the first vehicle component includes any one of a left front wheel, a left front door, a left rear wheel, a left rear door, and a left fender, and the first behavior intention prediction result includes one or more of the normal behavior, damaging the left front wheel, damaging the left front door, damaging the left rear wheel, damaging the left rear door, and damaging the left fender; If the first direction is the right direction of the vehicle, the first vehicle component includes any one of the right front wheel, the right front door, the right rear wheel, the right rear door, and the right fender, and the first behavior intention prediction result includes one or more of the normal behavior, damaging the right front wheel, damaging the right front door, damaging the right rear wheel, damaging the right rear door, and damaging the right fender.

9. The system according to claim 7 or 8, characterized in that: The first sensing device group includes the photographing device; the process of creating the first training set of the first behavior intention prediction model includes the following steps: Acquire a basic picture including image information of the first vehicle component and the user; According to whether there is abnormal behavior, the basic picture is divided into a first basic picture with abnormal behavior and a second basic picture without abnormal behavior; Performing a first ROI image region division on the first basic image to obtain a third basic image including image information of the first vehicle component and the user; creating a corresponding destructive behavior type label for the third basic image to obtain a first training sample; Performing a second ROI image region division on the second basic image to obtain a fourth basic image containing image information of the first vehicle component and the user; creating a corresponding normal behavior label for the fourth basic image to obtain a second training sample; The first training set is constructed according to the first training sample and the second training sample.

10. The system according to any one of claims 6 to 8, characterized in that: In terms of controlling the first interactive component of the vehicle to perform the first interactive reminder operation according to the first behavior intention prediction result, the vehicle system is specifically used to: According to the first behavior intention prediction result, a preset interaction reminder strategy set is queried to obtain a first interaction reminder strategy, wherein the interaction reminder strategy set includes a mapping relationship between the behavior intention prediction result and the interaction reminder strategy, and the interaction reminder strategy includes an interaction component to be enabled and information output content based on the interaction component; and, parsing the first interactive component and the first interactive information content in the first interactive reminder strategy; And, controlling the first interactive component of the vehicle to output the first interactive information content.

Images