Vehicle information processing method and related device, vehicle, and storage medium

Abstract

The present application discloses a vehicle information processing method and related devices, a vehicle, and a storage medium. The vehicle information processing method includes: obtaining the current position of the vehicle; determining the trajectory point corresponding to the current position on a preset route; and using the vehicle's onboard components to perform preset processing that matches the position of the trajectory point on the preset route. The above scheme realizes intelligent processing during the movement of the vehicle and improves the user experience.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle information processing method and related devices, vehicles, and storage media. Background Art

[0002] At present, the only function of the guided tour buses in various scenic spots is to carry people, and they cannot realize other functions besides carrying people, such as explaining the information of each scenic spot. As a result, when tourists are on the bus, they can only learn about the relevant information of the scenic spot through online search or manual explanation by the tour guide. Whether through online search or manual explanation by the tour guide, it is very inconvenient. Summary of the invention

[0003] The present application at least provides a vehicle information processing method and related devices, vehicles, and storage media.

[0004] The present application provides a vehicle information processing method, including: obtaining the current position of the vehicle; determining the trajectory point corresponding to the current position on a preset route; and using the vehicle's onboard components to execute preset processing that matches the position of the trajectory point on the preset route.

[0005] Therefore, by pre-setting the preset processing corresponding to each position on the route, the corresponding preset processing can be executed later according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0006] The preset route is composed of multiple line segments; determining the trajectory point corresponding to the current position on the preset route includes: determining a route in the preset route that matches the current position as a target line segment; and determining the trajectory point using the positional relationship between the current position and the target line segment.

[0007] Therefore, by first determining a target line segment matching the current position, and then determining the trajectory point according to the positional relationship between the current position and the target line segment, the trajectory point can be determined more accurately.

[0008] Among them, determining a route in a preset route that matches the current position includes: searching for a line segment between a first vector and a second vector that satisfies a first condition from multiple line segments, wherein the first vector is composed of a starting point of the line segment to the current position, and the second vector is composed of a starting point to an end point of the line segment; or searching for a line segment from multiple line segments whose distance from the current position satisfies a second condition.

[0009] Therefore, because vectors not only have size but also direction, the matching route is determined by the relationship between the vectors, making the determined route more accurate. In addition, the matching route is determined based on the distance from the current location, with fewer reference factors, and the entire process takes less time.

[0010] The step of searching for a line segment between the first vector and the second vector that satisfies the first condition from the multiple line segments is performed in response to the length of the first vector being less than or equal to a preset length.

[0011] Therefore, by first determining the size of the first vector before determining the line segment matching the current position, the accuracy of subsequent matching results can be improved.

[0012] The first condition is that the length of the first vector is smaller than the length of the second vector, and the angle between the first vector and the second vector is smaller than a preset angle.

[0013] Therefore, by comprehensively determining the matching line segment based on the length of the vector and the angle between the two, the accuracy of the matching result can be improved.

[0014] The second condition is that the distance between the multiple line segments and the current position is the shortest.

[0015] Therefore, by selecting the line segment closest to the current position as the line segment to be matched, the accuracy of matching can be improved.

[0016] The trajectory point is determined by utilizing the positional relationship between the current position and the target line segment, including: taking the intersection of the perpendicular line from the current position to the target line segment and the target line segment as the trajectory point.

[0017] Therefore, since the shortest distance from a point to a line is a perpendicular line segment, it is more accurate to use the intersection of the perpendicular line from the current position to the target line segment and the target line segment as the trajectory point.

[0018] Among them, before determining the trajectory point corresponding to the current position on the preset route, the method also includes: obtaining multiple frames of images obtained by shooting the environment where the planned route is located, and positioning information corresponding to the multiple frames of images; based on the multiple frames of images and the positioning information, obtaining the position information of several turning points on the planned route, and using the position information of the turning points, connecting the several turning points to form a preset route; or, transmitting the multiple frames of images and the positioning information to a local device, and obtaining the preset route from the local device.

[0019] Therefore, by first determining the turning points and then connecting the turning points to obtain a route consisting of several line segments, the former is faster and the established route is simpler than the route established by other methods. In addition, by transmitting multiple frames of images and positioning information to the local device and then receiving the preset route from the local device, the resource consumption of the vehicle can be reduced.

[0020] The multiple frames of images are captured by a shooting component of the vehicle or a shooting device that is communicatively connected to the vehicle.

[0021] Therefore, multiple methods can be used to capture images of the environment, which increases the flexibility of device selection during image acquisition.

[0022] Wherein, based on multiple frames of images and positioning information, the position information of several turning points on the planned route is obtained, including: analyzing multiple frames of images to determine images related to the turning points, or displaying multiple frames of images and determining images related to the turning points in response to user selection; based on the positioning information of the images related to the turning points, the position information of the turning points is obtained.

[0023] Therefore, the position information of each inflection point can be obtained by analyzing multiple frames of images, or the inflection point information can be determined by receiving the user's selection, thereby reducing the resource consumption of the execution device.

[0024] Among them, the vehicle's on-board component is used to execute preset processing that matches the position of the trajectory point on the preset route, including: detecting whether the trajectory point is located within a preset interval of a preset station in the preset route, wherein the preset interval includes at least one of an entry interval and an exit interval; in response to the trajectory point being located within the preset interval of the preset station, the vehicle-mounted component is used to execute preset processing that matches the preset interval of the preset station.

[0025] Therefore, by determining that the trajectory point is in a preset interval, the corresponding preset processing can be executed, and the whole process is convenient and fast.

[0026] Among them, the preset processing includes augmented reality processing of preset virtual information of environmental objects that match the position situation, and the preset virtual information comes from target map data; before using the vehicle-mounted components of the vehicle to execute the preset processing that matches the position situation of the trajectory point on the preset route, the method also includes: obtaining initial map data, wherein the initial map data is constructed based on multiple frames of images obtained by pre-shooting the environment of the planned route and the positioning information corresponding to the multiple frames of images; determining the setting information of at least one environmental object in the initial map data, wherein the setting information includes preset virtual information; associating the setting information of at least one environmental object with the initial map data to obtain target map data.

[0027] Therefore, by associating the setting information of the environmental objects with the initial map, the corresponding virtual information can be obtained according to the position of the vehicle later.

[0028] The present application provides a vehicle information processing device, including: a first positioning module, used to obtain the current position of the vehicle; a second positioning module, used to determine the trajectory point corresponding to the current position on the preset route; and a processing module, used to use the on-board components of the vehicle to execute preset processing that matches the position of the trajectory point on the preset route.

[0029] The present application provides a vehicle-mounted device applied to a vehicle, comprising a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned vehicle information processing method.

[0030] It also includes a vehicle-mounted component for performing augmented reality processing on preset virtual information.

[0031] The present application provides a vehicle, comprising: a vehicle subject and a vehicle-mounted device arranged on the vehicle subject, wherein the vehicle-mounted device is the above-mentioned vehicle-mounted device.

[0032] The present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned vehicle information processing method is implemented.

[0033] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0036] Figure 1 It is a flow chart of an embodiment of the vehicle information processing method of the present application;

[0037] Figure 2 is a schematic diagram showing a preset route according to an embodiment of the vehicle information processing method of the present application;

[0038] Figure 3 is a flow chart showing step S12 of an embodiment of the vehicle information processing method of the present application;

[0039] Figure 4 It is a structural schematic diagram of an embodiment of an information processing device for a vehicle of the present application;

[0040] Figure 5 It is a structural schematic diagram of an embodiment of the vehicle-mounted device of the present application applied to a vehicle;

[0041] Figure 6 It is a structural schematic diagram of an embodiment of a vehicle of the present application;

[0042] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0043] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0044] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0045] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0046] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the vehicle information processing method of the present application. Specifically, it may include the following steps:

[0047] Step S11: Determine the current position of the vehicle.

[0048] In the disclosed embodiments, there are multiple ways to determine the current position of the vehicle, for example, directly obtaining it based on a global positioning (GPS) device, or obtaining it through visual inertial positioning. Of course, if the camera of the vehicle is a depth camera or a three-dimensional (3D) camera, the current position of the vehicle can be directly determined based on visual positioning. There are multiple ways to obtain the current position of the vehicle, and the above-mentioned methods are only examples. In other embodiments, any other method that can achieve positioning can be used.

[0049] Step S12: Determine the trajectory point corresponding to the current position on the preset route.

[0050] Among them, the preset route is a pre-set vehicle route. Of course, the vehicle may not travel completely along the preset route. That is, in some application scenarios, the current position may not be on the preset route. If the preset route may be a line segment, and the actual road often has a certain width, there is a certain distance between the current position of the vehicle and the preset route. At this time, it is necessary to determine the trajectory point of the current position on the preset route. Of course, the ideal situation is that the current position overlaps with the preset route, then the current position can be directly used as the corresponding trajectory point.

[0051] Step S13: Utilizing the vehicle-mounted components, a preset process is executed to match the position of the trajectory point on the preset route.

[0052] The vehicle-mounted components of the vehicle include, but are not limited to, a display, an audio player, etc. The display may be a real display channel, or a projection device, etc.

[0053] Among them, the preset processing corresponding to the position of the trajectory point on the preset route can be preset. In some application scenarios, if the vehicle is a tour guide car in a scenic spot, the preset processing matching the position on the preset route can be a voice explanation of the scenic spot or special effects of environmental objects. For example, a display can be used to display the corresponding special effects, and / or an audio player can be used to play the corresponding audio.

[0054] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0055] Among them, before executing the preset processing that matches the position of the trajectory point on the preset route, the following steps can also be performed: obtaining initial map data. The initial map data is constructed based on multiple frames of images obtained by pre-shooting the environment of the planned route and the positioning information corresponding to the multiple frames of images. Among them, the multiple frames of images can be taken by the shooting component of the vehicle, or by a shooting device connected to the vehicle in communication. Multiple frames are at least two frames or more. The more images are taken, the more accurate the constructed initial map data is, but relatively speaking, it takes more time to build the initial map.

[0056] The initial map data can be obtained by constructing based on multiple frames of images, or multiple frames of images can be sent to a local device and the initial map data fed back by the local device can be received. The initial map data is three-dimensional map data. By constructing three-dimensional map data, the environment is also three-dimensional, so that three-dimensional preset virtual information about the environment can be constructed. The three-dimensional map data can be constructed by a sparse three-dimensional point cloud or a dense three-dimensional point cloud. Specifically, the initial map data can be constructed based on multiple frames of images by extracting and matching feature points of adjacent images, and triangulating each matching feature point based on the corresponding posture of each frame image to obtain the three-dimensional point corresponding to each matching feature point, and obtaining a three-dimensional point cloud map. Of course, the three-dimensional point cloud map can be further constructed as a structured plane. The so-called structured plane construction can specifically be to form a plane by forming three-dimensional points belonging to a plane. The posture of each frame image can be obtained by using a high-precision GPS device, wherein the high-precision GPS device can obtain the position and orientation of the second image. Each frame image and the corresponding posture can be synchronized by a timestamp. Each frame image and the corresponding posture are used as offline data, and the offline data can be transmitted to other devices to construct initial map data, and to determine the debugging of corresponding preset routes, triggering of preset processing and other conditions. Then, the final target map data, preset routes, preset processing and other related contents can be sent to the vehicle for final debugging on the vehicle. The initial map data is constructed by acquiring a second image captured by the vehicle or a shooting device connected to the vehicle for communication, so that the constructed map is more realistic. In addition, if the initial map is constructed using a local device, the resource consumption of the device executing the vehicle information processing method can be reduced.

[0057] Setting information of at least one environmental object in the initial map data is determined, wherein the setting information of the environmental object includes preset virtual information.

[0058] In the disclosed embodiment, the preset virtual information is a virtual three-dimensional model for augmented reality (AR) display. Specifically, the virtual three-dimensional model can be edited using development engine software. The development engine software can be Unity or UnrealEngine, etc. Specifically, in response to the user's second preset operation, the editing mode of the initial map data is entered. In the editing mode, the setting information of at least one environmental object in the initial map data is determined based on the user's input information. The user's second preset operation can be a startup instruction for the development engine software program, or a startup instruction for the editing mode, etc. The editing mode of the initial map data can be performed in real time according to the user's second preset operation, so that the user can adjust the setting information corresponding to each environmental object at any time. By setting the editing mode, the user can set the setting information of the environmental object in the editing mode, which can improve the convenience of the user during use.

[0059] The setting information of at least one environmental object is associated with the initial map data to obtain the target map data. By associating the setting information of the environmental object with the initial map, corresponding virtual information can be obtained according to the position of the vehicle later.

[0060] The method of associating the preset virtual information in the setting information of the environmental object with the initial map data may be to overlay the setting information of the environmental object on the corresponding environmental object in the three-dimensional map scene. For example, if the setting information of the environmental object is the appearance of a building, the method of associating the information with the initial map data may be to overlay the appearance of the building on the corresponding building in the initial map data, thereby changing the appearance of the building. If only the appearance of one side of the building is to be changed and the original appearance of the other sides is to be retained, the setting information may be determined only for the side to be changed, or transparent setting information may be set for the original appearance of the other sides to be retained. If overlaying is performed, the original appearance may also be retained.

[0061] Among them, the setting information of the environmental object also includes interactive virtual information and the first preset operation corresponding to the interactive virtual information. Among them, the interactive virtual information is used to replace the preset virtual information as the preset virtual information in response to the user's first preset operation. The interactive virtual information can also be used to perform an enhanced display operation together with the preset virtual information in response to the user's first preset operation. For example, the interactive virtual information and the preset virtual information are not the same type of information. For example, the preset virtual information can be the appearance of the environmental object, and the interactive virtual information can be a plate, a character model, audio information, etc. that explains the relevant information of the environmental object. In some application scenarios, the interactive virtual information can be hidden and rendered on the environmental object, and it is not displayed or played when the user's first preset operation is not received, and it is displayed or played when the user's first preset operation is received. By also setting the interactive virtual information and the first preset operation corresponding to the interactive virtual information in the setting information, so that the interaction can be realized according to the user's first preset operation later, the fun of augmented reality processing is improved.

[0062] Before determining the track point corresponding to the current position on the preset route, the following steps may be performed:

[0063] Acquire multiple frames of images captured of the environment where the planned route is located, and the positioning information corresponding to the multiple frames of images. The method for acquiring the positioning information corresponding to the multiple frames of images is as described above, and will not be repeated here.

[0064] Based on the multi-frame images and positioning information, the location information of several turning points on the planned route is obtained. And using the location information of the turning points, several turning points are connected to form a preset route. By first determining the turning points and then connecting the turning points, a route consisting of several line segments is obtained. Compared with the routes established by other methods, the former is faster and the established route is simpler. Alternatively, the multi-frame images and positioning information are transmitted to the local device, and the preset route from the local device is obtained. Among them, the local device can be any device other than the vehicle that obtains the multi-frame images. That is, the multi-frame images and the corresponding positioning information can generate offline data. By generating offline data, the local device can be a guided experience on the simulated route, which is convenient for development and testing, and there is no need to debug and develop on the vehicle all the time. By transmitting the multi-frame images and positioning information to the local device, and then receiving the preset route from the local device, the resource consumption of the vehicle can be reduced.

[0065] Among them, the multi-frame images are taken by the shooting component of the vehicle or the shooting device connected to the vehicle in communication. The shooting device connected to the vehicle in communication can be a drone, or a mobile phone, etc. Among them, the multi-frame images used to determine the preset route and the multi-frame images used to construct the initial map data can be the same or different. However, the images used by both are taken in the same environment. Therefore, even if the images used by the two are different, the preset route can be associated with the initial map data, that is, the location of the track points on the preset route in the initial map data is known. Therefore, multiple methods can be used to capture images of the environment, which improves the flexibility of equipment selection during image acquisition.

[0066] Among them, based on multiple frames of images and positioning information, a specific method for obtaining the position information of several turning points on the planned route can be: analyzing multiple frames of images to determine images related to the turning points. For example, performing parallax analysis on adjacent frame images, when the parallax meets the preset conditions, it is considered that the position of the previously captured image is the turning point. Specifically, the determination is made in combination with the first parallax of adjacent frame images, the travel speed and the rotation speed of the device that captured the image. For example, the next image frame is predicted based on the previously captured image frame through the travel speed and rotation speed of the device, and the second parallax between the previously captured image frame and the preset next image frame is obtained. It is determined whether the first parallax is greater than the second parallax. If so, the position corresponding to the previously captured image frame is considered to be the turning point.

[0067] Alternatively, multiple frames of images are displayed, and images related to the inflection point are determined in response to the user's selection. For example, the user selects the image corresponding to the inflection point from the multiple frames of images based on experience, and the positioning information corresponding to the image selected by the user is recorded.

[0068] Then, based on the positioning information of the image related to the inflection point, the position information of the inflection point is obtained. The position information of the inflection point can be the position information of the inflection point in the target map. Of course, the positioning information of the image related to the inflection point can be directly used as the position information of the inflection point. The position information of each inflection point can be obtained by analyzing multiple frames of images, or the inflection point information can be determined by receiving the user's selection, which can reduce the resource consumption of the execution device.

[0069] And, using the location information of the inflection points, several inflection points are connected to form a preset route. In some application scenarios, the above offline data is loaded, each frame of the image is rendered, and the corresponding posture is updated to the virtual camera. Among them, the virtual camera here refers to the camera component on the virtual vehicle that moves according to the posture in the target map. The virtual vehicle obtains the corresponding scene according to the posture in order to perform inflection point analysis.

[0070] Among them, each turning point is recorded in the preset route configuration for subsequent use.

[0071] After obtaining the preset route, the station location is further analyzed. The station location refers to the location corresponding to the more important environmental objects in the environment. Because of the area occupied by the environmental objects, the station location is generally an area. Further, the station entry interval and the exit interval are determined for the station location. Among them, in the embodiment of the present disclosure, the number of station entry intervals and exit intervals is the same, and each station entry interval and exit interval are alternately set. For a station, it includes at least one station entry interval and at least one exit interval. In response to the trajectory point being located within the preset interval of the preset station, the vehicle-mounted component is used to perform a preset process that matches the preset interval of the preset station. Among them, the length of the interval between the station entry interval and the exit interval can be determined according to the specific situation, for example, according to the time taken by the corresponding preset process. In some application scenarios, the distance between the station entry interval and the exit interval at least guarantees that the preset process can execute a preset ratio, which can be 80%. Of course, this is just an example. In other embodiments, other ratios can be used. The size of a single station entry interval and a single exit interval can also be determined according to the specific situation.

[0072] The entry section is used to start the corresponding operation, while the exit section is used to close the corresponding operation. For example, the entry section is used to start the audio explanation of the scenic spot, while the exit section is used to close the audio explanation of the scenic spot.

[0073] Each entry section and exit section and their corresponding preset processing are recorded in the preset route configuration, so that after obtaining the trajectory points of the vehicle on the preset route, the preset route configuration is searched, the corresponding preset processing is determined, and then the preset processing that matches the position of the trajectory points on the preset route is executed.

[0074] The preset route consists of multiple line segments. The multiple segments can be two or more. Figure 2 , Figure 2 FIG. 1 is a schematic diagram showing a preset route according to an embodiment of the vehicle information processing method of the present application. Figure 2 As shown, the preset route is composed of line segments connected by four inflection points, namely N0, N1, N2 and N3. The line segments include a line segment composed of N0 and N1, a line segment composed of N1 and N2, a line segment composed of N2 and N3, and a line segment composed of N3 and N0.

[0075] The entry section is composed of line segment AB, and the exit section is composed of line segment CD. Figure 2 This is just an example. In a specific application scenario, it may include multiple turning points as well as entry and exit sections.

[0076] Please also see Figure 3 , Figure 3FIG. 1 is a flow chart showing step S12 of an embodiment of the vehicle information processing method of the present application. Figure 3 As shown, the above step S12 includes the following sub-steps:

[0077] Step S121: Determine a line segment in the preset route that matches the current position as a target line segment.

[0078] Specifically, according to the current position, a line segment matching the current position is found from multiple line segments as the target line segment. There are multiple ways to determine the line segment matching the current position from multiple line segments:

[0079] 1. Find the line segment between the first vector and the second vector that meets the first condition from the multiple line segments. The first vector is composed of the starting point of the line segment to the current position. That is, the first vector is from the starting point of the line segment to the end point of the current position. The second vector is composed of the starting point to the end point of the line segment. Specifically, the second vector is from the starting point of the line segment to the end point of the same line segment. The starting point and the end point of the line segment can be set in advance. For example, the preset route is set in advance to set the starting point and the end point for each line segment in a clockwise order. In other disclosed embodiments, the starting point and the end point of each line segment can be determined according to the driving direction of the vehicle. Specifically, the angle between the line from the starting point to the end point of the line segment and the driving direction of the vehicle can be less than 90°. For example, if the line segment is a horizontal line and the driving direction of the vehicle is from left to right, the end point on the left side of the line segment is the starting point of the line segment, and the end point on the right side of the line segment is the end point of the line segment. Of course, in other embodiments, the angle between the line from the starting point to the end point of the line segment and the driving direction of the vehicle is greater than or equal to 90°. The first condition may be that the length of the first vector is less than the length of the second vector, and the angle between the first vector and the second vector is less than a preset angle. Optionally, the starting points of the first vector and the second vector are the same. Figure 2 As shown, point P is the current position, and the first vector can be The second vector can be Because vectors not only have size but also direction, the matching route is determined by the relationship between vectors, so that the determined route is more accurate. In some disclosed embodiments, the step of finding a line segment between a first vector and a second vector that satisfies a first condition from multiple line segments is performed in response to the length of the first vector being less than or equal to a preset length. For example, if there are a total of 5 first vectors, it is determined whether the lengths of these 5 vectors are less than or equal to the preset length. If only the lengths of two of the first vectors are less than or equal to the preset length, the second vectors corresponding to the first vectors are retained, and it is determined whether the first condition is satisfied between the first vector and the second vector. Of course, if the lengths of all first vectors are greater than the preset length, the subsequent steps are no longer performed. By first determining the size of the first vector before determining the line segment that matches the current position, the accuracy of the subsequent matching results can be improved.

[0080] 2. Find the line segment whose distance from the current position meets the second condition from the multiple line segments. The second condition may be the line segment that is closest to the current position among the multiple line segments. The way to obtain the line segment closest to the current distance may be to draw a circle at the current position to find the tangent, and use the line segment corresponding to the minimum radius as the target line segment, or to draw perpendicular lines from the current position to each line segment, and use the line segment corresponding to the shortest perpendicular line as the target line segment. Determine the matching route based on the distance from the current position, with fewer reference factors, and the less time the whole process takes. In addition, by selecting the line segment closest to the current position as the matching line segment, the matching accuracy can be improved.

[0081] Step S122: Determine the trajectory point using the positional relationship between the current position and the target line segment.

[0082] Specifically, the intersection of the perpendicular line from the current position to the target line segment and the target line segment is used as the trajectory point. Of course, the first vector can also be projected onto the target line segment, and the projection point of the current position is the trajectory point. Assuming that the target line segment is line segment N0N1, Figure 2 As shown, the trajectory point of the current position P on the target line segment is point P1. By first determining the target line segment matching the current position, and then determining the trajectory point according to the positional relationship between the current position and the target line segment, the trajectory point can be determined more accurately.

[0083] Also, because the shortest distance from a point to a line is a perpendicular line segment, it is more accurate to use the intersection of the perpendicular line from the current position to the target line segment and the target line segment as the trajectory point.

[0084] In some disclosed embodiments, the method of executing preset processing that matches the position of the trajectory point on the preset route using the vehicle-mounted component of the vehicle can be: detecting whether the trajectory point is located within a preset interval of a preset station in the preset route. Among them, the preset interval includes at least one of an entry interval and an exit interval. And, in response to the trajectory point being located within the preset interval of the preset station, the vehicle-mounted component is used to execute the preset processing that matches the preset interval. As mentioned above, the preset processing that matches the preset interval can be stored in the configuration of the preset route, and the trajectory point is matched with the configuration of the preset route to determine whether it is in the preset interval, and the corresponding preset processing is executed. Continuing with the above example, after judgment, the trajectory point P1 is exactly located in the entry interval AB, and the preset processing that matches the entry interval AB is executed. By judging that the trajectory point is in the preset interval, the corresponding preset processing can be executed, and the whole process is convenient and quick.

[0085] In some disclosed embodiments, the preset processing may be to perform augmented reality processing on preset virtual information of environmental objects that match the location situation. The preset virtual information comes from the target map data. The environmental objects may be objects with relatively fixed positions in the target map data. For example, buildings, trees, etc. The preset virtual information of the environmental objects may be any model constructed in advance for the environmental objects, such as a three-dimensional visualization model, or a sound effect model, etc.

[0086] For the preset virtual information of the same environmental object, the number of models constructed is not limited, and can be 1, or 2 or more. In addition, the target map data can include preset virtual information of multiple environmental objects. For example, preset virtual information can be set for all environmental objects, or preset virtual information can be set for some environmental objects in the target map data.

[0087] The method of using the vehicle's onboard components to perform augmented reality processing on the preset virtual information may be to combine the preset virtual information with the image actually captured by the vehicle to obtain an augmented reality processed image and display it on a display, or to use an audio player to play the corresponding audio. For example, the preset virtual information is combined with the real environment captured by the vehicle, and the combination of the preset virtual information and the real environment is displayed on the vehicle's display.

[0088] In some disclosed embodiments, the preset virtual information may be display information and / or audio information. The display information includes at least one of display effects, prompt information, and a three-dimensional model located outside the building. The display effects may be fireworks, virtual character models, particle effects, and the like. The prompt information may be at least one of text and images. For example, the prompt information may be in the form of a graphic pop-up window. The text-based prompt information may be used to introduce relevant information or safety protection knowledge of environmental objects, and the like. The image-based prompt information may be similar to traffic signs (stop, turn left, turn right, slow down, etc.), safety protection signs (no smoking, be careful of slippery ground, etc.), and other images used to convey information. The three-dimensional model located outside the building may be a three-dimensional model of the building's exterior. The audio-based information may be an explanation of the history of environmental objects, or an introduction to safety protection, and the like. For example, when a vehicle passes a location that may pose a certain threat to a person's body or property, the corresponding safety protection audio may be played to ensure the safety of passengers during the vehicle's driving. By setting multiple categories of virtual information, the fun of the subsequent augmented reality processing process can be improved.

[0089] Among them, the method of using the vehicle's on-board component to perform augmented reality processing on the preset virtual information can be, in response to the preset virtual information including display information, rendering the display information at a preset position of the environment in the first image. Among them, the preset position of the environment can be a preset position on the body of the environment, or a preset position relatively independent of the body of the environment. The former, for example, renders the display information on the outer wall of the building, and the latter, for example, renders the display information on the ground next to the building. The present embodiment takes the preset position on the body of the environment as an example. And, the vehicle's on-board component can be used to display the rendered first image. In the present embodiment, the first image can be taken by the vehicle's shooting component, or it can be taken by a shooting device connected to the vehicle in communication. The shooting device connected to the vehicle in communication can specifically be a drone, a mobile phone, a computer, a smart camera, etc. By obtaining the display information rendered at the preset position corresponding to the environment in the first image, the display effect in the first image can be improved.

[0090] Among them, the method of using the vehicle's on-board component to perform augmented reality processing on the preset virtual information can also be that in response to the preset virtual information including audio information, the audio information is played using the vehicle's on-board component. The on-board component that plays the audio information can be a car stereo. By playing the corresponding audio information according to the vehicle's positioning information, the fun of the vehicle's driving process can be improved. Among them, when the preset virtual information of the environmental object includes both display information and audio information, then according to the current positioning information of the vehicle, the vehicle's on-board component can be used to display the rendered first image, and the vehicle's on-board component can be used to play the audio information.

[0091] After the preset virtual information is processed by augmented reality using the vehicle's onboard components, the following steps may be performed:

[0092] In response to the user's first preset operation, interactive virtual information matching the preset operation is obtained from the preset environment. The preset virtual information is replaced by the interactive virtual information to update the preset virtual information of the environmental object. Among them, the logical response relationship between the first preset operation and the interactive virtual information can be preset. Optionally, the first preset operation can be performed on the preset virtual information or environmental object displayed on the display interface of the vehicle-mounted component. For example, the first preset operation instruction can be a single click, double click, drag, slide, etc. instruction on the environmental object or the displayed preset virtual information. Further, different interactive virtual information can be set according to different sliding directions of the user. Of course, the first preset operation can also be a voice operation. For example, when the preset voice information is recognized, it is considered that the user has made the first preset operation. For example, when the preset virtual information includes display information, the displayed information can be clicked to use other preset virtual information as new preset virtual information. By operating the preset virtual information or environmental object, or operating according to voice, the entire interactive process is convenient and fast. Of course, if the environmental object has only one preset virtual information, the preset operation used will not cause the preset virtual information to be replaced. For example, passengers can see images of real buildings processed with real-time special effects on the display screen, and interact with the preset virtual information superimposed on the building through the display screen, such as changing the appearance of the building, adding or switching real-time visual effects to the building, and so on.

[0093] After the preset virtual information is replaced by the interactive virtual information, the augmented reality processing of the preset virtual information using the vehicle's onboard components is re-executed. Specifically, the first image currently captured by the vehicle is augmented with the changed preset virtual information. By responding to the user's first preset operation, replacing the preset virtual information of the environment, and re-executing the step of augmenting the preset virtual information using the vehicle's onboard components, the interaction between the user and the device can be achieved, further improving the fun of the vehicle's driving process.

[0094] The vehicle information processing method provided in the embodiments of the present disclosure can be applied to a tour guide vehicle. The tour guide vehicle is a type of electric vehicle for regional use, and is a means of transportation for sightseeing and regional development in tourist attractions, parks, science and technology parks, large amusement parks, closed communities, campuses, resorts, urban pedestrian streets, and other areas. It can be an environmentally friendly electric passenger vehicle for transportation, and can also be used for connecting large enterprises and government industrial parks such as industrial parks and science and technology parks. The vehicle can be an AR vehicle, an autonomous driving vehicle, and the like.

[0095] The technical solution provided by the embodiments of the present disclosure can meet the needs of manned transfer and guided explanation in scenes such as scenic spots and industrial parks, and display industrial development, economic planning, science and technology, culture and other contents in a visual and intelligent way.

[0096] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0097] The execution subject of the vehicle information processing method may be an information processing device of the vehicle, for example, the vehicle information processing method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the vehicle information processing method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0098] See also Figure 4 , Figure 4 1 is a schematic diagram of the structure of an embodiment of the information processing device of the vehicle of the present application. The information processing device 100 of the vehicle includes a first positioning module 110, a second positioning module 120 and a processing module 130. The first positioning module 110 is used to obtain the current position of the vehicle; the second positioning module 120 is used to determine the trajectory point corresponding to the current position on the preset route; the processing module 130 is used to use the vehicle's on-board components to perform preset processing that matches the position of the trajectory point on the preset route.

[0099] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0100] In some disclosed embodiments, the preset route is composed of multiple line segments; the second positioning module 120 determines the trajectory point corresponding to the current position on the preset route, including: determining the line segment in the preset route that matches the current position as the target line segment; and determining the trajectory point using the positional relationship between the current position and the target line segment.

[0101] The above scheme first determines the target line segment matching the current position, and then determines the trajectory point according to the positional relationship between the current position and the target line segment, so that the trajectory point is determined more accurately.

[0102] In some disclosed embodiments, the second positioning module 120 determines a line segment in a preset route that matches the current position, including: searching for a line segment between a first vector and a second vector from multiple line segments that satisfies a first condition, wherein the first vector is composed of a distance from the starting point of the line segment to the current position, and the second vector is composed of a distance from the starting point to the end point of the line segment; or searching for a line segment from multiple line segments whose distance from the current position satisfies a second condition.

[0103] In the above scheme, since vectors not only have size but also direction, the matching route is determined by the relationship between the vectors, making the determined route more accurate. In addition, the matching route is determined based on the distance from the current position, with fewer reference factors, and the whole process takes less time.

[0104] In some disclosed embodiments, the step of searching for a line segment between the first vector and the second vector that satisfies a first condition from among the multiple line segments is performed in response to the length of the first vector being less than or equal to a preset length.

[0105] The above solution can improve the accuracy of subsequent matching results by first determining the size of the first vector before determining the line segment matching the current position.

[0106] In some disclosed embodiments, the first condition is that the length of the first vector is smaller than the length of the second vector, and the angle between the first vector and the second vector is smaller than a preset angle.

[0107] The above scheme can improve the accuracy of the matching result by comprehensively determining the matching line segment based on the length of the vector and the angle between the two.

[0108] In some disclosed embodiments, the second condition is that the distance between the multiple line segments and the current position is the shortest.

[0109] The above solution can improve the matching accuracy by selecting the line segment closest to the current position as the line segment to be matched.

[0110] In some disclosed embodiments, the positional relationship between the current position and the target line segment is used to determine the trajectory point, including: taking the intersection of the perpendicular line from the current position to the target line segment and the target line segment as the trajectory point.

[0111] Therefore, since the shortest distance from a point to a line is a perpendicular line segment, it is more accurate to use the intersection of the perpendicular line from the current position to the target line segment and the target line segment as the trajectory point.

[0112] In some disclosed embodiments, before determining the trajectory point corresponding to the current position on the preset route, the second positioning module 120 is also used to: obtain a plurality of frames of images captured of the environment in which the planned route is located, and positioning information corresponding to the plurality of frames of images; obtain the position information of several turning points on the planned route based on the plurality of frames of images and the positioning information, and connect the plurality of turning points to form a preset route using the position information of the turning points; or transmit the plurality of frames of images and the positioning information to a local device, and obtain the preset route from the local device.

[0113] The above scheme first determines the turning points and then connects the turning points to obtain a route composed of several line segments. Compared with the routes established by other methods, the former is faster and the established route is simpler.

[0114] In some disclosed embodiments, the second positioning module 120 obtains multiple frames of images by being captured by a shooting component of the vehicle or a shooting device that is communicatively connected to the vehicle; and / or obtains position information of several turning points on the planned route based on the multiple frames of images and the positioning information, including: analyzing the multiple frames of images to determine images related to the turning points, or displaying the multiple frames of images and determining images related to the turning points in response to user selection; and obtaining the position information of the turning points based on the positioning information of the images related to the turning points.

[0115] The above scheme can obtain the position information of each inflection point by analyzing multiple frames of images, or receive the user's selection to determine the inflection point information, which can reduce the resource consumption of the execution device.

[0116] In some disclosed embodiments, the processing module 130 utilizes the vehicle's on-board components to execute preset processing that matches the position of the trajectory point on a preset route, including: detecting whether the trajectory point is located within a preset interval of a preset station in the preset route, wherein the preset interval includes at least one of an entry interval and an exit interval; in response to the trajectory point being located within the preset interval of a preset station, utilizing the on-board components to execute preset processing that matches the preset interval of the preset station.

[0117] The above scheme can execute corresponding preset processing when it is determined that the trajectory point is in the preset interval, and the whole process is convenient and fast.

[0118] In some disclosed embodiments, the preset processing includes performing augmented reality processing on preset virtual information of environmental objects that matches the location conditions, wherein the preset virtual information comes from target map data; before using the vehicle's on-board components to execute the preset processing that matches the location conditions of the trajectory point on the preset route, the second positioning module 120 is also used to: obtain initial map data, wherein the initial map data is constructed based on multiple frames of images obtained by pre-shooting the environment of the planned route and the positioning information corresponding to the multiple frames of images; determine setting information of at least one environmental object in the initial map data, wherein the setting information includes preset virtual information; and associate the setting information of at least one environmental object with the initial map data to obtain target map data.

[0119] The above solution associates the setting information of the environmental objects with the initial map so that the corresponding virtual information can be obtained according to the position of the vehicle later.

[0120] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of an embodiment of the vehicle-mounted device of the present application applied to a vehicle. Figure 5 As shown, the vehicle-mounted device 10 includes a memory 200 and a processor 300, and the processor 300 is used to execute program instructions stored in the memory 200 to implement the steps in any of the above-mentioned vehicle information processing method embodiments.

[0121] Specifically, the processor 300 is used to control itself and the memory 200 to implement the steps in the above-mentioned information processing method embodiment of any vehicle. The processor 300 can also be called a CPU (Central Processing Unit). The processor 300 may be an integrated circuit chip with signal processing capabilities. The processor 300 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 300 can be implemented by an integrated circuit chip.

[0122] In some disclosed embodiments, the vehicle-mounted device 10 further includes a vehicle-mounted component 100 for performing augmented reality processing on preset virtual information. The vehicle-mounted component 100 may further include a display (not shown), an audio player (not shown), and other devices. The vehicle-mounted component 100 is connected to the processor 300, and the processor 300 may control the vehicle-mounted component 100 to perform corresponding operations. For example, the processor 300 controls the audio player in the vehicle-mounted component 100 to perform an operation of playing audio.

[0123] The display may be an interactive display. The display may respond to a first preset operation of the user, obtain interactive virtual information matching the preset operation from the target map data, and replace the non-preset virtual information with the interactive virtual information to update the preset virtual information of the environmental object, so that the vehicle-mounted device 10 re-executes the augmented reality processing of the preset virtual information using the vehicle-mounted component 100 of the vehicle.

[0124] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0125] See also Figure 6 , Figure 6 Schematic diagram of the structure of a vehicle of the present application. Figure 6 As shown, the vehicle 1 includes a vehicle body 20 and a vehicle-mounted device 10 disposed on the vehicle body 20. The vehicle-mounted device 10 here is described in the above-mentioned embodiment of the vehicle-mounted device 10, and will not be described again.

[0126] The vehicle-mounted device 10 includes a display screen (not shown). The display screen is arranged inside the vehicle 1. Specifically, the display screen is arranged on the side of the vehicle 1. The display surface of the display screen is parallel to the traveling direction of the vehicle 1. Optionally, except for the driver's seat, the remaining seats are arranged along the traveling direction of the vehicle 1. When a person sits on the seat, the body faces the display surface of the display screen.

[0127] The vehicle 1 provided in the embodiment of the present disclosure may be a tour guide vehicle. The tour guide vehicle is a type of electric vehicle for regional use, and is a means of transportation for sightseeing and use in the region developed for tourist attractions, parks, science and technology parks, large amusement parks, closed communities, campuses, resorts, urban pedestrian streets, etc. It may be an environmentally friendly electric passenger vehicle for transportation, and may also be used for connecting large enterprises and government industrial parks such as industrial parks and science and technology parks.

[0128] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0129] See also Figure 7 , Figure 7 The computer-readable storage medium 2 stores program instructions 21 that can be executed by a processor, and the program instructions 21 are used to implement the steps in any of the above-mentioned vehicle information processing method embodiments.

[0130] The above scheme, by pre-setting the preset processing corresponding to each position on the route, can subsequently execute the corresponding preset processing according to the current position of the vehicle, thereby realizing intelligent processing during the movement of the vehicle, thereby improving the experience of people in the vehicle and bringing certain convenience.

[0131] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0132] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0133] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0134] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A vehicle information processing method, characterized in that: include: Get the current location of the vehicle; Determine a trajectory point corresponding to the current position on a preset route; Utilizing the vehicle-mounted component of the vehicle, executing a preset process that matches the position of the trajectory point on the preset route; Before determining the track point corresponding to the current position on the preset route, the method further includes: Acquire multiple frames of images captured of the environment where the planned route is located, and positioning information corresponding to the multiple frames of images; Based on the multiple frames of images and the positioning information, the position information of several turning points on the planned route is obtained, and the position information of the turning points is used to connect the several turning points to form the preset route.

2. The method according to claim 1, characterized in that The preset route is composed of multiple line segments; and determining the trajectory point corresponding to the current position on the preset route includes: Determine the line segment in the preset route that matches the current position as a target line segment; The trajectory point is determined by using the positional relationship between the current position and the target line segment.

3. The method according to claim 2, characterized in that The determining the line segment in the preset route that matches the current position includes: Find the line segment between the first vector and the second vector that satisfies the first condition from the multiple line segments, wherein the first vector is composed of the starting point of the line segment to the current position, and the second vector is composed of the starting point to the end point of the line segment; or, The line segment whose distance from the current position satisfies the second condition is searched from the multiple line segments.

4. The method according to claim 3, characterized in that The step of searching for the line segment between the first vector and the second vector that satisfies the first condition from the plurality of line segments is performed in response to the length of the first vector being less than or equal to a preset length; And / or, the first condition is that the length of the first vector is smaller than the length of the second vector, and the angle between the first vector and the second vector is smaller than a preset angle; And / or, the second condition is that the distance between the multiple line segments and the current position is shortest.

5. The method according to claim 2, characterized in that: The determining the trajectory point by using the positional relationship between the current position and the target line segment includes: The intersection point of the perpendicular line from the current position to the target line segment and the target line segment is used as the trajectory point.

6. The method according to any one of claims 1 to 5, characterized in that: Before determining the track point corresponding to the current position on the preset route, the method further includes: The multiple frames of images and the positioning information are transmitted to a local device, and a preset route is obtained from the local device.

7. The method according to claim 6, characterized in that The multiple frames of images are captured by a capturing component of the vehicle or a capturing device that is in communication with the vehicle; And / or, obtaining the position information of several turning points on the planned route based on the multiple frames of images and the positioning information includes: Analyzing the multiple frames of images to determine the image associated with the inflection point, or displaying the multiple frames of images and determining the image associated with the inflection point in response to a user's selection; Based on the positioning information of the image related to the inflection point, the position information of the inflection point is obtained.

8. The method according to any one of claims 1 to 7, characterized in that: The using the vehicle-mounted component of the vehicle to perform a preset process matching the position of the track point on the preset route includes: Detecting whether the trajectory point is located within a preset interval of a preset station in the preset route, wherein the preset interval includes at least one of an entry interval and an exit interval; In response to the track point being located within a preset interval of the preset site, the vehicle-mounted component is used to perform a preset process that matches the preset interval of the preset site.

9. The method according to any one of claims 1 to 8, characterized in that: The preset processing includes performing augmented reality processing on preset virtual information of an environmental object matching the position situation, wherein the preset virtual information comes from target map data; Before using the vehicle-mounted component of the vehicle to perform a preset process that matches the position of the track point on the preset route, the method further includes: Acquiring initial map data, wherein the initial map data is constructed based on a plurality of frames of images taken in advance of the environment where the planned route is located and positioning information corresponding to the plurality of frames of images; Determining setting information of at least one environmental object in the initial map data, wherein the setting information includes the preset virtual information; The setting information of the at least one environmental object is associated with the initial map data to obtain target map data.

10. A vehicle information processing device, characterized in that: include: A first positioning module, used to obtain the current position of the vehicle; A second positioning module is used to determine the trajectory point corresponding to the current position on the preset route; A processing module, used to use the vehicle-mounted components of the vehicle to perform a preset process that matches the position of the trajectory point on the preset route; Before determining the trajectory point corresponding to the current position on the preset route, the second positioning module is also used to perform the following steps: obtaining multiple frames of images taken of the environment in which the planned route is located, and positioning information corresponding to the multiple frames of images; based on the multiple frames of images and the positioning information, obtaining the position information of several turning points on the planned route, and using the position information of the turning points, connecting the several turning points to form the preset route.

11. A vehicle-mounted device applied to a vehicle, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. The device according to claim 11, characterized in that It also includes an on-vehicle component for performing augmented reality processing on preset virtual information.

13. A vehicle, characterized in that: It includes a vehicle body and a vehicle-mounted device arranged on the vehicle body, wherein the vehicle-mounted device is the vehicle-mounted device according to claim 11 or 12.

14. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images