Gate passage state determination method and device, electronic equipment and readable storage medium

Abstract

The present disclosure provides a determination method and device for gate passing state, and relates to the technical field of artificial intelligence such as large model, deep learning, natural language processing, automatic driving and map navigation. The determination method for gate passing state comprises: determining a target local road network according to a target gate, and obtaining a gate entry track and a gate exit track corresponding to the target gate according to the target local road network; generating a gate entry track graph according to the target local road network and the gate entry track, and generating a gate exit track graph according to the target local road network and the gate exit track; using a multi-modal large model, generating a first answer according to the gate entry track graph, and generating a second answer according to the gate exit track graph; and obtaining the passing state of the target gate according to the first answer and the second answer. The present disclosure can enhance the robustness when judging the gate passing state, and improve the efficiency and accuracy of obtaining the gate passing state.

Description

Technical Field

[0001] This disclosure relates to the field of internet technology, and more particularly to the fields of artificial intelligence technologies such as large-scale models, deep learning, natural language processing, autonomous driving, and map navigation. It provides a method, apparatus, electronic device, and readable storage medium for determining the access status of a gate. Background Technology

[0002] In the entire navigation guidance chain, "destination guidance" is the final link in the user experience, and its presentation quality directly determines the user's satisfaction with the overall navigation service.

[0003] Gates (such as park entrances, community entrances, parking lot entrances, etc.) are key road segments at the end of navigation. They are characterized by diverse traffic directions, high scene complexity, and sensitivity to positioning errors. Their traffic capacity plays a decisive role in the accuracy of destination guidance.

[0004] Unlike the driving planning of general roads, the gate scene has the following unique characteristics: 1) The scene scale is small, the roads are dense but the shape is irregular; 2) The vehicle trajectory height is affected by non-standard behaviors such as parking, U-turns, and waiting on the side of the road; 3) User positioning errors of a few meters can significantly change the trajectory shape; 4) The attributes such as one-way / two-way / closed / internal passage change frequently and are highly dependent on timeliness.

[0005] Existing automation methods mainly determine the gate passage status by using trajectory crossing characteristics. After excluding scenarios with large trajectory binding errors, such as parallel roads, elevated roads, and short roads, the automation process uses a large amount of trajectory crossing data in the direction of entering / exiting the gate to determine the direction.

[0006] However, existing automated methods have the following problems when determining the passage status of gates: 1) The trajectory error is larger in the end scene: Due to factors such as positioning noise, low speed behavior, and slight changes in road structure, the trajectory shape in the end scene is extremely unstable. Even in simple scenarios, trajectory misbinding is easy to occur, leading to errors in the automated strategy; 2) Trajectory feature values ​​(such as directionality, passage volume, etc.) are indirect features and cannot intuitively express the actual passage of vehicles. They are greatly affected by noise and are prone to failure in scenarios such as gate excavation and change of closure (excavation and change of closure indicates that the discovered gate may no longer be passable).

[0007] Therefore, improving the accuracy of determining the access status of the gate is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] According to a first aspect of this disclosure, a method for determining the passage status of a gate is provided, comprising: determining a target local road network based on a target gate; obtaining a gate entry trajectory and a gate exit trajectory corresponding to the target gate based on the target local road network; generating a gate entry trajectory map based on the target local road network and the gate entry trajectory; generating a gate exit trajectory map based on the target local road network and the gate exit trajectory; using a multimodal large model, generating a first response based on the gate entry trajectory map and a second response based on the gate exit trajectory map; and obtaining the passage status of the target gate based on the first response and the second response.

[0009] According to a second aspect of this disclosure, an apparatus for determining the passage status of a gate is provided, comprising: an acquisition unit, configured to determine a target local road network based on a target gate, and acquire a gate entry trajectory and a gate exit trajectory corresponding to the target gate based on the target local road network; a first generation unit, configured to generate a gate entry trajectory map based on the target local road network and the gate entry trajectory, and generate a gate exit trajectory map based on the target local road network and the gate exit trajectory; a second generation unit, configured to use a multimodal large model to generate a first response based on the gate entry trajectory map and a second response based on the gate exit trajectory map; and a first determination unit, configured to obtain the passage status of the target gate based on the first response and the second response.

[0010] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0011] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0012] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0015] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0016] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0017] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0018] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0019] Figure 5 This is a block diagram of an electronic device used to implement the method for determining the gate passage status according to embodiments of the present disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0021] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the method for determining the gate access status in this embodiment specifically includes the following steps:

[0022] S101. Determine the target local road network based on the target gate, and obtain the gate entry trajectory and gate exit trajectory corresponding to the target gate based on the target local road network;

[0023] S102. Generate a gate entry trajectory map based on the target local road network and the gate entry trajectory; generate a gate exit trajectory map based on the target local road network and the gate exit trajectory.

[0024] S103. Using a multimodal large model, generate a first answer based on the gate entry trajectory map and a second answer based on the gate exit trajectory map;

[0025] S104. Based on the first answer and the second answer, the passage status of the target gate is obtained.

[0026] The gate passage status determination method in this embodiment generates gate entry trajectory maps and gate exit trajectory maps based on the target local road network and the gate entry and exit trajectories obtained from the target local road network. This visualizes the spatial relationship between the target gate's crossing trajectory and the map road network. Based on the corresponding trajectory maps, and leveraging the cross-modal understanding capabilities of the multimodal large model, a more accurate answer can be generated. Furthermore, the passage status of the target gate is obtained based on the first and second answers generated by the multimodal large model, which enhances the robustness in determining the gate passage status and improves the efficiency and accuracy of obtaining the gate passage status.

[0027] In this embodiment, when performing S101 to determine the target local road network based on the target gate, a geographical area of ​​a preset size can be defined with the location of the target gate as the center, and the local road network corresponding to the geographical area can be extracted as the target local road network.

[0028] In this embodiment, the geographical area of ​​a preset size centered on the gate location can be a circular area with a radius of a preset distance (e.g., 50m) centered on the gate location, or a square area with a side length of twice the preset distance centered on the gate location (i.e., the side length of the square area is 100m).

[0029] Specifically, in this embodiment, when executing S101 to obtain the gate entry trajectory and gate exit trajectory of the corresponding target gate based on the target local road network, the implementation method can be as follows: select candidate trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate; select the trajectory from the candidate trajectories whose driving direction is the direction of entering the target gate as the gate entry trajectory; select the trajectory from the candidate trajectories whose driving direction is the direction of leaving the target gate as the gate exit trajectory.

[0030] In this embodiment, the trajectory database includes multiple historical trajectories. The historical trajectory is an ordered sequence composed of timestamps and location coordinates, which is usually the driving trajectory of a vehicle. The driving trajectory usually has a driving direction. In this embodiment, the candidate trajectory selected by S101 simultaneously meets the location requirements (i.e., the location coordinates of the candidate trajectory are located within the geographical area corresponding to the target local road network) and the attribute requirements (i.e., the candidate trajectory needs to pass through the target gate).

[0031] In this embodiment, when executing S101 to select candidate trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate, the implementation method can be as follows: select trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate as initial trajectories; determine the number of trajectories passing through the target gate each day based on the initial trajectories and obtain the trajectory number fluctuation result; obtain the target time period corresponding to the trajectory number fluctuation result; select candidate trajectories based on the initial trajectories according to the target time period.

[0032] In this embodiment, the trajectory number fluctuation result includes either large trajectory number fluctuation or small trajectory number fluctuation. Different trajectory number fluctuation results correspond to different time periods. If the trajectory number fluctuation result is large, the target time period is shorter, such as 2 days, 3 days or 5 days from the current time. If the trajectory number fluctuation result is small, the target time period is longer, such as 10 days or 15 days from the current time.

[0033] In other words, this embodiment completes the selection of candidate trajectories through two selection processes, so that the selected candidate trajectories can meet the location and attribute requirements as well as the time requirements. Furthermore, the target time period for selecting candidate trajectories is obtained based on the trajectory number fluctuation results, which enhances the correlation between the selected candidate trajectories and the trajectory number fluctuation, improves the selection accuracy of candidate trajectories, and further improves the selection accuracy of entry and exit trajectories.

[0034] In this embodiment, when executing S101 to obtain the trajectory number fluctuation result based on the number of crossing trajectories of the target gate each day, if a sudden drop in the number of crossing trajectories occurs within a certain number of consecutive days (e.g., two consecutive days), for example, if the number of crossing trajectories is 10 on one day and 1 on another day, the large trajectory number fluctuation is taken as the trajectory number fluctuation result; if no sudden drop in the number of crossing trajectories occurs within a certain number of consecutive days, this embodiment takes the small trajectory number fluctuation as the trajectory number fluctuation result.

[0035] It is understood that after obtaining the initial trajectory in S101, this embodiment can also perform low-quality cleaning on the initial trajectory to remove trajectories of low quality (e.g., discontinuous trajectories) in the initial trajectory, thereby improving the trajectory quality of the selected candidate trajectories and further improving the trajectory quality of the selected entry and exit trajectories.

[0036] In this embodiment, after executing S101 to obtain the gate entry trajectory and gate exit trajectory of the corresponding target gate based on the target local road network, S102 is executed to generate a gate entry trajectory map based on the target local road network and the gate entry trajectory, and to generate a gate exit trajectory map based on the target local road network and the gate exit trajectory.

[0037] In other words, this embodiment visualizes the spatial relationship between the crossing trajectory of the target gate and the road network on the map by generating a trajectory map, making it possible to obtain the passage status of the target gate by leveraging the cross-modal understanding capability of the image using a multimodal large model.

[0038] In this embodiment, when generating the gate entry trajectory map in step S102, the gate entry trajectory can be superimposed on the road network base map corresponding to the target local road network, and the gate entry trajectory map is obtained based on the superposition result; the gate entry trajectory map in this embodiment includes the target local road network and the gate entry trajectory.

[0039] In this embodiment, when generating the gate exit trajectory map in step S102, the gate exit trajectory can be superimposed on the road network base map corresponding to the target local road network, and the gate exit trajectory map is obtained based on the superposition result; the gate exit trajectory map in this embodiment includes the target local road network and the gate exit trajectory.

[0040] In this embodiment, when generating the trajectory map in step S102, the location of the target gate can also be marked on the generated trajectory map using a preset method.

[0041] In this embodiment, after executing S102 to generate the gate entry trajectory map and the gate exit trajectory map, S103 is executed to use a multimodal large model to generate a first answer based on the gate entry trajectory map and a second answer based on the gate exit trajectory map.

[0042] In this embodiment, the multimodal large model is a large-scale deep learning model that can simultaneously understand, generate, or associate multiple modal information (such as text, images, audio, video, etc.). The multimodal large model in this embodiment can be a VLM (Vision Language Model). The VLM model has a high matching degree with the "image + question" input format, which can improve the robustness and automation capability of gate passage status judgment.

[0043] Specifically, in this embodiment, when executing S103 using a multimodal large model to generate a first answer based on the gate entry trajectory map, the implementation method can be as follows: obtain a first preset question, which is a question used to obtain a first answer (i.e., whether the target gate can be entered); input the first preset question and the gate entry trajectory map into the multimodal large model, and obtain the first answer based on the output result of the multimodal large model.

[0044] In this embodiment, when executing S103 using a multimodal large model to generate a second answer based on the gate exit trajectory map, the implementation method can be as follows: obtain a second preset question, which is a question used to obtain a second answer (i.e., the answer to whether the target gate can be exited); input the second preset question and the gate exit trajectory map into the multimodal large model, and obtain the second answer based on the output result of the multimodal large model.

[0045] In other words, this embodiment directly "concretizes" the spatial relationship between the trajectory and the gate road network, thereby constructing a new feature expression form that is adapted to multimodal large models. It eliminates the need to use abstract features such as passage volume, which are easily interfered with by end noise in existing technologies. This allows multimodal large models to utilize spatial structure understanding, motion trajectory and road network trend reasoning, noise trajectory filtering, and complex relationship scene inference capabilities to make judgments based on visual features such as trajectory density, directional trends, and road network structure in the trajectory map, thereby generating corresponding answers and improving the accuracy of the obtained answers.

[0046] In this embodiment, the first preset question and the second preset question can be the same question, such as "Please determine whether the gate location in the picture is passable? Please select from: Yes / No / Cannot determine."; the first preset question and the second preset question can also be different questions, for example, the first preset question can be "Please determine whether the gate location in the picture is passable? Please select from: Yes / No / Cannot determine.", and the second preset question can be "Please determine whether the gate location in the picture is passable? Please select from: Yes / No / Cannot determine."

[0047] The first or second answer obtained by executing S103 in this embodiment is one of yes, no, or cannot be determined. If the answer is "yes", it means that the target gate can be entered or exited. If the answer is "no", it means that the target gate cannot be entered or exited. If the answer is "cannot be determined", it means that the multimodal large model cannot determine whether the target gate can be entered or not based on the gate entry trajectory map, or cannot determine whether the target gate can be exited or not based on the gate exit trajectory map.

[0048] In this embodiment, after executing S103 to generate the first answer and the second answer, S104 is executed to obtain the passage status of the target gate based on the first answer and the second answer.

[0049] Specifically, in this embodiment, when executing S104 to obtain the passage status of the target gate based on the first answer and the second answer, the implementation method can be as follows: in response to determining that the target local road network belongs to a simple road network scenario and that the first answer and the second answer do not include the one that cannot be determined, the passage status of the target gate is obtained based on the first answer and the second answer.

[0050] In other words, this embodiment only obtains the passage status of the target gate based on the first and second answers generated by the multimodal large model when it is determined that the local road network near the target gate belongs to a simple road network scenario and the answer does not include the case where it cannot be determined. This avoids the problem of incorrectly obtaining the passage status when the road network near the target gate is more complex or the multimodal large model cannot generate a definite result, and further improves the accuracy of the obtained passage status.

[0051] In this embodiment, when executing S104, if it is determined that the target local road network belongs to a complex road network scenario or the answer includes "cannot be determined", the generated gate entry trajectory map and gate exit trajectory map are sent to the relevant personnel for manual verification of the gate passage status.

[0052] In this embodiment, whether the target local road network belongs to a simple road network scenario or a complex road network scenario can be determined based on the number of roads related to the target gate in the target local road network. If the number of roads related to the target gate exceeds a preset threshold, the target local road network is determined to belong to a complex road network scenario; otherwise, the target local road network is determined to belong to a simple road network scenario.

[0053] In this embodiment, when executing S104, if both the first and second answers are "yes", the passage status of the target gate is "the target gate is passable in both directions"; if both the first and second answers are "no", the passage status of the target gate is "the gate is closed and not passable"; if the first answer is "yes" and the second answer is "no", the passage status of the target gate is "the gate is passable in one direction when entering"; if the first answer is "no" and the second answer is "yes", the passage status of the target gate is "the gate is passable in one direction when exiting".

[0054] In other words, this embodiment obtains the access status of the target gate based on the preset accessibility judgment rules and the first and second answers generated by the multimodal large model, thereby achieving the purpose of automatically verifying the access status of the gate and effectively improving the accuracy of the obtained access status of the gate.

[0055] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown, after executing S101 "selecting candidate trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate", this embodiment may further include the following:

[0056] S201. Select the target vehicle trajectory from the candidate trajectories;

[0057] S202. Generate a target vehicle trajectory map based on the target local road network and the target vehicle trajectory;

[0058] S203. Using the multimodal large model, generate a third answer based on the target vehicle trajectory map;

[0059] S204. Based on the third answer, the open state of the target gate is obtained.

[0060] In other words, in addition to obtaining the passage status of the target gate based on the gate entry trajectory and gate exit trajectory selected from the candidate trajectories, this embodiment can further select the target vehicle trajectory, thereby using a multimodal large model to obtain a third answer based on the target vehicle trajectory map generated from the target vehicle trajectory, and then obtain the opening status of the target gate based on the third answer, thus achieving the purpose of obtaining both the passage status and the opening status of the target gate at the same time.

[0061] In this embodiment, the target vehicle is at least one of ride-hailing vehicles and taxis; the target vehicle trajectory is the gate entry trajectory and / or gate exit trajectory corresponding to the target vehicle in the candidate trajectories.

[0062] In this embodiment, when executing S202, the target vehicle trajectory can be superimposed on the road network base map corresponding to the target local road network, and the target vehicle trajectory map can be obtained based on the superposition result; the target vehicle trajectory map in this embodiment includes the target local road network and the target vehicle trajectory.

[0063] In this embodiment, when executing S203, the third preset question and the target vehicle trajectory map can be input into the multimodal large model. Based on the output of the multimodal large model, the third answer is obtained. The third preset question is a question used to obtain the third answer (i.e., the answer to whether the target gate is open to the outside).

[0064] In this embodiment, the third preset question can be "Please determine whether the gate in the picture is open to the outside? Please select from: Yes / No / Cannot determine."

[0065] In this embodiment, when executing S204, in response to determining that the target local road network belongs to a simple road network scenario and the third answer is not "cannot be determined", the open status of the target gate is obtained based on the third answer; if it is determined that the target local road network belongs to a complex road network scenario or the third answer is "cannot be determined", the generated target vehicle trajectory map is sent to the relevant personnel for manual verification of the gate's open status.

[0066] In this embodiment, the open state of the target gate is either "the gate is open to the outside" or "the gate is not open to the outside". If the third answer is "yes", the open state of the target gate is "the gate is open to the outside". If the third answer is "no", the open state of the target gate is "the gate is not open to the outside".

[0067] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure. Figure 3 The image shows a trajectory diagram generated in this embodiment; as shown below. Figure 3 As shown, the generated trajectory map includes the trajectory of crossing the target gate (one of the gate entry trajectory, gate exit trajectory, and target vehicle trajectory) and the target local road network corresponding to the target gate.

[0068] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, the gate passage status determination device 400 of this embodiment includes:

[0069] Acquisition unit 401 is used to determine the target local road network based on the target gate, and to acquire the gate entry trajectory and gate exit trajectory corresponding to the target gate based on the target local road network;

[0070] The first generation unit 402 is used to generate a gate entry trajectory map based on the target local road network and the gate entry trajectory, and to generate a gate exit trajectory map based on the target local road network and the gate exit trajectory;

[0071] The second generation unit 403 is used to generate a first answer based on the gate entry trajectory map and a second answer based on the gate exit trajectory map using a multimodal large model;

[0072] The first determining unit 404 is used to determine the passage status of the target gate based on the first answer and the second answer.

[0073] When determining the target local road network based on the target gate, the acquisition unit 401 can define a geographical area of ​​a preset size with the location of the target gate as the center, and extract the local road network corresponding to the geographical area as the target local road network.

[0074] In this embodiment, the geographical area of ​​a preset size centered on the gate location can be a circular area with a radius of a preset distance (e.g., 50m) centered on the gate location, or a square area with a side length of twice the preset distance centered on the gate location (i.e., the side length of the square area is 100m).

[0075] Specifically, when the acquisition unit 401 acquires the gate entry trajectory and gate exit trajectory of the corresponding target gate based on the target local road network, the following implementation method can be adopted: select candidate trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate; select the trajectory from the candidate trajectories whose driving direction is the direction of entering the target gate as the gate entry trajectory; select the trajectory from the candidate trajectories whose driving direction is the direction of leaving the target gate as the gate exit trajectory.

[0076] In this embodiment, the trajectory database includes multiple historical trajectories. The historical trajectory is an ordered sequence composed of timestamps and location coordinates, which is usually the driving trajectory of a vehicle. The driving trajectory usually has a driving direction. The candidate trajectory selected by the acquisition unit 401 simultaneously meets the location requirements (i.e., the location coordinates of the candidate trajectory are located within the geographical area corresponding to the target local road network) and the attribute requirements (i.e., the candidate trajectory needs to pass through the target gate).

[0077] Specifically, when the acquisition unit 401 selects candidate trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate, the following implementation method can be adopted: select trajectories from the trajectory database that match the geographical area corresponding to the target local road network and pass through the target gate as initial trajectories; determine the number of trajectories passing through the target gate each day based on the initial trajectories and obtain the trajectory number fluctuation result; obtain the target time period corresponding to the trajectory number fluctuation result; select the initial trajectories based on the target time period to obtain candidate trajectories.

[0078] In this embodiment, the trajectory number fluctuation result includes either large trajectory number fluctuation or small trajectory number fluctuation. Different trajectory number fluctuation results correspond to different time periods. If the trajectory number fluctuation result is large, the target time period is shorter, such as 2 days, 3 days or 5 days from the current time. If the trajectory number fluctuation result is small, the target time period is longer, such as 10 days or 15 days from the current time.

[0079] In other words, the acquisition unit 401 completes the selection of candidate trajectories through two selection processes, so that the selected candidate trajectories can meet the location requirements, attribute requirements, and time requirements. Furthermore, it obtains the target time period for selecting candidate trajectories based on the trajectory number fluctuation results, which enhances the correlation between the selected candidate trajectories and the trajectory number fluctuation, improves the selection accuracy of candidate trajectories, and further improves the selection accuracy of entry and exit trajectories.

[0080] When the acquisition unit 401 obtains the trajectory number fluctuation result based on the number of crossing trajectories of the target gate each day, it can take the case where the number of crossing trajectories drops sharply within a certain number of consecutive days (e.g., two consecutive days), for example, the number of crossing trajectories is 10 on one day and 1 on another day, and take the large trajectory number fluctuation as the trajectory number fluctuation result; if it is determined that there is no sharp drop in the number of crossing trajectories within a certain number of consecutive days, this embodiment takes the small trajectory number fluctuation as the trajectory number fluctuation result.

[0081] It is understandable that after acquiring the initial trajectory, the acquisition unit 401 can also perform low-quality cleaning on the initial trajectory to remove trajectories of lower quality (such as discontinuous trajectories), thereby improving the trajectory quality of the selected candidate trajectories and further improving the trajectory quality of the selected entry and exit trajectories.

[0082] In this embodiment, after the acquisition unit 401 acquires the gate entry trajectory and gate exit trajectory of the corresponding target gate based on the target local road network, the first generation unit 402 generates a gate entry trajectory map based on the target local road network and the gate entry trajectory, and generates a gate exit trajectory map based on the target local road network and the gate exit trajectory.

[0083] In other words, this embodiment visualizes the spatial relationship between the crossing trajectory of the target gate and the road network on the map by generating a trajectory map, making it possible to obtain the passage status of the target gate by leveraging the cross-modal understanding capability of the image using a multimodal large model.

[0084] When generating the gate entry trajectory map, the first generation unit 402 can overlay the gate entry trajectory onto the road network base map corresponding to the target local road network, and obtain the gate entry trajectory map based on the overlay result; the gate entry trajectory map in this embodiment includes the target local road network and the gate entry trajectory.

[0085] When generating the gate exit trajectory map, the first generation unit 402 can overlay the gate exit trajectory onto the road network base map corresponding to the target local road network, and obtain the gate exit trajectory map based on the overlay result; the gate exit trajectory map in this embodiment includes the target local road network and the gate exit trajectory.

[0086] When generating the trajectory map, the first generation unit 402 can also preset a method to mark the location of the target gate in the generated trajectory map.

[0087] In this embodiment, after the first generation unit 402 generates the gate entry trajectory map and the gate exit trajectory map, the second generation unit 403 uses a multimodal large model to generate a first answer based on the gate entry trajectory map and a second answer based on the gate exit trajectory map.

[0088] In this embodiment, the multimodal large model is a large-scale deep learning model that can simultaneously understand, generate, or associate multiple modal information (such as text, images, audio, video, etc.). The multimodal large model in this embodiment can be a VLM (Vision Language Model). The VLM model has a high matching degree with the "image + question" input format, which can improve the robustness and automation capability of gate passage status judgment.

[0089] Specifically, when the second generation unit 403 uses the multimodal large model to generate the first answer based on the gate entry trajectory map, the implementation method can be as follows: obtain a first preset question, which is a question used to obtain the first answer (i.e., the answer of whether the target gate can be entered); input the first preset question and the gate entry trajectory map into the multimodal large model, and obtain the first answer based on the output result of the multimodal large model.

[0090] When the second generation unit 403 uses the multimodal large model to generate the second answer based on the gate exit trajectory map, the implementation method can be as follows: obtain a second preset question, which is a question used to obtain the second answer (i.e., the answer to whether the target gate can be exited); input the second preset question and the gate exit trajectory map into the multimodal large model, and obtain the second answer based on the output result of the multimodal large model.

[0091] In other words, this embodiment directly "concretizes" the spatial relationship between the trajectory and the gate road network, thereby constructing a new feature expression form that is adapted to multimodal large models. It eliminates the need to use abstract features such as passage volume, which are easily interfered with by end noise in existing technologies. This allows multimodal large models to utilize spatial structure understanding, motion trajectory and road network trend reasoning, noise trajectory filtering, and complex relationship scene inference capabilities to make judgments based on visual features such as trajectory density, directional trends, and road network structure in the trajectory map, thereby generating corresponding answers and improving the accuracy of the obtained answers.

[0092] In this embodiment, the first preset question and the second preset question can be the same question or different questions.

[0093] The first or second answer obtained by the second generation unit 403 is one of yes, no or cannot be determined; if the answer is "yes", it means that the target gate can be entered or exited; if the answer is "no", it means that the target gate cannot be entered or exited; if the answer is "cannot be determined", it means that the multimodal large model cannot determine whether the target gate can be entered or not based on the gate entry trajectory map, or cannot determine whether the target gate can be exited or not based on the gate exit trajectory map.

[0094] In this embodiment, after the second generation unit 403 generates the first answer and the second answer, the first determination unit 404 determines the passage status of the target gate based on the first answer and the second answer.

[0095] Specifically, when the first determining unit 404 obtains the passage status of the target gate based on the first answer and the second answer, the implementation method can be as follows: in response to determining that the target local road network belongs to a simple road network scenario and that the first answer and the second answer do not include any that cannot be determined, the passage status of the target gate is obtained based on the first answer and the second answer.

[0096] In other words, the first determining unit 404 will only determine the passage status of the target gate based on the first and second answers generated by the multimodal large model if the target local road network near the target gate is a simple road network scenario and the answer does not include the case where it cannot be determined. This avoids the problem of incorrectly obtaining the passage status when the road network near the target gate is more complex or the multimodal large model cannot generate a definite result, and further improves the accuracy of the obtained passage status.

[0097] If the target local road network is determined to be a complex road network scenario or the answer includes "cannot be determined", the first determining unit 404 will send the generated gate entry trajectory map and gate exit trajectory map to the relevant personnel, who will then manually verify the gate passage status.

[0098] In this embodiment, whether the target local road network belongs to a simple road network scenario or a complex road network scenario can be determined based on the number of roads related to the target gate in the target local road network. If the number of roads related to the target gate exceeds a preset threshold, the target local road network is determined to belong to a complex road network scenario; otherwise, the target local road network is determined to belong to a simple road network scenario.

[0099] If both the first and second answers are "yes", the passage status of the target gate determined by the first determining unit 404 is "the target gate is passable in both directions"; if both the first and second answers are "no", the passage status of the target gate is "the gate is closed and not passable"; if the first answer is "yes" and the second answer is "no", the passage status of the target gate is "the gate is passable in one direction when entering"; if the first answer is "no" and the second answer is "yes", the passage status of the target gate is "the gate is passable in one direction when exiting".

[0100] In other words, the first determining unit 404 obtains the access status of the target gate based on the preset accessibility judgment rules and the first and second answers generated by the multimodal large model, thereby achieving the purpose of automatically verifying the access status of the gate and effectively improving the accuracy of the obtained access status of the gate.

[0101] The gate access status determination device 400 in this embodiment may further include a second determination unit 405, which performs the following: selecting a target vehicle trajectory from candidate trajectories; generating a target vehicle trajectory map based on the target local road network and the target vehicle trajectory; generating a third response based on the target vehicle trajectory map using a multimodal large model; and obtaining the open status of the target gate based on the third response.

[0102] In other words, in addition to obtaining the passage status of the target gate based on the gate entry trajectory and gate exit trajectory selected from the candidate trajectories, this embodiment can further select the target vehicle trajectory, thereby using a multimodal large model to obtain a third answer based on the target vehicle trajectory map generated from the target vehicle trajectory, and then obtain the opening status of the target gate based on the third answer, thus achieving the purpose of obtaining both the passage status and the opening status of the target gate at the same time.

[0103] In this embodiment, the target vehicle is at least one of ride-hailing vehicles and taxis; the target vehicle trajectory is the gate entry trajectory and / or gate exit trajectory corresponding to the target vehicle in the candidate trajectories.

[0104] The second determining unit 405 can overlay the target vehicle trajectory onto the road network base map corresponding to the target local road network, and obtain the target vehicle trajectory map based on the overlay result; in this embodiment, the target vehicle trajectory map includes the target local road network and the target vehicle trajectory.

[0105] The second determining unit 405 can input the third preset question and the target vehicle trajectory map into the multimodal large model, and obtain the third answer based on the output of the multimodal large model; wherein, the third preset question is the question used to obtain the third answer (i.e., the answer to whether the target gate is open to the outside).

[0106] In this embodiment, the third preset question can be "Please determine whether the gate in the picture is open to the outside? Please select from: Yes / No / Cannot determine."

[0107] In response to the determination that the target local road network belongs to a simple road network scenario and the third answer is not "cannot be determined", the second determining unit 405 obtains the opening status of the target gate based on the third answer; if the determination that the target local road network belongs to a complex road network scenario or the third answer is "cannot be determined", the second determining unit 405 sends the generated target vehicle trajectory map to the relevant personnel, who then manually verify the gate opening status.

[0108] In this embodiment, the open state of the target gate is either "the gate is open to the outside" or "the gate is not open to the outside". If the third answer is "yes", the open state of the target gate is "the gate is open to the outside". If the third answer is "no", the open state of the target gate is "the gate is not open to the outside".

[0109] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0110] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0111] like Figure 5 The diagram shown is a block diagram of an electronic device for determining a gate passage state according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0112] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0113] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of displays, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0114] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for determining the gate access status. For example, in some embodiments, the method for determining the gate access status may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508.

[0115] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the method for determining the gate access status described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform the method for determining the gate access status by any other suitable means (e.g., by means of firmware).

[0116] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other device for determining the access status of a programmable gate, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0120] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0121] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0122] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining the access status of a gate, comprising: The target local road network is determined based on the target gate, and the gate entry trajectory and gate exit trajectory corresponding to the target gate are obtained based on the target local road network. A gate entry trajectory map is generated based on the target local road network and the gate entry trajectory; a gate exit trajectory map is generated based on the target local road network and the gate exit trajectory. Using a multimodal large model, a first answer is generated based on the gate entry trajectory map, and a second answer is generated based on the gate exit trajectory map; Based on the first and second answers, the passage status of the target gate is obtained.

2. The method according to claim 1, wherein, The step of obtaining the gate entry trajectory and gate exit trajectory corresponding to the target gate based on the target local road network includes: Candidate trajectories that match the geographical region corresponding to the target local road network and pass through the target gate are selected from the trajectory database. The trajectory with the driving direction of entering the target gate is selected from the candidate trajectories and used as the gate entry trajectory; The trajectory with the driving direction of leaving the target gate is selected from the candidate trajectories and used as the gate exit trajectory.

3. The method according to claim 2, wherein, The selection of candidate trajectories from the trajectory database that match the geographical region corresponding to the target local road network and pass through the target gate includes: Select from the trajectory database a trajectory that matches the geographical region corresponding to the target local road network and passes through the target gate, and use it as the initial trajectory; The number of crossings of the target gate per day is determined based on the initial trajectory, and the fluctuation results of the number of trajectories are obtained. Obtain the target time period corresponding to the fluctuation results of the trajectory quantity; The initial trajectory is selected based on the target time period to obtain the candidate trajectory.

4. The method according to claim 1, wherein, The process of determining the access status of the target gate based on the first and second answers includes: In response to determining that the target local road network belongs to a simple road network scenario and that neither the first answer nor the second answer includes any cases where it cannot be determined, the passage status of the target gate is obtained based on the first answer and the second answer.

5. The method according to claim 2, further comprising: Select the target vehicle trajectory from the candidate trajectories; Based on the target local road network and the target vehicle trajectory, a target vehicle trajectory map is generated; Using the aforementioned multimodal large model, a third answer is generated based on the target vehicle trajectory map; Based on the third answer, the open state of the target gate is obtained.

6. The method according to claim 5, wherein, The process of determining the open state of the target gate based on the third answer includes: In response to determining that the target local road network belongs to a simple road network scenario and that the third answer is not "cannot be determined", the open state of the target gate is obtained based on the third answer.

7. A device for determining the passage status of a gate, comprising: The acquisition unit is used to determine the target local road network based on the target gate, and to acquire the gate entry trajectory and gate exit trajectory corresponding to the target gate based on the target local road network. The first generation unit is used to generate a gate entry trajectory map based on the target local road network and the gate entry trajectory, and to generate a gate exit trajectory map based on the target local road network and the gate exit trajectory. The second generation unit is used to generate a first answer based on the gate entry trajectory map and a second answer based on the gate exit trajectory map using a multimodal large model; The first determining unit is used to determine the passage status of the target gate based on the first answer and the second answer.

8. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images