Scene matching method, device, electronic device and storage medium

Abstract

The present invention discloses a scene matching method, device, electronic device and storage medium, the method comprising: determining a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame; determining at least one adjacent vehicle of the target vehicle according to the preset number of lane lines; constructing a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determining a target hash sequence based on the target vector map; wherein the driving data includes position information, current vehicle speed and heading angle; determining a to-be-matched scene video corresponding to the target type from a scene library according to the target type to which the target scene video frame belongs, and determining at least one scene video corresponding to the target scene video frame according to the target hash sequence and each to-be-matched scene video. Through the technical solution of the embodiment of the present invention, the technical effect of improving the efficiency and accuracy of scene matching is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a scene matching method, device, electronic device and storage medium. Background Art

[0002] As intelligent driving functions become more popular, the complexity of the testing and debugging process has gradually increased during the development of intelligent driving functions. In order to adapt to changing weather, complex traffic environments, diverse driving tasks, and dynamic driving conditions, repeated testing and debugging are required for different scenarios.

[0003] During the testing and debugging process, if a problematic scene is found, similar scenes can be found in the scene library for repeated testing. However, due to the huge amount of data in the scene library, the workload of finding similar scenes for repeated verification is large.

[0004] Currently, data in the scene library can be annotated manually or automatically, such as whether the scene category is driving on a curve or a straight road, whether the location type is a highway or an urban road, etc. Then, the developer further analyzes and determines the similarity between each data and the tested scene based on the annotated information, so as to determine the scene video similar to the tested scene from the scene library. The above method has the problems of high complexity, high human resource consumption and high subjectivity, which will have a great impact on subsequent repeated verification. Summary of the invention

[0005] The present invention provides a scene matching method, device, electronic device and storage medium to achieve the technical effect of improving scene matching efficiency and accuracy.

[0006] According to one aspect of the present invention, a scene matching method is provided, the method comprising:

[0007] Determining a preset number of lane lines corresponding to the target vehicle according to position information of the target vehicle in the target scene video frame;

[0008] Determining at least one adjacent vehicle of the target vehicle according to the preset number of lane lines;

[0009] Constructing a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determining a target hash sequence based on the target vector map; wherein the driving data includes location information, current vehicle speed and heading angle;

[0010] According to the target type to which the target scene video frame belongs, a to-be-matched scene video corresponding to the target type is determined from a scene library, and according to the target hash sequence and each to-be-matched scene video, at least one scene video corresponding to the target scene video frame is determined.

[0011] According to another aspect of the present invention, there is provided a scene matching device, the device comprising:

[0012] A lane line determination module is used to determine a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame;

[0013] An adjacent vehicle determination module, used to determine at least one adjacent vehicle of the target vehicle according to the preset number of lane lines;

[0014] A target hash sequence determination module, configured to construct a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determine a target hash sequence based on the target vector map; wherein the driving data includes location information, current vehicle speed and heading angle;

[0015] The scene video matching module is used to determine, from a scene library, a scene video to be matched corresponding to the target type to which the target scene video frame belongs, and to determine, based on the target hash sequence and each scene video to be matched, at least one scene video corresponding to the target scene video frame.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the scene matching method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the scene matching method described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention determines a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame to perform lane division, determines at least one adjacent vehicle of the target vehicle according to the preset number of lane lines to determine the related vehicles around the target vehicle, constructs a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determines a target hash sequence based on the target vector map to represent the current driving environment of the target scene video frame, determines a to-be-matched scene video corresponding to the target type from a scene library according to the target type to which the target scene video frame belongs, and determines at least one scene video corresponding to the target scene video frame according to the target hash sequence and each to-be-matched scene video, thereby solving the problems of high complexity, high human resource consumption and high subjectivity when searching for related videos from the scene library, and achieving the technical effect of improving the efficiency and accuracy of scene matching.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a scene matching method provided in Embodiment 1 of the present invention;

[0025] Figure 2 A schematic diagram of a target vehicle and adjacent vehicles provided in Embodiment 1 of the present invention;

[0026] Figure 3 A schematic diagram of a flow chart of a scene matching method provided in Embodiment 2 of the present invention;

[0027] Figure 4 A schematic diagram of a target vector diagram provided by Embodiment 2 of the present invention;

[0028] Figure 5 A schematic diagram of a flow chart of a scene matching method provided in Embodiment 3 of the present invention;

[0029] Figure 6 A schematic diagram of the structure of a scene matching device provided in Embodiment 4 of the present invention;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "target", "to be matched", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "for" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0034] Embodiment 1

[0035] Figure 1 A flow chart of a scene matching method provided in Embodiment 1 of the present invention is provided. This embodiment can be applied to the situation of searching a scene video that matches a target scene video frame in a scene library. The method can be executed by a scene matching device. The scene matching device can be implemented in the form of hardware and / or software. The scene matching device can be configured in an electronic device.

[0036] like Figure 1 As shown, the method includes:

[0037] S110. Determine a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame.

[0038] The target scene video frame may be a video frame of the current video to be searched for a match, or may be a frame in a scene video in which the target vehicle is tested. The target vehicle may be the main vehicle in the target scene video frame, i.e., the vehicle under test. The position information may be the position information of the target vehicle in the target scene video, such as coordinates, etc. The preset number of lane lines may be a certain number of lines pre-set for subsequent determination of adjacent vehicles, for example, four lane lines. It should be noted that four lane lines can determine three lanes, and the target vehicle is usually located in the middle lane of the three lanes.

[0039] Specifically, the position information of the target vehicle is determined in the target scene video frame, and a preset number of lane lines corresponding to the target vehicle are determined based on the position information.

[0040] S120. Determine at least one adjacent vehicle of the target vehicle according to a preset number of lane lines.

[0041] The adjacent vehicle may be a vehicle adjacent to the target vehicle in the target scene video frame.

[0042] Specifically, the position of each lane is determined according to a preset number of lane lines. According to the lane division, the vehicles adjacent to the target vehicle are regarded as adjacent vehicles, for example, eight vehicles around the target vehicle are regarded as adjacent vehicles.

[0043] For example, the schematic diagram of the target vehicle and the adjacent vehicles is as follows: Figure 2 As shown, the white vehicle numbered 1 is the target vehicle, and the black vehicles numbered 2-9 are adjacent vehicles.

[0044] S130, constructing a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determining a target hash sequence based on the target vector map.

[0045] The driving data may be data used to describe the movement of each vehicle, and the driving data may include location information, current vehicle speed, and heading angle. The target vector map may be a vector map describing the movement of the target vehicle and adjacent vehicles. The target hash sequence may be a hash sequence obtained by processing the target vector map using a perceptual hash algorithm.

[0046] Specifically, the driving data of the target vehicle and each adjacent vehicle in the target scene video frame is determined, and each vector is determined according to the determined driving data to construct a target vector map. The target vector map is processed based on the perceptual hash algorithm, and the sequence generated by the processing is used as the target hash sequence.

[0047] S140. According to the target type to which the target scene video frame belongs, determine from the scene library a scene video to be matched corresponding to the target type, and according to the target hash sequence and each scene video to be matched, determine at least one scene video corresponding to the target scene video frame.

[0048] The target type may be a general road type used to describe the current scene, such as a city road or a highway, etc. The scene video to be matched may be each scene video of the target type stored in the scene library. The scene library may be a database for storing videos in various scenes.

[0049] Specifically, the target type corresponding to the target scene video frame is determined automatically or manually, and each to-be-matched scene video corresponding to the target type is found in the scene library. Furthermore, the hash sequence of each video frame in each to-be-matched scene video can be matched with the target hash sequence respectively, and the to-be-matched scene video to which the video frame with the highest matching degree belongs is taken as the scene video corresponding to the target scene video frame. If the number of scene videos required is at least two, the determined scene video can be excluded, and then the scene video corresponding to the target scene video frame can be determined in the above manner until the number of determined scene videos meets the requirements.

[0050] Optionally, the hash sequence of each scene video frame in each scene video in the scene library may be predetermined, and specifically may be:

[0051] For each scene video in the scene library, a main vehicle in the scene video is determined, and based on the main vehicle, a to-be-matched hash sequence corresponding to each scene video frame in the scene video is determined.

[0052] The main vehicle may be a vehicle used for testing in the scene video and may be pre-configured in the scene video. The hash sequence to be matched may be a sequence generated by processing the vector map corresponding to the scene video frame through the perceptual hash algorithm.

[0053] Specifically, the same method can be used to process each scene video in the scene library to obtain a to-be-matched hash sequence corresponding to each scene video frame in the scene video. Therefore, one of the scene videos is used as an example for explanation. The main vehicle is determined according to the scene video, and then the adjacent vehicles of the main vehicle are determined. For each scene video frame in the scene video, a vector diagram corresponding to the scene video frame is constructed based on the pre-recorded driving data of the main vehicle and the adjacent vehicles, and the vector diagram is processed by the perceptual hash algorithm to generate a to-be-matched hash sequence corresponding to the scene video frame.

[0054] The technical solution of the embodiment of the present invention determines a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame to perform lane division, determines at least one adjacent vehicle of the target vehicle according to the preset number of lane lines to determine the related vehicles around the target vehicle, constructs a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determines a target hash sequence based on the target vector map to represent the current driving environment of the target scene video frame, determines a to-be-matched scene video corresponding to the target type from a scene library according to the target type to which the target scene video frame belongs, and determines at least one scene video corresponding to the target scene video frame according to the target hash sequence and each to-be-matched scene video, thereby solving the problems of high complexity, high human resource consumption and high subjectivity when searching for related videos from the scene library, and achieving the technical effect of improving the efficiency and accuracy of scene matching.

[0055] Embodiment 2

[0056] Figure 3 This is a flow chart of a scene matching method provided in the second embodiment of the present invention. Based on any optional technical solution in the embodiment of the present invention, this embodiment can optionally refer to the technical solution of the embodiment of the present invention for the specific implementation methods of the lane line determination method, the adjacent vehicle determination method, the target vector diagram component method, and the scene video matching method in the scene library. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0057] like Figure 3 As shown, the method includes:

[0058] S210. In the target scene video frame, a vehicle coordinate system is established with the position information of the target vehicle as the origin, the moving direction of the target vehicle as the horizontal axis, and the direction perpendicular to the moving direction as the vertical axis.

[0059] The traveling direction may be the speed direction of the target vehicle or the orientation direction of the wheels. The vehicle coordinate system may be a coordinate system established in the target scene video frame for subsequently determining the lane line position and the adjacent vehicles of the target vehicle.

[0060] S220. In the vehicle coordinate system, according to the current vehicle speed, yaw rate, and vehicle width of the target vehicle, determine the lane line positions of a preset number of lane lines corresponding to the target vehicle.

[0061] The current vehicle speed may be the speed of the target vehicle in the target scene video frame. The yaw rate may be the deflection of the target vehicle around the vertical axis in the target scene video frame. The vehicle width may be the distance between two corresponding doors of the target vehicle, or the length of the front bumper.

[0062] Specifically, in the vehicle coordinate system, the lane line equation of each lane line can be determined by calculation according to the current speed, yaw angular velocity and vehicle width of the target vehicle, and the lane line positions of a preset number of lane lines corresponding to the target vehicle in the target scene video frame can be determined according to the lane line equation.

[0063] Optionally, if the preset number is four, the lane line positions of the four lane lines may be determined by the following formula:

[0064]

[0065]

[0066]

[0067]

[0068] Among them, Y Lane1 Indicates the longitudinal coordinate of the first lane line, Y Lane2 Indicates the longitudinal coordinate of the second lane line, Y Lane3 Indicates the longitudinal coordinate of the third lane line, Y Lane4 Indicates the longitudinal coordinate of the fourth lane line, V host Indicates the current speed of the target vehicle, Yaw Rate represents the yaw rate of the target vehicle, X represents the lateral coordinate of the target vehicle, Width host Indicates the vehicle width of the target vehicle.

[0069] S230. Divide the target scene video frame according to a preset number of lane lines, and determine at least one adjacent vehicle adjacent to the target vehicle.

[0070] Specifically, the target scene video frame can be divided according to a preset number of lane lines to determine the area contained in each lane. Further, according to the division of the lane lines and the position information of each vehicle in the target scene video frame, the vehicle adjacent to the target vehicle can be regarded as the adjacent vehicle of the target vehicle.

[0071] Optionally, the adjacent vehicles may include: 1) the front vehicle and the rear vehicle in the same lane as the target vehicle; 2) vehicles in the adjacent lane of the target vehicle that are within a preset distance from the target vehicle in the direction of travel, i.e., the left vehicle and the right vehicle; 3) the left front vehicle and the left rear vehicle in the same lane as the left vehicle; 4) the right front vehicle and the right rear vehicle in the same lane as the right vehicle. The above is the case where the number of adjacent vehicles is 8. If at least one adjacent vehicle does not exist, the number of adjacent vehicles is less than 8.

[0072] S240: construct a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determine a target hash sequence based on the target vector map.

[0073] Specifically, the target vector map can be constructed according to the driving data of the target vehicle and the driving data of each adjacent vehicle through the following steps:

[0074] Step 1: Take the target vehicle and each adjacent vehicle as vector vehicles.

[0075] Step 2: For each vector vehicle, a vector vector corresponding to the vector vehicle is constructed with the position information of the vector vehicle as the starting point, the heading angle of the vector vehicle as the vector direction, and the current speed of the vector vehicle as the vector magnitude.

[0076] Step 3: Construct a target vector map based on the vectors of each vector vehicle.

[0077] For example, the target vector diagram is as follows: Figure 4 As shown, it includes the vectors of the target vehicle and each adjacent vehicle.

[0078] After constructing the target vector map, the target vector map is processed by the perceptual hashing algorithm and the generated sequence is used as the target hash sequence.

[0079] S250: According to the target type to which the target scene video frame belongs, determine a to-be-matched scene video corresponding to the target type from a scene library.

[0080] Specifically, the scene videos can be classified and stored in the scene library according to different types. After determining the target type to which the target scene video frame belongs, each scene video stored under the target type can be found in the scene library, and each found scene video is used as the scene video to be matched.

[0081] S260: for each scene video to be matched, respectively determine the matching degree between the target hash sequence and the matching hash sequence of each scene video frame in the scene video to be matched, and take the maximum value of the matching degrees as the scene matching degree of the scene video to be matched.

[0082] The matching degree may be sequence similarity, and the similarity may be cosine similarity, Euclidean distance, Mahalanobis distance, or Pearson correlation coefficient, etc. The specific similarity used may be determined according to actual needs and is not specifically limited in this embodiment. The scene matching degree may be a value used to measure the degree of matching between the target scene video frame and the scene video.

[0083] Specifically, the same method can be used to process each scene video in the scene library to obtain the matching degree between the target hash sequence and the hash sequence to be matched of each scene video frame in the scene video to be matched. Therefore, one of the scene videos is used as an example for explanation. Obtain the hash sequence to be matched corresponding to each scene video frame in the predetermined scene video, and then determine the matching degree between the target hash sequence and each hash sequence to be matched according to a predetermined similarity method. The maximum value of the determined matching degrees is used as the scene matching degree of the scene video to be matched, that is, the matching degree between the scene video to be matched and the target scene video frame.

[0084] S270. Determine at least one target scene matching degree corresponding to the target scene video frame according to the scene matching degrees of each to-be-matched scene video, and use the to-be-matched scene video corresponding to the target scene matching degree as the scene video corresponding to the target scene video frame.

[0085] The target scene matching degree may be a scene matching degree determined according to a preset matching degree screening condition. The preset matching degree screening condition may be screening according to a matching degree threshold or screening according to a preset number. For example, if the matching degree threshold is 80%, the scene matching degree with a scene matching degree greater than 80% may be used as the target scene matching degree; if the preset number is 10, the scene matching degrees may be arranged in descending order, and the top 10 scene matching degrees may be used as the target scene matching degrees.

[0086] Specifically, at least one target scene matching degree corresponding to the target scene video frame can be determined from the scene matching degrees of each to-be-matched scene video according to the preset matching degree screening condition. Then, the to-be-matched scene video with the target scene matching degree is used as the scene video corresponding to the target scene video frame.

[0087] The technical solution of the embodiment of the present invention is to establish a vehicle coordinate system in a target scene video frame, with the position information of the target vehicle as the origin, the traveling direction of the target vehicle as the horizontal axis, and the direction perpendicular to the traveling direction as the vertical axis; in the vehicle coordinate system, the lane line positions of a preset number of lane lines corresponding to the target vehicle are determined according to the current vehicle speed, yaw angular velocity and vehicle width of the target vehicle; the target scene video frame is divided according to the preset number of lane lines, and at least one adjacent vehicle adjacent to the target vehicle is determined; a target vector map is constructed according to the driving data of the target vehicle and the driving data of each adjacent vehicle; a target hash sequence is determined based on the target vector map; and a target type of the target scene video frame is determined from the target vector map. A to-be-matched scene video corresponding to the target type is determined in the scene library. For each to-be-matched scene video, the matching degree between the target hash sequence and the to-be-matched hash sequence of each scene video frame in the to-be-matched scene video is determined respectively. The maximum value of the matching degrees is used as the scene matching degree of the to-be-matched scene video. According to the scene matching degrees of each to-be-matched scene video, at least one target scene matching degree corresponding to the target scene video frame is determined, and the to-be-matched scene video corresponding to the target scene matching degree is used as the scene video corresponding to the target scene video frame. This solves the problems of high complexity, high human resource consumption and high subjectivity when searching for relevant videos from the scene library, and achieves the technical effect of improving the efficiency and accuracy of scene matching.

[0088] Embodiment 3

[0089] Figure 5 This is a flow chart of a scene matching method provided in Embodiment 3 of the present invention, wherein the explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0090] like Figure 5 As shown, the method includes:

[0091] Step 1: Annotate the target scene video frame and determine the location type (target type) corresponding to the target scene video frame. If each scene video in the scene library has annotation information, that is, the location type is annotated, then obtain the scene video with the same location type as the target scene video frame from the scene library for subsequent processing; if there is a scene video without annotation information in the scene library, then obtain the scene video without annotation information for subsequent processing. The obtained scene videos are used as the scene videos to be matched.

[0092] Step 2: For each frame of the scene video to be matched (scene video frame) selected in the scene library and the target scene video frame, lanes are divided on both sides of the driving track of the main vehicle to determine whether the vehicles around the main vehicle (adjacent vehicles) are in the lane of the main vehicle or in the adjacent lane. It should be noted that the target vehicle in the target scene video frame is the main vehicle, and the main vehicle in each scene video to be matched in the scene library can be pre-set.

[0093]

[0094]

[0095]

[0096]

[0097] Among them, Y Lane1 Indicates the longitudinal coordinate of the first lane line, Y Lane2 Indicates the longitudinal coordinate of the second lane line, Y Lane3 Indicates the longitudinal coordinate of the third lane line, Y Lane4 Indicates the longitudinal coordinate of the fourth lane line, V host Indicates the current speed of the host vehicle, Yaw Rate represents the yaw angular velocity of the vehicle, X represents the lateral coordinate of the vehicle, Width host Indicates the vehicle width of the host vehicle.

[0098] Step 3: Determine the adjacent vehicles around the main vehicle.

[0099] Step 4: Form a sequence according to the position, speed and heading angle of the main vehicle and each adjacent vehicle.

[0100] Step 5: Generate a vector map based on the above sequence. For each vehicle, use the position as the vector starting point, the speed as the vector magnitude, and the heading angle as the vector direction to construct the vehicle vector and form a vector map.

[0101] Step 6: Perform perceptual hashing algorithm calculation on the obtained vector graph to generate a sequence (the hash sequence to be matched and the target hash sequence).

[0102] Step 7: Compare the hash sequence of the target scene video frame (target hash sequence) with the hash sequence of each scene video frame of each scene video to be matched (to be matched hash sequence), and take the maximum value of the matching degree of each scene video frame in each scene video to be matched as the scene matching degree of the scene video to be matched. Traverse all the scene videos to be matched, sort them by the scene matching degree, and determine the target scene video.

[0103] The technical solution of the embodiment of the present invention determines a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame to perform lane division, determines at least one adjacent vehicle of the target vehicle according to the preset number of lane lines to determine the related vehicles around the target vehicle, constructs a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determines a target hash sequence based on the target vector map to represent the current driving environment of the target scene video frame, determines a to-be-matched scene video corresponding to the target type from a scene library according to the target type to which the target scene video frame belongs, and determines at least one scene video corresponding to the target scene video frame according to the target hash sequence and each to-be-matched scene video, thereby solving the problems of high complexity, high human resource consumption and high subjectivity when searching for related videos from the scene library, and achieving the technical effect of improving the efficiency and accuracy of scene matching.

[0104] Embodiment 4

[0105] Figure 6 This is a schematic diagram of the structure of a scene matching device provided by Embodiment 4 of the present invention. Figure 6 As shown, the device includes: a lane line determination module 610, an adjacent vehicle determination module 620, a target hash sequence determination module 630 and a scene video matching module 640.

[0106] Among them, the lane line determination module 610 is used to determine a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame; the adjacent vehicle determination module 620 is used to determine at least one adjacent vehicle of the target vehicle according to the preset number of lane lines; the target hash sequence determination module 630 is used to construct a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determine the target hash sequence based on the target vector map; wherein the driving data includes position information, current vehicle speed and heading angle; the scene video matching module 640 is used to determine the scene video to be matched corresponding to the target type from the scene library according to the target type to which the target scene video frame belongs, and determine at least one scene video corresponding to the target scene video frame according to the target hash sequence and each scene video to be matched.

[0107] Optionally, the lane line determination module 610 is also used to establish a vehicle coordinate system in the target scene video frame, with the position information of the target vehicle as the origin, the traveling direction of the target vehicle as the horizontal axis, and the direction perpendicular to the traveling direction as the vertical axis; in the vehicle coordinate system, the lane line positions of a preset number of lane lines corresponding to the target vehicle are determined according to the current vehicle speed, yaw angular velocity and vehicle width of the target vehicle.

[0108] Optionally, if the preset number is four, the lane line determination module 610 is further configured to determine the lane line positions of the four lane lines using the following formula:

[0109]

[0110]

[0111]

[0112]

[0113] Among them, Y Lane1 Indicates the longitudinal coordinate of the first lane line, Y Lane2 Indicates the longitudinal coordinate of the second lane line, Y Lane3 Indicates the longitudinal coordinate of the third lane line, Y Lane4 Indicates the longitudinal coordinate of the fourth lane line, V host Indicates the current speed of the target vehicle, Yaw Rate represents the yaw angular velocity of the target vehicle, X represents the lateral coordinate of the target vehicle, Width host Indicates the vehicle width of the target vehicle.

[0114] Optionally, the adjacent vehicle determination module 620 is further used to divide the target scene video frame according to the preset number of lane lines to determine at least one adjacent vehicle adjacent to the target vehicle.

[0115] Optionally, the target hash sequence determination module 630 is also used to take the target vehicle and each adjacent vehicle as a vector vehicle; for each vector vehicle, a vector vector corresponding to the vector vehicle is constructed with the position information of the vector vehicle as the starting point, the heading angle of the vector vehicle as the vector direction, and the current speed of the vector vehicle as the vector size; a target vector map is constructed based on the vector vectors of each vector vehicle.

[0116] Optionally, the scene video matching module 640 is also used to determine, for each scene video to be matched, the matching degree between the target hash sequence and the matching hash sequence of each scene video frame in the scene video to be matched, and take the maximum value of the matching degrees as the scene matching degree of the scene video to be matched; according to the scene matching degrees of each scene video to be matched, determine at least one target scene matching degree corresponding to the target scene video frame, and take the scene video to be matched corresponding to the target scene matching degree as the scene video corresponding to the target scene video frame.

[0117] Optionally, the device also includes: a scene library processing module, which is used to determine, for each scene video in the scene library, a main vehicle in the scene video, and determine, based on the main vehicle, a hash sequence to be matched corresponding to each scene video frame in the scene video.

[0118] The technical solution of the embodiment of the present invention determines a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame to perform lane division, determines at least one adjacent vehicle of the target vehicle according to the preset number of lane lines to determine the related vehicles around the target vehicle, constructs a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determines a target hash sequence based on the target vector map to represent the current driving environment of the target scene video frame, determines a to-be-matched scene video corresponding to the target type from a scene library according to the target type to which the target scene video frame belongs, and determines at least one scene video corresponding to the target scene video frame according to the target hash sequence and each to-be-matched scene video, thereby solving the problems of high complexity, high human resource consumption and high subjectivity when searching for related videos from the scene library, and achieving the technical effect of improving the efficiency and accuracy of scene matching.

[0119] The scene matching device provided in the embodiment of the present invention can execute the scene matching method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0120] Embodiment 5

[0121] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0122] like Figure 7As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0124] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a scene matching method.

[0125] In some embodiments, the scene matching method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the scene matching method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the scene matching method in any other appropriate manner (e.g., by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0131] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A scene matching method, characterized in that: include: Determining a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame; Determining at least one adjacent vehicle of the target vehicle according to the preset number of lane lines; Constructing a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determining a target hash sequence based on the target vector map; wherein the driving data includes location information, current vehicle speed and heading angle; According to the target type to which the target scene video frame belongs, determining a to-be-matched scene video corresponding to the target type from a scene library, and according to the target hash sequence and each to-be-matched scene video, determining at least one scene video corresponding to the target scene video frame; The step of determining at least one scene video corresponding to the target scene video frame according to the target hash sequence and each scene video to be matched includes: For each scene video to be matched, respectively determine the matching degree between the target hash sequence and the matching hash sequence of each scene video frame in the scene video to be matched, and take the maximum value of the matching degrees as the scene matching degree of the scene video to be matched; According to the scene matching degree of each to-be-matched scene video, at least one target scene matching degree corresponding to the target scene video frame is determined, and the to-be-matched scene video corresponding to the target scene matching degree is used as the scene video corresponding to the target scene video frame.

2. The method according to claim 1, characterized in that: The step of determining a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame includes: In the target scene video frame, a vehicle coordinate system is established with the position information of the target vehicle as the origin, the moving direction of the target vehicle as the horizontal axis, and the direction perpendicular to the moving direction as the vertical axis; In the vehicle coordinate system, lane line positions of a preset number of lane lines corresponding to the target vehicle are determined according to the current vehicle speed, yaw rate, and vehicle width of the target vehicle.

3. The method according to claim 2, characterized in that If the preset number is four, then in the vehicle coordinate system, according to the current vehicle speed, yaw rate and vehicle width of the target vehicle, determining the lane line positions of the preset number of lane lines corresponding to the target vehicle includes: The lane positions of the four lanes are determined by the following formula: Among them, Y Lane1 Indicates the longitudinal coordinate of the first lane line, Y Lane2 Indicates the longitudinal coordinate of the second lane line, Y Lane3 Indicates the longitudinal coordinate of the third lane line, Y Lane4 Indicates the longitudinal coordinate of the fourth lane line, V host Indicates the current speed of the target vehicle, Yaw Rate represents the yaw angular velocity of the target vehicle, X represents the lateral coordinate of the target vehicle, Width host Indicates the vehicle width of the target vehicle.

4. The method according to claim 1, characterized in that The step of determining at least one adjacent vehicle of the target vehicle according to the preset number of lane lines comprises: The target scene video frame is divided according to the preset number of lane lines to determine at least one adjacent vehicle adjacent to the target vehicle.

5. The method according to claim 1, characterized in that The step of constructing a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle includes: Taking the target vehicle and each adjacent vehicle as vector vehicles; For each vector vehicle, a vector vector corresponding to the vector vehicle is constructed with the position information of the vector vehicle as a starting point, the heading angle of the vector vehicle as a vector direction, and the current speed of the vector vehicle as a vector magnitude; Construct a target vector map based on the vectors of each vector vehicle.

6. The method according to claim 1, characterized in that Also includes: For each scene video in the scene library, a main vehicle in the scene video is determined, and based on the main vehicle, a to-be-matched hash sequence corresponding to each scene video frame in the scene video is determined.

7. A scene matching device, characterized in that: include: A lane line determination module is used to determine a preset number of lane lines corresponding to the target vehicle according to the position information of the target vehicle in the target scene video frame; An adjacent vehicle determination module, used to determine at least one adjacent vehicle of the target vehicle according to the preset number of lane lines; A target hash sequence determination module, configured to construct a target vector map according to the driving data of the target vehicle and the driving data of each adjacent vehicle, and determine a target hash sequence based on the target vector map; wherein the driving data includes location information, current vehicle speed and heading angle; A scene video matching module, configured to determine, from a scene library, a scene video to be matched corresponding to the target type according to the target type to which the target scene video frame belongs, and to determine, according to the target hash sequence and each scene video to be matched, at least one scene video corresponding to the target scene video frame; The step of determining at least one scene video corresponding to the target scene video frame according to the target hash sequence and each scene video to be matched includes: For each scene video to be matched, respectively determine the matching degree between the target hash sequence and the matching hash sequence of each scene video frame in the scene video to be matched, and take the maximum value of the matching degrees as the scene matching degree of the scene video to be matched; According to the scene matching degree of each to-be-matched scene video, at least one target scene matching degree corresponding to the target scene video frame is determined, and the to-be-matched scene video corresponding to the target scene matching degree is used as the scene video corresponding to the target scene video frame.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the scene matching method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the scene matching method according to any one of claims 1 to 6 when executed.

Images