Target inter-vehicle association detection method and device, storage medium, controller and vehicle

Abstract

The application relates to a target interrelation detection method and device, a storage medium, a controller and a vehicle. A first traffic image is acquired, and a relation matrix between targets in the first traffic image is constructed. Then, the first traffic image is input as training data into an initial detection model, and the relation matrix is used as label data to train the initial detection model, so as to obtain a target detection model. Finally, a second traffic image to be predicted is input into the target detection model, and a correlation detection result between traffic targets and lane targets in the second traffic image is generated, wherein the relation matrix represents the real correlation between the traffic targets and the lane targets in the first traffic image. Therefore, the application can improve the accuracy and efficiency of the correlation detection between the targets.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, storage medium, controller, and vehicle for detecting the correlation between targets. Background Technology

[0002] Currently, in the field of autonomous driving, the main methods for detecting the correlation between targets in traffic scenarios are logical rules, high-precision maps, and end-to-end methods, such as detecting whether the traffic sign ahead is related to the lane being driven.

[0003] However, rule-based methods cannot detect complex and dynamic traffic scenarios, high-precision map-based methods are difficult to configure, and end-to-end methods are complex and difficult to control in terms of development and debugging. This makes it difficult for existing detection methods to balance accuracy and efficiency in detecting inter-target correlations. Summary of the Invention

[0004] This application provides a method for detecting the correlation between targets, which can improve the detection of targets.

[0005] To achieve the above objectives, according to a first aspect of this application, a method for detecting the correlation between targets is provided, comprising:

[0006] A first traffic image is acquired, and a relationship matrix between targets in the first traffic image is constructed; the relationship matrix represents the true correlation between traffic targets and lane targets in the first traffic image.

[0007] The first traffic image is used as training data and input into the initial detection model. The relation matrix is ​​used as label data to train the initial detection model to obtain the target detection model.

[0008] The second traffic image to be predicted is input into the target detection model to generate the correlation detection results between traffic targets and lane targets in the second traffic image.

[0009] Optionally, constructing the relationship matrix between targets in the first traffic image includes:

[0010] Based on the actual correlation between the traffic target and the lane target, determine the correlation value for each pair of targets;

[0011] The relationship matrix is ​​constructed by using each traffic target as a row element of the matrix, each lane target as a column element of the matrix, and the association value of each pair of targets as a matrix element of the matrix.

[0012] Optionally, constructing the relationship matrix between targets in the first traffic image further includes:

[0013] If there is a correlation between the traffic target and the lane target, then the correlation value of the pair of targets is determined as the first value;

[0014] If there is no association between the traffic target and the lane target, then the association value of the pair of targets is determined as the second value.

[0015] Optionally, the step of inputting the first traffic image as training data into the initial detection model, and using the relation matrix as label data to train the initial detection model to obtain the target detection model includes:

[0016] The feature extraction network of the initial detection model is used to extract features from the first traffic image to obtain a feature map containing the location encoding of each target.

[0017] The feature map is input into the traffic target generator of the initial detection model to obtain traffic target features, and the feature map is input into the lane generator of the initial detection model to obtain lane target features;

[0018] A bipartite graph is constructed based on the traffic target features and lane target features, where each element in the bipartite graph represents the probability of a correlation between each traffic target feature and each lane target feature.

[0019] A loss function is constructed based on the probabilities in the bipartite graph and the correlation values ​​in the relation matrix. The initial detection model is then iteratively trained until the loss function converges, thus obtaining the target detection model.

[0020] Optionally, the step of inputting the feature map into the traffic target generator of the initial detection model to obtain traffic target features includes:

[0021] The traffic target features are generated by processing the feature map through the self-attention network of the traffic target generator.

[0022] Optionally, the step of inputting the feature map into the lane generator of the initial detection model to obtain lane target features includes:

[0023] The feature extraction network is used to extract features from the first traffic image to obtain the backbone features that characterize the visual information in the first traffic image.

[0024] The lane target features are generated by processing the backbone features and the traffic target features through the cross-attention network of the lane generator.

[0025] Optionally, the step of inputting the second traffic image to be predicted into the target detection model to generate the association detection result between traffic targets and lane targets in the second traffic image includes:

[0026] The second traffic image is processed by the target detection model to generate a first probability that there is a correlation between traffic targets and lane targets in the second traffic image;

[0027] If the first probability is less than or equal to a preset threshold, then the association detection result is that there is no association between the traffic target and the lane target;

[0028] If the first probability is greater than the preset threshold, then the association detection result indicates that there is an association between the traffic target and the lane target.

[0029] Optionally, the method further includes:

[0030] When the first probability is greater than the preset threshold, the target detection model outputs multiple second probabilities; each second probability represents the probability that the target detection model predicts the association type between the traffic target and the lane target;

[0031] Obtain the target probability with the largest value among multiple second probabilities, and determine the target association type corresponding to the target probability as the association type between the traffic target and the lane target.

[0032] According to a second aspect of this application, a target correlation detection device is provided, comprising:

[0033] A matrix construction module is used to acquire a first traffic image and construct a relationship matrix between targets in the first traffic image; the relationship matrix represents the true correlation between traffic targets and lane targets in the first traffic image.

[0034] The model training module is used to input the first traffic image as training data into the initial detection model, and use the relation matrix as label data to train the initial detection model to obtain the target detection model;

[0035] The association detection module is used to input the second traffic image to be predicted into the target detection model and generate association detection results between traffic targets and lane targets in the second traffic image.

[0036] According to a third aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described above.

[0037] According to a fourth aspect of this application, a controller is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described above.

[0038] According to a fifth aspect of this application, a vehicle is also provided, including the controller described above.

[0039] According to a sixth aspect of this application, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of the method described above.

[0040] In summary, the embodiments of this application, through the above technical solutions, annotate the real correlation relationships between various targets in traffic images, construct a relationship matrix between lane targets and traffic targets in traffic images, and obtain a target detection model by supervised learning training of the initial detection model based on the correlation values ​​between targets in the relationship matrix. This can provide effective supervision signals for model learning without relying on logical rules, high-precision maps, and end-to-end methods, ensuring that the model can efficiently learn the correlation rules between lane targets and traffic targets during the training process, thereby improving the accuracy and efficiency of target correlation detection.

[0041] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0044] Figure 1 This is a flowchart of the steps of a method for detecting the correlation between targets provided in an exemplary embodiment of this disclosure;

[0045] Figure 2 This is a schematic diagram illustrating the marking of targets in a traffic image according to an exemplary embodiment of this disclosure;

[0046] Figure 3 This is a schematic diagram illustrating the output of association detection results by a target detection model in an exemplary embodiment of this disclosure;

[0047] Figure 4This is a schematic diagram of the target association detection device provided in an exemplary embodiment of this disclosure;

[0048] Figure 5 This is a schematic diagram of the vehicle architecture provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0050] Based on the problems mentioned in the background technology, the current methods for handling the association rules between lane targets and traffic targets during vehicle driving mainly include the following: First, a logic rule-based approach, which uses pre-defined simple logic rules to handle the relationship between traffic targets and lanes. However, this approach has low reliability; for example, it cannot accurately measure distances, and the rules are relatively simple, making it difficult to cope with complex and ever-changing real-world traffic scenarios. Second, a high-precision map-based approach, which is currently the mainstream method for autonomous driving and is frequently used in patent searches and comparisons. It uses pre-marked traffic target information in high-precision maps and their association with lanes to guide vehicle driving. However, map maintenance costs are huge, and with technological advancements, map-free intelligent driving is gradually becoming a trend, highlighting the limitations of this approach. Third, an end-to-end approach, which does not directly identify the relationship between traffic targets and lanes but instead allows the model to learn the current vehicle's motion information. However, this approach has huge development costs, and each stage is a black box; once an error occurs, iterative repair is extremely difficult, hindering practical application and optimization. Therefore, none of the above methods can simultaneously meet the accuracy and efficiency requirements for detecting the correlation between targets.

[0051] This application provides a method for detecting the correlation between targets. Please refer to [link / reference]. Figure 1 The target association detection method provided in this application includes steps S101-S103, which will be described in detail below.

[0052] Step S101: Obtain the first traffic image and construct the relationship matrix between targets in the first traffic image.

[0053] Traffic images refer to images taken on-site or retrieved from storage that reflect traffic scenes.

[0054] In some embodiments, the target in this application scenario may include traffic targets and lane targets, such as Figure 2As shown, traffic targets in a traffic image can include static targets such as traffic signs, traffic lights, and road markings, as well as dynamic targets such as pedestrians and vehicles. Correspondingly, lane targets refer to the lanes that have been divided on the road surface; for example, the first lane and the second lane of a road are two different lane targets.

[0055] The relationship matrix represents the true correlation between traffic targets and lane targets in the first traffic image. A true correlation refers to whether there is a connection between the targets; for example, if traffic light A is used to indicate vehicles in lane a, then traffic light A and lane a are correlated.

[0056] In some embodiments, step S101 may include:

[0057] First, based on the actual correlation between traffic targets and lane targets, determine the correlation value for each pair of targets;

[0058] Next, a relation matrix is ​​constructed by using each traffic target as a row element of the matrix, each lane target as a column element of the matrix, and the correlation value of each pair of targets as a matrix element.

[0059] Specifically, in the data ground truth preparation stage of the associated values, traffic targets and lanes can be labeled separately. Traffic targets in the image are labeled, such as traffic signs, traffic lights, and road markings. When labeling lane targets, methods include, but are not limited to, bounding boxes and dot matrix methods. Afterwards, the relationships between lanes and objects can be labeled in detail, such as sign 1 associating with lane 1, sign 2 associating with lane 2, etc. Examples of labeling relationships are shown in the table below:

[0060] Sign 1 Associated Lane 1 Sign 2 Associated Lane 2 Sign 3 Associated Lane 3 Sign 4 Associated Lane 4 Sign 5: Unconnected lanes

[0061] Based on this annotation information, a corresponding truth relation matrix can be generated. The rows of the matrix represent traffic targets and the columns represent lane targets, or vice versa. A matrix element value of 1 indicates that the lane corresponding to that row is associated with the traffic target corresponding to that column, while a value of 0 indicates no association. An example relation matrix is ​​shown below:

[0062] Logo 1 Mark 2 Mark 3 Mark 4 Mark 5 Lane 1 1 0 0 0 0 Lane 2 0 1 0 0 0 Lane 3 0 0 1 0 0 Lane 4 0 0 0 1 0

[0063] As can be seen from the relationship matrix, traffic targets can be various traffic signs. Sign 1 is associated with lane 1, and sign 2 is associated with lane 2. However, it should be noted that one traffic target can be associated with multiple lane targets, and similarly, one lane target can be associated with multiple traffic targets. Therefore, this application embodiment does not limit the number of targets associated with each target.

[0064] In subsequent model training, the relation matrix can serve as the ground truth for the association values ​​between targets in the model's bipartite graph, used to supervise the learning of auxiliary model features. The model can continuously adjust its parameters to make its output relationships between lane targets and traffic targets as close as possible to the true association relationships in the relation matrix, thereby achieving accurate judgment of lane-level traffic target identification and association. In the bipartite graph construction stage, the feature similarity between lane targets and traffic targets is calculated to obtain a list of relationships, and the relation matrix provides a standard for evaluating the accuracy of these relationships, helping the model better learn the true association patterns between traffic targets and lane targets.

[0065] In some embodiments, the relation matrix can also be constructed through the following steps:

[0066] If there is a correlation between traffic targets and lane targets, then the correlation value of that pair of targets is determined as the first value;

[0067] If there is no correlation between traffic targets and lane targets, then the correlation value between the two targets is determined as the second value.

[0068] It is understood that, by assigning specific values ​​to different associated states, the embodiments of this application can clearly distinguish the relationship between traffic targets and lane targets, providing clear supervision signals for the model. The model can learn and optimize based on these values, improving the accuracy of lane-level traffic target identification and association judgment.

[0069] As mentioned above, in practice, the "first value" can be set to 1 and the "second value" to 0. Taking the annotation of the relationship between lane targets and traffic targets and the generation of the relationship matrix mentioned above as an example, assuming there are signs 1-5 and lanes 1-4, if sign 1 is associated with lane 1, then in the relationship matrix, the element value at the intersection of lane 1 and sign 1 is 1. Correspondingly, if sign 5 and lane 1 are not associated, the element value at that position is 0.

[0070] In the subsequent bipartite graph construction phase of the model, after calculating the similarity of lane and traffic target features, a list of pairwise relationships between target queues can be obtained, with each relationship value normalized to between 0 and 1. At this point, by comparing with the set first and second values, and according to an appropriate threshold, target pairs with high confidence association are retained, thereby determining the final relationship between lanes and traffic targets.

[0071] It should also be noted that the first and second values ​​are not limited to being 0 and 1. As long as the first and second values ​​are set to two different values, this application embodiment does not impose any restrictions on this.

[0072] Step S102: Input the first traffic image as training data into the initial detection model, and use the relation matrix as label data to train the initial detection model to obtain the target detection model.

[0073] Understandably, the relationship matrix meticulously records the true correlations between traffic targets and lane targets. During training, it serves as label data, allowing the initial detection model to continuously adjust its parameters to make its predictions as close as possible to the true situation depicted in the relationship matrix. The constructed relationship matrix clearly identifies which traffic signs are associated with which lanes. By learning this information, the initial detection model gradually masters how to accurately determine the relationships between traffic targets and lane targets.

[0074] During initial training, the detection model processes a first traffic image, extracting features of traffic targets and lane targets, and attempting to determine the relationships between them. The predicted relationships are compared with the actual relationships in the relationship matrix, and the difference is calculated, typically using a loss function to measure this difference. Based on the calculated loss, optimization algorithms are used to adjust the model's parameters. This process is repeated continuously; as training progresses, the model gradually learns how to better extract features and determine relationships, making the predictions increasingly closer to the labeled data in the relationship matrix, thereby improving the model's accuracy in identifying and associating lane-level traffic targets.

[0075] By using the relationship matrix as label data to train the initial detection model, valuable supervisory information can be provided, enabling the model to continuously optimize through learning from large amounts of data and master the complex correlation patterns between traffic targets and lanes. This helps improve the model's adaptability and accuracy in complex traffic scenarios, reduces false positives and false negatives, and lays a solid foundation for achieving reliable lane-level traffic target recognition and association functions, ultimately enhancing the safety and reliability of autonomous driving systems.

[0076] In some embodiments, step S102 may include:

[0077] First, the feature extraction network of the initial detection model is used to extract features from the first traffic image to obtain a feature map containing the location encoding of each target.

[0078] Next, the feature map is input into the traffic target generator of the initial detection model to obtain traffic target features, and the feature map is input into the lane generator of the initial detection model to obtain lane target features;

[0079] Then, a bipartite graph is constructed based on traffic target features and lane target features. Each element in the bipartite graph represents the probability of a correlation between each traffic target feature and each lane target feature.

[0080] Finally, a loss function is constructed based on the probability in the bipartite graph and the correlation value in the relation matrix. The initial detection model is then iteratively trained until the loss function converges, resulting in the target detection model.

[0081] like Figure 3 As shown, the initial detection model's feature extraction network can be used to process the first traffic image; for example, the feature extraction network could be a CNN network. The feature extraction network captures various visual features of traffic targets (such as traffic signs and traffic lights) and lane targets in the image, such as shape, color, and texture. Since the positional information of traffic targets and lanes is crucial for determining the relationship between them, positional encoding can be performed on each target during feature extraction. For example, a transformer encoder can be used to encode the position of the targets. The resulting feature map not only contains the visual features of the targets but also incorporates positional information, providing a more comprehensive data foundation for subsequent accurate determination of the relationship between targets.

[0082] In some embodiments, the feature map can be processed by the self-attention network of the traffic target generator to generate traffic target features.

[0083] Specifically, the feature maps with location encoding can be input into the traffic target generator and lane generator of the initial detection model, respectively. The traffic target generator uses a self-attention module to analyze the information in the feature maps and generate traffic target features, which focus on the key characteristics of traffic targets. The lane generator uses a cross-attention module to combine the backbone features and traffic target features to generate lane target features, enabling the extraction of lane features to integrate multiple aspects of information and more accurately reflect the actual situation of the lanes.

[0084] In some embodiments, a bipartite graph can be constructed based on the generated traffic target features and lane target features. Each element in the bipartite graph represents the probability that there is a correlation between each traffic target feature and each lane target feature. Similarity indices between two target features can be calculated and converted into probability values ​​between 0 and 1 to fill the bipartite graph. For example, an element value of 0.9 represents a 90% probability that there is a correlation between the traffic target and the lane target. If a traffic target feature and a lane target feature have high similarity at the feature level, the corresponding bipartite graph element probability value is high, indicating a high probability that they are correlated; conversely, the probability value is low.

[0085] In some embodiments, a loss function can be constructed based on the probabilities in the bipartite graph and the correlation values ​​in the relation matrix. The relation matrix records the true correlation between traffic targets and lane targets; for example, correlation values ​​of 0 and 1 can represent the presence or absence of a correlation. The loss function measures the difference between the model's prediction and the actual situation by comparing the probabilities in the bipartite graph and the true correlation values ​​in the relation matrix. During iterative training, the parameters of the initial detection model are continuously adjusted to gradually reduce the value of the loss function. When the loss function converges, it means that the difference between the model's prediction and the actual situation has reached a small and stable state. The model obtained at this point is the target detection model, which has the ability to accurately identify and correlate lane-level traffic targets.

[0086] In some embodiments, the first traffic image can be first extracted using a feature extraction network to obtain the backbone features that characterize the visual information in the first traffic image. Then, the backbone features and traffic target features can be processed by the cross-attention network of the lane generator to generate lane target features.

[0087] Specifically, a feature extraction network can be used to process the first traffic image, extracting the core features that represent the visual information of the image. The feature extraction network can use components such as convolutional layers and pooling layers to progressively process pixel information in the image, capturing various elements in the traffic scene, such as the shape, color, and texture of traffic targets (like traffic signs and traffic lights), as well as lane lines and markings. This complex visual information can be transformed into more abstract and representative core features, providing foundational data for subsequent analysis.

[0088] Step S103: Input the second traffic image to be predicted into the target detection model to generate the correlation detection results between traffic targets and lane targets in the second traffic image.

[0089] The second traffic image refers to the traffic image that predicts the relationship between the targets. After obtaining the trained target detection model, the second traffic image can be input into the target detection model to obtain the corresponding association detection result. For example, the association detection result can be whether there is a relationship between traffic target B and lane target b in the second traffic image.

[0090] In some embodiments, step S103 may include:

[0091] First, the second traffic image is processed by an object detection model to generate a first probability that there is a correlation between traffic objects and lane objects in the second traffic image;

[0092] If the first probability is less than or equal to the preset threshold, the correlation detection result is that there is no correlation between the traffic target and the lane target;

[0093] If the first probability is greater than the preset threshold, the correlation detection result is that there is a correlation between the traffic target and the lane target.

[0094] Specifically, after inputting the second traffic image, the model can identify and analyze traffic targets and lane targets in the image based on its internally learned features and patterns. Through a series of complex calculations and processing, it generates a first probability indicating a correlation between the traffic targets and lane targets. This probability is based on the model's understanding and judgment of various visual information in the image, representing the likelihood that the model believes there is a correlation between the two.

[0095] The preset threshold is a pre-defined reference value used to measure the magnitude of the first probability to determine whether there is a correlation between the traffic target and the lane target. For example, the preset threshold can be set to 0.99. If the first probability is less than or equal to the preset threshold, it means that the model believes there is a low probability of a correlation between the traffic target and the lane target, and the correlation detection result determines that there is no correlation between the two. Conversely, if the first probability is greater than the preset threshold, it means that the model believes there is a high probability of a correlation between the two, and the correlation detection result determines that there is a correlation between the traffic target and the lane target.

[0096] The setting of preset thresholds usually needs to be adjusted based on the actual application scenario and requirements. In autonomous driving scenarios with high safety requirements, the preset threshold may be set relatively high to reduce false positives of correlation and prevent the vehicle from making incorrect decisions; while in some scenarios where detection accuracy requirements are lower but detection efficiency is more important, the preset threshold may be set relatively low.

[0097] In some embodiments, the method of this application may further include:

[0098] First, when the first probability is greater than a preset threshold, the target detection model outputs multiple second probabilities; each second probability represents the probability that the target detection model predicts the association type between traffic targets and lane targets.

[0099] Next, the target probability with the largest value among multiple second probabilities is obtained, and the target association type corresponding to the target probability is determined as the association type between traffic targets and lane targets.

[0100] Specifically, once the target detection model determines that the first probability is greater than a preset threshold, indicating a correlation between the traffic target and the lane target, it can further analyze and predict the correlation type. The model can output multiple second probabilities, each representing the probability of a possible correlation type being predicted. Correlation types can include various types such as jurisdiction, guidance, and warning. For example, if traffic target C is a sign indicating "No Entry to Lane Target C," then the correlation type between traffic target C and lane target C is "Jurisdiction." These probabilities are derived by the model based on learned knowledge and analysis of current traffic image features, reflecting the likelihood of each correlation type occurring in the current scenario.

[0101] In some embodiments, the target probability with the highest value can be selected from multiple second probabilities. The association type corresponding to this target probability is determined as the actual association type between the current traffic target and the lane target. This is because the highest probability means that, in the model's judgment, this association type is most likely to be true in the current scenario. If the second probability corresponding to the "guidance" association type has the highest value among all association types, then the association type between the current traffic target and the lane target can be determined as "guidance," meaning that the traffic target guides the lane target, helping the driver to clarify the driving direction, etc. This method of determining the association type provides more detailed and accurate traffic scenario information for the autonomous driving system or driver, helping to make more reasonable decisions.

[0102] In summary, the embodiments of this application, through the above technical solutions, annotate the real correlation relationships between various targets in traffic images, construct a relationship matrix between lane targets and traffic targets in traffic images, and obtain a target detection model by supervised learning training of the initial detection model based on the correlation values ​​between targets in the relationship matrix. This can provide effective supervision signals for model learning without relying on logical rules, high-precision maps, and end-to-end methods, ensuring that the model can efficiently learn the correlation rules between lane targets and traffic targets during the training process, thereby improving the accuracy and efficiency of target correlation detection.

[0103] According to a second aspect of this disclosure, an inter-target correlation detection device is provided. Figure 4 This is a schematic diagram of the structure of a target correlation detection device provided in an embodiment of this application. Please refer to... Figure 4 The correlation detection device between targets may include a matrix construction module 201, a model training module 202, and a correlation detection module 203, as detailed below:

[0104] The matrix construction module 201 is used to acquire the first traffic image and construct a relationship matrix between targets in the first traffic image; the relationship matrix represents the real correlation between traffic targets and lane targets in the first traffic image.

[0105] Model training module 202 is used to input the first traffic image as training data into the initial detection model, and to train the initial detection model using the relation matrix as label data to obtain the target detection model;

[0106] The association detection module 203 is used to input the second traffic image to be predicted into the target detection model and generate the association detection results between traffic targets and lane targets in the second traffic image.

[0107] Among them, the matrix construction module 201, the model training module 202 and the association detection module 203 can be used to execute steps S101-S103 in the embodiments of the above-mentioned association detection method between targets. For the specific implementation of these modules and more details, please refer to the corresponding method section, which will not be elaborated here.

[0108] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the aforementioned target association detection method.

[0109] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0111] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0112] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0113] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0114] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0115] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated communication signals and carrier waves.

[0116] like Figure 5 The diagram shown is a schematic representation of a vehicle architecture provided in an embodiment of this application. In this embodiment, the vehicle 400 may include a controller 300, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described above. In this embodiment, the vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc., and this disclosure does not specifically limit it in this way.

[0117] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0118] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0119] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0120] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method for detecting correlations between targets, characterized in that, include: Acquire a first traffic image and construct a relationship matrix between targets in the first traffic image; The relationship matrix represents the true correlation between traffic targets and lane targets in the first traffic image; The first traffic image is used as training data and input into the initial detection model. The relation matrix is ​​used as label data to train the initial detection model to obtain the target detection model. The second traffic image to be predicted is input into the target detection model to generate the correlation detection results between traffic targets and lane targets in the second traffic image.

2. The method according to claim 1, characterized in that, The construction of the relationship matrix between targets in the first traffic image includes: Based on the actual correlation between the traffic target and the lane target, determine the correlation value for each pair of targets; The relationship matrix is ​​constructed by using each traffic target as a row element of the matrix, each lane target as a column element of the matrix, and the association value of each pair of targets as a matrix element of the matrix.

3. The method according to claim 2, characterized in that, The construction of the relationship matrix between targets in the first traffic image further includes: If there is a correlation between the traffic target and the lane target, then the correlation value of the pair of targets is determined as the first value; If there is no association between the traffic target and the lane target, then the association value of the pair of targets is determined as the second value.

4. The method according to claim 2, characterized in that, The step of inputting the first traffic image as training data into the initial detection model, and using the relation matrix as label data to train the initial detection model to obtain the target detection model includes: The feature extraction network of the initial detection model is used to extract features from the first traffic image to obtain a feature map containing the location encoding of each target. The feature map is input into the traffic target generator of the initial detection model to obtain traffic target features, and the feature map is input into the lane generator of the initial detection model to obtain lane target features; A bipartite graph is constructed based on the traffic target features and lane target features, where each element in the bipartite graph represents the probability of a correlation between each traffic target feature and each lane target feature. A loss function is constructed based on the probabilities in the bipartite graph and the correlation values ​​in the relation matrix. The initial detection model is then iteratively trained until the loss function converges, thus obtaining the target detection model.

5. The method according to claim 4, characterized in that, The step of inputting the feature map into the traffic target generator of the initial detection model to obtain traffic target features includes: The traffic target features are generated by processing the feature map through the self-attention network of the traffic target generator.

6. The method according to claim 4, characterized in that, The step of inputting the feature map into the lane generator of the initial detection model to obtain lane target features includes: The feature extraction network is used to extract features from the first traffic image to obtain the backbone features that characterize the visual information in the first traffic image. The lane target features are generated by processing the backbone features and the traffic target features through the cross-attention network of the lane generator.

7. The method according to claim 1, characterized in that, The step of inputting the second traffic image to be predicted into the target detection model to generate the association detection result between traffic targets and lane targets in the second traffic image includes: The second traffic image is processed by the target detection model to generate a first probability that there is a correlation between traffic targets and lane targets in the second traffic image; If the first probability is less than or equal to a preset threshold, then the association detection result is that there is no association between the traffic target and the lane target; If the first probability is greater than the preset threshold, then the association detection result indicates that there is an association between the traffic target and the lane target.

8. The method according to claim 7, characterized in that, The method further includes: When the first probability is greater than the preset threshold, the target detection model outputs multiple second probabilities; each second probability represents the probability that the target detection model predicts the association type between the traffic target and the lane target; Obtain the target probability with the largest value among multiple second probabilities, and determine the target association type corresponding to the target probability as the association type between the traffic target and the lane target.

9. A target correlation detection device, characterized in that, include: A matrix construction module is used to acquire a first traffic image and construct a relationship matrix between targets in the first traffic image; The relationship matrix represents the true correlation between traffic targets and lane targets in the first traffic image; The model training module is used to input the first traffic image as training data into the initial detection model, and use the relation matrix as label data to train the initial detection model to obtain the target detection model; The association detection module is used to input the second traffic image to be predicted into the target detection model and generate association detection results between traffic targets and lane targets in the second traffic image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

11. A controller having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

12. A vehicle, characterized in that, Includes the controller as described in claim 11.

13. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images