A data processing method and system for smart lamp pole equipment based on vehicle-road cloud

Abstract

The present invention provides a data processing method and system for smart lamp pole equipment based on vehicle-road cloud. In the network training link, an image sample library is loaded into a basic image processing network; based on the network, basic features of the graph units corresponding to multiple graph units are constructed, and the involved connection lines are constructed. According to the shared debugging parameters of the involved connection lines and the basic features of the graph units, the basic features of the connection lines corresponding to the involved connection lines can be determined; in addition, a basic graph is constructed, and according to the basic features of the graph units corresponding to the two object graph units, the identified object tuples and the basic graph, the network parameters of the basic image processing network can be updated to obtain an image processing network for identifying object tuples containing identified object relations in traffic images. The present invention can increase the generalization and flexibility of the network, and at the same time improve the training speed of the network.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a vehicle-road-cloud-based smart lamp pole equipment data processing method and system. Background Art

[0002] With the rapid development of intelligent transportation systems, there are more and more smart devices on the road, such as roadside infrastructure in the vehicle-road-cloud system. These devices, such as sensors, communication equipment, cables, and checkpoints, can be deployed on smart lamp poles. In the intelligent transportation system, the processing and analysis of traffic images plays a vital role. It can not only help traffic management departments to grasp traffic conditions in real time, but also provide key information support for applications such as autonomous driving and intelligent navigation. With the rise of deep learning technology, image processing technology based on deep learning models such as convolutional neural networks (CNNs) has made significant progress. It can automatically extract high-level feature representations from raw images, effectively improving the accuracy and robustness of image recognition. However, in the field of traffic image processing, especially in complex scenes involving multiple objects and their relationships, it is difficult to meet actual needs to simply rely on models such as CNN for object detection and classification. Traffic images not only contain a large number of objects (such as vehicles, pedestrians, traffic signs, etc.), but also have complex spatial relationships and interactive behaviors (such as following, overtaking, crossing, etc.) between these objects. Therefore, how to effectively capture and represent these relationships has become the key to improving the performance of traffic image processing. Summary of the invention

[0003] The purpose of the present invention is to provide a data processing method and system for smart lamp pole equipment based on vehicle-road cloud. The present invention is implemented as follows:

[0004] In a first aspect, the present invention provides a data processing method for smart lamp pole equipment based on vehicle-road cloud, comprising: obtaining an image sample library; the image sample library comprises traffic image samples, multiple identified objects in the traffic image samples, and identified object tuples containing identified object relationships; the image sample library is loaded into a basic image processing network, and based on the basic image processing network, basic features of atlas units corresponding to multiple atlas units are constructed; the multiple atlas units comprise regional atlas units corresponding to a first number of sample image areas belonging to the traffic image samples, and object atlas units corresponding to the multiple identified objects; based on the multiple atlas units, an involved connection line is constructed, and the connection line corresponding to the involved connection line is determined based on the shared debugging parameters of the involved connection line and the basic features of the atlas units corresponding to the two atlas units at both ends of the involved connection line. Basic features; construct a basic map including multiple map unit basic features and the connecting line basic features, update the network parameters of the basic image processing network according to the map unit basic features corresponding to each pair of object map units, the identified object binary and the basic map, and obtain an image processing network for identifying object binary groups containing the identified object relationship in the traffic image; the network parameters of the basic image processing network include the shared debugging parameters; obtain a traffic image uploaded by a smart lamp pole device, the traffic image includes multiple objects to be identified, the traffic image is collected by a data acquisition device based on the smart lamp pole device on a target road section, and uploaded based on a network device of the smart lamp pole device; load the traffic image into the image processing network, and identify whether the traffic image contains the object binary group containing the identified object relationship.

[0005] In a second aspect, the present invention provides a computer system comprising: one or more processors; a memory; one or more computer programs; wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, the method as described above is implemented.

[0006] Beneficial effects of the present invention: In the present invention, in the network training link, an image sample library is loaded into a basic image processing network, wherein the image sample library includes traffic image samples, multiple identified objects in the traffic image samples, and identified object tuples containing identified object relationships; based on the basic image processing network, basic features of atlas units corresponding to multiple atlas units can be constructed, wherein the multiple atlas units include regional atlas units corresponding to a first number of sample image areas belonging to the traffic image samples, and object atlas units corresponding to multiple identified objects; the multiple atlas units include object atlas units and regional atlas units. According to a plurality of graph units, a connection line is constructed, and according to the shared debugging parameters of the connection line and the basic features of the graph units corresponding to the two graph units at both ends of the connection line, the basic features of the connection line corresponding to the connection line can be determined; in addition, a basic graph including a plurality of basic features of the graph units and basic features of the connection line is constructed, and according to the basic features of the graph units corresponding to the two object graph units, the identified object tuples and the basic graph, the network parameters of the basic image processing network can be updated to obtain an image processing network for identifying object tuples containing identified object relations in traffic images, wherein the network parameters of the basic image processing network include the shared debugging parameters. Based on this, the present invention introduces learnable shared debugging parameters in the connection line, which can increase the generalization and flexibility of the basic image processing network. At the same time, the present invention does not need to predict the basic features of the connection line, which can reduce the consumption of computing power. At the same time, because the basic features of the connection line are not predicted, the training speed of the network can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a flow chart of a data processing method of a smart lamp pole device based on a vehicle-road-cloud provided in an embodiment of the present invention.

[0008] Figure 2 It is a schematic diagram of the composition of a computer system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] The execution subject of the vehicle-road-cloud-based smart lamp pole device data processing method in the embodiment of the present invention is a computer system, including but not limited to a server, a personal computer, a laptop, a tablet computer, a smart phone, etc. Figure 1 As shown, the method includes:

[0010] Step S100: Acquire an image sample library, where the image sample library includes traffic image samples, a plurality of identified objects in the traffic image samples, and identified object tuples containing relationships between the identified objects.

[0011] The image sample library is a comprehensive database used to train basic image processing networks. Traffic image samples can be images from various traffic scenes, such as an intersection of urban roads, a section of a highway, or an image inside a parking lot. Taking an intersection as an example, this traffic image sample may contain multiple lanes, traffic lights, pedestrians, and various types of vehicles (such as cars, buses, motorcycles, etc.).

[0012] The sample image library also includes multiple recognized objects in traffic image samples. These recognized objects are elements that the computer system can clearly distinguish and mark. In the above image of the intersection, the recognized objects can be each individual car, each pedestrian, the different states of traffic lights (red light, green light, yellow light), etc. For example, for a car, the computer system can recognize its model, color, general direction of travel, etc.; for pedestrians, it can recognize their clothing color, walking direction, etc.

[0013] In addition, the image sample library also contains identified object tuples of identified object relations. In traffic scenes, an identified object tuple represents a specific relationship between two identified objects. For example, in the scene of an intersection, there is a relationship between cars and traffic lights. When the traffic light is red, the car should stop. There is also a relationship between cars and pedestrians. In certain areas (such as crosswalks), cars need to give way to pedestrians. These relationships constitute an identified object tuple.

[0014] For the identified objects and the identified object tuples, they can be used as label information during the model training process. The features of the identified objects can be represented by vectors. For example, the color features of a car can be represented by a three-dimensional vector, assuming that red is [255, 0, 0], green is [0, 255, 0], blue is [0, 0, 255], etc. The vehicle model features can be represented by a multidimensional vector. Different vehicle models correspond to different vector values. This vector may contain information such as the size and shape of the vehicle. The relationship between the identified object tuples can also be represented by vectors or arrays. For example, the relationship between a car and a traffic light can be represented by a binary array, [car status (driving / stopping, etc.), traffic light status (red / green / yellow)]. This representation method can facilitate the computer system to process and analyze object relationships.

[0015] Step S200: Load the image sample library into the basic image processing network, and construct basic features of the atlas units corresponding to multiple atlas units based on the basic image processing network; the multiple atlas units include regional atlas units corresponding to a first number of sample image areas belonging to the traffic image sample, and object atlas units corresponding to multiple identified objects.

[0016] In an embodiment of the present application, step S200 loads the image sample library obtained in step S100 into the basic image processing network, and then constructs the basic features of the atlas units corresponding to the multiple atlas units. First, the image sample library is loaded into the basic image processing network. This basic image processing network is a pre-built but not yet converged neural network designed to process traffic image samples. For example, if the basic image processing network is a convolutional neural network (CNN), the traffic image samples in the image sample library will be adjusted in data format according to the input requirements of the network, and then input into the network.

[0017] Next, the computer system constructs basic features of atlas units corresponding to multiple atlas units based on this basic image processing network. The basic features of the atlas units are the initial features of the atlas units, wherein the multiple atlas units include regional atlas units corresponding to the first number of sample image areas belonging to the traffic image sample, and object atlas units corresponding to multiple identified objects. Taking the traffic image sample of an intersection as an example, the computer system divides the image into several sample image areas. Assume that the intersection image is divided into four quadrant areas, each quadrant area is a sample image area, and a regional atlas unit is constructed for each sample image area. For the construction of the regional atlas unit, the computer system will extract feature information related to the area from the basic image processing network. For example, in CNN, after being processed by the convolution layer and the pooling layer, the feature map corresponding to each area can be used as a part of the feature source of the regional atlas unit. These features may include traffic flow characteristics in the area (which can be represented by a vector, such as [number of vehicles, number of pedestrians]), road facility characteristics (such as whether there is a zebra crossing, represented by a binary value [1] for existence and [0] for non-existence), etc.

[0018] For the object map units corresponding to multiple identified objects, taking the intersection as an example, identified objects such as cars, pedestrians, and traffic lights will all have corresponding object map units. For the car object map unit, the computer system will obtain feature information about the car from the basic image processing network. The features of the car can be represented by vectors, such as the position information of the car (coordinates [X, Y] in the image), speed information (represented by the number of pixels moved by the car in each frame of the image, such as [5] pixels / frame), color features (such as [255, 0, 0] for red), etc. For the pedestrian object map unit, its features may include the position of the pedestrian ([X1, Y1]), walking direction (represented by angles, such as [45°]), etc. The features of the traffic light object map unit include the color of the light (such as [0, 255, 0] for green), state (on / off, represented by [1 / 0]), etc.

[0019] Step S200 loads the image sample library into the basic image processing network to construct the basic features of the atlas unit, thereby achieving preliminary structured processing of the traffic image samples, laying a solid foundation for subsequent atlas construction, relationship analysis, and the entire smart lamp pole equipment data processing process.

[0020] Step S300: construct an involved connection line according to a plurality of graph units, and determine a connection line basic feature corresponding to the involved connection line according to a shared debugging parameter of the involved connection line and graph unit basic features corresponding to two graph units at both ends of the involved connection line.

[0021] In step S300 of the embodiment of the present application, first, an involved connection line is constructed based on multiple atlas units. These atlas units include regional atlas units and object atlas units constructed in the previous steps. For example, in the image processing of a traffic scene, the regional atlas unit may correspond to different road areas, such as the four quadrant areas of an intersection; the object atlas unit may correspond to identified objects such as vehicles, pedestrians, and traffic lights. The basis for the computer system to construct the involved connection line is the intrinsic connection between the atlas units. For example, a car (object atlas unit) located in a quadrant area (regional atlas unit) has an association between the two, and the computer system will construct an involved connection line to represent this relationship. For another example, if a pedestrian is located in the crosswalk area (regional atlas unit), then an involved connection line will also be constructed between the pedestrian and this regional atlas unit.

[0022] Then, the computer system determines the basic features of the connection line corresponding to the connection line according to the shared debugging parameters of the connection line and the basic features of the atlas units corresponding to the two atlas units at both ends of the connection line. The shared debugging parameters can be regarded as a universal training parameter, which helps to adjust and optimize the entire image processing process. For example, the present application adopts a neural network model, and the shared debugging parameters are like parameters such as weights and biases in the neural network, which have an important impact on the output results of the model.

[0023] The process of determining the basic features of a connection line is described by taking the involved connection line between a vehicle (object graph unit) and the road area (region graph unit) where it is located as an example. The basic features of the vehicle graph unit may include the vehicle's speed vector (such as [10, 0] means the speed is 10 in the x-axis direction and 0 in the y-axis direction), the type of vehicle (represented by an integer, such as [1] for a sedan), etc.; the basic features of the road area graph unit may include the traffic flow in the area (represented by an array, such as [number of vehicles: 5, number of pedestrians: 3]), the road type (such as [0] for ordinary roads), etc. The shared debugging parameter is assumed to be a weight vector [0.5, 0.3], and the computer system performs calculations based on this information.

[0024] Constructing involved connection lines and determining their corresponding basic features of connection lines can better integrate the relationship information between graph units. In traffic scenes, the relationships between different graph units (such as vehicles, pedestrians, road areas, etc.) are intricate. By constructing involved connection lines and determining their features, these relationships can be expressed in a quantitative and computable way. This helps the computer system to understand the structure and dynamic changes of the entire traffic scene more comprehensively and accurately. Secondly, by introducing shared debugging parameters, the generalization and flexibility of the network are increased. In different traffic scenes or data collected by different smart lamp pole devices, even if there are some differences, the shared debugging parameters can be adjusted according to the actual situation so that the model can adapt to these changes. For example, in different cities, traffic rules and traffic flow may be different. The shared debugging parameters can adjust the basic features of the connection lines according to these differences, thereby ensuring that the model can effectively process traffic image data in various situations.

[0025] Step S400: construct a basic graph including multiple graph unit basic features and connecting line basic features, update the network parameters of the basic image processing network according to the graph unit basic features corresponding to each two object graph units, the identified object tuples and the basic graph, and obtain an image processing network for identifying object tuples containing identified object relations in traffic images; the network parameters of the basic image processing network include shared debugging parameters.

[0026] In step S400 of the embodiment of the present application, a basic map including multiple basic features of the map unit and basic features of the connecting line is first constructed. Taking the traffic scene as an example, the multiple basic features of the map unit include the basic features of the regional map unit and the basic features of the object map unit constructed in the previous step. The basic features of the regional map unit may involve different regional information in the traffic scene, such as the traffic flow characteristics of different quadrants of the intersection (represented by an array, such as [number of vehicles: 5, number of pedestrians: 3] represents the number of vehicles and pedestrians in a quadrant), the type of road (such as [0] represents an ordinary road), etc.; the basic features of the object map unit are for the identified objects, for example, the basic features of the car object map unit may include the speed vector of the car (such as [10, 0] represents the speed of 10 in the x-axis direction and the speed of 0 in the y-axis direction), the type of vehicle (represented by an integer, such as [1] represents a sedan), etc. The basic features of the connecting line reflect the relationship between the graph units. For example, the basic features of the connecting line between the car (object graph unit) and the area where it is located (region graph unit) may include the distance relationship between the two (expressed as a numerical value, such as [5] indicating that the distance is 5 units). The computer system integrates these basic features of the graph units and the basic features of the connecting line to construct a basic graph, which is like an overall mapping diagram of the relationship between traffic scenes.

[0027] Next, the computer system updates the network parameters of the basic image processing network according to the basic features of the atlas units corresponding to the two object atlas units, the identified object tuples, and the basic atlas. Assume that the basic image processing network is a convolutional neural network (CNN), and the network parameters include the weights and biases of the convolution kernels. Take the relationship between vehicles and pedestrians as an example. The basic features of the vehicle object atlas unit are the speed and type mentioned above, and the basic features of the pedestrian object atlas unit may include the position of the pedestrian (such as the coordinate vector [3, 5]), the walking direction (such as [45°]), etc. The identified object tuple is assumed to be the avoidance relationship between the vehicle and the pedestrian (such as whether the vehicle should avoid the pedestrian). The basic atlas contains information such as the basic features of the area where the vehicle and the pedestrian are located and the connecting line between the two.

[0028] The process by which the computer system updates the network parameters based on this information is similar to adjusting parameters in the back-propagation algorithm in CNN. For example, when the identified object binary indicates that the vehicle should give way to pedestrians, and the current network prediction result does not match the actual result, the computer system will adjust the convolution kernel weights and other parameters in the CNN based on the differences between the basic features of the object map units and the relationship information in the basic map. If the vehicle is moving too fast and is close to the pedestrian (based on the basic features of the object map units and the relationship information in the basic map), and the current network predicts that the vehicle does not need to give way to the pedestrian, the computer system will increase the weights of certain convolution kernels so that the network can more accurately identify the avoidance relationship between the vehicle and the pedestrian in the subsequent learning process.

[0029] The construction of the basic graph integrates various relationship information in the traffic scene, and represents the regions, objects and the relationships between them in a unified structure. This helps the computer system understand the complexity of the traffic scene as a whole and provides a comprehensive information basis for accurately identifying the relationship between object tuples. For example, in a complex intersection traffic scene, the basic graph can clearly show the relationship between various regions, vehicles, pedestrians, etc., avoiding the problem of processing objects individually and ignoring the overall relationship. Secondly, updating the network parameters of the basic image processing network can improve the network's ability to recognize the relationship between object tuples in traffic images. By utilizing the rich information such as the basic features of the pairwise object graph units, the identified object tuples and the basic graph, the network can continuously adjust its own parameters so that its output results are more in line with the actual traffic scene relationship.

[0030] Step S500: Acquire a traffic image uploaded by the smart light pole device, the traffic image including a plurality of objects to be identified, the traffic image being acquired by collecting data from a target road section by a data collection device based on the smart light pole device and uploading the data by a network device based on the smart light pole device.

[0031] Smart light pole equipment is equipped with data acquisition devices and network devices. The data acquisition device is used to collect traffic images of the target road section. For example, on the main roads in the city, the data acquisition device of the smart light pole equipment may be a high-definition camera, which can capture the traffic conditions on the road in real time, including the driving of vehicles, the movement of pedestrians, and the status of traffic lights. These collected traffic images contain multiple objects to be identified, such as cars, motorcycles, bicycles, pedestrians, traffic signs, etc.

[0032] The traffic images acquired by the computer system are uploaded through the network device of the smart light pole device. Taking a city traffic monitoring system as an example, the smart light pole device sends the collected traffic images to a designated server or processing center through a wired or wireless network (such as Wi-Fi, 4G / 5G network, etc.), and the computer system receives these images at this processing center.

[0033] From the perspective of data structure, the features of the objects to be identified in traffic images can be represented in many ways. For example, for the object to be identified, the features of the car can be represented by a vector. Assume that the position information of the car is represented by a two-dimensional coordinate vector [X, Y], the speed of the car can be represented by a scalar value, such as

[30] (in kilometers per hour), and the color information is represented by a three-dimensional vector, such as [255, 0, 0] for red. For pedestrians, their position can be represented by a coordinate vector [X1, Y1], and the walking direction can be represented by an angle value, such as [45°].

[0034] Step S600: Load the traffic image into the image processing network to identify whether the traffic image contains an object tuple with an identified object relationship.

[0035] In step S600 of the embodiment of the present application, the traffic image here is collected and uploaded by the smart light pole device in step S500, and contains multiple objects to be identified. For example, in an urban traffic scene, the traffic image may contain objects such as cars, pedestrians, and traffic lights. The image processing network is constructed and updated through a series of operations in step S400, and it has the ability to identify specific object relationships in traffic images.

[0036] For example, suppose that in a traffic image, the computer system recognizes three vehicles (vehicles A, B, and C) and two pedestrians (pedestrians X and Y) through a trained model. The model not only outputs the type and location information of these objects, but also infers the following relationships based on their relative positions and movement directions:

[0037] Vehicle A is following vehicle B.

[0038] Vehicle C is about to overtake vehicle B from the right.

[0039] Pedestrians X and Y are standing at a zebra crossing waiting for an opportunity to cross the road.

[0040] These recognition results are then used in practical applications such as adjusting the lighting strategy of smart light poles, issuing traffic warning information, or controlling changes in traffic lights. At the same time, the system also collects feedback from traffic participants and evaluation results from traffic management departments to evaluate the accuracy and practicality of the model and guide subsequent training and optimization work.

[0041] As an implementation manner, step S300, determining the connection line basic features corresponding to the connection line involved according to the shared debugging parameters of the connection line involved and the basic features of the graph units corresponding to the two graph units at both ends of the connection line involved, may include:

[0042] Step S310: determining the pseudo-standardized connection line features corresponding to the connection line according to the shared debugging parameters of the connection line involved and the basic features of the graph units corresponding to the two graph units at both ends of the connection line respectively;

[0043] Step S320: standardize the quasi-standardized connection line features corresponding to the involved connection lines to obtain the connection line basic features corresponding to the involved connection lines.

[0044] In step S310, the computer system calculates the pseudo-standardized connection line features of the connection line based on the shared debugging parameters involved in the connection line and the basic features of the atlas units at both ends of the connection line. This step is the basis for forming the connection line features and prepares for the subsequent standardization process.

[0045] Shared debugging parameters (such as weights, scaling factors, etc.) are pre-defined or learned during model training, and they are universal for links of different types or attributes. These parameters are used to adjust the contribution of different graph unit features in forming link features, thereby enhancing the generalization and adaptability of the model.

[0046] For example, in a traffic image, if a link connects a vehicle graph unit and a pedestrian graph unit, the shared debugging parameters of this link can take into account the relative size, speed, direction, etc. of the vehicle and pedestrian. Specifically, these parameters may include a weight matrix , where n and m are the dimensions of the feature vectors of the vehicle and pedestrian graph units, respectively, and a scaling factor γ.

[0047] After determining the shared debugging parameters, the computer system needs to extract basic features of the atlas units at both ends of the connection line. These features are calculated by the basic image processing network in step S200, and include visual, spatial or semantic information of the atlas units.

[0048] Continuing with the example of vehicles and pedestrians, assume that the basic feature vector of the vehicle graph unit is , the basic feature vector of the pedestrian atlas unit is These features may include color histograms, texture features, edge information, and high-level semantic features extracted by deep learning models.

[0049] With the shared debugging parameters and basic features of the graph units, the computer system starts to calculate the quasi-standardized connection line features. This process involves operations such as feature weighting, splicing, and nonlinear transformation.

[0050] For example, the quasi-normalized link features can be calculated in the following way: First, the pedestrian feature vector is weighted using the weight matrix W to adjust its contribution to the link features. The weighted feature vector can be expressed as Then, the weighted pedestrian feature vector is concatenated with the vehicle feature vector to form a longer feature vector. The concatenated feature vector can be expressed as In order to introduce nonlinear factors and enhance the expressive power of the model, the concatenated feature vectors can be transformed nonlinearly. This is achieved, for example, by a nonlinear activation function, such as ReLU, sigmoid, or tanh. For example, the feature vector after nonlinear transformation using the ReLU activation function can be expressed as Whether to perform nonlinear transformation depends on the specific design requirements of the model.

[0051] The final quasi-normalized connecting line features can be the concatenated feature vector z (if no nonlinear transformation is performed) or the nonlinearly transformed feature vector z' (if a nonlinear transformation is performed). In practical applications, these features will be used as input for subsequent normalization processing.

[0052] In step S320, the computer system performs standardization processing on the quasi-standardized connecting line features calculated in step S310 to eliminate the influence of feature scale differences on model training and accelerate the convergence process.

[0053] The specific method of standardization depends on the nature of the features and the requirements of the model. Common standardization methods include Min-Max Scaling, Z-score standardization, etc.

[0054] Taking Z-score normalization as an example, this method calculates the mean μ and standard deviation σ of the eigenvector, then subtracts the mean from each eigenvalue and divides it by the standard deviation, thereby scaling the eigenvalues ​​to a range with a mean of 0 and a standard deviation of 1.

[0055] For the quasi-standardized connecting line feature vector z (or z'), its Z-score standardization process can be expressed as:

[0056] ;

[0057] Among them, μ is the mean vector of the feature vector z, and σ is the standard deviation vector of the feature vector z. In order to simplify the calculation and improve the efficiency, the features of the entire data set can be globally standardized, that is, the mean and standard deviation of the entire data set are used to standardize the feature vector of each sample.

[0058] In the scenario of traffic image processing, due to the high dimensionality and complexity of image data, a practical method is to use batch normalization technology to perform local normalization on each small batch of data. This can not only reduce the problem of internal covariate shift, but also speed up the model training process and improve model performance.

[0059] After standardization, the computer system obtains the standardized connection line feature vector z norm , which is the basic feature of the link involved. These features will be used as input for subsequent steps (such as building a basic graph, updating network parameters, etc.) to further analyze and understand the object relationship in the traffic image.

[0060] For example, suppose that in a traffic image, there is a connecting line connecting a vehicle map unit and a pedestrian map unit. The basic feature vector of the vehicle map unit is (This is just an example, the actual feature vector dimension may be higher), the basic feature vector of the pedestrian atlas unit is At the same time, suppose that the weight matrix has been obtained in some way and the global mean vector and the global standard deviation vector .

[0061] In step S310, the weighted pedestrian feature vector is: .

[0062] The concatenated feature vector is: .

[0063] In step S320, the feature vector is normalized: The final z norm That is the basic feature of the connection line involved.

[0064] As an implementation manner, step S310, determining the pseudo-standardized connection line features corresponding to the connection line according to the shared debugging parameters of the connection line and the basic features of the atlas units corresponding to the two atlas units at both ends of the connection line, may include:

[0065] Step S311: determining shared debugging parameters and a proposed processing feature set corresponding to the involved connection line according to the types of the two graph units corresponding to the two graph units at both ends of the involved connection line; the proposed processing feature set includes the graph unit basic features corresponding to the two graph units at both ends of the involved connection line; the shared debugging parameters include shared influence parameters and shared gain parameters;

[0066] Step S312: Based on the shared influencing parameters, feature weighting is performed on the feature set to be processed to obtain an adjusted feature set;

[0067] Step S313: combining the adjustment features in the adjustment feature set to obtain a pseudo-gain adjustment feature, and performing gain adjustment on the pseudo-gain adjustment feature based on a shared gain parameter to obtain a pseudo-standardized connecting line feature corresponding to the involved connecting line.

[0068] In step S311, the computer system determines the corresponding shared debugging parameters and the feature set to be processed according to the types of the graph units at both ends of the connection line (object graph unit, region graph unit, similar graph unit). The shared debugging parameters include shared influence parameters (weight parameters) and shared gain parameters (scaling parameters), which are used to adjust the contribution and importance of different graph unit features in forming the connection line features.

[0069] In an embodiment of the present application, different types of atlas units represent different levels of visual and semantic information in the image. For example, an object atlas unit corresponds to a specific identifiable object in the image (such as a vehicle, pedestrian, etc.), a regional atlas unit represents a specific area in the image (such as an intersection, lane, etc.), and a similar atlas unit refers to multiple instances of the same type of object in different forms of expression (such as vehicles of the same brand and model). The computer system identifies the type of each atlas unit by traversing the annotated data in the image sample library or using a pre-trained classification model. For example, for a traffic image containing vehicles and pedestrians, the system can automatically identify vehicle object atlas units and pedestrian object atlas units, and can also identify specific areas in the image (such as intersection area atlas units).

[0070] The specific values ​​of the shared debugging parameters are automatically learned by the back propagation algorithm during the model training process, but can also be initialized based on prior knowledge or empirical rules. In this step, the computer system selects appropriate shared debugging parameters according to the type of graph unit.

[0071] The shared influence parameter is used to adjust the weights of different atlas unit features in the process of forming the link feature. For example, if a link connects a vehicle object atlas unit and an intersection area atlas unit, the vehicle feature has a higher weight because the vehicle is a dynamically changing subject, while the intersection exists more as a static background. The shared gain parameter is used to further scale the weighted features to ensure the numerical stability and consistency of the link features.

[0072] The feature set to be processed includes the basic features of the atlas units at both ends of the connection line. These features are extracted by the basic image processing network in step S200, and they represent the visual, spatial or semantic information of the atlas unit. For example, the basic features of a vehicle object atlas unit may include color histogram, texture features, edge information, and high-level semantic features extracted by a deep learning model; while the basic features of an intersection area atlas unit may focus more on the shape, size, position, and relative relationship of the area to the surrounding environment.

[0073] After determining the shared influence parameter and the feature set to be processed, the computer system performs weighted processing on the features in the feature set to be processed according to the shared influence parameter. The purpose of the weighted operation is to adjust the representation strength of the graph units in the connecting line features according to their importance and contribution.

[0074] For different types of atlas unit combinations, the computer system can construct different weighting matrices. The dimensions and element values ​​of these matrices depend on the dimensions of the atlas unit feature vectors and the specific values ​​of the shared influence parameters. For example, suppose the feature vector dimension of the vehicle object atlas unit is n, the feature vector dimension of the intersection area atlas unit is m, and the corresponding weight matrix has been determined according to the atlas unit type. Then, for the feature vector of the vehicle object graph unit and the feature vector of the intersection area map unit , the weighted regional atlas unit feature can be expressed as .

[0075] After weighted processing, the computer system obtains the adjusted feature vectors of the atlas units, which numerically reflect the importance and contribution of the atlas units in the formation of the connection line features.

[0076] After completing the feature weighting, the computer system combines the weighted features to form a quasi-gain-adjusted feature. Then, the quasi-gain-adjusted feature is gain-adjusted (i.e., scaled) according to the shared gain parameter to finally obtain a quasi-normalized connecting line feature.

[0077] The purpose of feature combination is to fuse the weighted atlas unit features into a unified feature vector for subsequent gain adjustment and other processing. The combination method can be simple concatenation, summation, averaging or other more complex feature fusion methods.

[0078] Let’s continue to take the vehicle object map unit and the intersection area map unit as examples. Assume that the weighted vehicle feature vector is The weighted intersection area feature vector is , then they can be combined into a longer feature vector by concatenation: .

[0079] Gain adjustment is the process of adjusting the numerical range of the pseudo-gain adjustment feature by scaling it. The purpose of this step is to ensure the numerical stability and consistency of the connecting line feature for subsequent standardization or other processing. Gain adjustment is achieved, for example, by multiplying a scaling factor. This scaling factor can be a predefined constant, a parameter learned through model training, or a value calculated based on some heuristic rule. In this step, the computer system scales the pseudo-gain adjustment feature according to the shared gain parameter.

[0080] Let's continue with the example of feature vector z. Assuming that the scaling factor γ has been determined based on the atlas unit type and model training results, the feature vector after gain adjustment can be expressed as: In practical applications, gain adjustment can be closely combined with the feature combination process to form a unified feature fusion and adjustment framework. For example, in the attention mechanism, the scaling factor itself is part of the feature weight learned by the model; in bilinear pooling, the scaling operation may be implicit in the process of extracting second-order statistical information between features.

[0081] As an implementation manner, step S311, according to the graph unit types corresponding to the two graph units at both ends of the involved connection line, respectively, determines the shared debugging parameter and the proposed processing feature set corresponding to the involved connection line, which may include:

[0082] Step S3111: if the graph unit types corresponding to the two graph units at both ends of the connection line are inconsistent, then the mixed shared debugging parameter is determined as the shared debugging parameter, and the graph unit basic features corresponding to the two graph units at both ends of the connection line are determined as the feature set to be processed;

[0083] Step S3112: If the types of the graph units respectively corresponding to the two graph units at both ends of the connection line are consistent, and the two graph units at both ends of the connection line belong to the first number of regional graph units, then the regional shared debugging parameter is determined as the shared debugging parameter, and the basic features of the graph units respectively corresponding to the two graph units at both ends of the connection line and the correlation features between the two graph units at both ends of the connection line are determined as the feature set to be processed;

[0084] Step S3113: If the graph unit types corresponding to the two graph units at both ends of the connection line are consistent, and the two graph units at both ends of the connection line belong to the same type of graph unit set, then the same type of shared debugging parameters are determined as shared debugging parameters, and the graph unit basic features corresponding to the two graph units at both ends of the connection line, the autocorrelation semantic features of the two graph units at both ends of the connection line, and the correlation features between the two graph units at both ends of the connection line are determined as the feature set to be processed; the same type of image elements corresponding to the same type of graph units in the same type of graph unit set belong to traffic image samples.

[0085] In step S311, the computer system determines corresponding shared debugging parameters and feature sets to be processed according to the types of graph units at both ends of the connection line.

[0086] In step S3111, when the types of graph units corresponding to the two graph units at both ends of the connection line are inconsistent (for example, one end is a vehicle object graph unit and the other end is an intersection area graph unit), the computer system determines the hybrid shared debugging parameters as shared debugging parameters, and determines the basic features of the graph units corresponding to the two graph units as the feature sets to be processed.

[0087] The hybrid shared debugging parameters are obtained through training of machine learning algorithms (such as neural networks, support vector machines, etc.) based on the interaction characteristics and statistical laws between different types of graph units. These parameters are designed to balance the contribution of different types of graph units in forming connection line features to ensure that the connection line features can accurately reflect the relationship between objects. During the training process, the system uses a large amount of labeled data (i.e., traffic image samples with known relationships between objects) to guide the optimization of the machine learning model. The model updates the parameter values ​​through continuous iteration to minimize the difference between the predicted relationship and the true relationship (for example, by minimizing the loss function). Finally, the parameter values ​​obtained when the model converges are the hybrid shared debugging parameters.

[0088] When the types of the atlas units are inconsistent, the system directly extracts the basic features of the two atlas units as the feature set to be processed. These basic features are calculated by the basic image processing network in step S200, and they represent the visual, spatial or semantic information of the atlas units.

[0089] For example, for a vehicle object map unit, its basic features may include color histogram, texture features, edge information, and high-level semantic features extracted by a deep learning model; while for an intersection area map unit, its basic features may focus more on the shape, size, position, and relative relationship of the area to the surrounding environment.

[0090] In step S3112, when the two atlas units at both ends of the connection line respectively correspond to the same atlas unit type and both belong to regional atlas units, the system determines the regional shared debugging parameters as shared debugging parameters, and determines the basic features of the two atlas units and the correlation features between them (such as distance vectors) as the feature sets to be processed.

[0091] The regional shared debugging parameters are also trained by machine learning algorithms based on a large amount of labeled data. Unlike the hybrid shared debugging parameters, these parameters are specifically used to process the relationship between regional atlas units of the same type. Since the regional atlas units represent specific areas in the image (such as intersections, lanes, etc.), during the training process, the system pays special attention to these correlation features between regional atlas units, and adjusts the values ​​of the regional shared debugging parameters through optimization algorithms to ensure that the connecting line features can accurately reflect the relationship between regions.

[0092] For the case where the types are consistent and both are regional atlas units, in addition to extracting the basic features of the two atlas units, the system also needs to calculate the correlation features between them. These correlation features include, for example, distance vectors, direction vectors, relative position relationships, etc., which describe the spatial relationship and layout of regional atlas units. For example, for two intersection regional atlas units, the system can calculate the center point distance between them as part of the distance vector; at the same time, it can also calculate the direction vector and relative position relationship between them based on the pixel coordinates in the image. These correlation features will constitute the feature set to be processed together with the basic features.

[0093] In step S3113, when the two graph units at both ends of the connection line correspond to the same graph unit type and both belong to the same type of graph unit set (i.e. different manifestations of the same type of object), the system adopts a more refined processing method. At this time, the system determines the same type of shared debugging parameters as shared debugging parameters, and determines the basic features of the two graph units, the self-correlated semantic features (i.e. context features) and the correlation features between them as the feature set to be processed.

[0094] The same type of shared debugging parameters are also trained by machine learning algorithms based on labeled data. These parameters are specifically used to process the relationship between the same type of graph units. Since the same type of graph units represent different manifestations of the same type of objects (such as vehicles of the same brand and model), the autocorrelation semantic features and correlation features between them are crucial for understanding the subtle differences and overall relationships between objects.

[0095] During the training process, the system pays special attention to these feature differences and similarities between similar graph units, and adjusts the values ​​of similar shared debugging parameters through optimization algorithms to ensure that the connection line features can accurately reflect the relationship between similar objects.

[0096] For the case of graph units of the same type, the system needs to extract the basic features, self-correlated semantic features and correlation features between the two graph units. Self-correlated semantic features include, for example, contextual information, attribute descriptions, category labels, etc. of the object, which describe the semantic commonalities and differences between graph units of the same type.

[0097] For example, for two vehicle graph units of the same brand, their self-correlation semantic features may include brand name, model code, production date, etc., while the correlation features may include their relative position, direction, speed difference, etc. in the image. These features will together constitute the feature set to be processed for subsequent connection line feature calculation.

[0098] To more specifically illustrate the implementation process of step S311 , the following is a detailed example in combination with a traffic image processing scenario.

[0099] Assume that in a traffic image, there are two involved connection lines. The first connection line connects a vehicle object map unit (denoted as V1) and an intersection area map unit (denoted as R1); the second connection line connects two vehicle object map units of the same brand (denoted as V2 and V3).

[0100] Processing the first connection line (V1-R1):

[0101] 1. Determine the shared debugging parameters: Since the graph unit types of V1 and R1 are inconsistent (one is an object graph unit and the other is a region graph unit), the system selects hybrid shared debugging parameters as shared debugging parameters. These parameters are obtained through the optimization algorithm during the model training process.

[0102] 2. Construct the feature set to be processed: The system extracts the basic features of V1 and R1 as the feature set to be processed. Assume that the basic feature vector of V1 is , the basic eigenvector of R1 is , then the feature set to be processed is .

[0103] Processing the second connection line (V2-V3):

[0104] 1. Determine the shared debugging parameters: Since the graph unit types of V2 and V3 are the same (both are object graph units) and belong to the same graph unit set (the same brand of vehicle), the system selects the same shared debugging parameters as the shared debugging parameters. These parameters are also obtained during the model training process.

[0105] 2. Construct the feature set to be processed: The system not only extracts the basic features of V2 and V3, but also calculates the correlation features (such as relative position vectors) and autocorrelation semantic features (such as brand name, model code, etc.) between them. Assume that the basic feature vectors of V2 and V3 are and , the relative position vector is , the self-correlated semantic features are the brand name "BrandX" and the model code "ModelY", then the feature set to be processed is .

[0106] As an implementation manner, the involved connection line includes a first connection line, and the two atlas units at both ends of the first connection line include a first atlas unit and a second atlas unit; based on this, step S320, standardizing the quasi-standardized connection line features corresponding to the involved connection line to obtain the connection line basic features corresponding to the involved connection line, may include:

[0107] Step S321: if the atlas unit type corresponding to the first atlas unit is inconsistent with the atlas unit type corresponding to the second atlas unit, then a second connection line is obtained from the involved connection lines, wherein, if the first atlas unit belongs to a plurality of object atlas units and the second atlas unit belongs to a first number of regional atlas units, then the two atlas units at both ends of the second connection line include the first atlas unit and the third atlas unit; wherein the third atlas unit belongs to the first number of regional atlas units; if the first atlas unit belongs to a set of similar atlas units, then the two atlas units at both ends of the second connection line include the first atlas unit and the fourth atlas unit, the atlas unit type corresponding to the fourth atlas unit is consistent with the atlas unit type corresponding to the second atlas unit, and similar image elements corresponding to similar atlas units in the set of similar atlas units belong to traffic image samples;

[0108] Step S322: acquiring the quasi-standardized connection line features corresponding to the second connection line from the quasi-standardized connection line features corresponding to the involved connection line;

[0109] Step S323: Based on the quasi-standardized connecting line features corresponding to the second connecting line, the quasi-standardized features corresponding to the first connecting line are standardized to obtain the connecting line basic features corresponding to the first connecting line.

[0110] In step S320, the core task of the computer system is to further standardize the calculated quasi-standardized connection line features to ensure the consistency and comparability of the connection line features, thereby providing a stable and reliable basis for subsequent network parameter updates and image recognition tasks.

[0111] In step S321, the computer system first determines whether the types of the atlas units at both ends of the first connection line are consistent. If not, a second connection line is obtained according to a specific rule to assist in completing the standardization process in subsequent steps.

[0112] Case 1: Connection between object graph unit and region graph unit.

[0113] Assume that the first atlas unit is a vehicle object atlas unit (denoted as V1), and the second atlas unit is an intersection area atlas unit (denoted as R1). Since their types are inconsistent, the computer system needs to find a regional atlas unit connected to V1 as the third atlas unit (denoted as R2) to construct the second connecting line V1-R2. Here, R2 can be a regional atlas unit that is adjacent or related to V1 in spatial position.

[0114] Select the third atlas unit: The basis for selecting R2 may include the position of V1 in the image, the relative position relationship between R2 and V1 (such as distance, direction, etc.), and the properties of R2 itself (such as size, shape, etc.). In practical applications, these bases may be determined by a series of preset rules or machine learning models.

[0115] Constructing the second connection line: Once R2 is determined, the system constructs the second connection line V1-R2. This connection line will be used in the subsequent standardization process as a reference to match the type of the first connection line V1-R1.

[0116] Case 2: Connection between the same type of atlas units and inconsistent type of atlas units.

[0117] If the first atlas unit is a similar atlas unit (such as the same vehicle object atlas unit under different viewing angles or different lighting conditions, denoted as VC1), and the second atlas unit is a regional atlas unit (denoted as R1), the system needs to find a similar atlas unit of the same type as VC1 as the fourth atlas unit (denoted as VC2) to construct the second connecting line VC1-VC2.

[0118] Selecting the fourth atlas unit: The selection basis of VC2 may include similarity with VC1 (such as consistency of features such as appearance, size, color, etc.), position relationship in the image, etc. Similarly, these bases can be determined by preset rules or machine learning models.

[0119] Constructing the second connection line: Once VC2 is determined, the system constructs the second connection line VC1-VC2. This connection line will serve as a reference of consistent type for subsequent standardization of the first connection line VC1-R1.

[0120] In step S322, the computer system retrieves the feature vector corresponding to the second connecting line constructed in step S321 from the calculated quasi-standardized connecting line features. For the second connecting line V1-R2 (or VC1-VC2), the system searches for the feature vector matching this connecting line in its corresponding quasi-standardized connecting line feature set. This feature vector is calculated in step S310 through weighting, combination, gain adjustment and other operations, and it reflects the visual, spatial or semantic characteristics of the second connecting line.

[0121] In step S323, the computer system uses the quasi-standardized connection line features of the second connection line to assist in completing the standardization of the first connection line. The purpose of standardization is to eliminate the difference in the numerical value of the feature vector and make the connection line features comparable between different types of atlas units.

[0122] The choice of normalization method depends on the specific application scenario and the nature of the feature vector. Common normalization methods include Min-Max Scaling, Z-score normalization, etc. In this embodiment, since inconsistent types of atlas units are involved, the system can use normalization methods such as scaling, translation or rotation based on the reference vector.

[0123] Taking Z-score normalization as an example, it may not be appropriate to directly perform Z-score normalization on the quasi-normalized link features of the first link, because the feature distributions of the first link and the second link may be significantly different. Therefore, the system may need to calculate the mean of the second link first. and standard deviation , and then adjust the features of the first connecting line according to these statistics. Since the original features of the first connecting line are not directly Z-score standardized, but are processed using its quasi-standardized features and combined with the features of the second connecting line, the actual process may be more complicated. One possible method is to first try to align the quasi-standardized feature vector of the first connecting line with the quasi-standardized feature vector of the second connecting line in some sense. This alignment may be based on similarity, correlation or other metrics between features. Based on the alignment results, a scaling factor (used to adjust the amplitude of the feature) and a translation vector (used to adjust the center position of the feature) are calculated to map the feature vector of the first connecting line to a space that is closer to the feature vector of the second connecting line. In the mapped space, the feature vector of the first connecting line is standardized (although it may no longer be the traditional Z-score standardization at this time). The standardization process may involve further scaling, translation or other linear or nonlinear transformations to ensure that the connecting line features are comparable between different types of atlas units.

[0124] For example, suppose that in a traffic image, there is a first connecting line connecting a vehicle object atlas unit V1 and an intersection area atlas unit R1. According to step S321, the system selects another area atlas unit R2 adjacent to V1 as the third atlas unit and constructs a second connecting line V1-R2. In step S322, the system retrieves a feature vector matching the second connecting line V1-R2 from the quasi-standardized connecting line feature set. Then, in step S323, the system uses To assist in completing the standardization process of the first connecting line V1 - R1 .

[0125] Assume that the quasi-normalized link line eigenvector of the first link line is , and the quasi-normalized connecting line eigenvector of the second connecting line is .

[0126] To simplify the explanation, assume that the system adopts a simplified normalization strategy: first calculate The mean and standard deviation , then Each element in is normalized as follows:

[0127] ;

[0128] Among them, α, β, and γ are scaling factors and translation parameters, which can be obtained through machine learning model training or determined by some heuristic rules.

[0129] Finally, the normalized feature vector It will become the basic feature of the first connection line V1-R1, which will be used for subsequent network parameter updates and image recognition tasks.

[0130] As an implementation manner, step S323, based on the quasi-standardized connection line feature corresponding to the second connection line, standardizing the quasi-standardized feature corresponding to the first connection line to obtain the connection line basic feature corresponding to the first connection line, may include:

[0131] Step S3231: performing a nonlinear transformation on the quasi-standardized connecting line feature corresponding to the first connecting line to obtain a nonlinear transformation feature corresponding to the first connecting line, and performing a nonlinear transformation on the quasi-standardized connecting line feature corresponding to the second connecting line to obtain a nonlinear transformation feature corresponding to the second connecting line;

[0132] Step S3232: performing exponential mapping on the nonlinear transformation feature corresponding to the first connecting line to obtain the mapping feature corresponding to the first connecting line, and performing exponential mapping on the nonlinear transformation feature corresponding to the second connecting line to obtain the mapping feature corresponding to the second connecting line;

[0133] Step S3233: adding the mapping features corresponding to the second connecting line to obtain a merged mapping feature corresponding to the second connecting line;

[0134] Step S3234: taking the ratio of the mapping feature corresponding to the first connecting line to the merged mapping feature corresponding to the second connecting line as the connecting line basic feature corresponding to the first connecting line.

[0135] In step S323, the computer system performs complex standardization processing on the quasi-standardized features of the first connecting line based on the quasi-standardized connecting line features of the second connecting line to obtain the connecting line basic features of the first connecting line. This process involves multiple steps such as nonlinear transformation, exponential mapping, and feature ratio calculation, aiming to eliminate scale differences between features and improve feature comparability and model generalization.

[0136] In step S3231, the computer system first performs a nonlinear transformation on the quasi-standardized connecting line features of the first connecting line and the second connecting line. The purpose of the nonlinear transformation is to introduce nonlinear factors so that the transformed features can better capture the complex structure and nonlinear relationship in the data.

[0137] There are many methods of nonlinear transformation, such as Sigmoid function, Tanh function, ReLU function and more complex neural network layers. Assume that the quasi-normalized link feature vector of the first link is , the quasi-normalized connecting line feature vector of the second connecting line is Here n and m may not be equal, because the types of the atlas units at both ends of the first connecting line and the second connecting line may be different, resulting in inconsistent feature dimensions.

[0138] For the first connecting line, apply a nonlinear transformation function Get the nonlinear transformation feature vector:

[0139] ;

[0140] Similarly, applying the same nonlinear transformation function to the second connecting line yields:

[0141] ;

[0142] For example, assuming the nonlinear transformation function is the Sigmoid function:

[0143] ;

[0144] Then for a certain eigenvalue z of the first connecting line 11 , and its value after nonlinear transformation is:

[0145] ;

[0146] Likewise, the same nonlinear transformation may be performed on the eigenvalues ​​of the second connecting line.

[0147] In step S3232, the computer system performs exponential mapping on the features after nonlinear transformation to further adjust the scale of the features and introduce nonlinear characteristics. Exponential mapping can, for example, amplify smaller differences and compress larger differences, which helps the model better capture the subtle differences between features.

[0148] The exponential mapping can be simply implemented by the exponential function, that is, g(x)=e x In practical applications, in order to balance the scale of features and avoid numerical overflow, a more robust mapping function can be used, such as a scaled exponential function or a combination of a logarithmic function and an exponential function.

[0149] The nonlinear transformation eigenvector h of the first connecting line 1 Apply the exponential mapping function Get the mapping feature vector:

[0150] ;

[0151] Similarly, the nonlinear transformation eigenvector h of the second connecting line is 2 Performing exponential mapping yields:

[0152] ;

[0153] For example, suppose a simple exponential function is used as the mapping function:

[0154] ;

[0155] Then for the eigenvalue h after a nonlinear transformation of the first connecting line 11 , the value after index mapping is:

[0156] ;

[0157] In step S3233, since the feature dimensions of the first connecting line and the second connecting line may be inconsistent, it may not be feasible to directly perform operations between the features. Therefore, in this step, the computer system performs some form of merging or aggregation on the mapping features of the second connecting line in order to perform subsequent operations with the mapping features of the first connecting line. The method of merging mapping features depends on the specific application scenario and feature properties. Common methods include summation, averaging, maximum pooling, etc. In this embodiment, in order to simplify the description, summation is used as the merging method. However, it should be noted that in actual applications, more complex merging strategies may be required to retain more information.

[0158] The mapping feature vector e of the second connecting line 2 Perform summation and merging to obtain the merged mapping feature scalar:

[0159] ;

[0160] In practical applications, weighted summation or other more complex merging strategies can be adopted.

[0161] In step S3234, the computer system calculates the ratio between the mapping feature of the first connecting line and the merged mapping feature of the second connecting line, and uses the ratio as the basic connecting line feature of the first connecting line. The purpose of this step is to eliminate the scale difference between features through proportional calculation and make the connecting line features comparable between different types of atlas units.

[0162] Since the mapping feature of the first connecting line e 1 is a vector, and the merged mapping feature e of the second connecting line sum,2 is a scalar, so we need to decide how to calculate the ratio between them. A simple way is to divide each mapped feature value of the first connecting line by the combined mapped feature scalar:

[0163] ;

[0164] This approach can introduce numerical stability issues (such as division by zero or very small numbers). Therefore, in practice, a small positive number (such as ) to avoid division by zero:

[0165] ;

[0166] in, is a small positive number (such as 10 -9 ).

[0167] For example, suppose that a certain mapping feature value of the first connecting line is e 11 , the merged mapping eigenvalue of the second connecting line is e sum,2 (and e sum,2 ≠0 to avoid division by zero), the corresponding element in the connecting line basis eigenvector of the first connecting line is:

[0168] ;

[0169] Or, to avoid numerical stability issues:

[0170] .

[0171] As another implementation of step S320, the involved connection line includes a third connection line; the two atlas units at both ends of the third connection line include a fifth atlas unit and a sixth atlas unit; based on this, step S320, standardizing the quasi-standardized connection line features corresponding to the involved connection line to obtain the connection line basic features corresponding to the involved connection line, may include:

[0172] Step S32A: If the atlas unit type corresponding to the fifth atlas unit is consistent with the atlas unit type corresponding to the sixth atlas unit, then obtain the fourth and fifth connecting lines in the involved connecting lines, wherein, if the fifth atlas unit and the sixth atlas unit both belong to the first number of regional atlas units, then the two atlas units at both ends of the fourth connecting line include the fifth atlas unit and the seventh atlas unit, the two atlas units at both ends of the fifth connecting line include the sixth atlas unit and the eighth atlas unit, and the seventh atlas unit and the eighth atlas unit both belong to the first number of regional atlas units; if the fifth atlas unit and the sixth atlas unit both belong to a set of similar atlas units, then the two atlas units at both ends of the fourth connecting line include the fifth atlas unit and the ninth atlas unit, the two at both ends of the fifth connecting line include the sixth atlas unit and the tenth atlas unit; the ninth atlas unit and the tenth atlas unit both belong to a set of similar atlas units; similar image elements corresponding to similar atlas units in the set of similar atlas units belong to traffic image samples;

[0173] Step S32B: acquiring the quasi-standardized connection line feature corresponding to the fourth connection line and the quasi-standardized connection line feature corresponding to the fifth connection line from the quasi-standardized connection line features corresponding to the involved connection lines;

[0174] Step S32C: based on the quasi-standardized connecting line feature corresponding to the fourth connecting line, the quasi-standardized feature corresponding to the third connecting line is standardized to obtain the first connecting line basic feature corresponding to the third connecting line;

[0175] Step S32D: based on the quasi-standardized connecting line feature corresponding to the fifth connecting line, the quasi-standardized feature corresponding to the third connecting line is standardized to obtain the second connecting line basic feature corresponding to the third connecting line;

[0176] Step S32E: Determine the first connection line basic feature and the second connection line basic feature as the connection line basic features corresponding to the third connection line.

[0177] In another embodiment of step S320, when the types of the atlas units at both ends of the involved connection line (such as the third connection line) are consistent, the computer system adopts a more complex standardization strategy. This strategy introduces additional connection lines (the fourth connection line and the fifth connection line) to assist in completing the standardization process, aiming to more accurately adjust the characteristics of the third connection line to maintain consistency and comparability between different types of atlas units.

[0178] In step S32A, the computer system first determines whether the types of the fifth and sixth atlas units at both ends of the third connection line are consistent. If consistent, the system will construct additional connection lines according to the specific types of the atlas units (regional atlas units or similar atlas units).

[0179] Case 1: Connections between regional atlas units.

[0180] Assume that the fifth atlas unit (denoted as R5) and the sixth atlas unit (denoted as R6) are both regional atlas units, and they belong to the same set of regional atlas units. In order to standardize the characteristics of the third connecting line R5-R6, the system needs to construct two additional connecting lines: the fourth connecting line R5-R7 and the fifth connecting line R6-R8, where R7 and R8 are also members of the set of regional atlas units.

[0181] Selecting the seventh and eighth atlas units: The selection of R7 and R8 may be based on a variety of factors, such as relative positional relationship with R5 and R6, region size, shape similarity, etc. In practical applications, these factors may be determined by preset rules, heuristic algorithms, or machine learning models.

[0182] Constructing the fourth and fifth connecting lines: Once R7 and R8 are determined, the system constructs the fourth connecting line R5-R7 and the fifth connecting line R6-R8. These two connecting lines will be used in the subsequent steps to assist in completing the standardization of the third connecting line R5-R6.

[0183] Case 2: Connections between similar graph units.

[0184] If the fifth atlas unit (denoted as VC5) and the sixth atlas unit (denoted as VC6) are both members of the same atlas unit set (representing different manifestations of the same type of object), the system also needs to construct two additional connecting lines: the fourth connecting line VC5-VC9 and the fifth connecting line VC6-VC10, where VC9 and VC10 are also members of the same atlas unit set, and the corresponding same type of image elements belong to traffic image samples.

[0185] Selection of the ninth and tenth atlas units: The selection of VC9 and VC10 may be determined based on similarity to VC5 and VC6 (e.g., consistency in features such as appearance, size, color, etc.). Similarly, these factors may be evaluated by preset rules, heuristic algorithms, or machine learning models.

[0186] Constructing the fourth and fifth connection lines: After determining VC9 and VC10, the system constructs the fourth connection line VC5-VC9 and the fifth connection line VC6-VC10. These two connection lines will be used for the subsequent standardization of the third connection line VC5-VC6.

[0187] In step S32B, the computer system retrieves feature vectors corresponding to the fourth connecting line (such as R5-R7 or VC5-VC9) and the fifth connecting line (such as R6-R8 or VC6-VC10) from the calculated pseudo-standardized connecting line feature set.

[0188] The system traverses the feature set of the quasi-standardized connection lines, matches and retrieves the corresponding feature vectors according to the identification of the connection lines (such as the atlas unit numbers at both ends of the connection lines).

[0189] In steps S32C and S32D, the computer system uses the quasi-standardized link features of the fourth link and the fifth link to standardize the quasi-standardized features of the third link, respectively. This process may involve complex nonlinear transformation, scaling, translation or other feature processing methods.

[0190] However, since the specific implementation of this process is highly dependent on data characteristics and task requirements, only a conceptual framework is provided here to describe the possible processing flow.

[0191] The selection of the normalization method depends on many factors, including the properties of the feature vector (such as dimension, distribution, etc.), the task goal (such as classification, regression, clustering, etc.), and the model architecture (such as neural network, support vector machine, decision tree, etc.). In this embodiment, it is assumed that the system has selected a suitable normalization method based on these factors.

[0192] To illustrate this process, assume that the system adopts a reference vector-based normalization strategy. The basic idea of ​​this strategy is to first calculate some statistics (such as mean, median, mode, etc.) of the quasi-normalized link features of the additional link as a reference vector; then, adjust the quasi-normalized feature vector of the third link based on this reference vector so that it is consistent with the reference vector in some sense.

[0193] In step S32E, the computer system integrates or selects the two standardized feature vectors (ie, the first connecting line basic feature and the second connecting line basic feature) obtained in steps S32C and S32D to obtain a final connecting line basic feature of the third connecting line.

[0194] The selection of the integration method also depends on many factors, including the nature of the feature vector, the task objectives, and the model architecture. Common integration methods include concatenation, summation, averaging, maximum pooling, etc. However, in this embodiment, no specific integration method is limited, but it is assumed that the system has selected a suitable method according to the specific situation.

[0195] To illustrate this process, assume that the system uses concatenation as an integration method. That is, the first connecting line basic feature vector and the second connecting line basic feature vector are concatenated into a longer vector as the connecting line basic feature of the third connecting line. In practical applications, the system can use attention mechanisms, bilinear pooling, etc. to integrate these two feature vectors.

[0196] For example, assuming that in a traffic image, there is a third connecting line connecting two intersection area map units R5 and R6. According to step S32A, the system constructs a fourth connecting line R5-R7 and a fifth connecting line R6-R8, where R7 and R8 are other intersection area map units adjacent to R5 and R6.

[0197] In step S32B, the system retrieves the quasi-normalized connecting line feature vectors of the fourth connecting line and the fifth connecting line: R5R7 and z R6R8 Then, in steps S32C and S32D, the system uses the two feature vectors to calculate the quasi-normalized feature vector z of the third connecting line. R5R6 A normalization process was performed (the exact process is not detailed as it depends on the chosen normalization method).

[0198] Finally, in step S32E, the system assumes that concatenation is used as the integration method, and concatenates the two normalized feature vectors into a connecting line basic feature vector of the third connecting line:

[0199] ;

[0200] in, and They are respectively the first connecting line basic feature and the second connecting line basic feature after being standardized based on the feature vectors of the fourth connecting line and the fifth connecting line.

[0201] As an implementation mode, step S400, based on the basic features of the atlas units corresponding to each of the two object atlas units, the identified object tuples and the basic atlas, updates the network parameters of the basic image processing network to obtain an image processing network for identifying object tuples containing identified object relations in the traffic image, which may include:

[0202] Step S410: constructing reasoning relationships corresponding to the two object graph units based on the basic features of the graph units corresponding to the two object graph units;

[0203] Step S420: constructing a priori connection lines corresponding to two object graph units based on the identified object tuples;

[0204] Step S430: Based on the inference relationship corresponding to the pairwise object map units, the prior connection lines corresponding to the pairwise object map units, and the basic map, the network parameters of the basic image processing network are updated to obtain the image processing network; the image processing network is used to identify object tuples containing identified object relationships in the traffic image.

[0205] In step S400, the computer system updates the network parameters of the basic image processing network according to the basic features of the two-by-two object graph units, the identified object tuples and the basic graph, thereby obtaining an image processing network that can identify object tuples containing specific object relationships in traffic images.

[0206] In step S410, the computer system constructs the inference relationship between the two object graph units based on their basic features. These inference relationships reflect the possible spatial, semantic or logical connections between objects, which are important basis for subsequent relationship identification and network parameter update.

[0207] The method of constructing the reasoning relationship can be selected according to the specific application scenario and model architecture. A common method is to use graph neural networks (GNNs) to capture the interaction information between objects and update the feature representation of the objects through a message passing mechanism. In GNNs, each object graph unit is regarded as a node in the graph, and the edges between the nodes represent a certain relationship between them. By iteratively aggregating the information of neighboring nodes, the feature representation of each node will gradually incorporate the information of its neighbors, thereby forming a richer reasoning relationship.

[0208] In other implementations of S410, a similarity metric (such as cosine similarity, Euclidean distance, etc.) may be used to evaluate the similarity between two object graph units, or a rule-based reasoning method may be used to infer the relationship between the objects based on their types and attributes.

[0209] For example, suppose there are two object graph units: vehicle object graph unit V1 and pedestrian object graph unit P1. Their basic feature vectors are and To construct the inference relationship between V1 and P1, we can calculate the cosine similarity between them:

[0210] ;

[0211] In step S420, the computer system constructs prior connection lines between two pairs of object graph unit pairs based on the identified object pairs. These prior connection lines represent the known relationship between objects, and they will serve as supervisory signals to guide the training process of the model.

[0212] The construction of a priori connection lines depends on, for example, annotated data or domain knowledge. In an embodiment of the present application, the annotated data may contain a large number of traffic images in which the relationships between objects have been marked. These relationships can be different types of interactions such as following, collision, and intersection. By traversing the object tuples in the annotated data, a priori connection lines between object graph units can be constructed and stored in some form of knowledge base.

[0213] In practical applications, the construction of a priori connection lines requires converting the relationship descriptions in the labeled data into a format that the model can understand (such as a list of edges, adjacency matrix, etc.) and ensuring a one-to-one correspondence between them and the object graph units.

[0214] For example, suppose there is a record in the annotated data indicating that vehicle V1 is following pedestrian P1. Then, when constructing the prior connection line, the edge between V1 and P1 can be marked as a "follow" relationship and stored in the knowledge base. This knowledge base may be a graph structure, in which nodes represent object graph units and edges represent the relationship between them. When it is necessary to construct a priori connection lines for a specific scenario, it is only necessary to retrieve the edges related to the current scenario from the knowledge base.

[0215] However, it should be noted that not all object pairs appear in the labeled data. For those object pairs that do not appear in the labeled data, some heuristic rules or default values ​​may be needed to construct their prior connection lines (although the reliability of these connection lines may be lower).

[0216] In addition, in practical applications, the construction of prior connection lines may also need to consider the directionality of the relationship. For example, a "follow" relationship is directional (i.e., A follows B), while a "collision" relationship may be undirectional (i.e., A and B collide). When constructing prior connection lines, it is necessary to ensure that the directionality of the relationship is correctly represented.

[0217] In step S430, the computer system updates the network parameters of the basic image processing network based on the reasoning relationship between the two-tuple of object map units, the prior connection line and the basic map. The purpose of this process is to enable the model to learn how to predict the relationship between the objects in the input image based on their features.

[0218] The updating of network parameters is achieved, for example, through the back-propagation algorithm. During the training process, the model calculates the prediction error (such as cross entropy loss, mean square error, etc.) based on the input image and the corresponding annotation data (i.e., the prior connection line), and updates the network parameters through optimization algorithms such as gradient descent to minimize the error.

[0219] In an independently implementable solution of S430, a multi-task learning model including a relationship prediction branch and a feature extraction branch can be designed. The relationship prediction branch is responsible for predicting the relationship type between objects based on the reasoning relationship; the feature extraction branch is responsible for extracting the basic features of the object from the input image. By sharing the parameters of the feature extraction branch and introducing the reasoning relationship and the prior connection line as additional input or constraint conditions in the relationship prediction branch, the model can learn the feature representation of the object and the relationship pattern between them at the same time during the training process.

[0220] For example, suppose there is a basic image processing network M, which includes components such as convolutional layers, pooling layers, and fully connected layers. In step S430, this network needs to be trained to enable it to recognize object relationships in traffic images. To achieve this goal, the network can be modified as follows:

[0221] 1. Add a relationship prediction branch: Add an additional branch after the fully connected layer of the network to predict the relationship type between objects. The input of this branch can be the basic feature vector of the object graph unit and the inference relationship representation between them (such as similarity value, interaction type probability distribution, etc.). The output is the probability distribution of the relationship type between objects.

[0222] 2. Customized loss function: Design a loss function that contains two parts of loss: one is classification loss (used to evaluate the accuracy of relationship prediction) and the other is regression loss (optional, used to evaluate the consistency between the inference relationship and the prior connection line). Classification loss can be calculated using cross entropy loss; regression loss can be calculated using mean square error or other appropriate regression loss functions.

[0223] 3. Introducing prior connection lines as supervisory signals: During the training process, the prior connection lines are used as supervisory signals to guide the training of the relationship prediction branch. Specifically, the relationship type in the prior connection line is used as the target label to train the classifier of the relationship prediction branch; at the same time, the prior connection line can also be used to constrain the calculation process of the inference relationship (for example, by introducing a regularization term to encourage the consistency between the inference relationship and the prior connection line).

[0224] 4. Joint optimization: In each iteration, the back propagation algorithm is used to calculate the gradient of the entire network, and the network parameters are updated through optimization algorithms such as gradient descent. Since the feature extraction branch and the relationship prediction branch share some parameters (such as the parameters of the convolution layer and the pooling layer), their optimization processes are interdependent. By jointly optimizing the parameters of these two branches, the model can learn the relationship pattern between objects while extracting useful features.

[0225] Through the training process of the above steps, we can obtain an image processing network that can recognize object pairs containing specific object relationships in traffic images. This network can not only accurately detect objects in images, but also predict the type of relationship between them based on the features of the objects and the reasoning relationship between them. This is of great significance for realizing applications such as intelligent traffic monitoring and autonomous driving.

[0226] As an implementation mode, the basic image processing network includes a basic relationship reasoning component; the two object graph units include a first object graph unit and a second object graph unit; based on this, step S410, based on the basic features of the graph units corresponding to the two object graph units, constructs the reasoning relationship corresponding to the two object graph units, which may include:

[0227] Step S411: Based on the basic relationship reasoning component, the basic features of the graph unit corresponding to the first object graph unit and the basic features of the graph unit corresponding to the second object graph unit are combined to obtain a combined feature;

[0228] Step S412: perform feature weighting on the combined features to obtain quasi-nonlinear transformation features, perform nonlinear transformation on the quasi-nonlinear transformation features, and obtain the inference relationship corresponding to the first object graph unit and the second object graph unit.

[0229] In step S411, the computer system combines the basic features of the first object map unit and the second object map unit using the basic relationship reasoning component to form a combined feature. The basic relationship reasoning component is a key part of the basic image processing network, which is specifically used to process the relationship reasoning task between objects.

[0230] The method of feature combination can be selected according to the specific application scenario and data characteristics. A common method is feature concatenation, which directly concatenates the feature vectors of two objects to form a longer vector. Alternatively, in order to balance the richness of feature representation and the efficiency of the model, the computer system can use feature fusion technology to fuse the features of two objects through weighted summation, bilinear pooling, or attention mechanism.

[0231] For example, suppose the basic feature vector of the first object atlas unit (such as vehicle V1) is , the basic feature vector of the second object atlas unit (such as pedestrian P1) is To simplify the explanation, it is assumed that feature concatenation is used as the combination method.

[0232] Then the combined eigenvector c is:

[0233] ;

[0234] In practical applications, the eigenvector and It may contain different types of features (such as color, texture, shape, position, etc.), which together describe the comprehensive properties of V1 and P1.

[0235] In step S412, the computer system performs feature weighting processing on the combined features to adjust the importance of different features in the reasoning relationship; then, the weighted features are nonlinearly transformed to capture the complex interactions between the features and generate a reasoning relationship representation.

[0236] Feature weighting is achieved through a weight vector or weight matrix. These weights can be pre-set fixed values ​​or dynamically adjusted through learning algorithms (such as back propagation). The purpose of weighting is to highlight important features and suppress the influence of minor features on the inference results.

[0237] Nonlinear transformation is one of the key steps in building complex models, which enables the model to capture nonlinear relationships in the data. Common nonlinear transformation methods include Sigmoid function, ReLU function, Tanh function, and more complex neural network layers.

[0238] In relational reasoning, nonlinear transformations help the model learn complex interaction patterns between objects, such as following, collision, intersection, etc. These transformations can map combined features to a new feature space, making the originally linearly inseparable relationships linearly separable.

[0239] For example, let's continue with the example of vehicle V1 and pedestrian P1. Assume there is a weight vector It is used to perform weighted processing on the combined feature c. The weighted feature vector h is:

[0240] ;

[0241] Among them, ⊙ represents element-wise multiplication operation.

[0242] Next, the weighted feature vector h is transformed nonlinearly. Assume that the Sigmoid function is used as the nonlinear transformation function:

[0243] ;

[0244] Then the transformed reasoning relationship r is:

[0245] ;

[0246] Among them, W r is a learnable weight matrix, b r is a bias term. Note that a linear transformation W is introduced here r h+b r As the input of the Sigmoid function, this is to increase the flexibility of the model and allow the model to learn more complex feature combinations.

[0247] In practical applications, the representation of inferred relations may not require the explicit calculation of a vector r. Instead, the model may directly output a probability distribution of relation types (such as following, collision, no relation, etc.), which is obtained by applying one or more fully connected layers (possibly with nonlinear activation functions) and softmax functions on the weighted features.

[0248] As an implementation mode, the pairwise object graph unit includes a first object graph unit and a second object graph unit; based on this, step S420, based on the identified object tuples, constructing a priori connection lines corresponding to the pairwise object graph units, may include:

[0249] Step S421: determining the object graph units corresponding to the two identified objects in the identified object binary group as a first object graph unit binary group;

[0250] Step S422: constructing a second object graph unit tuple including the first object graph unit and the second object graph unit;

[0251] Step S423: if the first object graph unit binary contains the same object graph unit binary as the second object graph unit binary, then determining the positive sample prior connection line as the prior connection line corresponding to the second object graph unit binary;

[0252] Step S424: If the first object graph unit binary does not include an object graph unit binary that is identical to the second object graph unit binary, then the negative example a priori connecting line is determined as the a priori connecting line corresponding to the second object graph unit binary.

[0253] In step S421, the computer system first traverses the set of identified object tuples, extracts the object graph units corresponding to the two identified objects in each tuple, and forms a first object graph unit tuple. These tuples directly reflect the relationship between objects in the annotated data and are the basis for constructing a priori connection lines.

[0254] For example, assume that the set of identified object binary tuples includes the following two binary tuples: {vehicle V1, pedestrian P1} and {vehicle V2, vehicle V3}. Based on these binary tuples, the computer system can determine two first object atlas unit binary tuples: (V1, P1) and (V2, V3). Among them, V1, P1, V2, and V3 represent the object atlas units corresponding to different objects in the image.

[0255] In step S422, after determining the first object map unit binary, the computer system constructs a second object map unit binary including the first object map unit and the second object map unit. The "second object map unit" here is a general term, indicating any other object map unit that may exist in the image in addition to the first object map unit. By constructing such a binary, the computer system can evaluate the relationship between all potential object pairs in the image.

[0256] For example, continuing with the above example, assume that in addition to V1, P1, V2, and V3, there is another pedestrian object map unit P2 in the image. Then, the computer system can construct the following second object map unit tuples: (V1, P2), (V1, V2), (V1, V3), (P1, V2), (P1, V3), (P1, P2), (V2, P2), etc. These tuples cover all possible object pair combinations in the image.

[0257] In step S423, the computer system traverses all constructed second object graph unit binary tuples and compares them with the first object graph unit binary tuples. If a second object graph unit binary tuple is exactly the same as a first object graph unit binary tuple (i.e., contains the same object graph units and the order is consistent), the connecting line corresponding to the second object graph unit binary tuple is marked as a positive sample prior connecting line (i.e., a positive sample label connecting line).

[0258] For example, let's continue with the previous example. Assume that during the comparison process, the computer system finds that the second object graph unit pair (V1, P1) is exactly the same as the first object graph unit pair (V1, P1). Therefore, the computer system marks the connection line corresponding to (V1, P1) as a positive example prior connection line. This means that during the training process, this connection line will be regarded as a positive sample representing a true object relationship.

[0259] In step S424, for those second object graph unit pairs that are not identical to any first object graph unit pairs, the computer system marks their corresponding connection lines as negative sample prior connection lines (i.e., negative sample label connection lines). These connection lines represent object relationships that are not observed in the labeled data or are believed to not exist.

[0260] For example, let's continue with the previous example. Assume that during the comparison process, the computer system finds that the second object graph unit pair (V1, P2) is not exactly the same as any of the first object graph unit pairs. Therefore, the computer system marks the connection line corresponding to (V1, P2) as a negative example prior connection line. This means that during the training process, this connection line will be regarded as a negative sample representing a non-real object relationship.

[0261] Due to the complexity and diversity of traffic scenes, even if two objects are not marked as related in the annotated data, there may be some potential interaction or influence between them. Therefore, when constructing negative sample prior connection lines, the computer system needs to be cautious to avoid introducing too much noise or mislabeled samples.

[0262] To solve this problem, computer systems can adopt some strategies to enhance the reliability and diversity of negative examples. For example:

[0263] 1. Rule-based generation: Generate some reasonable negative examples based on domain knowledge or heuristic rules. For example, in a traffic scene, two vehicles that are far apart and driving in opposite directions will not be marked as related.

[0264] 2. Data augmentation: Generate more negative examples by performing appropriate transformations on the labeled data (such as translation, rotation, scaling, etc.). Although these transformed samples are similar to the original samples, they may be different in relationship.

[0265] 3. Hard Example Mining: Dynamically identify samples that are difficult to classify as negative examples during training and focus on training. These samples, for example, are located near the decision boundary and are important for improving the generalization ability of the model.

[0266] Prior connection lines play a vital role as supervisory signals in the training process. They not only guide the optimization of network parameters but also help the model learn how to distinguish between real and false object relationships.

[0267] As an implementation mode, step S430, based on the reasoning relationship corresponding to the pairwise object graph units, the prior connection lines corresponding to the pairwise object graph units, and the basic graph, updates the network parameters of the basic image processing network to obtain the image processing network, which may include:

[0268] Step S431: determining the network training cost according to the reasoning relationship corresponding to each pair of object graph units and the prior connection lines corresponding to each pair of object graph units;

[0269] Step S432: Update the network parameters of the basic image processing network according to the network training cost and the basic graph to obtain the image processing network.

[0270] In step S431, the computer system determines the network training cost based on the reasoning relationship between the two-tuple of object graph units and the prior connection line. The training cost is an indicator to measure the difference between the model prediction result and the actual result. By minimizing the training cost, the model can gradually learn the inherent laws of the data.

[0271] First, for each pair of object graph unit tuples (such as vehicle V1 and pedestrian P1), the computer system generates an inference relationship representation (such as the inference relationship vector r obtained in step S410). V1P1 and a priori link line labels (such as positive sample labels indicating a following relationship or negative sample labels indicating no relationship).

[0272] Next, the computer system needs to define a loss function to calculate the difference between the inference relationship and the prior connection line label. Common loss functions include cross-entropy loss, mean squared error loss, etc. Here, it is assumed that the cross entropy loss function is used because it performs well in handling classification problems.

[0273] For binary classification problems (such as determining whether there is a specific relationship between two objects), the calculation formula of the cross entropy loss function can be expressed as:

[0274] ;

[0275] Where N is the number of samples, y i is the true label (0 or 1) of the i-th sample, p i is the probability that the model predicts that the i-th sample is a positive sample.

[0276] Assume that for the object graph unit pair (V1, P1), the inference relation vector output by the model is , then the corresponding probability value can be calculated by the sigmoid function:

[0277] ;

[0278] If the prior connection line corresponding to (V1, P1) is a positive sample (that is, there is a follow-up relationship), then the true label y V1P1 =1; if it is a negative sample (i.e., there is no follow-up relationship), then y V1P1 =0.

[0279] For the entire dataset, the network training cost L is the average of all sample losses:

[0280] ;

[0281] Where D is the set of all object graph unit pairs.

[0282] In step S432, after determining the network training cost, the computer system calculates the gradient through the backpropagation algorithm and updates the network parameters of the basic image processing network according to the gradient. The backpropagation algorithm is an effective method for calculating the gradient of all weights in the neural network with respect to the loss function, and propagates the gradient from the output layer to the input layer through the chain rule.

[0283] During backpropagation, for each trainable parameter (such as weights and biases), the computer system calculates its gradient with respect to the loss function. These gradients indicate how the parameter should be adjusted to minimize the loss function.

[0284] Assuming there is a simple neural network layer, its forward propagation process can be expressed as:

[0285] ;

[0286] Where w is the weight vector, x is the input vector, b is the bias term, σ(·) is the activation function (which can be a sigmoid function or something else), z is the weighted sum, and a is the output after activation.

[0287] During backpropagation, it is necessary to calculate the gradient of the loss function L with respect to the weights w and biases b. This involves, for example, the application of the chain rule and the calculation of the derivative of the activation function.

[0288] The gradient of weight w can be expressed as:

[0289] ;

[0290] in, is the gradient of the loss function with respect to the output a (which can be obtained by backpropagating the gradient of subsequent layers), is the derivative of the activation function with respect to z (this depends on the choice of activation function), is the gradient of the weighted sum with respect to the weights.

[0291] Similarly, the gradient of bias b can be expressed as:

[0292] ;

[0293] in, Because the bias term is added directly to the weighted sum.

[0294] Once the gradients of all trainable parameters are calculated, the computer system can use optimization algorithms (such as gradient descent, Adam, etc.) to update the values ​​of these parameters. Taking gradient descent as an example, the update rule can be expressed as:

[0295] ;

[0296] Among them, α is the learning rate, which controls the step size of parameter update.

[0297] As an implementation mode, the plurality of object graph units include an object graph unit D m ; m≥1, and m is not greater than the number of the plurality of object graph units; Based on this, in step S300, constructing the involved connection line according to the plurality of graph units may include:

[0298] Step S301: Obtain the object graph unit D from the first number of regional graph units. m The first involved area map unit involved is constructed to include an object map unit D m and a first involved connection line of the first involved area atlas unit; the sample image area corresponding to the first involved area atlas unit includes the object atlas unit D m the corresponding identified object;

[0299] Step S302: if the first number is greater than 1, then constructing a second involved connection line based on the two-by-two area map units in the first number of area map units;

[0300] Step S303: construct an involved connection line according to the first involved connection line and the second involved connection line.

[0301] In the implementation of step S300, the computer system constructs the involved connection lines between multiple atlas units (including object atlas units and region atlas units) based on them. These connection lines not only reflect the spatial relationship between the atlas units, but also imply the potential interaction or influence between the objects.

[0302] In step S301, the goal of the computer system is to find the graph unit (such as D m ) related regional atlas units and construct the first involved connection lines between them. These connection lines directly connect the object with the atlas units in the region where it is located, providing a basis for subsequent relationship reasoning.

[0303] First, the computer system needs to traverse all the region graph units and find those that are consistent with the object graph unit D. m Regional atlas units that are adjacent or related in space. This relationship can be based on distance (such as the object atlas unit is located inside or on the edge of a regional atlas unit) or semantics (such as the function of the object atlas unit is closely related to a regional atlas unit).

[0304] Assume that there are multiple regional atlas units, marked as R 1 , R 2 , ..., R n , and an object graph unit D m (For example, a car). Through spatial analysis and semantic matching, the system determines R 3 Yes and D m The most relevant regional atlas units (e.g., D m lane area).

[0305] Once the first involved region map unit (such as R 3 ), the computer system can construct the object graph unit D m to R 3 The first involved connection line of D m With R 3 The direct relationship between them may involve information such as spatial location, functional attributes or potential interactions.

[0306] In practice, the first connection may not only be a simple mark or tag, but also a structure containing multiple features. These features may include the type of connection (such as object-region connection), weight (indicating the strength of the association), direction (if any), and possible metadata (such as the creation time and update frequency of the connection).

[0307] To simplify the description, it is assumed that the first involved link consists of only one identifier and a set of basic feature vectors. 1 `, the basic eigenvector is `[f 1 , f 2 , ..., f k ]`, where each feature component reflects D m With R 3 The strength or nature of a particular type of association between two entities.

[0308] In step S302, if there are multiple regional atlas units in the image (i.e., the first number is greater than 1), the computer system also needs to consider the relationship between these regional atlas units. In step S302, the system constructs a second involved connection line based on the regional atlas units to capture the spatial layout, adjacent relationship or functional dependency between regions.

[0309] For a given set of regional atlas units {R 1 , R 2 , ..., R n}, the computer system needs to generate all possible pairwise combinations. These combinations represent the possible direct connections between regions. For example, for three region map units R 1 , R 2 and R 3 , possible pairwise combinations include (R 1 , R 2 )、(R 1 , R 3 ) and (R 2 , R 3 ).

[0310] For each pair of regional atlas units, the computer system constructs a second involved connection line to represent the relationship between them. These connection lines may be defined based on a variety of factors such as spatial distance, shape similarity, functional complementarity, etc. between regions.

[0311] Assume that the regional atlas unit R has been determined 1 and R 2 There is some kind of relationship between them (for example, they are adjacent and together form part of an intersection). Then, the system can construct a 1 to R2 (or vice versa) the second involves the connection line `L 2 `. Similar to the first involved connection line, `L 2 `may also contain a set of basis eigenvectors describing R in detail 1 and R 2 The correlation characteristics between them.

[0312] In step S303, the computer system integrates the first involved connection line and the second involved connection line to form a complete involved connection line network. This network includes not only the direct association between the object graph unit and the regional graph unit (the first involved connection line), but also the indirect association between the regional graph units (the second involved connection line).

[0313] When integration involves connecting wires, computer systems use certain strategies to ensure the consistency and integrity of the network. These strategies include, for example:

[0314] Deduplication: Ensure that no duplicate connections exist. If two connections describe essentially the same relationship, only one is retained.

[0315] Weight adjustment: The weight of the link is adjusted according to its type and characteristics. For example, the direct connection between an object and a region (the first involved link) may be more important than the indirect connection between regions (the second involved link).

[0316] Direction processing: For directional connection lines (such as the direction of vehicle travel), it is necessary to ensure the consistency of the direction.

[0317] Loop detection: Avoid loops in the network of connected lines, which may cause logical errors or performance degradation when reasoning about relationships.

[0318] For example, suppose the following scenario: a traffic image contains a car (object atlas unit D m ) and three regional atlas units (R 1 :Sidewalk, R 2 : Lane, R 3 : intersection). Through step S301, a m to R 3 The first involved connection line `L 1 ` (indicates that the car is located at the intersection). Through step S302, three second involved connection lines `L are constructed 2 `(R 1 -R 2 )、`L 3 `(R 2 -R 3 ) and `L 4 `(Assuming R1 With R 3 There is also some kind of connection between them, such as visual continuity).

[0319] In step S303, the computer system integrates these connection lines into a network. 1 `Directly connects the positional relationship between the car and the intersection; `L 2 `、`L 3 ` and `L 4 The ` describes the spatial layout and adjacency between intersections, lanes, and sidewalks. The entire network provides rich contextual information for subsequent relational reasoning and image recognition tasks.

[0320] Although this embodiment mainly focuses on the construction and integration strategy of connecting lines, it is worth noting that in practical applications, each connecting line may be accompanied by a set of basic feature vectors to represent its specific associated attributes. These feature vectors may be extracted from the original image data through machine learning algorithms (such as convolutional neural networks, graph neural networks, etc.), or may be defined based on preset rules or domain knowledge.

[0321] For example, for the first involved connection line `L 1 `, whose feature vector may include:

[0322] Position feature: represents the object graph unit D m In the regional atlas unit R 3 The specific location in (such as center coordinates, bounding box, etc.).

[0323] Shape features: describing object graph unit D m shape properties (such as aspect ratio, area, etc.), which may be related to the regional atlas unit R 3 The shape features of the object are combined to infer the object's motion state or behavioral intention.

[0324] Semantic features: Representing object graph unit D m The semantic features of the object are helpful for understanding the role and importance of the object in a specific scene.

[0325] Similarly, for the second involved connection line (such as `L 2 `、`L 3 `、`L 4 `), whose feature vectors may focus on describing the spatial relationship between regions, shape similarity, functional complementarity, etc. The specific content and dimensions of these feature vectors will depend on the needs of the application scenario and the design choices of the model.

[0326] As an implementation manner, the plurality of atlas units include a second number of similar atlas units, and the similar image elements corresponding to the second number of similar atlas units belong to the traffic image sample; based on this, step S303, constructing the involved connection line according to the first involved connection line and the second involved connection line, may include:

[0327] Step S3031: If the second number of similar graph units includes the object graph unit D m The involved graph units are similar to the involved graph units, so the construction includes the involved graph units and the object graph units D m The third involved connection line involves the same type of image elements corresponding to the same type of atlas units as the object atlas unit D m the corresponding image elements of the identified object;

[0328] Step S3032: obtaining a second involved regional atlas unit involved with the involved similar atlas unit from the first number of regional atlas units, and constructing a fourth involved connection line including the involved similar atlas unit and the second involved regional atlas unit; the sample image region corresponding to the second involved regional atlas unit includes similar image elements corresponding to the involved similar atlas unit;

[0329] Step S3033: if the second number is greater than 1, and the second number of similar atlas units includes a pair of similar atlas units involved, then construct a fifth involving connection line including the pair of similar atlas units involved; the two similar image elements corresponding to the pair of similar atlas units involved are in the same sample image region;

[0330] Step S3034: determine the first involved connection line, the second involved connection line, the third involved connection line, the fourth involved connection line and the fifth involved connection line as involved connection lines.

[0331] In step S303, the computer system further integrates multiple connection lines including similar graph units based on the constructed first involved connection lines and second involved connection lines to comprehensively capture the complex relationship between objects and between objects and regions in the traffic image. Similar graph units refer to graph units that represent the same type of objects but may have different representation forms (such as different viewing angles, different lighting conditions, etc.).

[0332] In step S3031, the computer system first checks whether there is a corresponding object graph unit D. m If there is such a unit, the system will construct a path from D m A third involving connection line to the involving similar atlas unit.

[0333] Assume that in the traffic image example, in addition to the object graph unit D mIn addition to the image of a car (such as a red car), there are also multiple similar image units, which represent the same car from different perspectives (for example, the front, side, rear, etc.). Although these similar image units have different forms of expression, they are essentially the same type of object. The system compares the features of these units (such as color, shape, size, etc.) with D m characteristics, we can determine which units are related to D m have sufficient similarity to be considered as involving the same type of graph units.

[0334] Once the involved graph units are identified, the system can construct m The third connection line to the unit. This connection line represents D m Potential associations between similar objects. In practical applications, the third-party link may contain a variety of features, such as similarity score, association strength, connection type, etc. These features are helpful for subsequent relationship reasoning and image recognition tasks.

[0335] For example, suppose D m represents the front view of a red car, while the involved similar graph unit represents the side view of the car. The system can construct a third involved connection line `L3`, whose features may include the similarity score between the two views (calculated by comparing color histograms, edge features, etc.) and the connection type (such as "similar object connection").

[0336] In step S3032, after constructing the third involving connection line, the system further checks whether there is a regional atlas unit related to the involving same type of atlas unit (ie, the second involving regional atlas unit), and constructs a corresponding fourth involving connection line.

[0337] The system traverses all regional atlas units and finds areas that are spatially adjacent to or have some functional association with the involved similar atlas units. These areas may be lanes, intersections, parking lots, etc. where the object is located. By comparing the characteristics of the regional atlas units (such as location, shape, size, etc.) with the characteristics of the involved similar atlas units, the system can determine which areas have a direct association with the involved similar atlas units.

[0338] Once the second involved region atlas unit is determined, the system can construct a fourth involved connection line from the involved similar atlas unit to the region. This connection line represents the spatial or functional relationship between the similar object and the region where it is located. Similar to the third involved connection line, the fourth involved connection line may also contain multiple features to describe this relationship. For example, suppose the involved similar atlas unit represents a side view of a car, and the car is located in an intersection area. The system can construct a fourth involved connection line `L 4`, whose features may include the spatial location relationship between cars and intersections (such as distance, direction, etc.) and connection type (such as "object-area connection").

[0339] In step S3033, if there are multiple similar graph units (ie, the second number is greater than 1), and there is a certain specific association between these units (eg, they represent different objects in the same scene), the system needs to construct a fifth involvement connection line to capture this association.

[0340] The system traverses all similar graph units and finds pairs of units that are spatially adjacent, have similar attributes, or participate in the same event (such as a traffic accident, traffic jam, etc.). These unit pairs are considered to involve similar graph unit dyads. By comparing the features of unit pairs and the relationship patterns between them, the system can determine which dyads are related.

[0341] For each pair of related involved homogeneous graph units, the system constructs a fifth involved link to connect them. This link represents the interaction or mutual influence between homogeneous objects. Similar to the previous link, the fifth involved link may also contain multiple features to describe the type and strength of the relationship between objects. For example, suppose there are two adjacent cars in a traffic image (represented by two homogeneous graph units respectively), and they are driving in the same lane. The system can construct a fifth involved link `L 5 ` to connect these two similar graph units, whose characteristics may include their relative position, speed difference, driving direction, etc., as well as the connection type (such as "same object-same object connection").

[0342] In step S3034, after all relevant connection lines are constructed, the system integrates them into a complete network of involved connection lines. This network contains various relationship information between objects and between objects and regions, providing rich context support for subsequent relationship reasoning and image recognition tasks.

[0343] During the integration process, the system uses certain strategies to ensure the consistency and validity of the connection network. These strategies may include deduplication (avoiding the existence of duplicate connections), weight adjustment (adjusting the weight of the connection according to its type and characteristics), directionality processing (ensuring the directionality of the connection is consistent), and loop detection (avoiding the formation of logical loops).

[0344] For example, suppose there is a traffic image example containing a red car (object atlas unit D m ), multiple similar atlas units representing the car under different viewing angles, and multiple regional atlas units (such as lanes, intersections, etc.). By executing each sub-step in step S303, the system can construct the following connecting lines:

[0345] The first connection line: from D m to the intersection area atlas unit that contains it.

[0346] The second involves connecting lines: connecting the spatial relationships between map units in different regions (such as the adjacent relationship between lanes and intersections).

[0347] The third involved connection line: from D m to its side view of the same atlas unit.

[0348] The fourth involves a connection line: from the same type of atlas unit in the side view to the atlas unit in the intersection area.

[0349] The fifth involves connecting lines: connecting pairs of similar graph units that represent different cars (but belong to the same class of objects).

[0350] Ultimately, these connections are integrated into a complex network that fully describes the object relationships, spatial layout, and potential events or scenes in traffic images. This network not only helps with subsequent relationship reasoning and image recognition tasks, but also provides important reference information for decision-making in intelligent transportation systems.

[0351] As an implementation method, the basic image processing network includes a basic image feature extraction component and a basic atlas generation component; based on this, step S200, based on the basic image processing network, constructs atlas unit basic features corresponding to a plurality of atlas units, which may include:

[0352] Step S210: Based on the basic image feature extraction component, the traffic image sample is divided into blocks to obtain a plurality of image blocks, and the plurality of image blocks are quantified respectively to obtain basic image block features corresponding to the plurality of image blocks respectively;

[0353] Step S220: embedding the basic features of multiple image blocks respectively to obtain image block embedding features corresponding to the multiple image blocks respectively, and loading the multiple image block embedding features into the basic atlas generation component;

[0354] Step S230: generating object atlas units corresponding to a plurality of identified objects and region atlas units corresponding to a first number of sample image regions based on the basic atlas generation component;

[0355] Step S240: construct multiple atlas units based on multiple object atlas units and a first number of region atlas units, and construct atlas unit basic features corresponding to the multiple atlas units respectively based on multiple image block embedding features.

[0356] In step S210, the computer system first uses the basic image feature extraction component to perform image segmentation processing on the traffic image sample. Image segmentation is to divide the entire image into multiple smaller areas (i.e., image blocks) so that each area can be processed and analyzed independently. This processing method helps to reduce the amount of calculation, improve processing efficiency, and better capture the local features of the image.

[0357] Image segmentation can be done in many ways, such as uniform grid segmentation, content-based adaptive segmentation, etc. Here, it is assumed that the uniform grid segmentation method is used, that is, the image is divided into multiple rectangular areas of equal size. For example, for a traffic image of size 1920×1080 pixels, it can be divided into multiple image blocks of 128×128 pixels (Note: the aspect ratio and boundary effect of the image may need to be considered during the actual segmentation).

[0358] After the image is divided into blocks, the computer system quantizes each image block. Quantization is the process of mapping the pixel values ​​of an image to a finite number of values ​​in order to reduce the amount of data and facilitate subsequent processing. Common quantization methods include uniform quantization and non-uniform quantization. Here, a simple uniform quantization method can be used, that is, mapping the grayscale value (or color value) of each pixel to a smaller integer range.

[0359] For example, for a grayscale image, assuming that the original pixel value range is [0, 255], it can be quantized to the range of [0, 25], that is, each original pixel value is divided by 10 and rounded down. For a color image, each color channel can be quantized separately.

[0360] The quantized image blocks are converted into a series of numerical features, which constitute the basic features of the image blocks. These features may include grayscale histograms, color moments, texture features, etc., which describe the visual properties of the image blocks.

[0361] In step S220, after obtaining the basic features of the plurality of image blocks, the computer system embeds these features. Feature embedding is the process of mapping the original feature space to a new, for example, low-dimensional feature space. This mapping helps capture the nonlinear relationship between features and reduces the redundancy and noise of features.

[0362] Feature embedding can be implemented in many ways, such as principal component analysis (PCA), linear discriminant analysis (LDA), auto encoder, and embedding layer in deep learning. Here, it is assumed that the embedding layer in deep learning is used to implement feature embedding.

[0363] The embedding layer is, for example, a fully connected layer or a convolutional layer, whose input is the basic feature vector of the image block and whose output is the transformed embedded feature vector. The weights of the embedding layer are learned through the training process, and they can capture the complex relationship between the features of the image blocks.

[0364] For example, assuming that the length of the basic feature vector of each image block is 100 (i.e., it contains 100 quantized pixel values ​​or feature statistics), an embedding layer can be designed to map it to an embedded feature vector of length 50. This embedding layer may contain multiple neurons and activation functions (such as ReLU) to ensure the nonlinearity and expressiveness of the output features.

[0365] After feature embedding is completed, the computer system loads the embedded features of multiple image blocks into the basic atlas generation component. The basic atlas generation component is the key part responsible for generating atlas units based on the embedded features. It may contain a series of graph processing algorithms and data structures (such as graph databases, graph neural networks, etc.) to efficiently manage and operate atlas data.

[0366] In step S230, in the basic atlas generation component, the computer system generates a plurality of atlas units according to the embedded features. These atlas units include two types: object atlas units and region atlas units, which correspond to the identified objects and sample image regions in the traffic image, respectively.

[0367] For each identified object (such as a vehicle, pedestrian, etc.), the computer system will look for feature vectors related to the object in the embedded features, and generate object graph units based on these feature vectors. Object graph units not only contain the visual features of the object (such as shape, color, texture, etc.), but may also contain semantic information of the object (such as category labels, attribute descriptions, etc.). For example, assuming that a red car is identified in the image, the computer system will find the embedded feature vectors related to the car (these vectors may capture features such as the shape outline and color distribution of the car), and generate an object graph unit based on these features. This unit will contain the car's category information (such as "car"), color information (such as "red"), and possible attribute descriptions (such as "four-door sedan", etc.).

[0368] In addition to object atlas units, the computer system also generates region atlas units corresponding to sample image regions. These regions may represent specific scenes or backgrounds in the image (such as intersections, roads, buildings, etc.). Similar to object atlas units, region atlas units also contain visual features and semantic information of the region. For example, there may be an intersection region in a traffic image, which contains elements such as multiple roads and traffic lights. The computer system will generate a region atlas unit based on the embedded features of the region. The unit will contain the category information of the intersection (such as "intersection"), location information (such as coordinate range), and possible contextual descriptions (such as "busy traffic hub", etc.).

[0369] In step S240, after generating a plurality of object atlas units and region atlas units, the computer system needs to construct their basic features according to these units and the corresponding image block embedding features. These basic features will be used in the subsequent relationship reasoning and network training process.

[0370] First, the computer system integrates the object atlas units and the region atlas units to form a complete set of atlas units. This set contains representations of all identified objects and sample image regions in the image. Each atlas unit corresponds to an entity (object or region) in the image and contains visual and semantic information about the entity. Next, the computer system constructs basic features for each atlas unit based on multiple image patch embedding features. These basic features may include statistics directly extracted from the embedded features (such as mean, variance, maximum / minimum values, etc.), low-dimensional representations obtained by clustering or dimensionality reduction methods, or deep features calculated by advanced models such as graph neural networks.

[0371] For example, for an object atlas unit (such as a red car), the computer system can extract all image patch embedding features related to the object and calculate the mean and variance of these features as one of the basic features of the object. In addition, the system may also use graph neural networks to capture the spatial relationships between the object and the surrounding area or other objects, and encode these relationships as part of the basic features of the object.

[0372] Similarly, for regional atlas units (such as intersections), the computer system will also extract the image block embedding features related to the region and calculate the corresponding statistics or deep features to construct the basic features of the region. These features will reflect the visual attributes of the region (such as texture complexity, color distribution, etc.) and spatial layout (such as road direction, traffic flow pattern, etc.).

[0373] As an embodiment, the plurality of identified objects include the identified object K n; n≥1, and not greater than the number of multiple identified objects; Based on this, the method also includes: if the traffic image sample includes an identified object K n Corresponding image elements of the same type R o , then generate the same type of image elements R o Corresponding similar atlas units.

[0374] Based on this, step S240, based on the plurality of object atlas units and the first number of regional atlas units, constructing a plurality of atlas units, including: o Corresponding atlas units of the same type are determined as multiple atlas units.

[0375] In the process of traffic image processing, the computer system first identifies multiple recognized objects in the image (such as vehicles, pedestrians, etc.) and generates a corresponding object map unit for each object. However, sometimes the image may also contain other image elements of the same type as the recognized objects. These elements may not be directly recognized as independent objects due to factors such as occlusion, perspective changes or lighting conditions, but they are still important for understanding the entire scene.

[0376] For example, suppose that in a traffic image, the computer system has identified a red car (recorded as the identified object K 1 ) as an independent object and generate a corresponding object atlas unit for it. At the same time, the image also contains other parts of the red car (such as the rear end, rearview mirror, etc.), which are not directly recognized as independent vehicle objects due to being connected to the body or blocked by the viewing angle, but they still belong to the same type of image elements of the red car.

[0377] To make full use of this information, the computer system checks whether the traffic image sample contains the identified object K n (In this case, K 1 ) of the same image element R o . If such elements are detected, the system will generate corresponding similar atlas units to represent these elements. Similar atlas units are similar to object atlas units in that they contain visual features and semantic information of the image, but the difference between them is that similar atlas units may only represent a part or perspective of the object, rather than a complete independent object. In step S240, the task of the computer system is to construct a complete set of atlas units based on multiple object atlas units, a first number of regional atlas units, and possible similar atlas units. The following is a detailed expansion of this step:

[0378] 1. Collect all types of graph units. First, the computer system needs to collect all types of graph units generated in the previous steps. These units include:

[0379] Object graph unit: represents the identified independent objects in the traffic image, such as vehicles, pedestrians, etc. Each object graph unit contains the visual features (such as shape, color, texture, etc.) and semantic information (such as category labels, attribute descriptions, etc.) of the object.

[0380] Region atlas unit: represents a specific area in a traffic image, such as a lane, intersection, parking lot, etc. The region atlas unit also contains the visual features and semantic information of the region, as well as the spatial relationship between the region and the object.

[0381] Similar atlas units (if present): represent other image elements of the same type as the identified object. These elements may not be recognized as independent objects due to occlusion, viewpoint changes, etc., but they are still important for understanding the entire scene.

[0382] 2. Integrate graph units. After collecting all types of graph units, the computer system needs to integrate them into a unified graph structure. This structure can be a graph database, graph neural network, or other data structure suitable for representing complex relationships. During the integration process, the system needs to ensure that each graph unit can be uniquely identified and the relationship between them can be accurately recorded.

[0383] For example, in a traffic image, if the computer system has identified a red car (object atlas unit) and multiple image elements of the same type as the car (same type atlas units), and the image also contains multiple regions (regional atlas units), then the system needs to integrate these units into a single atlas. In this atlas, the red car object atlas unit will establish a connection relationship with the same type atlas units representing its parts (such as a "part-whole" relationship), and it will also establish a spatial relationship with the region atlas unit where it is located (such as a "located in" relationship).

[0384] 3. Construct basic features of graph units. After integrating all types of graph units, the computer system needs to construct basic features for each unit. These features will be used in subsequent relationship reasoning and network training processes. The construction of basic features may involve a variety of technologies and methods, such as feature extraction, feature dimensionality reduction, feature fusion, etc.

[0385] For object atlas units and similar atlas units, their basic features may include visual features extracted from image blocks (such as color histograms, texture features, shape descriptors, etc.) and deep features obtained through deep learning models (such as convolutional neural network features, graph neural network features, etc.). These features together describe the appearance, shape, texture and other properties of the object as well as their spatial information such as position and size in the image.

[0386] For region atlas units, their basic features may focus more on describing the layout, shape, size, and relationship between regions and objects. These features may include the bounding box coordinates, shape descriptors, and relative positional relationships with surrounding objects.

[0387] For example, suppose that in a traffic image, the computer system has identified a red car (object atlas unit K 1 ) and multiple image elements of the same type as the car (such as the rear of the car, rearview mirror, etc., to generate corresponding similar graph units R o1 ,R o2 ,...). At the same time, the image also contains areas such as lanes and intersections (regional atlas unit R 1 , R 2 ,...).

[0388] In step S240, the computer system first converts K 1 , R o1 , R o2 ,... and R 1 , R o , ... and other units into a graph structure. Then, the system constructs basic features for each unit. For example:

[0389] For K 1 For example, its basic features may include visual features such as color histogram, texture features, shape descriptors extracted from car images, as well as deep features obtained through deep learning models.

[0390] For R o1 For the rear part of the vehicle, its basic features may include visual features such as shape descriptors and color information of the rear area, as well as K 1 The "part-whole" relationship characteristics between them.

[0391] For R 1 For lane areas, the basic features may include the lane’s bounding box coordinates, shape descriptors, and K 1 The spatial position relationship characteristics between them (such as relative distance, direction, etc.).

[0392] Finally, these graph units and their basic features will be used in subsequent relational reasoning and network training processes to achieve more accurate traffic image understanding and analysis.

[0393] By processing the same type of image elements of the identified objects and generating corresponding same type of graph units, the computer system can capture the information in the traffic image more comprehensively. In step S240, the system integrates multiple object graph units, regional graph units and similar graph units into a unified graph structure and constructs basic features for each unit. These graph units and basic features provide important support for subsequent relationship reasoning and network training, which helps to achieve more accurate traffic image understanding and analysis.

[0394] An embodiment of the present invention provides a computer system, such as Figure 2 As shown, the computer system 100 includes: a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, such as through a bus 102. Optionally, the computer system 100 may also include a transceiver 104. It should be noted that in actual applications, the transceiver 104 is not limited to one, and the structure of the computer system 100 does not constitute a limitation on the embodiments of the present invention.

[0395] An embodiment of the present invention provides a computer system. The computer system in the embodiment of the present invention includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more programs are executed by the processor, the above method is implemented.

Claims

1. A data processing method for smart lamp pole equipment based on vehicle-road cloud, characterized in that: include: Acquire an image sample library; the image sample library includes traffic image samples, a plurality of identified objects in the traffic image samples, and identified object tuples containing identified object relationships; Loading the image sample library into a basic image processing network, and constructing basic features of atlas units corresponding to a plurality of atlas units based on the basic image processing network; The plurality of atlas units include area atlas units corresponding to a first number of sample image regions belonging to the traffic image sample, and object atlas units corresponding to the plurality of identified objects respectively; Constructing an involved connection line according to the multiple graph units, and determining a connection line basic feature corresponding to the involved connection line according to a shared debugging parameter of the involved connection line and graph unit basic features respectively corresponding to two graph units at both ends of the involved connection line; Constructing a basic graph including a plurality of basic features of graph units and the basic features of the connecting lines, updating the network parameters of the basic image processing network according to the basic features of the graph units corresponding to each of the object graph units, the identified object tuples and the basic graph, and obtaining an image processing network for identifying object tuples containing the identified object relations in a traffic image; the network parameters of the basic image processing network include the shared debugging parameters; Acquire a traffic image uploaded by a smart light pole device, wherein the traffic image includes a plurality of objects to be identified, and the traffic image is acquired by collecting a target road section based on a data collection device of the smart light pole device and uploading the traffic image based on a network device of the smart light pole device; Loading the traffic image into the image processing network, identifying whether the traffic image contains the object tuple of the identified object relationship; The determining of the connection line basic features corresponding to the involved connection line according to the shared debugging parameters of the involved connection line and the basic features of the graph units corresponding to the two graph units at both ends of the involved connection line respectively includes: Determine the pseudo-standardized connection line feature corresponding to the involved connection line according to the shared debugging parameter of the involved connection line and the basic features of the atlas units respectively corresponding to the two atlas units at both ends of the involved connection line; The quasi-standardized connection line features corresponding to the involved connection lines are standardized to obtain the connection line basic features corresponding to the involved connection lines.

2. The method according to claim 1, characterized in that The determining of the pseudo-standardized connection line features corresponding to the involved connection line according to the shared debugging parameters of the involved connection line and the basic features of the atlas units respectively corresponding to the two atlas units at both ends of the involved connection line comprises: Determine the shared debugging parameter and the proposed processing feature set corresponding to the involved connection line according to the graph unit types corresponding to the two graph units at both ends of the involved connection line; the proposed processing feature set includes the graph unit basic features corresponding to the two graph units at both ends of the involved connection line; the shared debugging parameter includes a shared influence parameter and a shared gain parameter; Based on the shared influencing parameter, feature weighting is performed on the feature set to be processed to obtain an adjusted feature set; The adjustment features in the adjustment feature set are combined to obtain a pseudo gain adjustment feature, and based on the shared gain parameter, the pseudo gain adjustment feature is gain-adjusted to obtain a pseudo-standardized connecting line feature corresponding to the involved connecting line.

3. The method according to claim 2, characterized in that The determining, according to the graph unit types respectively corresponding to the two graph units at both ends of the involved connection line, the shared debugging parameter and the proposed processing feature set corresponding to the involved connection line comprises: If the types of graph units respectively corresponding to the two graph units at both ends of the involved connection line are inconsistent, then the mixed shared debugging parameter is determined as the shared debugging parameter, and the basic features of the graph units respectively corresponding to the two graph units at both ends of the involved connection line are determined as the feature set to be processed; If the types of the graph units respectively corresponding to the two graph units at both ends of the involved connection line are consistent, and the two graph units at both ends of the involved connection line belong to the first number of regional graph units, then the regional shared debugging parameter is determined as the shared debugging parameter, and the basic features of the graph units respectively corresponding to the two graph units at both ends of the involved connection line and the correlation features between the two graph units at both ends of the involved connection line are determined as the feature set to be processed; If the two atlas units at both ends of the involved connecting line respectively correspond to the same atlas unit type, and the two atlas units at both ends of the involved connecting line belong to the same atlas unit set, then the same shared debugging parameter is determined as the shared debugging parameter, and the atlas unit basic features corresponding to the two atlas units at both ends of the involved connecting line, the autocorrelation semantic features of the two atlas units at both ends of the involved connecting line, and the correlation features between the two atlas units at both ends of the involved connecting line are determined as the feature set to be processed; the same type of image elements corresponding to the same type of atlas units in the same type of atlas unit set belong to the traffic image sample.

4. The method according to claim 1, characterized in that: The involved connection line includes a first connection line, and the two atlas units at both ends of the first connection line include a first atlas unit and a second atlas unit; the standardizing of the quasi-standardized connection line features corresponding to the involved connection line to obtain the connection line basic features corresponding to the involved connection line includes: If the atlas unit type corresponding to the first atlas unit is inconsistent with the atlas unit type corresponding to the second atlas unit, then a second connection line is obtained from the involved connection line, wherein if the first atlas unit belongs to a plurality of object atlas units and the second atlas unit belongs to a first number of regional atlas units, then the two atlas units at both ends of the second connection line include the first atlas unit and the third atlas unit; The third atlas unit belongs to the first number of regional atlas units; if the first atlas unit belongs to a set of similar atlas units, then the two atlas units at both ends of the second connecting line include the first atlas unit and the fourth atlas unit, the atlas unit type corresponding to the fourth atlas unit is consistent with the atlas unit type corresponding to the second atlas unit, and the similar image elements corresponding to the similar atlas units in the set of similar atlas units belong to the traffic image sample; Acquire the quasi-standardized connection line features corresponding to the second connection line from the quasi-standardized connection line features corresponding to the involved connection line; Based on the quasi-standardized connecting line features corresponding to the second connecting line, the quasi-standardized features corresponding to the first connecting line are standardized to obtain the connecting line basic features corresponding to the first connecting line.

5. The method according to claim 4, characterized in that The step of normalizing the quasi-standardized features corresponding to the first connecting line based on the quasi-standardized connecting line features corresponding to the second connecting line to obtain the connecting line basic features corresponding to the first connecting line includes: Performing a nonlinear transformation on the quasi-standardized connecting line feature corresponding to the first connecting line to obtain a nonlinear transformation feature corresponding to the first connecting line, and performing a nonlinear transformation on the quasi-standardized connecting line feature corresponding to the second connecting line to obtain a nonlinear transformation feature corresponding to the second connecting line; Performing exponential mapping on the nonlinear transformation feature corresponding to the first connecting line to obtain a mapping feature corresponding to the first connecting line, and performing exponential mapping on the nonlinear transformation feature corresponding to the second connecting line to obtain a mapping feature corresponding to the second connecting line; Adding the mapping features corresponding to the second connecting lines to obtain a merged mapping feature corresponding to the second connecting lines; The ratio of the mapping feature corresponding to the first connecting line to the combined mapping feature corresponding to the second connecting line is used as the connecting line basic feature corresponding to the first connecting line.

6. The method according to claim 1, characterized in that The involved connection line includes a third connection line; the two graph units at both ends of the third connection line include a fifth graph unit and a sixth graph unit; The step of normalizing the quasi-standardized connection line features corresponding to the involved connection lines to obtain the connection line basic features corresponding to the involved connection lines includes: If the atlas unit type corresponding to the fifth atlas unit is consistent with the atlas unit type corresponding to the sixth atlas unit, then a fourth connection line and a fifth connection line are obtained in the involved connection line, wherein, if the fifth atlas unit and the sixth atlas unit both belong to the first number of regional atlas units, then the two atlas units at both ends of the fourth connection line include the fifth atlas unit and the seventh atlas unit, the two atlas units at both ends of the fifth connection line include the sixth atlas unit and the eighth atlas unit, and the seventh atlas unit and the eighth atlas unit both belong to the first number of regional atlas units; if the fifth atlas unit and the sixth atlas unit both belong to a set of similar atlas units, then the two atlas units at both ends of the fourth connection line include the fifth atlas unit and the ninth atlas unit, the two atlas units at both ends of the fifth connection line include the sixth atlas unit and the tenth atlas unit; the ninth atlas unit and the tenth atlas unit both belong to the set of similar atlas units; similar image elements corresponding to similar atlas units in the set of similar atlas units belong to the traffic image sample; From the quasi-standardized connection line features corresponding to the involved connection lines, obtain the quasi-standardized connection line features corresponding to the fourth connection line and the quasi-standardized connection line features corresponding to the fifth connection line; Based on the quasi-standardized connecting line feature corresponding to the fourth connecting line, the quasi-standardized feature corresponding to the third connecting line is standardized to obtain the first connecting line basic feature corresponding to the third connecting line; Based on the quasi-standardized connecting line feature corresponding to the fifth connecting line, the quasi-standardized feature corresponding to the third connecting line is standardized to obtain the second connecting line basic feature corresponding to the third connecting line; The first connecting line basic feature and the second connecting line basic feature are determined as connecting line basic features corresponding to the third connecting line.

7. The method according to claim 1, characterized in that The updating of the network parameters of the basic image processing network according to the basic features of the atlas units corresponding to each of the two object atlas units, the identified object tuples and the basic atlas, so as to obtain an image processing network for identifying the object tuples containing the identified object relationship in the traffic image, comprises: Based on the basic features of the graph units corresponding to the two object graph units, construct the reasoning relationship corresponding to the two object graph units; Based on the identified object binary, construct a priori connection lines corresponding to the pairwise object graph units; Based on the inference relationship corresponding to the pairwise object graph units, the prior connection lines corresponding to the pairwise object graph units and the basic graph, the network parameters of the basic image processing network are updated to obtain an image processing network; the image processing network is used to identify object tuples containing the identified object relationship in the traffic image.

8. The method according to claim 7, characterized in that The basic image processing network includes a basic relational reasoning component; the two-to-two object graph units include a first object graph unit and a second object graph unit; The constructing the reasoning relationship corresponding to the pairwise object graph units based on the graph unit basic features corresponding to the pairwise object graph units respectively includes: Based on the basic relationship reasoning component, combining the basic features of the graph unit corresponding to the first object graph unit and the basic features of the graph unit corresponding to the second object graph unit to obtain a combined feature; Performing feature weighting on the combined features to obtain quasi-nonlinear transformation features, performing nonlinear transformation on the quasi-nonlinear transformation features to obtain an inference relationship corresponding to the first object graph unit and the second object graph unit; The two-by-two object graph units include a first object graph unit and a second object graph unit; The step of constructing the prior connection lines corresponding to the two-by-two object graph units based on the identified object binary groups includes: Determine the object graph units corresponding to the two identified objects in the identified object binary group as a first object graph unit binary group; Constructing a second object graph unit tuple including the first object graph unit and the second object graph unit; If the first object graph unit binary contains the same object graph unit binary as the second object graph unit binary, then determining the positive sample prior connection line as the prior connection line corresponding to the second object graph unit binary; If the first object graph unit binary does not include an object graph unit binary that is the same as the second object graph unit binary, then determining the negative sample prior connection line as the prior connection line corresponding to the second object graph unit binary; The updating of the network parameters of the basic image processing network based on the reasoning relationship corresponding to the pairwise object graph units, the priori connection lines corresponding to the pairwise object graph units and the basic graph to obtain the image processing network includes: Determining a network training cost according to the reasoning relationship corresponding to the pairwise object graph units and the prior connection lines corresponding to the pairwise object graph units; According to the network training cost and the basic graph, the network parameters of the basic image processing network are updated to obtain an image processing network.

9. A computer system, characterized in that: include: one or more processors; Memory; one or more computer programs; The one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, the method according to any one of claims 1 to 8 is implemented.

Images