A target matching method based on instance segmentation

Abstract

The embodiment of the present application relates to a kind of target matching methods based on instance segmentation, the method comprises:1, obtain first image;2, first image is carried out pedestrian target color distribution feature extraction processing;3, obtain second image;4, second image is carried out pedestrian target color distribution feature extraction processing;5, according to current time, previous time feature tensor carries out pedestrian target matching processing;And for each first matching target group corresponding two pedestrian targets of previous and next time establish target association;6, to previous time feature tensor carries out unvanishing target feature vector update and has disappeared target feature vector and obtains once updated feature tensor, to once updated feature tensor carries out new target feature vector and adds and obtains secondary update feature tensor;7, secondary update feature tensor is used as new previous time feature tensor and is transferred to 3. By the present application, the feature difference of pedestrian target can be increased, and the accuracy of pedestrian target matching can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a target matching method based on instance segmentation. BACKGROUND

[0002] In an automatic driving system, a perception module can perform pedestrian target matching processing on perception data continuously output by a visual sensor, i.e., a camera, i.e., a visual image captured by the camera, to obtain tracking trajectories of each pedestrian target. In a conventional case, the typical processing steps of the perception module for pedestrian target matching are as follows: first, performing pedestrian target detection processing on a visual image at a current time to obtain a plurality of pedestrian target bounding boxes (bounding boxes, bboxes) at the current time; second, performing one-step prediction on each pedestrian target bounding box bbox at a previous time based on a Kalman filter to obtain a corresponding predicted pedestrian target bounding box bbox; third, calculating an intersection-over-union of each pedestrian target bounding box bbox and each predicted pedestrian target bounding box bbox to obtain a corresponding intersection-over-union matrix; fourth, performing matching degree calculation based on a Hungarian algorithm according to the intersection-over-union matrix to obtain a corresponding matching degree matrix; fifth, finding a predicted pedestrian target bounding box bbox with the maximum matching degree for each pedestrian target bounding box bbox based on the matching degree matrix; sixth, finding a pedestrian target bounding box bbox with the maximum matching degree for each predicted pedestrian target bounding box bbox based on the matching degree matrix; seventh, regarding a pedestrian target bounding box bbox and a pedestrian target bounding box bbox with the maximum matching degree greater than a preset matching degree threshold as a pair of mutually matched bounding box matching pairs; eighth, regarding a pair of pedestrian target bounding boxes bbox and bbox of each bounding box matching pair as two target detection results of the same pedestrian target at the previous and current times, and creating a corresponding target association relationship for the two bboxes and bboxes; and ninth, performing target matching processing on the perception data based on the target association relationship. now pre * now now * now now * now now pre pre now pre now pre now

[0003] ​​​​​​​​​​​​​​​However, through practice, we have found that this conventional approach often results in target matching errors when there are many pedestrians. The main reason is that for pedestrian targets that are close together, the shapes and sizes of their corresponding bounding boxes are similar. If the traditional method uses the intersection-union ratio (IUU) of the pedestrian bounding boxes from different time steps as the matching condition, mismatches are likely to occur. For example, if the bounding box corresponding to pedestrian A is detected in the previous time step... pre,1 If, at the current moment, passerby B walks out from behind passerby A, and passerby B's height and build are similar to passerby A's, then the two bounding boxes corresponding to passerby A and passerby B will be detected. now,1 and bbox now,2 In practice, we have found that the perception model based on conventional methods may encounter the following two types of target matching errors: 1) It assumes that the bounding box... now,1 bbox now,2 and bbox pre,1 Both are matching targets, and for bbox now,1 -bbox pre,1 and bbox now,2 -bbox pre,1 Both form a target association; 2) It is believed that bbox now,2 and bbox pre,1 It matches the target and is for the bbox. now,2 -bbox pre,1 Form a target association.

[0004] To reduce the likelihood of the perception module making similar target matching errors, it is necessary to select other features that better reflect the differences between pedestrian targets as matching conditions during the target matching process, instead of using the intersection-union ratio of pedestrian target detection boxes as the matching condition in the conventional approach. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a target matching method, electronic device, and computer-readable storage medium based on instance segmentation. This invention, when performing target detection on visual images, also performs instance segmentation on each pedestrian target and extracts color distribution feature vectors from each pedestrian target instance. Target matching is then performed using the color distribution feature vectors of each pedestrian target instance at different time points as matching conditions, and target association relationships are established for the pedestrian target bounding boxes at different time points based on the target matching results. Furthermore, this invention selects a color distribution feature vector that better reflects the differences between pedestrian targets as the matching condition for target matching. This processing method increases the feature differences of pedestrian targets and improves the accuracy of pedestrian target matching, thereby reducing the probability of target matching errors in the perception module.

[0006] To achieve the above objectives, a first aspect of the present invention provides a target matching method based on instance segmentation, the method comprising:

[0007] Step 1: Acquire the first image at the initial time;

[0008] Step 2: Extract the pedestrian target color distribution features from the first image to obtain the earliest previous time-series feature tensor;

[0009] Step 3: Acquire the second image at the next moment;

[0010] Step 4: Extract pedestrian target color distribution features from the second image to obtain the corresponding pedestrian feature tensor at the current time.

[0011] Step 5: Perform pedestrian target matching processing based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups; and establish target association relationships for the pedestrian targets at two previous and next times corresponding to each first matching target group.

[0012] Step 6: Based on all the first matching target groups, update the feature vectors of the previous time step by updating the feature vectors of the undisappeared targets and delete the feature vectors of the disappeared targets to obtain the corresponding first-time updated feature tensor; and based on all the first matching target groups and the pedestrian feature tensor at the current time step, add new target feature vectors to the first-time updated feature tensor to obtain the corresponding second-time updated feature tensor.

[0013] Step 7: Use the second-updated feature tensor as the new feature tensor of the previous time step; and proceed to step 3 to continue the loop processing for the next time step.

[0014] Preferably, the shape of the feature tensor at the previous time step is A×N. 3 A represents the number of pedestrian targets at the previous moment, and N represents the number of preset feature intervals, where 1 ≤ A and 1 ≤ N; the feature tensor at the previous moment includes A vectors of length N. 3 The pedestrian feature vector F at the previous moment i where i is the pedestrian target index at the previous time step, and 1≤i≤A;

[0015] The shape of the pedestrian feature tensor at the current moment is B×N. 3 B represents the number of pedestrian targets at the current moment, 1 ≤ B; the pedestrian feature tensor at the current moment includes B vectors of length N. 3 The pedestrian feature vector F at the current time j j is the pedestrian target index at the current time, 1≤j≤B;

[0016] The first matching target group includes a pedestrian feature vector F at the current time.j and the pedestrian feature vector F from the previous time step i .

[0017] Preferably, the pedestrian target color distribution feature extraction process specifically includes:

[0018] The first or second image input this time is taken as the corresponding current image;

[0019] The current image is processed by pedestrian target detection and pedestrian instance segmentation to obtain a semantic segmentation map with multiple pedestrian target detection boxes; the detection target type corresponding to each pedestrian target detection box is a pedestrian target type; the semantic segmentation map includes multiple first pixels; each first pixel includes a set of three-channel RGB pixel values ​​and a pixel semantic type; the three-channel RGB pixel values ​​include R channel pixel values, G channel pixel values ​​and B channel pixel values; the pixel semantic type includes pedestrian semantic type and multiple other non-pedestrian semantic types;

[0020] The image regions covered by each pedestrian target detection box on the semantic segmentation map are extracted as the corresponding pedestrian semantic segmentation maps;

[0021] Color distribution feature vector extraction is performed on each of the pedestrian semantic segmentation maps to obtain the corresponding first feature vector; the length of the first feature vector is N. 3 ;

[0022] The number of pedestrian target detection boxes in the semantic segmentation map is counted to obtain the corresponding first number;

[0023] If the current image is the first image, then the corresponding number of pedestrian targets A at the previous moment is set to the first number, and each of the first feature vectors is used as the corresponding pedestrian feature vector F at the previous moment. i The vector length of the obtained number of pedestrian targets A at the previous moment is N. 3 The pedestrian feature vector F of the previous moment i The earliest output of the feature tensor from the previous time step;

[0024] If the current image is the second image, then the corresponding number of pedestrian targets B at the current time is set to the first number, and each of the first feature vectors is used as the corresponding pedestrian feature vector F at the current time. j The vector length of the current pedestrian target number B is N. 3 The pedestrian feature vector F at the current time j The corresponding pedestrian feature tensor output is formed at the current time.

[0025] Furthermore, the step of extracting color distribution feature vectors from each of the pedestrian semantic segmentation maps to obtain the corresponding first feature vector specifically includes:

[0026] A three-dimensional RGB grid space, denoted as the first grid space, is constructed based on the preset value ranges of the R, G, and B channels and the number of feature intervals N. The value ranges of the X, Y, and Z axes of the first grid space correspond to the value ranges of the R, G, and B channels, respectively. The X-axis of the first grid space is divided into N X-axis feature intervals, each corresponding to a feature interval index x_id, 1 ≤ x_id ≤ N. The Y-axis of the first grid space is divided into N Y-axis feature intervals, each corresponding to a feature interval index y_id, 1 ≤ y_id ≤ N. The Z-axis of the first grid space is divided into N Z-axis feature intervals, each corresponding to a feature interval index z_id, 1 ≤ z_id ≤ N. The first grid space consists of N... 3 The first grid g x_id,y_id,z_id constitute;

[0027] For each of the first grids g in the first grid space x_id,y_id,z_id Assign a corresponding first grid counter; and initialize the count value of each first grid counter to a preset counter initial value;

[0028] The first pixel in the pedestrian semantic segmentation map whose pixel semantic type is pedestrian semantic type is recorded as the corresponding first pedestrian pixel.

[0029] The process iterates through each of the first pedestrian pixels. During the iteration, the currently iterated first pedestrian pixel is recorded as the corresponding current pixel. The R-channel pixel value, G-channel pixel value, and B-channel pixel value of the current pixel are extracted and used as the corresponding X, Y, and Z-axis coordinates x1, y1, and z1 to form the corresponding first point coordinates (x1, y1, z1). The first grid g containing the first point coordinates (x1, y1, z1) in the first grid space is then defined. x_id,y_id,z_id As the corresponding current grid; and increment the count value of the first grid counter corresponding to the current grid by 1;

[0030] At the end of the traversal, N corresponding to the first grid space 3 The count values ​​of the first grid counters form a sequence of length N. 3 The count value vector is denoted as the corresponding first count value vector; and the first count value vector is normalized to obtain the corresponding vector of length N. 3 The normalized count value vector is generated, and the normalized count value vector is output as the corresponding first feature vector.

[0031] Preferably, the step of performing pedestrian target matching processing based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups specifically includes:

[0032] For each of the current time pedestrian feature vectors F in the current time pedestrian feature tensor j and the previous time step pedestrian feature vector F of the previous time step feature tensor i The first cosine distance d is obtained by calculating the cosine distance. j,i ,

[0033] From the pedestrian feature vector F at each of the current times j The first cosine distance d corresponding to the number of pedestrian targets A at the previous moment j,i The maximum value is selected as the corresponding first maximum cosine distance d. j,max ;

[0034] For each of the pedestrian feature vectors F at the current time j The corresponding first maximum cosine distance d j,max The system identifies pedestrians by determining whether their distance exceeds a preset cosine distance threshold; if so, it identifies them by the pedestrian feature vector F at the current time. j and the first maximum cosine distance d j,max The corresponding pedestrian feature vector F from the previous moment i Form a corresponding first matching target group;

[0035] Output all the first matching target groups obtained.

[0036] Preferably, establishing the target association relationship for pedestrian targets at two consecutive time points corresponding to each of the first matching target groups specifically includes:

[0037] The pedestrian feature vector F from the first matching target group at the previous moment i The corresponding pedestrian target detection box is denoted as the corresponding pedestrian target of the previous time step, and the pedestrian feature vector F of the first matching target group at the current time step is... j The corresponding pedestrian target detection box is used as the corresponding pedestrian target at the current time, and a common target association relationship is created between the pedestrian target at the previous time and the pedestrian target at the current time.

[0038] Preferably, the step of updating the feature vectors of undisappeared targets and deleting the feature vectors of disappeared targets based on all the first matching target groups to obtain the corresponding updated feature tensor specifically includes:

[0039] The pedestrian feature vector F from the first matching target group at the previous time step i The index value of the pedestrian target index i at the previous moment is extracted to form the corresponding first index value sequence;

[0040] In the previous time-phase feature tensor, the previous time-phase pedestrian feature vector F whose index value of the previous time-phase pedestrian target index i does not satisfy the first index value sequence is included. i Let be the corresponding disappeared target feature vector, and let the index value of the pedestrian target index i at the previous time satisfy the first index value sequence of the pedestrian feature vector F at the previous time. i Let be the corresponding feature vector of the undisappeared target;

[0041] The process iterates through each of the first matching target groups. During the iteration, the currently traversed first matching target group is taken as the corresponding current matching target group, and the pedestrian feature vector F from the previous time step in the feature tensor of the previous time step is used to match the current matching target group. i The corresponding non-disappearing target feature vector is used as the corresponding current non-disappearing target feature vector, and the pedestrian feature vector F of the current matching target group at the previous moment is used. i and the pedestrian feature vector F at the current time j The extracted values ​​are used as the corresponding state variables of the previous time step and the observations of the current time step; and based on a preset state filter, the state variable of the current time step is predicted according to the state variables of the previous time step and the observations of the current time step to obtain the corresponding state variable of length N. 3 The current state quantity is obtained; and the current non-disappearing target feature vector in the feature tensor of the previous time step is updated to the corresponding current state quantity; the state filter includes a Kalman filter and an exponential moving average filter;

[0042] At the end of the traversal, the pedestrian feature vector F corresponding to all the disappeared target feature vectors in the previous time step feature tensor is obtained. i Delete; and extract the remaining pedestrian feature vector F from the previous time step in the previous time step feature tensor. i The number of pedestrian targets A in the previous moment is statistically analyzed to obtain the corresponding second number, and the number of pedestrian targets A in the previous moment feature tensor is updated to the corresponding second number.

[0043] The feature tensor of the previous time step, which completes the update of the feature vector of the target that has not disappeared and the deletion of the feature vector of the target that has disappeared, is output as the corresponding updated feature tensor.

[0044] Preferably, the step of adding new target feature vectors to the first-time updated feature tensor based on all the first matching target groups and the pedestrian feature tensor at the current time to obtain the corresponding second-time updated feature tensor specifically includes:

[0045] The pedestrian feature vector F at the current time in the first matching target group j The index value of the pedestrian target index j at the current time is extracted to form the corresponding second index value sequence;

[0046] The pedestrian feature vector F at the current time whose index value of the pedestrian target index j in the current time pedestrian feature tensor does not satisfy the second index value sequence is defined as follows: j Let denot be the corresponding newly added target feature vector; and let each of the newly added target feature vectors be a newly added pedestrian feature vector F from the previous time step. i Add to the updated feature tensor;

[0047] For each update of the feature tensor, all the pedestrian feature vectors F from the previous time step are updated. i The number of pedestrian targets A at the previous moment is statistically analyzed to obtain the corresponding third number, and the number of pedestrian targets A in the first updated feature tensor is updated to the corresponding third number.

[0048] The updated feature tensor of the first update is output as the corresponding updated feature tensor of the second update.

[0049] A second aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0050] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0051] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0052] A third aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0053] This invention provides a target matching method based on instance segmentation, an electronic device, and a computer-readable storage medium. When detecting targets in a visual image, this invention performs instance segmentation on each pedestrian target and extracts color distribution feature vectors from each pedestrian target instance. Target matching is then performed using the color distribution feature vectors of each pedestrian target instance at different time points as matching conditions. Based on the target matching results, target association relationships are established between the pedestrian target bounding boxes at different time points. Furthermore, this invention selects a color distribution feature vector that better reflects the differences between pedestrian targets as the matching condition for target matching. This processing method increases the feature differences of pedestrian targets, improves the accuracy of pedestrian target matching, and reduces the probability of target matching errors in the perception module. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a target matching method based on instance segmentation provided in Embodiment 1 of the present invention;

[0055] Figure 2 This is a schematic diagram of the structure of an electronic device provided in Embodiment 2 of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0057] Embodiment 1 of the present invention provides a target matching method based on instance segmentation, such as Figure 1 The schematic diagram of a target matching method based on instance segmentation provided in Embodiment 1 of the present invention is shown. The method mainly includes the following steps:

[0058] Step 1: Acquire the first image at the start time.

[0059] In this embodiment of the invention, the perception module of the autonomous driving system continuously obtains multiple frames of visual images from the perception sensor, i.e., the camera. The first frame of visual image obtained at the beginning is the first image.

[0060] Step 2: Extract the pedestrian target color distribution features from the first image to obtain the earliest previous time-series feature tensor;

[0061] The shape of the feature tensor at the previous time step is A×N. 3A represents the number of pedestrian targets in the previous time step, and N represents the number of preset feature intervals, where 1 ≤ A and 1 ≤ N. The feature tensor in the previous time step includes A vectors of length N. 3 The pedestrian feature vector F at the previous moment i where i is the pedestrian target index at the previous time step, and 1≤i≤A;

[0062] Specifically, this includes: Step 21, taking the first image input this time as the corresponding current image;

[0063] Step 22: Perform pedestrian target detection and pedestrian instance segmentation on the current image to obtain a semantic segmentation map with multiple pedestrian target detection boxes;

[0064] Among them, the detection target type corresponding to each pedestrian target detection box is the pedestrian target type; the semantic segmentation map includes multiple first pixels; each first pixel includes a set of three-channel RGB pixel values ​​and a pixel semantic type; the three-channel RGB pixel values ​​include R channel pixel values, G channel pixel values ​​and B channel pixel values; the pixel semantic type includes pedestrian semantic type and multiple other non-pedestrian semantic types;

[0065] Here, in this embodiment of the invention, when performing pedestrian target detection and pedestrian instance segmentation on the current image, it can be implemented based on an end-to-end target detection and semantic segmentation model, such as the Mask R-CNN model. Processing with this type of model can directly obtain a semantic segmentation map with multiple pedestrian target detection boxes. Alternatively, it can be implemented by combining two independent target detection models and semantic segmentation models, such as using the YOLO series models for target detection and the Unet model for semantic segmentation. Based on this implementation, multiple pedestrian target detection boxes and a semantic segmentation map will be obtained respectively. In this case, the position and size of each pedestrian target detection box can be determined on the semantic segmentation map based on the positional correspondence between each pedestrian target detection box and the original image (i.e., the current image) and the image ratio between the semantic segmentation map and the original image (i.e., the current image). As can be seen from the output structure of various publicly available models with target detection functions, the second image is sent to... After the model is built, multiple object detection boxes can be obtained, and each object detection box corresponds to a detection object type. In this embodiment of the invention, the object detection box with the detection object type of pedestrian object is extracted as the corresponding pedestrian object detection box. As can be seen from the output structure of various public models with semantic segmentation function, after the second image is fed into the model, a semantic segmentation map can be obtained. Each pixel point on the semantic segmentation map, i.e., the first pixel point, has a semantic feature in addition to an RGB pixel feature, i.e., a three-channel RGB pixel value, as well as a semantic feature, i.e., a pixel semantic type. This pixel semantic type includes pedestrian semantic type and several other non-pedestrian semantic types. Common non-pedestrian semantic types include car semantic type, building semantic type, animal semantic type, background semantic type, etc.

[0066] Step 23: Extract the image region covered by each pedestrian target detection box on the semantic segmentation map as the corresponding pedestrian semantic segmentation map;

[0067] Here, the current step is actually to perform image matting on the image region covered by each pedestrian target detection box on the semantic segmentation map;

[0068] Step 24: Extract the color distribution feature vector from each pedestrian semantic segmentation image to obtain the corresponding first feature vector;

[0069] Wherein, the length of the first eigenvector is N. 3 ;

[0070] Specifically, it includes: Step 241, constructing a three-dimensional RGB grid space based on the preset R, G, B channel value range and the number of feature intervals N, denoted as the corresponding first grid space;

[0071] Among them, the value ranges of the X, Y, and Z axes in the first grid space correspond to the value ranges of the R, G, and B channels, respectively;

[0072] The X-axis of the first grid space is divided into N X-axis feature intervals, and each X-axis feature interval corresponds to a feature interval index x_id, 1≤x_id≤N;

[0073] The Y-axis of the first grid space is divided into N Y-axis feature intervals, and each Y-axis feature interval corresponds to a feature interval index y_id, 1≤y_id≤N;

[0074] The Z-axis of the first grid space is divided into N Z-axis feature intervals, and each Z-axis feature interval corresponds to a feature interval index z_id, 1≤z_id≤N;

[0075] The first grid space consists of N 3 The first grid g x_id,y_id,z_id constitute;

[0076] Here, the R, G, and B channel value ranges are the three preset primary color pixel value ranges. The R channel value range is the range of red pixel values, the G channel value range is the range of green pixel values, and the B channel value range is the range of blue pixel values. The number of feature intervals N is a preset positive integer that can be set based on actual application requirements. In this embodiment of the invention, when dividing the grid in the first grid space, the feature interval lengths of the X, Y, and Z axes are used as the corresponding three-dimensional side lengths of the grid for grid division. Since the X, Y, and Z axes of the first grid space are divided into N feature intervals, the first grid g... x_id,y_id,z_id The number is N*N*N=N 3 The first grid space here is actually a visual color distribution space, and each first grid g x_id,y_id,z_id For a small color distribution region in this visual color distribution space, the larger the value of the number of feature intervals N, the greater the density of the color distribution region in this visual color distribution space.

[0077] Step 242, for each of the first grids g in the first grid space x_id,y_id,z_id Assign a corresponding first grid counter; and initialize the count value of each first grid counter to a preset initial counter value;

[0078] Here, the default initial value of the counter is 0, but it can be set to other values ​​based on actual application requirements;

[0079] Step 243: Record the first pixel in the pedestrian semantic segmentation map whose pixel semantic type is pedestrian semantic type as the corresponding first pedestrian pixel.

[0080] Here, since the embodiments of the present invention only need to extract the color distribution feature vector for each pedestrian target instance, the pixels that are only related to the pedestrian target instance, namely the first pedestrian pixel, need to be marked in advance.

[0081] Step 244: Traverse each first pedestrian pixel; during traversal, record the currently traversed first pedestrian pixel as the corresponding current pixel; extract the R channel pixel value, G channel pixel value, and B channel pixel value of the current pixel as the corresponding X, Y, and Z axis coordinate values ​​x1, y1, and z1 to form the corresponding first point coordinates (x1, y1, z1); and set the first grid g containing the first point coordinates (x1, y1, z1) in the first grid space. x_id,y_id,z_id As the corresponding current grid; and increment the counter value of the first grid corresponding to the current grid by 1;

[0082] Here, it can be seen from the current step that each first grid counter is used to track the occurrences of each first grid g. x_id,y_id,z_id The number of pixels representing the first person in the area is counted.

[0083] Step 245, at the end of the traversal, N corresponding to the first grid space 3 The count values ​​of the first grid counters form a sequence of length N. 3 The count value vector is denoted as the corresponding first count value vector; and the first count value vector is normalized to obtain the corresponding vector of length N. 3 The normalized count value vector is generated, and the normalized count value vector is output as the corresponding first feature vector.

[0084] Here, the first count vector obtained in the current step is the color distribution feature vector of a pedestrian target instance. Since the visual color of each pedestrian target is different, the color distribution feature vector of each pedestrian target, i.e., the first count vector, will not be similar or the same. Furthermore, the larger the value of the number of feature intervals N, the greater the difference between the color distribution feature vectors of each pedestrian target, i.e., the first count vector. Normalizing the first count vector is a common vector processing method. There are many common normalization methods, such as normalization based on a preset standard maximum value, normalization based on the maximum value within the vector, normalization based on the mean, etc. This embodiment of the invention does not limit the specific implementation method of the normalization process.

[0085] Step 25: Count the number of pedestrian target detection boxes in the semantic segmentation map to obtain the corresponding first count;

[0086] Here, the first number obtained in the current step is actually the number of pedestrian targets in the current image;

[0087] Step 26: If the current image is the first image, then set the number of pedestrian targets A in the previous time step as the first number, and use each first feature vector as the corresponding pedestrian feature vector F in the previous time step. i The vector length of the obtained number of pedestrian targets A at the previous moment is N. 3 The pedestrian feature vector F at the previous moment i The earliest feature tensor output from the previous time step is formed.

[0088] Step 3: Acquire the second image at the next moment.

[0089] Here, the second image and the first image in step 1 above are both visual images output by the same camera.

[0090] Step 4: Extract pedestrian target color distribution features from the second image to obtain the corresponding pedestrian feature tensor at the current time.

[0091] The shape of the pedestrian feature tensor at the current time is B×N. 3 B represents the number of pedestrian targets at the current time, 1 ≤ B; the pedestrian feature tensor at the current time consists of B vectors of length N. 3 The pedestrian feature vector F at the current time j j is the pedestrian target index at the current time, 1≤j≤B;

[0092] Here, the current step of extracting pedestrian target color distribution features from the second image is the same as the previous step 2 of extracting pedestrian target color distribution features from the first image.

[0093] Specifically, this includes: Step 41, taking the second image input this time as the corresponding current image;

[0094] Step 42: Perform pedestrian target detection and pedestrian instance segmentation on the current image to obtain a semantic segmentation map with multiple pedestrian target detection boxes;

[0095] Among them, the detection target type corresponding to each pedestrian target detection box is the pedestrian target type; the semantic segmentation map includes multiple first pixels; each first pixel includes a set of three-channel RGB pixel values ​​and a pixel semantic type; the three-channel RGB pixel values ​​include R channel pixel values, G channel pixel values ​​and B channel pixel values; the pixel semantic type includes pedestrian semantic type and multiple other non-pedestrian semantic types;

[0096] Here, the current step is processed in the same way as step 22 mentioned above, and will not be repeated here;

[0097] Step 43: Extract the image region covered by each pedestrian target detection box on the semantic segmentation map as the corresponding pedestrian semantic segmentation map;

[0098] Here, the current step is processed in the same way as step 23 mentioned above, and will not be repeated here;

[0099] Step 44: Extract the color distribution feature vector from each pedestrian semantic segmentation image to obtain the corresponding first feature vector;

[0100] Wherein, the length of the first eigenvector is N. 3 ;

[0101] Here, the current step is processed in the same way as step 24 mentioned above;

[0102] Specifically, it includes: Step 441, constructing a three-dimensional RGB grid space based on the preset R, G, B channel value range and the number of feature intervals N, denoted as the corresponding first grid space;

[0103] In this first grid space, the X, Y, and Z axes correspond to the R, G, and B channel ranges, respectively. The X-axis is divided into N equal X-axis feature intervals, each with an index x_id (1 ≤ x_id ≤ N). The Y-axis is divided into N equal Y-axis feature intervals, each with an index y_id (1 ≤ y_id ≤ N). The Z-axis is divided into N equal Z-axis feature intervals, each with an index z_id (1 ≤ z_id ≤ N). 3 The first grid g x_id,y_id,z_id constitute;

[0104] Here, the current step is processed in the same way as step 241 mentioned above, and will not be repeated here;

[0105] Step 442, for each of the first grids g in the first grid space x_id,y_id,z_id Assign a corresponding first grid counter; and initialize the count value of each first grid counter to a preset initial counter value;

[0106] Here, the current step is processed in the same way as step 242 mentioned above, and will not be repeated here;

[0107] Step 443: Record the first pixel in the pedestrian semantic segmentation map whose pixel semantic type is pedestrian semantic type as the corresponding first pedestrian pixel.

[0108] Here, the current step is processed in the same way as step 243 mentioned above, and will not be repeated here;

[0109] Step 444: Traverse each first pedestrian pixel; during traversal, record the currently traversed first pedestrian pixel as the corresponding current pixel; extract the R channel pixel value, G channel pixel value, and B channel pixel value of the current pixel as the corresponding X, Y, and Z axis coordinate values ​​x1, y1, and z1 to form the corresponding first point coordinates (x1, y1, z1); and set the first grid g containing the first point coordinates (x1, y1, z1) in the first grid space. x_id,y_id,z_id As the corresponding current grid; and increment the counter value of the first grid corresponding to the current grid by 1;

[0110] Here, the current step is processed in the same way as step 244 mentioned above, and will not be repeated here;

[0111] Step 445, at the end of the traversal, N corresponding to the first grid space 3 The count values ​​of the first grid counters form a sequence of length N. 3 The count value vector is denoted as the corresponding first count value vector; and the first count value vector is normalized to obtain the corresponding vector of length N. 3 The normalized count value vector is generated, and the normalized count value vector is output as the corresponding first feature vector.

[0112] Here, the current step is processed in the same way as step 245 mentioned above, and will not be repeated here;

[0113] Step 45: Count the number of pedestrian target detection boxes in the semantic segmentation map to obtain the corresponding first count;

[0114] Here, the current step is processed in the same way as step 25 mentioned above, and will not be repeated here;

[0115] Step 46: If the current image is the second image, then set the number of pedestrian targets B at the current time as the first number, and use each first feature vector as the corresponding pedestrian feature vector F at the current time. j The vector length of the current pedestrian target number B is N. 3 The pedestrian feature vector F at the current time j The corresponding pedestrian feature tensor output is formed at the current moment.

[0116] Step 5: Perform pedestrian target matching processing based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups; and establish target association relationships for the pedestrian targets at the two previous and next time times corresponding to each first matching target group.

[0117] Specifically, it includes: Step 51, performing pedestrian target matching processing based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups;

[0118] Here, the current step is to perform target matching processing using the color distribution feature vectors of each pedestrian target instance at previous and subsequent time points as matching conditions.

[0119] Specifically, this includes: Step 511, processing each current-time pedestrian feature vector F of the current-time pedestrian feature tensor. j The pedestrian feature vectors F from the previous time step and the feature tensor from the previous time step. i The first cosine distance d is obtained by calculating the cosine distance. j,i ,

[0120] Step 512, from the pedestrian feature vector F at each current time... j The first cosine distance d corresponding to the number of pedestrian targets A at the previous moment j,i The maximum value is selected as the corresponding first maximum cosine distance d. j,max ;

[0121] Step 513, for each current time pedestrian feature vector F j The corresponding first maximum cosine distance d j,max The system identifies pedestrians based on whether their distance exceeds a preset cosine distance threshold; if so, it identifies them based on the pedestrian feature vector F at the current time. j The first maximum cosine distance d j,max The corresponding pedestrian feature vector F from the previous moment i Form a corresponding first matching target group;

[0122] Here, the embodiments of the present invention specify that not every first maximum cosine distance d j,max The corresponding pedestrian feature vector F from the previous moment i Both can be compared with the pedestrian feature vector F at the current time. j A first matching target group is formed only if the first maximum cosine distance d is reached. j,max Only after the distance is sufficiently large, exceeding the preset cosine distance threshold, can this first maximum cosine distance d be... j,max The corresponding pedestrian feature vector F from the previous moment i and the pedestrian feature vector F at the current time j Form a corresponding first matching target group; the cosine distance threshold here is a pre-set large cosine distance value, which can be set according to actual application requirements;

[0123] Step 514: Output all the first matching target groups obtained;

[0124] Step 52: Establish target association relationships for pedestrian targets at two consecutive time points corresponding to each first matching target group;

[0125] Specifically, this includes: matching the pedestrian feature vector F from the first matching target group at the previous moment... i The corresponding pedestrian target detection box is denoted as the corresponding pedestrian target at the previous time step, and the pedestrian feature vector F in the first matching target group at the current time step is... j The corresponding pedestrian target detection box is used as the corresponding pedestrian target at the current time, and a common target association relationship is created between the pedestrian target at the previous time and the pedestrian target at the current time.

[0126] Step 6: Based on all the first matching target groups, update the feature vectors of the previous time step by updating the feature vectors of the undisappeared targets and delete the feature vectors of the disappeared targets to obtain the corresponding first-time updated feature tensor; and based on all the first matching target groups and the pedestrian feature tensor at the current time step, add new target feature vectors to the first-time updated feature tensor to obtain the corresponding second-time updated feature tensor.

[0127] Specifically, it includes: Step 61, based on all the first matching target groups, performing update of the feature vectors of the undisappeared target and deletion of the feature vectors of the disappeared target to obtain the corresponding updated feature tensor;

[0128] Specifically, this includes: Step 611, which involves processing the pedestrian feature vectors F from the previous time step in all the first matching target groups. i The index value of the pedestrian target index i in the previous moment is extracted to form the corresponding first index value sequence;

[0129] Step 612: In the feature tensor of the previous time step, select the pedestrian feature vector F from the previous time step whose index value of the pedestrian target index i does not satisfy the first index value sequence. i Let F be the corresponding vanished target feature vector, and let F be the pedestrian feature vector F whose index value of the pedestrian target index i in the previous time step satisfies the first index value sequence. i Let be the corresponding feature vector of the undisappeared target;

[0130] Step 613: Traverse each first matching target group; during traversal, take the currently traversed first matching target group as the corresponding current matching target group, and take the pedestrian feature vector F from the previous time step in the feature tensor of the previous time step that matches the current matching target group. i The corresponding feature vector of the non-disappearing target is used as the corresponding current feature vector of the non-disappearing target, and the pedestrian feature vector F of the current matching target group at the previous time step is used. i and the pedestrian feature vector F at the current time j The values ​​are extracted as the corresponding state variables of the previous time step and the observations of the current time step; and based on a preset state filter, the state variable of the current time step is predicted according to the state variables of the previous time step and the observations of the current time step to obtain the corresponding state variable of length N. 3The current state quantity is determined; and the current non-disappearing target feature vector in the feature tensor of the previous time step is updated to the corresponding current state quantity.

[0131] Among them, the state filters include Kalman filters and exponential moving average filters;

[0132] Here, in this embodiment of the invention, the state filter is used to predict the state quantity at the current time based on the state quantity at the previous time and the observation at the current time in order to correct the error of the observation at the current time;

[0133] If the state filter is a Kalman filter, this embodiment of the invention will pre-set a Kalman filter equation set for predicting the normalized color distribution feature vector, i.e., the pedestrian feature vector. As can be seen from the known Kalman filter principle, the Kalman filter equation set consists of state transition equations and state-observation transition equations. Given the state quantity at the previous time step and the observation at the current time step, the state quantity at the previous time step and the observation at the current time step are substituted into the Kalman filter equation set and the equation set is solved according to the known Kalman filter equation set solution method to obtain the state quantity at the current time step, i.e., the observation at the current time step. The observation at the current time step is the corrected pedestrian feature vector at the current time step. Then, based on the state quantity at the current time step, the current non-disappearing target feature vector in the feature tensor at the previous time step is updated to reduce the observation error of the non-disappearing target feature vector.

[0134] If the state filter is an exponential moving average filter, this embodiment of the invention will pre-set an exponential moving average filter equation for predicting the normalized color distribution feature vector, i.e., the pedestrian feature vector, similar to: S t =α*Q t +(1-α)*S t-1 Where α is the preset filter index, S t-1 Q is the state variable from the previous time step. t For the current observation, S t The current state quantity is given. Given the state quantity of the previous time step and the observation quantity of the current time step, the current state quantity of the previous time step and the observation quantity of the current time step can be obtained by substituting them into the exponential moving average filter equation. The current observation quantity of the current time step is the corrected pedestrian feature vector of the current time step. Then, the observation error of the non-disappearing target feature vector in the feature tensor of the previous time step can be reduced by updating the current non-disappearing target feature vector based on the current state quantity of the current time step.

[0135] Step 614: When the traversal ends, extract the pedestrian feature vectors F from the previous time step corresponding to all the disappeared target feature vectors in the feature tensor of the previous time step. i Delete; and extract the remaining pedestrian feature vectors F from the previous time step feature tensor. iThe number of pedestrian targets A in the feature tensor of the previous time step is statistically analyzed to obtain the corresponding second number, and the number of pedestrian targets A in the feature tensor of the previous time step ...

[0136] Step 615: Output the feature tensor of the previous moment before the completion of updating the feature vector of the undisappeared target and deleting the feature vector of the disappeared target as the corresponding updated feature tensor.

[0137] Step 62: Based on all the first matching target groups and the pedestrian feature tensor at the current time, add new target feature vectors to the first update feature tensor to obtain the corresponding second update feature tensor;

[0138] Specifically, this includes: Step 621, which involves generating pedestrian feature vectors F from all the first matching target groups at the current time. j The index value of the pedestrian target index j at the current moment is extracted to form the corresponding second index value sequence;

[0139] Step 622: Extract the current pedestrian feature vector F from the current pedestrian feature tensor where the index value of the current pedestrian target index j does not satisfy the second index value sequence. j Let F be the corresponding newly added target feature vector; and let each newly added target feature vector be a newly added pedestrian feature vector F from the previous time step. i Add to the feature tensor in a single update;

[0140] Step 623: Update all pedestrian feature vectors F from the previous time step in the feature tensor. i The number of pedestrian targets A at the previous moment before updating the feature tensor is statistically analyzed to obtain the corresponding third number, and the number of pedestrian targets A at the previous moment is updated to the corresponding third number.

[0141] Step 624: Output the updated feature tensor as the corresponding updated feature tensor after the first update.

[0142] Step 7: Use the second-updated feature tensor as the new feature tensor of the previous time step; and return to step 3 to continue the loop processing for the next time step.

[0143] Figure 2 This is a schematic diagram of an electronic device provided in Embodiment 2 of the present invention. This electronic device can be the aforementioned terminal device or server, or it can be a terminal device or server connected to the aforementioned terminal device or server that implements the method of the embodiments of the present invention. Figure 2As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing method embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0144] exist Figure 2 The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The symbol is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0145] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0146] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0147] This invention also provides a chip for executing instructions, which is used to perform the processing steps described in the foregoing method embodiments.

[0148] This invention provides a target matching method based on instance segmentation, an electronic device, and a computer-readable storage medium. When detecting targets in a visual image, this invention performs instance segmentation on each pedestrian target and extracts color distribution feature vectors from each pedestrian target instance. Target matching is then performed using the color distribution feature vectors of each pedestrian target instance at different time points as matching conditions. Based on the target matching results, target association relationships are established between the pedestrian target bounding boxes at different time points. Furthermore, this invention selects a color distribution feature vector that better reflects the differences between pedestrian targets as the matching condition for target matching. This processing method increases the feature differences of pedestrian targets, improves the accuracy of pedestrian target matching, and reduces the probability of target matching errors in the perception module.

[0149] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0150] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0151] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target matching method based on instance segmentation, characterized in that, The method includes: Step 1: Acquire the first image at the initial time; Step 2: Extract the pedestrian target color distribution features from the first image to obtain the earliest previous time-series feature tensor; Step 3: Acquire the second image at the next moment; Step 4: Extract pedestrian target color distribution features from the second image to obtain the corresponding pedestrian feature tensor at the current time. Step 5: Perform pedestrian target matching processing based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups; and establish target association relationships for the pedestrian targets at two previous and next times corresponding to each first matching target group. Step 6: Based on all the first matching target groups, update the feature vectors of the previous time step by updating the feature vectors of the undisappeared targets and delete the feature vectors of the disappeared targets to obtain the corresponding first-time updated feature tensor; and based on all the first matching target groups and the pedestrian feature tensor at the current time step, add new target feature vectors to the first-time updated feature tensor to obtain the corresponding second-time updated feature tensor. Step 7: Use the second-updated feature tensor as the new feature tensor of the previous time step; and return to step 3 to continue the loop processing for the next time step; The shape of the feature tensor at the previous time step is A×N. 3 A represents the number of pedestrian targets at the previous moment, and N represents the number of preset feature intervals, where 1 ≤ A and 1 ≤ N; the feature tensor at the previous moment includes A vectors of length N. 3 The pedestrian feature vector F at the previous moment i where i is the pedestrian target index at the previous time step, and 1≤i≤A; The shape of the pedestrian feature tensor at the current moment is B×N. 3 B represents the number of pedestrian targets at the current moment, 1 ≤ B; the pedestrian feature tensor at the current moment includes B vectors of length N. 3 The pedestrian feature vector F at the current time j j is the pedestrian target index at the current time, 1≤j≤B; The first matching target group includes a pedestrian feature vector F at the current time. j and the pedestrian feature vector F from the previous time step i ; The pedestrian target color distribution feature extraction process specifically includes: The first or second image input this time is taken as the corresponding current image; The current image is processed by pedestrian target detection and pedestrian instance segmentation to obtain a semantic segmentation map with multiple pedestrian target detection boxes; the detection target type corresponding to each pedestrian target detection box is a pedestrian target type; the semantic segmentation map includes multiple first pixels; each first pixel includes a set of three-channel RGB pixel values ​​and a pixel semantic type; the three-channel RGB pixel values ​​include R channel pixel values, G channel pixel values ​​and B channel pixel values; the pixel semantic type includes pedestrian semantic type and multiple other non-pedestrian semantic types; The image regions covered by each pedestrian target detection box on the semantic segmentation map are extracted as the corresponding pedestrian semantic segmentation maps; Color distribution feature vector extraction is performed on each of the pedestrian semantic segmentation maps to obtain the corresponding first feature vector; the length of the first feature vector is N. 3 ; The number of pedestrian target detection boxes in the semantic segmentation map is counted to obtain the corresponding first number; If the current image is the first image, then the corresponding number of pedestrian targets A at the previous moment is set to the first number, and each of the first feature vectors is used as the corresponding pedestrian feature vector F at the previous moment. i The vector length of the obtained number of pedestrian targets A at the previous moment is N. 3 The pedestrian feature vector F of the previous moment i The earliest output of the feature tensor from the previous time step; If the current image is the second image, then the corresponding number of pedestrian targets B at the current time is set to the first number, and each of the first feature vectors is used as the corresponding pedestrian feature vector F at the current time. j The vector length of the current pedestrian target number B is N. 3 The pedestrian feature vector F at the current time j The corresponding pedestrian feature tensor output is formed at the current time.

2. The target matching method based on instance segmentation according to claim 1, characterized in that, The step of extracting color distribution feature vectors from each of the pedestrian semantic segmentation maps to obtain the corresponding first feature vector specifically includes: A three-dimensional RGB grid space, denoted as the first grid space, is constructed based on the preset value ranges of the R, G, and B channels and the number of feature intervals N. The value ranges of the X, Y, and Z axes of the first grid space correspond to the value ranges of the R, G, and B channels, respectively. The X-axis of the first grid space is divided into N X-axis feature intervals, each corresponding to a feature interval index x_id, 1 ≤ x_id ≤ N. The Y-axis of the first grid space is divided into N Y-axis feature intervals, each corresponding to a feature interval index y_id, 1 ≤ y_id ≤ N. The Z-axis of the first grid space is divided into N Z-axis feature intervals, each corresponding to a feature interval index z_id, 1 ≤ z_id ≤ N. The first grid space consists of N... 3 The first grid g x_id,y_id,z_id constitute; For each of the first grids g in the first grid space x_id,y_id,z_id Assign a corresponding first grid counter; and initialize the count value of each first grid counter to a preset counter initial value; The first pixel in the pedestrian semantic segmentation map whose pixel semantic type is pedestrian semantic type is recorded as the corresponding first pedestrian pixel. The process iterates through each of the first pedestrian pixels. During the iteration, the currently iterated first pedestrian pixel is recorded as the corresponding current pixel. The R-channel pixel value, G-channel pixel value, and B-channel pixel value of the current pixel are extracted and used as the corresponding X, Y, and Z-axis coordinates x1, y1, and z1 to form the corresponding first point coordinates (x1, y1, z1). The first grid g containing the first point coordinates (x1, y1, z1) in the first grid space is then defined. x_id,y_id,z_id As the corresponding current grid; and increment the count value of the first grid counter corresponding to the current grid by 1; At the end of the traversal, N corresponding to the first grid space 3 The count values ​​of the first grid counters form a sequence of length N. 3 The count value vector is denoted as the corresponding first count value vector; and the first count value vector is normalized to obtain the corresponding vector of length N. 3 The normalized count value vector is generated, and the normalized count value vector is output as the corresponding first feature vector.

3. The target matching method based on instance segmentation according to claim 1, characterized in that, The process of performing pedestrian target matching based on the pedestrian feature tensor at the current time and the feature tensor at the previous time to obtain multiple first matching target groups specifically includes: For each of the current time pedestrian feature vectors F in the current time pedestrian feature tensor j and the previous time step pedestrian feature vector F of the previous time step feature tensor i The first cosine distance d is obtained by calculating the cosine distance. j,i , From the pedestrian feature vector F at each of the current times j The first cosine distance d corresponding to the number of pedestrian targets A at the previous moment j,i The maximum value is selected as the corresponding first maximum cosine distance d. j,max ; For each of the pedestrian feature vectors F at the current time j The corresponding first maximum cosine distance d j,max The system identifies pedestrians by determining whether their distance exceeds a preset cosine distance threshold; if so, it identifies them by the pedestrian feature vector F at the current time. j and the first maximum cosine distance d j,max The corresponding pedestrian feature vector F from the previous moment i Form a corresponding first matching target group; Output all the first matching target groups obtained.

4. The target matching method based on instance segmentation according to claim 1, characterized in that, The step of establishing target association relationships for pedestrian targets at two consecutive time points corresponding to each of the first matching target groups specifically includes: The pedestrian feature vector F from the first matching target group at the previous moment i The corresponding pedestrian target detection box is denoted as the corresponding pedestrian target of the previous time step, and the pedestrian feature vector F of the first matching target group at the current time step is... j The corresponding pedestrian target detection box is used as the corresponding pedestrian target at the current time, and a common target association relationship is created between the pedestrian target at the previous time and the pedestrian target at the current time.

5. The target matching method based on instance segmentation according to claim 1, characterized in that, The step of updating the feature vectors of undisappeared targets and deleting the feature vectors of disappeared targets based on all the first matching target groups to obtain the corresponding updated feature tensor specifically includes: The pedestrian feature vector F from the first matching target group at the previous time step i The index value of the pedestrian target index i at the previous moment is extracted to form the corresponding first index value sequence; In the previous time-phase feature tensor, the previous time-phase pedestrian feature vector F whose index value of the previous time-phase pedestrian target index i does not satisfy the first index value sequence is included. i Let be the corresponding disappeared target feature vector, and let the index value of the pedestrian target index i at the previous time satisfy the first index value sequence of the pedestrian feature vector F at the previous time. i Let be the corresponding feature vector of the undisappeared target; The process iterates through each of the first matching target groups. During the iteration, the currently traversed first matching target group is taken as the corresponding current matching target group, and the pedestrian feature vector F from the previous time step in the feature tensor of the previous time step is used to match the current matching target group. i The corresponding non-disappearing target feature vector is used as the corresponding current non-disappearing target feature vector, and the pedestrian feature vector F of the current matching target group at the previous moment is used. i and the pedestrian feature vector F at the current time j The extracted values ​​are used as the corresponding state variables of the previous time step and the observations of the current time step; and based on a preset state filter, the state variable of the current time step is predicted according to the state variables of the previous time step and the observations of the current time step to obtain the corresponding state variable of length N. 3 The current state quantity is obtained; and the current non-disappearing target feature vector in the feature tensor of the previous time step is updated to the corresponding current state quantity; the state filter includes a Kalman filter and an exponential moving average filter; At the end of the traversal, the pedestrian feature vector F corresponding to all the disappeared target feature vectors in the previous time step feature tensor is obtained. i Delete; and extract the remaining pedestrian feature vector F from the previous time step in the previous time step feature tensor. i The number of pedestrian targets A in the previous moment is statistically analyzed to obtain the corresponding second number, and the number of pedestrian targets A in the previous moment feature tensor is updated to the corresponding second number. The feature tensor of the previous time step, which completes the update of the feature vector of the target that has not disappeared and the deletion of the feature vector of the target that has disappeared, is output as the corresponding updated feature tensor.

6. The target matching method based on instance segmentation according to claim 1, characterized in that, The step of adding new target feature vectors to the first-time updated feature tensor based on all the first matching target groups and the pedestrian feature tensor at the current time to obtain the corresponding second-time updated feature tensor specifically includes: The pedestrian feature vector F at the current time in the first matching target group j The index value of the pedestrian target index j at the current time is extracted to form the corresponding second index value sequence; The pedestrian feature vector F at the current time whose index value of the pedestrian target index j in the current time pedestrian feature tensor does not satisfy the second index value sequence is defined as follows: j Let denot be the corresponding newly added target feature vector; and let each of the newly added target feature vectors be a newly added pedestrian feature vector F from the previous time step. i Add to the updated feature tensor; For each update of the feature tensor, all the pedestrian feature vectors F from the previous time step are updated. i The number of pedestrian targets A at the previous moment is statistically analyzed to obtain the corresponding third number, and the number of pedestrian targets A in the first updated feature tensor is updated to the corresponding third number. The updated feature tensor of the first update is output as the corresponding updated feature tensor of the second update.

7. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-6; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-6.

Images