Trajectory tracking segmentation method and device, electronic equipment and storage medium
By using the method of combining features and entropy right values in video image target tracking, the lack of deep learning models in target feature details and background factors are solved, and the accuracy of trajectory tracking is improved.
Patent Information
- Application Number
- CN202411279501.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2025-01-24
AI Technical Summary
The existing deep learning tracking model has errors in the details extraction of target features, and does not consider the background factors of the video image, resulting in errors in tracking of the action trajectory.
By obtaining the initial tracing image of the monitoring system, the initial feature diagram is obtained by convolution, and the target pool feature diagram and score pooling feature diagram is obtained according to the preset score scores. Calculate the similarity scores and entropy power values, make the fusion update of the feature diagram, divide the useless background information in the initial feature diagram, and improve the extraction accuracy of the information to be tracked.
It reduces the interference of the use of information extraction of useless background information, and improves the accuracy of trajectory tracking.
Smart Images
Figure CN119359760A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of video image target tracking, and in particular to a trajectory tracking and segmentation method, device, electronic device and storage medium. Background Art
[0002] Video image target tracking and segmentation aims to extract the target features of the target to be tracked in the video image through the segmentation algorithm, and then use it to track the target's action trajectory in the video. As a key step in the field of computer vision, video image target tracking and segmentation lays the foundation for high-level computer vision tasks. Video image target tracking and segmentation can be widely used in the detection and tracking of multiple targets in videos and the detection of diseased bodies in medicine. There are three types of methods for video target tracking: classical tracking algorithms, tracking algorithms based on kernel correlation filtering (KCF), and tracking algorithms based on deep learning. Classic tracking algorithms mainly model the target or search for target features. The commonly used method is feature matching. After extracting the target features, similar features are found in subsequent frames and located. Common target features include: scale-invariant feature transform (SIFT), speeded up robust features (SURF), Harris corners, etc. With the deepening of research, the method based on target model modeling processes the entire image, which has poor real-time performance. Therefore, the prediction algorithm is added to the tracking, and the target search is performed near the predicted value, which reduces the search range. The correlation filter in the communication field is introduced into target tracking. The correlation filter is used to measure the similarity of two signals. Some tracking algorithms based on correlation filtering have also emerged, such as MOSSE, CSK, KCF, BACF, SAMF, etc., which can reach hundreds of frames per second and can be widely used in real-time tracking systems. There are many trackers with excellent tracking performance. With the rapid development of deep learning, we began to consider applying correlation filtering to target tracking. We use deep learning to establish a new tracking framework for target tracking. In the context of big data, the convolution feature output obtained by training the network model with deep learning has stronger expression ability. In target tracking, the initial application method is to directly apply the features learned by the network to the tracking framework of correlation filtering or Struck to obtain better tracking results.
[0003] However, many current deep learning tracking models have certain errors in extracting detail of target features and do not consider background factors of video images, resulting in errors in tracking of motion trajectories. Summary of the invention
[0004] The present invention provides a trajectory tracking and segmentation method, device, electronic device and storage medium to solve the above-mentioned technical problem of trajectory tracking errors caused by background factors.
[0005] In one embodiment of the present application, the present application provides a trajectory tracking and segmentation method, including: obtaining an initial image to be tracked of a monitoring system, and convolving the initial image to be tracked to obtain an initial feature map; pooling the initial image to be tracked along different color channels according to multiple preset proportion scores to obtain a target pooling feature map and multiple score pooling feature maps; respectively calculating the proportion of multiple similarity score values in the sum of all similarity score values to obtain multiple proportion score values, each of which is used to characterize the similarity score of each score pooling feature map and the initial feature map; performing entropy value calculation on each of the proportion score values to obtain multiple entropy weights, and fusing and updating the initial feature map according to the target pooling feature map and each of the entropy weights to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
[0006] In one embodiment of the present application, the initial feature map is fused and updated according to the target pooling feature map and each of the entropy weights to obtain a segmentation feature map, including: multiplying each of the entropy weights with the target pooling feature map respectively, and accumulating to obtain a second feature map; adjusting the size of the second feature map; multiplying the initial feature map and the adjusted second feature map to obtain a third feature map; and determining the sum of the third feature map and the initial feature map as the segmentation feature map.
[0007] In one embodiment of the present application, entropy calculation is performed on each of the proportion score values to obtain multiple entropy weights, including: entropy calculation is performed on each of the proportion score values to obtain the redundancy margin of each of the proportion score values; and the entropy weight corresponding to each of the proportion score values is determined based on each of the redundancy margins and each of the proportion score values.
[0008] In one embodiment of the present application, the method for determining the entropy weight includes:
[0009]
[0010] Among them, W i is the entropy weight of the i-th proportional pooling channel, I i is the preset unit value of the i-th ratio pooling channel, P i is the proportional score value of the i-th proportional pooling channel, m is the total number of proportional pooling channels, and there is a corresponding relationship between the proportional pooling channel and the preset proportional score.
[0011] In one embodiment of the present application, action trajectory tracking is performed based on the position information to be tracked in the segmentation feature map, including: comparing the segmentation feature map with the convolution feature maps of multiple next-frame images to be tracked for similarity, to obtain multiple changed position information of the position information to be tracked; and generating an action trajectory based on the position information to be tracked and all changed position information.
[0012] In one embodiment of the present application, the initial image to be tracked is pooled along different color channels according to multiple preset proportion scores to obtain a target pooling feature map and multiple score pooling feature maps, including: performing maximum pooling on the initial image to be tracked along different color channels to obtain multiple initial pooling feature maps; averaging all target pooling feature maps to obtain a target pooling feature map; multiplying the target pooling feature map by each of the preset proportion scores to obtain score pooling feature maps of multiple proportion pooling channels.
[0013] In one embodiment of the present application, the method for determining the similarity score value includes: multiplying a fractional pooling feature map by the initial feature map to obtain a first feature map; dividing the first feature map by the feature dimension corresponding to the initial feature map to obtain a similarity score value.
[0014] In one embodiment of the present application, the present application provides a trajectory tracking and segmentation device, including: an acquisition module, used to acquire an initial image to be tracked of a monitoring system, and convolve the initial image to be tracked to obtain an initial feature map; an image pooling module, used to pool the initial image to be tracked along different color channels according to multiple preset proportion scores, to obtain a target pooling feature map and multiple score pooling feature maps; a similarity determination module, used to respectively calculate the proportion of multiple similarity score values in the sum of all similarity score values, to obtain multiple proportion score values, each of the similarity score values is used to characterize the similarity score of each score pooling feature map and the initial feature map; an entropy weighted pooling segmentation module, used to perform entropy value calculation on each of the proportion score values, to obtain multiple entropy weights, and to fuse and update the initial feature map according to the target pooling feature map and each of the entropy weights, to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
[0015] In one embodiment of the present application, the present application provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the trajectory tracking and segmentation method as described in any of the above embodiments.
[0016] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the trajectory tracking and segmentation method described in any one of the above embodiments.
[0017] Beneficial effects of embodiments of the present invention: The present invention provides a trajectory tracking and segmentation method, device, electronic device and storage medium. The embodiments of the present invention obtain a target pooling feature map and multiple score pooling feature maps through pooling, obtain multiple proportional score values according to the similarity scores of each score pooling feature map and the initial feature map, and fuse and update the initial feature map according to the target pooling feature map and the entropy weights corresponding to each proportional score value, thereby segmenting useless background information in the initial feature map, reducing the interference of useless background information on the extraction of position information to be tracked, thereby improving the extraction accuracy of the position information to be tracked, and improving the accuracy of trajectory tracking.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0020] Figure 1 A schematic diagram of a process of a trajectory tracking and segmentation method according to an embodiment of the present application is shown;
[0021] Figure 2 A flow chart of a method for entropy weight pooling tracking segmentation according to an embodiment of the present application is shown;
[0022] Figure 3 A block diagram of a trajectory tracking and segmentation device according to an embodiment of the present application is shown;
[0023] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION
[0024] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0025] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0026] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.
[0027] In related technologies, many deep learning tracking models have certain errors in extracting details of target features and do not consider background factors of video images, resulting in errors in tracking action trajectories.
[0028] In order to solve the above technical problems, the present application provides a trajectory tracking and segmentation method, device, electronic device and storage medium. The implementation details of the technical solution of the embodiment of the present application are elaborated in detail below.
[0029] See also Figure 1 , Figure 1 FIG. 2 shows a flow chart of a trajectory tracking and segmentation method according to an embodiment of the present application. Figure 1 As shown, in an exemplary embodiment, the trajectory tracking and segmentation method includes at least steps S110 to S140, which are described in detail as follows:
[0030] Step S110, obtaining an initial image to be tracked of the monitoring system, and performing convolution on the initial image to be tracked to obtain an initial feature map.
[0031] In one embodiment of the present application, the monitoring system includes a road monitoring system. A camera in the road monitoring system is used to capture and record real-time videos of vehicles and pedestrians on the road.
[0032] In one embodiment of the present application, multiple video frame images to be tracked are extracted frame by frame from the surveillance video, and an initial video frame image to be tracked is determined as the initial image to be tracked.
[0033] In one embodiment of the present application, the initial image to be tracked is used to represent an initial frame image in which a target vehicle and a target pedestrian exist.
[0034] In one embodiment of the present application, the video frame image to be tracked is in red-green-blue (RGB) format.
[0035] Step S120, pooling the initial image to be tracked along different color channels according to a plurality of preset proportion scores to obtain a target pooling feature map and a plurality of score pooling feature maps.
[0036] In one embodiment of the present application, the initial image to be tracked is pooled along different color channels according to multiple preset proportion scores to obtain a target pooling feature map and multiple score pooling feature maps, including: performing maximum pooling on the initial image to be tracked along different color channels to obtain multiple initial pooling feature maps; averaging all target pooling feature maps to obtain a target pooling feature map; multiplying the target pooling feature map by each preset proportion score respectively to obtain score pooling feature maps of multiple proportion pooling channels.
[0037] In one embodiment of the present application, the color channel is used to represent the color channel of the initial image to be tracked.
[0038] In one embodiment of the present application, performing maximum pooling on the initial image to be tracked along different color channels includes: performing maximum pooling on the initial image to be tracked along a red channel, a green channel, and a blue channel, respectively.
[0039] In one embodiment of the present application, the pooling vectors corresponding to all fractional pooling feature maps are shown as follows:
[0040] R={R1,R2,…,R m}, R i =α i A formula (1)
[0041] Among them, R is the pooling vector, i1 is the fractional pooling feature map of the first ratio pooling channel, R2 is the fractional pooling feature map of the second ratio pooling channel, R3 is the fractional pooling feature map of the third ratio pooling channel, R m is the fractional pooling feature map of the mth ratio pooling channel, R i is the fractional pooling feature map of the i-th ratio pooling channel, α i is the preset proportion score of the i-th proportion pooling channel, and A is the target pooling feature map.
[0042] In one embodiment of the present application, the size of A is C×1×1, where C is the number of color channels.
[0043] In one embodiment of the present application, the total number of proportional pooling channels may be 10.
[0044] Step S130, respectively calculating the proportion of multiple similarity score values in the sum of all similarity score values to obtain multiple proportional score values, each similarity score value is used to represent the similarity score between each fractional pooling feature map and the initial feature map.
[0045] In one embodiment of the present application, a method for determining a similarity score value includes: multiplying a fractional pooling feature map by an initial feature map to obtain a first feature map; and dividing the first feature map by a feature dimension corresponding to the initial feature map to obtain a similarity score value.
[0046] In one embodiment of the present application, the similarity score value is as follows:
[0047]
[0048] Among them, S i is the similarity score of the i-th ratio pooling channel, Q is the initial feature map, d k is the feature dimension of the initial feature map, R i is the fractional pooling feature map of the i-th ratio pooling channel.
[0049] In one embodiment of the present application, the proportional score value is as follows:
[0050]
[0051] Among them, P i is the proportional score value of the i-th proportional pooling channel, S i is the similarity score value of the i-th proportional pooling channel, and m is the total number of proportional pooling channels.
[0052] In one embodiment of the present application, the sum of all similarity score values is obtained by pixel-by-pixel summation.
[0053] Step S140, entropy values are calculated for each proportional score value to obtain multiple entropy weights, and the initial feature map is fused and updated according to the target pooling feature map and each entropy weight to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
[0054] In one embodiment of the present application, entropy calculation is performed on each proportional score value to obtain multiple entropy weights, including: entropy calculation is performed on each proportional score value to obtain the redundancy margin of each proportional score value; and the entropy weight corresponding to each proportional score value is determined according to each redundancy margin and each proportional score value.
[0055] In one embodiment of the present application, the entropy weight is determined by:
[0056]
[0057] Among them, W i is the entropy weight of the i-th proportional pooling channel, I i is the preset unit value of the i-th ratio pooling channel, P iis the proportional score value of the i-th proportional pooling channel, m is the total number of proportional pooling channels, and there is a corresponding relationship between the proportional pooling channels and the preset proportional scores.
[0058] In one embodiment of the present application, the initial feature map is fused and updated according to the target pooling feature map and each entropy weight to obtain a segmentation feature map, including: multiplying each entropy weight with the target pooling feature map respectively, and accumulating to obtain a second feature map; adjusting the size of the second feature map; multiplying the initial feature map and the adjusted second feature map to obtain a third feature map; and determining the sum of the third feature map and the initial feature map as the segmentation feature map.
[0059] In one embodiment of the present application, the segmentation feature map is shown as follows:
[0060]
[0061] Among them, O is the segmentation feature map, A is the target pooling feature map, Q is the initial feature map, and W i is the entropy weight of the i-th proportional pooling channel, and m is the total number of proportional pooling channels.
[0062] In one embodiment of the present application, action trajectory tracking is performed based on the position information to be tracked in the segmentation feature map, including: comparing the segmentation feature map with the convolution feature maps of multiple next-frame images to be tracked for similarity, to obtain multiple changed position information of the position information to be tracked; generating an action trajectory based on the position information to be tracked and all the changed position information.
[0063] In one embodiment of the present application, the convolution feature map is obtained based on convolution of the next frame image to be tracked.
[0064] In one embodiment of the present application, see Figure 2 , Figure 2 FIG. 4 is a flow chart of a method for tracking and segmenting entropy weight pooling according to an embodiment of the present application. Figure 2As shown, step S210, the initial frame image of the road monitoring video: the monitoring video in the road monitoring system is frame-by-frame extracted with multiple video frames to be tracked, and the initial frame image recording the target vehicle and the target pedestrian is obtained; step S220, after pooling at different proportions, the similarity score is calculated with the original feature map: the initial image to be tracked is pooled along different color channels according to multiple preset proportion scores to obtain a target pooled feature map and multiple score pooled feature maps, and the proportion of multiple similarity score values in the sum of all similarity score values is calculated respectively to obtain multiple proportion score values, and each similarity score value is used to characterize the similarity score of each score pooled feature map with the initial feature map; step S230, the entropy weight pool is obtained by calculation The pooled feature map is multiplied with the original feature map and then added to record the position information of vehicles and pedestrians: the entropy value of each proportional score is calculated to obtain multiple entropy weights, and the initial feature map is fused and updated according to the target pooled feature map and each entropy weight to obtain a segmentation feature map, and the position information to be tracked is obtained according to the segmentation feature map; step S240, the initial frame feature map and the next frame image are analyzed for similarity to record the target position change: the segmentation feature map is compared with the convolution feature maps of multiple next frames of images to be tracked in turn for similarity to obtain multiple changed position information; step S250, finally obtaining the movement trajectory of vehicles and pedestrians: the action trajectory of the target vehicle and the target pedestrian is generated according to the position information to be tracked and all the changed position information.
[0065] See also Figure 3 , Figure 3 A block diagram of a trajectory tracking and segmentation device according to an embodiment of the present application is shown. The device can be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.
[0066] like Figure 3 As shown, a trajectory tracking and segmentation device 300 according to an embodiment of the present application includes: an acquisition module 301, an image pooling module 302, a similarity determination module 303 and an entropy weight pooling segmentation module 304.
[0067] The acquisition module 301 is used to acquire an initial image to be tracked of the monitoring system, and perform convolution on the initial image to be tracked to obtain an initial feature map;
[0068] An image pooling module 302 is used to pool the initial image to be tracked along different color channels according to a plurality of preset proportion scores to obtain a target pooling feature map and a plurality of score pooling feature maps;
[0069] A similarity determination module 303 is used to calculate the proportion of multiple similarity score values in the sum of all similarity score values, and obtain multiple proportional score values, each similarity score value is used to represent the similarity score of each fractional pooling feature map and the initial feature map;
[0070] The entropy weighted pooling segmentation module 304 is used to calculate the entropy value of each proportional score value to obtain multiple entropy weights, and to fuse and update the initial feature map according to the target pooling feature map and each entropy weight to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
[0071] It should be noted that the trajectory tracking and segmentation device provided in the above embodiment and the trajectory tracking and segmentation method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the trajectory tracking and segmentation device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.
[0072] An embodiment of the present application also provides an electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the trajectory tracking and segmentation method provided in the above-mentioned embodiments.
[0073] See also Figure 4 , Figure 4 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 4 The computer system 400 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0074] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, ROM 402 and RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0075] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.
[0076] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.
[0077] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0078] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0079] The units involved in the embodiments described in the present application can be implemented by software or by hardware, and the units described can also be set in a processor. Among them, the names of these units do not constitute a limitation on the units themselves under certain circumstances. Therefore, the technical solution according to the implementation mode of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation mode of the present application.
[0080] Another aspect of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the trajectory tracking and segmentation methods provided in the above-mentioned embodiments. The computer-readable storage medium may be included in the electronic device described in the above-mentioned embodiments, or may exist independently without being assembled into the electronic device.
[0081] In the above embodiments, unless otherwise specified, by using serial numbers such as "first" and "second" to describe common objects, it only means that they refer to different instances of the same object, rather than indicating that the objects being described must adopt a given order, whether in time, space, sorting or any other way.
[0082] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A trajectory tracking and segmentation method, characterized in that: The method comprises: Acquire an initial image to be tracked of the monitoring system, and perform convolution on the initial image to be tracked to obtain an initial feature map; Pooling the initial image to be tracked along different color channels according to a plurality of preset proportion scores to obtain a target pooling feature map and a plurality of fraction pooling feature maps; Respectively calculating the proportion of multiple similarity score values in the sum of all similarity score values to obtain multiple proportional score values, each of the similarity score values is used to characterize the similarity score of each of the fractional pooling feature maps and the initial feature map; The entropy value of each of the proportion score values is calculated to obtain a plurality of entropy weights, and the initial feature map is fused and updated according to the target pooling feature map and each of the entropy weights to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
2. The trajectory tracking and segmentation method according to claim 1, characterized in that: The initial feature map is fused and updated according to the target pooling feature map and each of the entropy weights to obtain a segmentation feature map, including: Multiplying each of the entropy weights by the target pooling feature map respectively, and accumulating them to obtain a second feature map; Adjusting the size of the second feature map; Multiplying the initial feature map and the adjusted second feature map to obtain a third feature map; The sum of the third feature map and the initial feature map is determined as a segmentation feature map.
3. The trajectory tracking and segmentation method according to claim 1, characterized in that: The entropy value is calculated for each of the proportion score values to obtain multiple entropy weights, including: Calculating the entropy value of each of the proportion score values respectively to obtain the redundancy margin of each of the proportion score values; The proportions of multiple redundant redundancies in the sum of all redundant redundancies are calculated respectively to obtain multiple entropy weights.
4. The trajectory tracking and segmentation method according to claim 3, characterized in that: The method for determining the entropy weight includes: Among them, W i is the entropy weight of the i-th proportional pooling channel, I i is the preset unit value of the i-th ratio pooling channel, P i is the proportional score value of the i-th proportional pooling channel, m is the total number of proportional pooling channels, and there is a corresponding relationship between the proportional pooling channel and the preset proportional score.
5. The trajectory tracking and segmentation method according to claim 1, characterized in that: Tracking the action trajectory according to the position information to be tracked in the segmented feature map includes: The segmentation feature map is respectively compared with the convolution feature maps of multiple next frames of images to be tracked for similarity, so as to obtain multiple change position information of the position information to be tracked; An action trajectory is generated according to the position information to be tracked and all the changed position information.
6. The trajectory tracking and segmentation method according to any one of claims 1 to 5, characterized in that: Pooling the initial image to be tracked along different color channels according to a plurality of preset proportion scores to obtain a target pooling feature map and a plurality of score pooling feature maps, including: Performing maximum pooling on the initial image to be tracked along different color channels to obtain multiple initial pooling feature maps; Average all target pooling feature maps to obtain the target pooling feature map; The target pooling feature map is multiplied by each of the preset proportion scores respectively to obtain the fractional pooling feature maps of multiple proportion pooling channels.
7. The trajectory tracking and segmentation method according to any one of claims 1 to 5, characterized in that: The method for determining the similarity score value includes: Multiplying a fractional pooling feature map by the initial feature map to obtain a first feature map; The first feature map is divided by the feature dimension corresponding to the initial feature map to obtain a similarity score value.
8. A trajectory tracking and segmentation device, characterized in that: The device comprises: An acquisition module is used to acquire an initial image to be tracked of the monitoring system, and convolve the initial image to be tracked to obtain an initial feature map; An image pooling module, used for pooling the initial image to be tracked along different color channels according to a plurality of preset proportion scores to obtain a target pooling feature map and a plurality of score pooling feature maps; A similarity determination module, used to respectively calculate the proportion of multiple similarity score values in the sum of all similarity score values to obtain multiple proportional score values, each of which is used to characterize the similarity score between each of the score pooling feature maps and the initial feature map; The entropy weighted pooling segmentation module is used to calculate the entropy value of each of the proportional score values to obtain multiple entropy weights, and to fuse and update the initial feature map according to the target pooling feature map and each of the entropy weights to obtain a segmentation feature map, so as to track the action trajectory according to the position information to be tracked in the segmentation feature map.
9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device, used to store one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the trajectory tracking and segmentation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the trajectory tracking and segmentation method according to any one of claims 1 to 7.