Image processing method, vehicle tracking method, computer terminal and storage medium

Abstract

The present application discloses an image processing method, a vehicle tracking method, a computer terminal and a storage medium. The present application relates to the field of image processing, and the method comprises: acquiring multiple images, wherein different images are acquired by different image acquisition devices from at least one target in a preset area, and there is an overlapping area in the image acquisition ranges of different image acquisition devices; identifying the targets in the multiple images to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; and using a target fusion model to fuse the recognition results corresponding to the multiple images to obtain a target fusion result. The present application solves the technical problem of low target recognition accuracy in the related art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an image processing method, a vehicle tracking method, a computer terminal and a storage medium. Background Art

[0002] In the field of cross-camera target tracking, current methods mainly focus on using position information and temporal trajectory information for target association. A common method is to achieve target handover through color space background weighting and improved field of view boundary. This method can achieve independent tracking of images collected by different cameras, and then use the target handover algorithm to achieve data communication between cameras. However, the current method only achieves cross-domain tracking between two cameras. When the camera is offset, it will affect the accuracy of the target handover algorithm, thereby reducing the accuracy of target recognition.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, a vehicle tracking method, a computer terminal, and a storage medium to at least solve the technical problem of low target recognition accuracy in the related art.

[0005] According to one aspect of an embodiment of the present application, there is provided an image processing method, comprising: acquiring multiple images, wherein different images are obtained by different image acquisition devices among multiple image acquisition devices capturing at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of the different image acquisition devices; identifying targets in the multiple images to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; and fusing the recognition results corresponding to the multiple images using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0006] According to one aspect of an embodiment of the present application, a vehicle tracking method is provided, including: acquiring multiple monitoring images, wherein different monitoring images are obtained by collecting at least one vehicle in a traffic area by different monitoring devices, and there is an overlapping area in the monitoring ranges of different monitoring devices; identifying vehicles in the multiple monitoring images to obtain recognition results corresponding to the multiple monitoring images, wherein the recognition result corresponding to each monitoring image is used to characterize the spatial position of the vehicle contained in the corresponding monitoring image in the corresponding device space; using a vehicle recognition model to fuse the recognition results corresponding to the multiple monitoring images to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one vehicle in the bird's-eye view space, and the vehicle recognition model is a model obtained by self-supervised learning using synthetic data; based on the target fusion result, tracking at least one vehicle to generate a driving trajectory of at least one vehicle.

[0007] According to one aspect of an embodiment of the present application, an image processing method is provided, including: in response to an input instruction acting on an operation interface, displaying multiple images on the operation interface, wherein different images are obtained by acquiring at least one target in a preset area by multiple image acquisition devices, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; in response to a processing instruction acting on the operation interface, displaying a target fusion result on the operation interface, wherein the target fusion result is used to characterize the spatial position of at least one target in a preset space, and the target fusion result is a result obtained by fusing recognition results corresponding to multiple images using a target fusion model, and the target fusion model is a model obtained by self-supervised learning using synthetic data, and the recognition result corresponding to each image is a result obtained by identifying the target in the corresponding image, and the recognition result is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space.

[0008] According to one aspect of an embodiment of the present application, an image processing method is provided, comprising: acquiring multiple images by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes multiple images, different images are acquired by different image acquisition devices acquiring at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; identifying targets in the multiple images to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; fusing the recognition results corresponding to the multiple images using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data; outputting the target fusion result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target fusion result.

[0009] According to one aspect of an embodiment of the present application, a training method for a target fusion model is provided, including: obtaining a preset fusion result at a preset moment, wherein the preset fusion result is used to characterize the spatial positions of multiple objects in a preset area in a preset space at the preset moment; generating preset fusion results for multiple moments based on the preset fusion result at the preset moment, wherein the preset moment is the first moment of the multiple moments; generating a set of preset recognition results for multiple moments based on the preset fusion result at the preset moment, wherein different preset recognition results in the set of preset recognition results at each moment are used to characterize the spatial positions of multiple objects in different device spaces at the corresponding moment; generating synthetic data based on the set of preset recognition results at any one of the multiple moments and the preset fusion results of historical moments before any one of the moments; performing self-supervised learning on the initial fusion model using the synthetic data to obtain a target fusion model, wherein the target fusion model is used to execute the method of any one of the above embodiments.

[0010] According to another aspect of the embodiments of the present application, a computer terminal is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present application is executed when the program is running.

[0011] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in each embodiment of the present application.

[0012] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0013] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0014] According to another aspect of the embodiments of the present application, a computer program is also provided, and when the computer program is executed by a processor, the methods in the various embodiments of the present application are implemented.

[0015] In an embodiment of the present application, multiple images are acquired, wherein different images are acquired by different image acquisition devices for acquiring at least one target in a preset area, and there is an overlapping area in the image acquisition ranges of different image acquisition devices; the targets in the multiple images are identified to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data, thereby improving the accuracy of target recognition. The recognition results corresponding to the multiple images are fused using the target fusion model, thereby improving the accuracy of target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0016] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application;

[0019] Figure 2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application;

[0020] Figure 3 is a flowchart of an image processing method according to Embodiment 1 of the present application;

[0021] Figure 4is a schematic diagram of constructing synthetic data according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the structure of a target fusion model according to an embodiment of the present application;

[0023] Figure 6 is a flow chart of a vehicle tracking method according to Embodiment 2 of the present application;

[0024] Figure 7 is a flowchart of an image processing method according to Embodiment 3 of the present application;

[0025] Figure 8 is a flowchart of an image processing method according to Embodiment 4 of the present application;

[0026] Fig. 9 is a schematic diagram of an image processing device according to Embodiment 5 of the present application;

[0027] Fig.10 is a schematic diagram of a vehicle tracking device according to Embodiment 6 of the present application;

[0028] Fig.11 is a schematic diagram of an image processing device according to Embodiment 7 of the present application;

[0029] Fig.12 is a schematic diagram of an image processing device according to Embodiment 8 of the present application;

[0030] Fig.13 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0031] Fig.14 is a flowchart of an image processing method according to Embodiment 11 of the present application;

[0032] Fig.15 It is a schematic diagram of an image processing device according to Example 12 of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0036] Self-supervised learning: The data set used to train the neural network model does not require any manual annotation, and the characteristics of the data itself or the relationship between the data are used as supervisory signals to train the model.

[0037] Cross-camera target fusion: Multiple cameras cover a certain range of perception space, and there are overlapping areas between cameras. In this case, the target in the overlapping area appears in more than two camera images at the same time. Each camera detects the target separately, and the same target in different cameras needs to be fused to restore the real target position.

[0038] Bird's Eye View (Bev): Usually converts different camera perspectives into a 3D representation of the same bird's eye view. This representation method is intuitive and removes complex perspective effects, which helps provide a clearer and more consistent representation of the environment.

[0039] In the related technology, feature points in the scene are extracted by using an algorithm for extracting local features in images (Scale-Invariant Feature Transform, abbreviated as SIFT), and the field of view boundary can be determined by feature point matching. This method can be based on traditional visual features. When the difference between two perspectives is large, the commonality of visual features is weakened, resulting in a decrease in matching accuracy.

[0040] Currently, a moving target relay tracking algorithm using multiple pan-tilt-zoom (PTZ) cameras can also be used. The intrinsic parameter matrix of the pan-tilt camera is estimated through camera self-calibration, and the field of view dividing line between adjacent pan-tilt cameras is set. The regression model is used as the classification function, combined with the mean shift algorithm to achieve target tracking. The processing of this method relies on the intrinsic parameter calibration of the pan-tilt camera. The calculation of the field of view dividing line requires the use of the geometric relationship of the picture, and the tracked target is single.

[0041] Currently, image feature information can also be used. This information is of a single type and the tracking effect is subject to the model accuracy of the feature extraction model.

[0042] The cross-camera target fusion method based on self-supervised learning involved in the present application is more efficient than the manual labeling method. It can use synthetic data for preliminary training and adopt a spatiotemporal consistency self-supervised loss function to reduce the cost of manual data labeling. Example

[0043] According to an embodiment of the present application, an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Considering that the machine learning model consumes a lot of computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application. Figure 1 In the application scenario shown, the machine learning model is deployed in the server 10, and the server 10 can be connected to one or more clients 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client 20 here may include but is not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client 20 can interact with the user through a graphical user interface to call the large model, thereby implementing the method provided in the embodiment of the present application.

[0045] It should be noted that, when the operating resources of the client device can meet the deployment and operating conditions of the large model, the embodiments of the present application can be carried out in the client device.

[0046] Figure 2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application, such as Figure 2As shown, computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ..., in the figure). Different computing nodes contain local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in computing environment 201, representing services "A", "D", "E" and "H" respectively.

[0047] The end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and / or request can be provided to the entry gateway 230. The entry gateway 230 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 201).

[0048] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0049] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 2 As shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). Pods can include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0050] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service. Figure 2As shown, service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 may call service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0051] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0052] Under the above operating environment, this application provides Figure 3 It should be noted that the image processing method of this embodiment can be Figure 1 The server implementation of the illustrated embodiment. Figure 3 is a flow chart of an image processing method according to Embodiment 1 of the present application. Figure 3 As shown, the method may include the following steps:

[0053] Step S302, acquiring multiple images;

[0054] The different images are obtained by capturing at least one target in a preset area by different image capturing devices among different image capturing devices, and there is an overlapping area in the image capturing ranges of the different image capturing devices.

[0055] The above-mentioned preset area may be a pre-set area where target recognition is required. In a vehicle scene, the preset area may be a road area; in a building scene, the preset area may be a building area; in a crowd scene, the preset area may be a shopping mall area. The type of the preset area is not limited here, and the above-mentioned preset area may be determined according to the type of the target actually detected.

[0056] The above-mentioned multiple image acquisition devices may be of the same type or of different types. There is no limitation on the types of the multiple image acquisition devices herein. The image acquisition devices include but are not limited to cameras, drones, scanners and other devices that can perform image acquisition.

[0057] The above-mentioned different image acquisition devices can be located at different positions to capture at least one target in a preset area to obtain multiple images. Among them, a single image acquisition device can continuously capture multiple consecutive images, and the images captured by different image acquisition devices at the same time point can be combined to obtain multiple images captured by different image acquisition devices at the same time point.

[0058] The at least one target may be a dynamic target or a static target in the preset area. A dynamic target refers to a target that moves dynamically in the preset area, and a static target refers to a target that is stationary in the preset area. The target may be determined according to the target type that needs to be identified.

[0059] In the vehicle scene, the target to be identified in the preset area may be a vehicle, which may be a moving vehicle or a stationary vehicle. In the building scene, the target to be identified in the preset area may be a construction personnel, which may be a moving construction personnel or a stationary construction personnel. In the crowd scene, the target to be identified in the preset area may be mobile personnel and stationary personnel in the shopping mall. This is only an example, and the specific application scenario and target type can be determined according to the actual image processing situation.

[0060] In an optional embodiment, multiple image acquisition devices may be used to acquire images of at least one target in a preset area at the same time point, and multiple images may be obtained. The multiple image acquisition devices may be set at different positions. Multiple image acquisition devices may also be used to acquire images of at least one target in a preset area within the same time period, and multiple images may be obtained. The acquisition time points for acquiring images of at least one target in a preset area by multiple image acquisition devices may be set according to actual conditions, and may be the same or different.

[0061] Since the image acquisition range of an image acquisition device is limited, it is necessary to set up multiple image acquisition devices to capture at least one target in a preset area, and obtain multiple images captured by multiple image acquisition devices, so as to achieve more extensive and comprehensive image acquisition. By acquiring multiple images, the targets in the multiple images can be identified so as to restore the real position of the target in the scene.

[0062] It should be noted that there are overlapping areas in the image acquisition ranges of multiple image acquisition devices. When there are overlapping areas in the image acquisition ranges of multiple image acquisition devices, more extensive and comprehensive image acquisition can be achieved. By fusion and processing of data in overlapping areas, the clarity, accuracy and overall picture of the image can be improved, thereby obtaining more comprehensive and detailed information. In addition, the overlapping areas can also be used to correct and modify images, thereby improving the quality and accuracy of data in subsequent image processing and analysis. Therefore, the existence of overlapping areas in the image acquisition ranges of multiple image acquisition devices can improve the effect and quality of image acquisition. Step S304, identifying targets in multiple images to obtain recognition results corresponding to the multiple images;

[0063] The recognition result corresponding to each image is used to represent the spatial position of the target contained in the corresponding image in the corresponding device space.

[0064] The recognition results corresponding to the above-mentioned multiple images may be the spatial position of the target in the device space determined according to the device point position, wherein the device point position refers to the position and direction of the image acquisition device.

[0065] The device space of the above-mentioned image acquisition device refers to the spatial range of the image or video that the image acquisition device can capture. The device space includes but is not limited to the field of view, resolution, color depth and other parameters of the device, which can affect the quality and detail of the image captured by the device. The larger the device space, the wider the image range and richer the detail information that the device can capture.

[0066] The above-mentioned spatial position of the target in the corresponding device space refers to the position of the target in the coordinate system of the image acquisition device. When performing target recognition, the spatial position of the target in the image acquisition device can be determined by analyzing and comparing the features of the target in the image, that is, the position coordinates and posture information of the target in the three-dimensional space. The positional relationship of the target relative to the image acquisition device can be described by the spatial position of the target in the corresponding device space, so that the target can be located.

[0067] In an optional embodiment, targets in multiple images can be identified through various current recognition methods to obtain recognition results corresponding to the multiple images, wherein the recognition methods include but are not limited to using current image recognition models, neural network models, etc. to perform image recognition.

[0068] The spatial position of the target contained in the image in the corresponding device space can be determined by identifying multiple images. It should be noted that the same target may appear in different images, and the spatial position of the same target in different images may also be different. After acquiring multiple images, the targets in the multiple images can be identified first to determine the spatial position of the target in the device space corresponding to multiple image acquisition devices. By analyzing the identification results corresponding to the multiple images, the real position of the target can be finally determined.

[0069] Step S306, using the target fusion model to fuse the recognition results corresponding to the multiple images to obtain a target fusion result.

[0070] The target fusion result is used to characterize the spatial position of at least one target in a preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0071] The above-mentioned preset space can be a three-dimensional space, and the three-dimensional space can be a bird's-eye view space (Bird's Eye View, referred to as BEV). The bird's-eye view space can be used to represent the conversion of images collected from different perspectives into a three-dimensional space representation under the same bird's-eye view perspective. This representation method can be intuitive and remove complex perspective effects, which helps to provide a clearer and more consistent environment representation.

[0072] The above-mentioned preset space can also be a plane view space (Top-Down View), a stereoscopic view space (3DView), a grid view space (Grid View), a roaming view space (Roaming View), and a timeline view space (Timeline View).

[0073] Among them, the plane view perspective space is similar to the bird's-eye view perspective space, but the perspective is more directly downward, similar to the top view on a map. The stereoscopic view perspective space can represent multiple images in three-dimensional space, so that the observer can view these images from different angles and directions. The grid view perspective space arranges multiple images in a space according to a grid, so that these images can be quickly browsed and compared. The roaming view perspective space allows the observer to move and navigate freely in a virtual environment, so as to explore the relationship and details between images in more depth. The timeline view perspective space can arrange multiple images in a space in chronological order, so that the observer can see the evolution and changes of the images.

[0074] The target fusion model mentioned above may be a neural network model, and the model type of the target fusion model is not limited here. The target fusion model refers to a model in the field of computer vision and machine learning that integrates information from different data sources or different features to improve model performance and generalization ability.

[0075] The above-mentioned synthetic data refers to data generated by a computer. This data can be used to supplement real data, increase the amount of data, or be used for data enhancement. In this application, a certain amount of synthetic data can be used in a self-supervised learning method to reduce the cost of data annotation and achieve better results in the continuity and authenticity of the generated trajectory.

[0076] The self-supervised learning mentioned above is an unsupervised learning method that does not require labeled data, but instead trains the model by generating labels from the data itself. The goal of self-supervised learning is to allow the model to learn the intrinsic representation of the data, thereby capturing the key information in the data.

[0077] In an optional embodiment, synthetic data may be used to generate training samples, which may contain information from different data sources or different features. Self-supervised learning tasks may be designed, such as the fusion of the spatial positions of targets in different device spaces, so as to learn the intrinsic representation of data from synthetic data. Self-supervised learning tasks may be used to train models so that the models learn the associations and complementarities between different data sources or different features. During the training process, model parameters are continuously adjusted to optimize model performance and generalization capabilities. After the training is completed, the target fusion model obtained can be used to fuse the spatial positions of target device spaces in multiple images to obtain the spatial positions of the targets in a preset space.

[0078] In the application scenario of highways, the deep learning model can be used in multiple cameras on the highway to track targets across cameras. The multiple cameras can be the multiple image acquisition devices mentioned above, and the multiple cameras can be monitoring cameras fixed on the highway to monitor the driving conditions and traffic flow of vehicles. The vehicles can be targets in the above-mentioned preset areas. The target fusion model can accurately identify and track the spatial positions of targets that appear repeatedly in multiple cameras in the preset space, and restore the vehicle trajectory in the preset space based on the spatial positions.

[0079] In the application scenario of the tunnel, the deep learning model can be used in multiple cameras on the tunnel to track targets across cameras. The multiple cameras can be the multiple image acquisition devices mentioned above. The multiple cameras can be monitoring cameras fixed in the tunnel, fixed on the upper side wall of the tunnel, or fixed on the side walls on both sides of the tunnel. The specific location of the monitoring camera is not limited. The above-mentioned target can be a vehicle, and the above-mentioned preset area can be an area inside the tunnel. The driving conditions and traffic flow of vehicles in the tunnel can be monitored by monitoring cameras. The target fusion model can accurately identify and track the spatial positions of targets that appear repeatedly in multiple cameras in the preset space, and restore the vehicle trajectory in the preset space based on the spatial position.

[0080] The above method can help traffic management departments monitor and manage vehicle driving conditions on highways in real time, improve traffic management efficiency and safety. By accurately tracking vehicle trajectories, intelligent traffic management can be achieved, and traffic congestion and accident risks can be warned in advance, providing drivers and passengers with a safer and more convenient travel experience.

[0081] Through the above steps, multiple images are obtained, wherein different images are obtained by different image acquisition devices capturing at least one target in a preset area, and there is an overlapping area in the image acquisition ranges of different image acquisition devices; the targets in the multiple images are identified to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data, thereby improving the accuracy of target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0082] In the above embodiments of the present application, the target fusion model includes at least a backbone module and a fusion module; the target fusion model is used to fuse the recognition results corresponding to multiple images to obtain a target fusion result, including: obtaining a historical fusion result, wherein the historical fusion result is used to characterize the historical spatial position of at least one target in a preset space; using the backbone module to perform feature extraction on the recognition results corresponding to multiple images to obtain image features of multiple images, and using the backbone module to perform feature extraction on the historical fusion results to obtain historical image features; using the fusion module to fuse the image features of multiple images and the historical image features to obtain the target fusion result.

[0083] The above-mentioned backbone module can be a backbone network structure (backbone) for extracting features. The backbone module is usually composed of a series of convolutional layers, pooling layers and activation function layers, which are used to extract features and represent input data. The backbone module can be a residual network (ResNet), a deep convolutional neural network, etc., which is not limited here, and the required backbone module can be selected according to the actual scenario.

[0084] The above-mentioned historical spatial position can be used to represent the spatial position of at least one target in a preset space predicted at adjacent time points. The adjacent time point can be the previous time point adjacent to the current time point at which multiple images are acquired. The adjacent time point can also be any time point before the current time point. No specific limitation is made to the adjacent time points here.

[0085] In an optional embodiment, the position of the target in the preset space can be more accurately characterized by extracting image features and historical image features from multiple images and fusing them. Using the backbone module to extract features from multiple images can improve the accuracy and stability of the recognition results. At the same time, using the historical fusion results for feature extraction can retain information about the spatial position of the target in the preset space under the historical state, which helps to more comprehensively understand the movement trajectory and change trend of the target. Furthermore, by fusing the feature information of multiple images with the historical image features through the fusion module, a more comprehensive and accurate target fusion result can be obtained. This fusion method can comprehensively utilize information at different time points and different angles to improve the accuracy and robustness of subsequent target detection and tracking.

[0086] In the above embodiment of the present application, the method also includes: obtaining a preset fusion result of a preset moment, wherein the preset fusion result is used to characterize the spatial positions of multiple objects in a preset area in a preset space at a preset moment; generating preset fusion results of multiple moments based on the preset fusion result of the preset moment, wherein the preset moment is the first moment of the multiple moments; generating a set of preset recognition results of multiple moments based on the preset fusion result of the preset moment, wherein different preset recognition results in the set of preset recognition results of each moment are used to characterize the spatial positions of multiple objects in different device spaces at the corresponding moment; generating synthetic data based on the set of preset recognition results of any moment among the multiple moments, and the preset fusion results of historical moments before any moment; and performing self-supervised learning on the initial fusion model using the synthetic data to obtain a target fusion model.

[0087] The historical moment before any of the above moments refers to a moment among the multiple moments.

[0088] In an optional embodiment, a preset fusion result at a preset moment can be obtained first, and the preset fusion result is used to characterize the spatial positions of multiple objects in a preset area in a preset space. The preset fusion result at the preset moment can be expanded in the time domain to form a time domain segment with good consistency, which is used to simulate real trajectory data to obtain the preset fusion results of the above multiple moments, and the preset fusion result at the preset moment can be split in the spatial domain to obtain the spatial positions of multiple objects in different device spaces at the corresponding moment, thereby obtaining a set of preset recognition results at multiple moments; the preset recognition results at multiple moments can be expanded by reverse data splitting. In the process of generating synthetic data, the perception results of different camera points can be determined according to the set of preset recognition results at any moment in the multiple moments and the preset fusion results at the historical moments before any moment. The preset fusion results and preset recognition results can be effectively used to train the target fusion model, thereby improving the accuracy and generalization ability of the model. By using information and historical data at multiple moments, the model can better understand the positional relationship of objects at different moments and spaces, thereby better performing fusion and recognition tasks.

[0089] By collecting preset recognition result sets at multiple moments and preset fusion results at historical moments, a more comprehensive and accurate data set can be obtained. Using these data sets to generate synthetic data can help the model better learn the associations and patterns between data. By performing self-supervised learning on the initial fusion model with synthetic data, the model parameters can be continuously adjusted and updated by comparing the model prediction results with the actual results of the synthetic data, thereby obtaining a more accurate target fusion model. Synthetic data is generated by combining data from multiple moments and historical information, and then the model is continuously improved through self-supervised learning, thereby improving the performance of the model in fusion tasks.

[0090] Figure 4 is a schematic diagram of constructing synthetic data according to an embodiment of the present application, such as Figure 4 As shown, the synthetic data construction method of the present application is based on a special camera topology, that is, adjacent points can be overlapped and spread out, and the targets in the target domain are split in the input domain in the camera overlapping area, that is, the targets in the preset space are split in the device space. Figure 4 The target domain data in the preset space is a series of trajectories Bev (P0, P1, P2, ... P t ), the input domain data is the spatial position in different device spaces. At the state at time t, the construction of synthetic data adopts the reverse splitting method, first randomly scattering points on the target domain, and then reversely splitting into multiple different points in the overlapping area of ​​the input domain, which can be represented as follows:

[0091] P t = St (i,C n ) + U n ;

[0092] Among them, C n = (n mod 3) represents the channel position of the nth camera on the input domain, S t (i,C n ) is the point position of the target in the input domain. In this application, the input can be modeled as a 3-channel tensor, and different camera indices are distributed in different channels in sequence; U n Represents the position offset between the point on camera n and the point P on the target domain. This position offset is obtained by random sampling within a certain range (x:[-5,5], y:[-2,2]). The range can be adjusted according to actual conditions.

[0093] In the above-mentioned embodiment of the present application, based on the preset fusion result of the preset moment, the preset fusion results of multiple moments are generated, including: determining the offset corresponding to the first moment, wherein the first moment is used to characterize the next moment after the preset moment among the multiple moments, and the offset corresponding to the first moment is used to characterize the movement speed of multiple objects at the first moment; superimposing the offset corresponding to the first moment with the preset fusion result of the preset moment to obtain the preset fusion result of the first moment; determining the offset corresponding to the second moment after the first moment; superimposing the offset corresponding to the second moment with the preset fusion result of the first moment to obtain the preset fusion result of the second moment; taking the second moment as the new first moment, and repeating the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with the preset fusion result of the first moment, until the second moment is the last moment among the multiple moments.

[0094] The above-mentioned offset may be an offset D, and the offset may represent a moving speed V of the target.

[0095] In an optional embodiment, the offset corresponding to the first moment and the preset fusion result of the preset moment can be superimposed to obtain the preset fusion result of the first moment. The displacement distance of the target between the preset moment and the first moment can be determined by the offset, and the displacement corresponding to the second moment after the first moment, that is, the displacement distance of the target between the first moment and the second moment, can be determined. The offset corresponding to the second moment can be superimposed with the preset fusion result of the first moment to obtain the preset fusion result of the second moment. In this way, the preset fusion results of two consecutive moments can be determined. When calculating the preset fusion results of other moments after the second moment in multiple moments, the second moment can be used as a new first moment, and the moment after the second moment can be used as a new second moment. The steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with the preset fusion result of the first moment are repeatedly performed until the second moment is the last moment in the multiple moments. By repeatedly performing the steps of determining the offset and superimposing the offset and the preset fusion result, the motion trajectory of the target at multiple moments can be obtained, and the preset fusion results of multiple moments can be obtained. The preset fusion result of the preset moment can be expanded based on the offset in the time domain to obtain the preset fusion results of multiple moments.

[0096] In the above embodiment of the present application, based on the preset fusion result at the preset moment, a preset recognition result set of multiple moments is generated, including: based on the device space corresponding to the multiple image acquisition devices, the preset fusion result of the preset moment is split to obtain the preset recognition result set of the preset moment; determine the offset corresponding to the first moment, wherein the first moment is used to characterize the next moment after the preset moment among the multiple moments, wherein the offset corresponding to the first moment is used to characterize the movement speed of multiple objects at the first moment; superimpose the offset corresponding to the first moment with different preset recognition results in the preset recognition result set of the preset moment to obtain the preset recognition result set of the first moment; determine the offset corresponding to the second moment after the first moment; superimpose the offset corresponding to the second moment with different preset recognition results in the preset recognition result set of the first moment to obtain the preset recognition result set corresponding to the second moment; take the second moment as the new first moment, and repeat the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with different preset recognition results in the preset recognition result set of the first moment, until the second moment is the last moment among the multiple moments.

[0097] In an optional embodiment, the preset fusion results at the preset moment can be split according to the device space corresponding to the multiple image acquisition devices to obtain the preset recognition result set at the preset moment. Optionally, the preset recognition result set at the preset moment can be obtained by splitting the targets in the overlapping area of ​​the image acquisition range of the multiple image acquisition devices. The offset corresponding to the first moment can be determined, and the offset and the preset recognition result set at the preset moment can be superimposed to determine the preset recognition result combination at the first moment according to the moving speed of the target. The offset corresponding to the second moment after the first moment can be determined, and the offset corresponding to the second moment can be superimposed with different preset recognition results in the preset recognition result set of the first moment to obtain the preset recognition result set corresponding to the second moment. In this way, the preset recognition result sets of the continuous first moment and second moment can be obtained. The second moment can be used as a new first moment, and the moment after the original second moment can be used as a new second moment, and the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with different preset recognition results in the preset recognition result set of the first moment are repeated until the second moment is the last moment of the multiple moments. The steps of determining the offset corresponding to the second moment after the first moment and superimposing the offset with the preset recognition result set of the first moment can be repeatedly performed, so as to determine the spatial position of the target in the corresponding device space at multiple moments, thereby determining the preset fusion results at multiple moments.

[0098] By splitting the preset fusion results at the preset moments as described above, a preset recognition result set at the preset moments is obtained. The spatial position of the target in the device space can be obtained inversely based on the spatial position of the target in the preset space. The motion trajectory of the target in the device space at different moments can be determined through the spatial position and offset of the target in the device space, thereby obtaining a preset recognition result set at multiple moments.

[0099] After the preset fusion result at the preset time is determined, the preset fusion result can be expanded in the time domain to form a time domain segment with good consistency, which is used to simulate the real track data. Specifically, an offset D (D: [0, 5], mean value Du) can be sampled for the spatial domain P and the temporal domain S. This offset represents the moving speed V of the target and can be expressed by the following formula:

[0100] Pt = Pt-1 + D, St = St-1 + D;

[0101] In a vehicle environment, in order to simulate real traffic conditions, the mean basis of the random distribution of D can be mapped according to the number of vehicles N on the same road section, Du = F(N). In traffic engineering, this relationship F is usually described by a basic graph, which is an important concept in traffic flow theory, used to express the relationship between traffic density (or the number of vehicles N), traffic flow (the number of vehicles passing a point) and vehicle speed (V).

[0102] The above basic diagram usually includes three key parts: free flow area, transition flow area, and congestion flow area. In the free flow area, the vehicle speed is high and the vehicle density is basically irrelevant, that is, the flow is low; in the transition flow area, as the vehicle density increases, the vehicle speed begins to decrease and the flow increases; in the congestion flow area, when the vehicle density is very high, the vehicle speed drops severely, the flow reaches or approaches the maximum value, and then begins to decrease as the density increases further.

[0103] The specific mathematical relationship between V and N mentioned above may vary greatly depending on the region, time, highway design and traffic rules. Traffic engineers use traffic simulation software and actual traffic survey data to analyze and predict the relationship between V and N for a specific road section. In practice, the F curve is empirically fitted to cover all real traffic conditions as much as possible.

[0104] In the above-mentioned embodiment of the present application, based on the device space corresponding to the multiple image acquisition devices, the preset fusion result at the preset moment is split to obtain the preset recognition result set at the preset moment, including: based on the device space corresponding to the multiple image acquisition devices and the preset fusion result at the preset moment, determining the target device space to which the multiple objects belong; when there is only one target device space to which any one of the multiple objects belongs, converting the spatial position of any one of the objects in the preset fusion result at the preset moment into the spatial position of any one of the objects in the target device space; when there are multiple target device spaces to which any one of the multiple objects belongs, splitting the spatial position of any one of the objects in the preset fusion result at the preset moment into the spatial positions of any one of the objects in multiple target device spaces.

[0105] The above-mentioned multiple objects may be objects in an overlapping area in a preset space, or may be objects in an area outside the preset space. The overlapping area may be an area where image acquisition ranges of any multiple image acquisition devices overlap.

[0106] If an object is in an overlapping area, when determining the spatial position of the object in the target device space, the object can be split to determine the spatial position of the object in different target device spaces. If an object is not in an overlapping area, when determining the spatial position of the object in the target device space, the object does not need to be split, and the spatial position of the object in the target device space can be directly determined.

[0107] The target device spaces to which the multiple objects belong can be determined based on the device spaces corresponding to the multiple image acquisition devices and the preset fusion results at the preset time, wherein any object among the multiple objects can be determined to belong to one target device space or multiple target device spaces based on the area in which it is located. If any object is in an overlapping area, it is determined that the object belongs to multiple target device spaces, and if any object is not in an overlapping area, it is determined that the object belongs to one target device space.

[0108] When the target device space to which any one of multiple objects belongs is one, it means that the any object is not in the overlapping area of ​​the acquisition ranges of multiple image acquisition devices, and the spatial position of any object in the preset fusion result at a preset moment can be converted into the spatial position of any object in the target device space.

[0109] When any one of multiple objects belongs to multiple target device spaces, it means that the any one object is in the overlapping area of ​​the acquisition ranges of multiple image acquisition devices, and the spatial position of any one object in the preset fusion result at a preset moment can be converted into the spatial position of any one object in the target device space.

[0110] In the above embodiment of the present application, the spatial position of any object in the preset fusion result at a preset moment is split into the spatial positions of any object in multiple target device spaces, including: determining second offset parameters corresponding to multiple target device spaces, wherein the second offset parameters are used to characterize the offset between the spatial position of any object in the corresponding target device space and the spatial position of any object in the preset space; superimposing the spatial position of any object with the second offset parameters corresponding to multiple target device spaces, respectively, to obtain the spatial positions of any object in multiple target device spaces.

[0111] The above-mentioned second offset parameter can be used to represent the offset between the spatial position of any object in the corresponding target device space and the spatial position of any object in the preset space, that is, the second offset parameter can refer to the offset between the spatial positions between different spaces.

[0112] The second offset parameter may be obtained by random sampling within a preset range. The second offset parameter may also be preset. The preset range may be a preset range, for example, (x: [-5, 5], y: [-2, 2]), but is not limited thereto.

[0113] The spatial position of any object can be superimposed with the second offset parameters corresponding to multiple target device spaces, so as to convert the spatial position of any object in a preset space to the spatial position in multiple target device spaces. By superimposing the second offset parameters, the position deviation in the space conversion process can be taken into account, thereby obtaining a relatively accurate spatial position of any object in multiple target device spaces.

[0114] In the above embodiment of the present application, determining the offset corresponding to the first moment includes: determining the mean of the random distribution based on the number of multiple objects within a preset range; and sampling in the random distribution corresponding to the mean of the random distribution to obtain the offset.

[0115] The above-mentioned random distribution mean may refer to the average value of each value in a random distribution. In a random distribution, each value has a certain probability of appearing, and the average value of these values ​​is the mean of the random distribution. The mean of the random distribution can be obtained by calculating the average value of all values ​​in the random distribution.

[0116] The random distribution corresponding to the random distribution mean mentioned above refers to the average value of the random distribution determined according to the number of preset multiple objects within a given range. Sampling in the distribution corresponding to this average value will result in a random distribution characteristic. This method can be used to generate random data that meets specific requirements and is used to simulate random variables in actual situations.

[0117] In an optional embodiment, random sampling may be performed in the random distribution corresponding to the mean of the random distribution to obtain an offset, or random sampling may be performed in the random distribution corresponding to the mean of the random distribution according to preset rules based on actual needs to obtain an offset.

[0118] In the above embodiment of the present application, the initial fusion model is self-supervised learned using synthetic data to obtain a target fusion model, including: inputting the synthetic data into the initial fusion model to obtain the predicted fusion result of any moment output by the initial fusion model, a set of predicted recognition results at any moment, and the predicted fusion results at historical moments; constructing a spatial consistency loss function based on the predicted recognition result set at any moment and the preset recognition result set; constructing a temporal consistency loss function based on the predicted fusion result at the previous moment and the preset fusion result; adjusting the model parameters of the initial fusion model based on the spatial consistency loss function and the temporal consistency loss function to obtain the target fusion model.

[0119] In an optional embodiment, the synthesized data can be input into the initial fusion model to obtain the predicted fusion results at any moment, the predicted recognition result set, and the predicted fusion results at historical moments output by the initial fusion model. Based on the predicted recognition result set and the preset recognition result set at any moment, a spatial consistency loss function is constructed to evaluate the prediction accuracy of the model in space. At the same time, based on the predicted fusion results and the preset fusion results at the previous moment, a temporal consistency loss function is constructed to evaluate the prediction consistency of the model in time. The spatial consistency loss function and the temporal consistency loss function are combined to adjust the model parameters of the initial fusion model to obtain the target fusion model to improve the prediction accuracy and consistency of the model. The whole process can help adjust the performance of the fusion model to make it more suitable for actual application scenarios.

[0120] By constructing the temporal consistency loss function and the spatial consistency loss function, the model can perform self-supervised learning on real data, thereby improving the trajectory fusion effect in real scenarios.

[0121] In the above embodiment of the present application, the synthesized data is input into the initial fusion model to obtain the predicted fusion result of any moment output by the initial fusion model, the predicted recognition result set at any moment, and the predicted fusion result at the historical moment, including: inputting the synthesized data into the initial fusion model to obtain the predicted fusion result of any moment output by the initial fusion model, the spatial offset set at any moment, and the time domain offset at any moment, wherein the spatial offset set is used to characterize the offset of the predicted fusion result at any moment relative to the predicted recognition result set at any moment, and the time domain offset is used to characterize the offset of the predicted fusion result at any moment relative to the predicted fusion result at the historical moment; based on the spatial offset set at any moment, the predicted fusion result at any moment is distorted and transformed to obtain the predicted recognition result set at any moment; based on the time domain offset at any moment, the predicted fusion result at any moment is distorted and transformed to obtain the predicted fusion result at the historical moment.

[0122] The above distortion change can be achieved through matrix change operation. Among them, distortion transformation refers to a nonlinear transformation operation on the prediction fusion result to obtain the prediction fusion result of the historical moment. This transformation can be achieved through matrix change. Assuming that the prediction fusion result is a matrix P and the prediction fusion result of the historical moment is a matrix H, the distortion transformation can be expressed as:

[0123] H = f(P);

[0124] Among them, f represents the specific operation of the distortion transformation. Through matrix changes, the distortion transformation can be expressed as a matrix multiplication form:

[0125] H = A * P;

[0126] Among them, A is the distortion transformation matrix, which can be obtained by performing a series of nonlinear transformation operations on P. This distortion transformation can help us take into account the nonlinear characteristics of the prediction fusion results at the current moment when making predictions at historical moments, thereby obtaining more accurate prediction results.

[0127] In an optional embodiment, the synthetic data can be input into the initial fusion model to obtain the predicted fusion result at any moment output by the initial fusion model, the set of spatial offsets at any moment, and the time domain offset at any moment. The predicted fusion result at any moment can be distorted according to the set of spatial offsets to consider the position offset in the spatial position conversion process, thereby obtaining a set of predicted recognition results at any moment with higher accuracy and in line with the actual situation. The predicted fusion result at any moment can be distorted according to the time domain offset at any moment to consider the time offset in the spatial position conversion process, thereby obtaining a predicted fusion result at a historical moment with higher accuracy and in line with the actual situation. The training object can be the initial fusion model, and the synthetic data can be input into the initial fusion model to obtain the predicted fusion result at any moment output by the initial fusion model, the set of predicted recognition results at any moment, and the predicted fusion result at the historical moment; different types of loss functions can be constructed according to the above results, so that the model parameters of the initial fusion model can be adjusted through the loss function to obtain a target fusion model with higher accuracy.

[0128] Figure 5 is a schematic diagram of the structure of a target fusion model according to an embodiment of the present application, such as Figure 5As shown in the figure, the recognition results corresponding to multiple images can be used as input and modeled into 3-channel input tensors respectively. The corresponding channel Channel = n mod 3, n is the serial number of the sequential image acquisition device. The target fusion model extracts features from the recognition results corresponding to the input multiple images through the backbone network, and outputs 4 predicted Heads, among which Head0 has C channels, predicting the heat map (HeatMap) of the target in the Bev space. The responses of different channels correspond to the predicted target types. The position in the HeatMap where the response is greater than the threshold (th=0.8) is the two-dimensional coordinate of the target in the Bev space, thereby obtaining the predicted spatial position P^ of the target.

[0129] The above HeatMap refers to a tensor of multiple channels, for example, shape = [3, 256, 256], where 256x256 is the size of the BEV space, 3 means that there are 3 categories of different targets, and HeatMap has a local maximum response at a certain position, for example, [1, 100, 100], which means that there is a first-category target (i.e., truck) at the position (100, 100).

[0130] like Figure 5 Head1, Head2, and Head3 in the above respectively predict the spatial offsets of points in P^ and the original input 3 channels, thereby obtaining the point pair matching results in the spatial domain, that is, the fusion process of the same target; among them, Head4 represents the point offset in the time domain, corresponding to the offset of the point Pt-1 at time t-1 from the point Pt at time t, that is, Head4 = P t -P t-1 , thus obtaining a matching result with time domain consistency.

[0131] like Figure 5Head1, Head2, Head3, and Head4 are all responsible for predicting the offset of a certain position in the Bev space (spatial offset and temporal offset); this spatial offset predicts the deviation between the perception results of multiple camera points and the results of the fused BEV, that is, the error between the spatial position of the target in the device space and the spatial position of the target in the preset space. The position offset (the offset of each position predicted by Head1, Head2, and Head3) is transformed by matrix transformation (Warping) to transform the Heatmap of Head0 through the m in the channel direction. The ax operation reduces the channel to 1, thus warping it to the corresponding spatial positions of Head1, Head2, and Head3. The L1 loss function is calculated on the heatmap after the warping (getting Warp1, Warp2, and Warp3) and the original three-channel heatmap. The time domain offset is similar. The offset predicted by Head4 warps the Heatmap of Head0 (after the max operation in the channel direction reduces the channel to 1) to the position at time T-1. The L1 loss is calculated on the heatmap after the warping (Warp4) and the fusion result heatmap at time T-1. Since the input T=t-1 is the fusion result of the previous time t-1, Figure 5 A channel is shown in FIG.

[0132] In the process of pre-training with synthetic data, Head0, Head1, Head2, Head3, and Head4 all have true labels (GroundTruth), and can be trained with the mean square error loss function (mse_loss), while adding the spatiotemporal consistency losses Ls and Lt; in the process of training (finetune) with real data, due to the lack of true labels, it is necessary to rely on the spatial consistency loss (Ls) and temporal consistency loss (Lt) to enhance the spatiotemporal consistency of the trajectory on the real data. The loss function for distance needs to use the smooth L1 loss function (Smooth L1 Loss).

[0133] The calculation method of the above Ls is as follows:

[0134] Ls = L1(Warping(Hmt, H1), C1) + L1(Warping(Hm, H2), C2) + L1(Warping(Hm, H3), C3);

[0135] The above Lt is calculated as follows:

[0136] Lt = L1(Warping(Hmt,H4),Hmt-1).

[0137] Among them, Hm represents the single-channel feature map obtained after the predicted HeatMap is operated in the channel direction, Hmt represents the single-channel feature map obtained after the predicted HeatMap is operated in the channel direction at time t, Hmt-1 represents the single-channel feature map obtained after the predicted HeatMap is operated in the channel direction at time t-1, Warping is the transformation matrix, H1, H2, H3, H4 represent Head1, Head2, Head3, Head4 respectively, which can be offset relative to a certain position in the HeatMap (spatial offset and temporal offset). C1, C2, C3 represent the original channels at time T, that is, 3-channel input tensors.

[0138] In the above embodiment of the present application, the method also includes: outputting a target fusion result; receiving a modified fusion result, wherein the modified fusion result is a result obtained by modifying the target fusion result; and updating the target fusion model based on the modified fusion result and multiple images.

[0139] In an optional embodiment, the target fusion result can be output so that the user can check whether the target fusion result meets the requirements or whether the target fusion result is accurate. If the target fusion result is inaccurate or needs to be adjusted, the target fusion result can be modified to obtain a modified fusion result. The target fusion model can be updated based on the modified fusion result and multiple images to improve the model accuracy of the target fusion model.

[0140] In the above embodiment of the present application, the method also includes: outputting preset fusion results of multiple moments and preset recognition result sets of multiple moments; receiving modified fusion results of multiple moments and modified recognition result sets of multiple moments, wherein the modified fusion results of multiple moments are the results obtained by modifying the preset fusion results of multiple moments, and the modified recognition result sets of multiple moments are the results obtained by modifying the preset recognition result sets of multiple moments; generating synthetic data based on the modified recognition result set of any moment and the modified fusion results of historical moments.

[0141] In an optional embodiment, preset fusion results of multiple moments and preset recognition result sets of multiple moments can be output so that the user can determine whether the preset fusion results and preset recognition result sets need to be adjusted. If it is considered that the preset fusion results or the preset recognition result sets need to be adjusted, the preset fusion results of multiple moments can be modified to obtain modified fusion results of multiple moments, and the preset recognition result sets of multiple moments can be modified to obtain modified recognition result sets of multiple moments. More accurate synthetic data can be obtained based on the modified recognition result set of any moment and the modified fusion results of historical moments, so that the initial fusion model can be trained with more accurate synthetic data to obtain a target fusion model with higher fusion accuracy.

[0142] In this application, a synthetic data and self-supervised learning method can be constructed to fully explore the relationship between the device space and the preset space, so that the target fusion model can achieve a matching accuracy of more than 95% in low-speed scenarios with high traffic volume, and the continuity of the trajectory is greatly enhanced in congestion.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0144] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0145] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application. Example

[0146] According to an embodiment of the present application, a vehicle tracking method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0147] Figure 6 is a flow chart of a vehicle tracking method according to Embodiment 2 of the present application, such as Figure 6 As shown, the method includes:

[0148] Step S602, acquiring multiple monitoring images;

[0149] The different monitoring images are obtained by collecting data from at least one vehicle in a traffic area by different monitoring devices, and there are overlapping areas in the monitoring ranges of different monitoring devices.

[0150] The above-mentioned monitoring equipment can be set up on the road to monitor the traffic area on the road.

[0151] In an optional embodiment, multiple monitoring devices may be used to collect images of vehicle flows in a traffic area to obtain the multiple monitoring images mentioned above. The multiple monitoring images are used to represent the vehicle flow conditions in the traffic area.

[0152] Step S604, identifying vehicles in the multiple monitoring images to obtain identification results corresponding to the multiple monitoring images;

[0153] The recognition result corresponding to each monitoring image is used to characterize the spatial position of the vehicle contained in the corresponding monitoring image in the corresponding device space.

[0154] The above recognition result can be used to represent the spatial position in the device space corresponding to the monitoring angle of the monitoring device corresponding to the vehicle.

[0155] Step S606, using the vehicle recognition model to fuse the recognition results corresponding to the multiple monitoring images to obtain a target fusion result;

[0156] The target fusion result is used to characterize the spatial position of at least one vehicle in the bird's-eye view space, and the vehicle recognition model is a model obtained by self-supervised learning using synthetic data.

[0157] The above-mentioned bird's-eye view space can be used to represent the spatial position of at least one vehicle viewed from above the at least one vehicle.

[0158] Step S608: Based on the target fusion result, track at least one vehicle and generate a driving trajectory of at least one vehicle.

[0159] In an optional embodiment, the spatial positions of at least one vehicle at multiple time points changing with time in the bird's-eye view space can be determined through the target fusion result, and the driving trajectory of at least one vehicle can be determined based on the spatial positions at the multiple time points.

[0160] Optionally, the driving trajectory of the at least one vehicle can be used to determine whether the at least one vehicle has violated traffic regulations or the like, and the driving trajectory of the at least one vehicle can also be used to determine the accident responsibility of the at least one vehicle in a traffic accident. The driving trajectory can also be used to handle other related matters, which are not specifically limited here.

[0161] Furthermore, a driving trajectory of at least one vehicle may be generated in the bird's-eye view space.

[0162] Through the above steps, a plurality of monitoring images are obtained, wherein different monitoring images are obtained by collecting data of at least one vehicle in a traffic area by different monitoring devices, and there are overlapping areas in the monitoring ranges of different monitoring devices; vehicles in a plurality of monitoring images are identified to obtain recognition results corresponding to the plurality of monitoring images, wherein the recognition result corresponding to each monitoring image is used to characterize the spatial position of the vehicle contained in the corresponding monitoring image in the corresponding device space; the recognition results corresponding to the plurality of monitoring images are fused using a vehicle recognition model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one vehicle in the bird's-eye view space, and the vehicle recognition model is a model obtained by self-supervised learning using synthetic data; based on the target fusion result, at least one vehicle is tracked to generate a driving trajectory of at least one vehicle, thereby improving the accuracy of target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0163] In the above embodiments of the present application, the vehicle recognition model includes at least a backbone module and a fusion module; the vehicle recognition model is used to fuse the recognition results corresponding to multiple monitoring images to obtain a target fusion result, including: obtaining a historical fusion result, wherein the historical fusion result is used to characterize the historical spatial position of at least one vehicle in the bird's-eye view space; using the backbone module to perform feature extraction on the recognition results corresponding to the multiple monitoring images to obtain image features of the multiple monitoring images, and using the backbone module to perform feature extraction on the historical fusion results to obtain historical image features; using the fusion module to fuse the image features of the multiple monitoring images and the historical image features to obtain the target fusion result.

[0164] The above-mentioned historical spatial position can be used to represent the spatial position of at least one vehicle in the bird's-eye view space predicted at adjacent time points. The adjacent time point can be the previous time point adjacent to the current time point at which multiple monitoring images are obtained. The adjacent time point can also be any time point before the current time point. There is no specific limitation on the adjacent time points here.

[0165] In the above embodiment of the present application, the method also includes: obtaining a preset fusion result at a preset moment, wherein the preset fusion result is used to characterize the spatial positions of multiple test vehicles in a preset area in the bird's-eye view space at the preset moment; based on the preset fusion result at the preset moment, generating preset fusion results for multiple moments, wherein the preset moment is the first moment of the multiple moments; based on the preset fusion result at the preset moment, generating a set of preset recognition results for multiple moments, wherein different preset recognition results in the set of preset recognition results at each moment are used to characterize the spatial positions of multiple test vehicles in different equipment spaces at the corresponding moment; based on the set of preset recognition results at any one of the multiple moments, and the preset fusion results of the historical moments before any one moment, generating synthetic data; using the synthetic data to perform self-supervised learning on the initial fusion model to obtain a vehicle recognition model.

[0166] In an optional embodiment, the preset fusion result is used to characterize the spatial positions of multiple test vehicles in a preset area in the bird's-eye view space, and the preset fusion result at a preset moment can be expanded in the time domain to obtain preset fusion results at multiple moments.

[0167] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0168] Example 3

[0169] According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0170] Figure 7 is a flowchart of an image processing method according to Embodiment 3 of the present application, such as Figure 7 As shown, the method includes:

[0171] Step S702, in response to an input instruction applied to the operation interface, displaying a plurality of images on the operation interface;

[0172] The different images are obtained by capturing at least one target in a preset area by multiple image capturing devices, and there are overlapping areas in the image capturing ranges of different image capturing devices.

[0173] The above-mentioned operation interface can be used to display multiple images, wherein the operation interface can include a variety of different controls for the user to operate, so as to display multiple images on the operation interface.

[0174] Step S704, responding to the processing instruction acting on the operation interface, and displaying the target fusion result on the operation interface.

[0175] Among them, the target fusion result is used to characterize the spatial position of at least one target in a preset space. The target fusion result is the result obtained by fusing the recognition results corresponding to multiple images using a target fusion model. The target fusion model is a model obtained by self-supervised learning using synthetic data. The recognition result corresponding to each image is the result obtained by identifying the target in the corresponding image. The recognition result is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space.

[0176] The above-mentioned processing instructions may be instructions for fusing the recognition results corresponding to multiple images. According to the processing instructions, multiple images may be first recognized to obtain the recognition results corresponding to the multiple images, and then the recognition results corresponding to the multiple images may be fused to obtain the target fusion result.

[0177] Through the above steps, in response to the input instruction on the operation interface, multiple images are displayed on the operation interface, wherein different images are obtained by multiple image acquisition devices acquiring at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; in response to the processing instruction on the operation interface, a target fusion result is displayed on the operation interface, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, the target fusion result is the result obtained by fusing the recognition results corresponding to multiple images using the target fusion model, the target fusion model is a model obtained by self-supervised learning using synthetic data, the recognition result corresponding to each image is the result obtained by recognizing the target in the corresponding image, and the recognition result is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space, thereby improving the accuracy of target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0178] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0179] According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0180] Figure 8 is a flowchart of an image processing method according to Embodiment 4 of the present application, such as Figure 8 As shown, the method includes:

[0181] Step S802, acquiring multiple images by calling the first interface;

[0182] The first interface includes a first parameter, the parameter value of the first parameter includes multiple images, different images are acquired by different image acquisition devices from capturing at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices.

[0183] The above-mentioned first interface can be an interface for data interaction between the cloud server and the client. The client can pass multiple images into the interface function as the first parameter of the interface function to achieve the purpose of uploading multiple images to the cloud server.

[0184] Step S804, identifying the targets in the multiple images to obtain the recognition results corresponding to the multiple images;

[0185] The recognition result corresponding to each image is used to represent the spatial position of the target contained in the corresponding image in the corresponding device space.

[0186] Step S806, using the target fusion model to fuse the recognition results corresponding to the multiple images to obtain a target fusion result;

[0187] The target fusion result is used to characterize the spatial position of at least one target in a preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0188] Step S808: output the target fusion result by calling the second interface.

[0189] The second interface includes a second parameter, and a parameter value of the second parameter includes a target fusion result.

[0190] The above-mentioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the target fusion result to the interface function as the second parameter of the interface function to achieve the purpose of sending the target fusion result to the client.

[0191] Through the above steps, multiple images are acquired by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes multiple images, and different images are acquired by different image acquisition devices to acquire at least one target in a preset area, and there is an overlapping area in the image acquisition ranges of different image acquisition devices; the targets in the multiple images are identified to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data; the target fusion result is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target fusion result, thereby improving the accuracy of target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0192] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0193] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig. 9 is a schematic diagram of an image processing device according to Embodiment 5 of the present application, such as Fig. 9 As shown, the device 900 includes: an acquisition module 902 , an identification module 904 , and a fusion module 906 .

[0194] Among them, the acquisition module is used to acquire multiple images, wherein different images are obtained by different image acquisition devices capturing at least one target in a preset area, and there is an overlapping area in the image acquisition ranges of different image acquisition devices; the recognition module is used to identify targets in multiple images and obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the fusion module is used to fuse the recognition results corresponding to multiple images using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0195] It should be noted that the acquisition module 902, the identification module 904, and the fusion module 906 correspond to steps S302 to S306 in Example 1, and the three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the modules may also be part of a device and run in the AR / VR device provided in Example 1.

[0196] In the above embodiments of the present application, a fusion module is used to obtain historical fusion results, wherein the historical fusion results are used to characterize the historical spatial position of at least one target in a preset space; a backbone module is used to perform feature extraction on recognition results corresponding to multiple images to obtain image features of multiple images, and the backbone module is used to perform feature extraction on historical fusion results to obtain historical image features; a fusion module is used to fuse image features of multiple images and historical image features to obtain target fusion results.

[0197] In the above embodiments of the present application, the device also includes: a generation module and a supervision module.

[0198] Among them, the acquisition module is used to obtain the preset fusion result of the preset moment, wherein the preset fusion result is used to characterize the spatial positions of multiple objects in the preset area in the preset space at the preset moment; the generation module is used to generate the preset fusion results of multiple moments based on the preset fusion result of the preset moment, wherein the preset moment is the first moment of the multiple moments; the generation module is used to generate a set of preset recognition results of multiple moments based on the preset fusion result of the preset moment, wherein different preset recognition results in the set of preset recognition results of each moment are used to characterize the spatial positions of multiple objects in different device spaces at the corresponding moment; the generation module is used to generate synthetic data based on the set of preset recognition results of any moment among the multiple moments, and the preset fusion results of the historical moments before any moment; the supervision module is used to use the synthetic data to perform self-supervised learning on the initial fusion model to obtain the target fusion model.

[0199] In the above embodiment of the present application, the generation module is also used to determine the offset corresponding to the first moment, wherein the first moment is used to characterize the next moment after the preset moment among multiple moments, and the offset corresponding to the first moment is used to characterize the movement speed of multiple objects at the first moment; the offset corresponding to the first moment is superimposed with the preset fusion result of the preset moment to obtain the preset fusion result of the first moment; determine the offset corresponding to the second moment after the first moment; superimpose the offset corresponding to the second moment with the preset fusion result of the first moment to obtain the preset fusion result of the second moment; take the second moment as the new first moment, and repeat the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with the preset fusion result of the first moment, until the second moment is the last moment among the multiple moments.

[0200] In the above-mentioned embodiment of the present application, the generation module is also used to split the preset fusion results of the preset moment based on the device space corresponding to the multiple image acquisition devices to obtain the preset recognition result set of the preset moment; determine the offset corresponding to the first moment, wherein the first moment is used to characterize the next moment after the preset moment among the multiple moments, and the offset corresponding to the first moment is used to characterize the movement speed of the multiple objects at the first moment; superimpose the offset corresponding to the first moment with the different preset recognition results in the preset recognition result set of the preset moment to obtain the preset recognition result set of the first moment; determine the offset corresponding to the second moment after the first moment; superimpose the offset corresponding to the second moment with the different preset recognition results in the preset recognition result set of the first moment to obtain the preset recognition result set corresponding to the second moment; take the second moment as the new first moment, and repeat the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with the different preset recognition results in the preset recognition result set of the first moment, until the second moment is the last moment among the multiple moments.

[0201] In the above-mentioned embodiment of the present application, the generation module is also used to determine the target device space to which multiple objects belong based on the device space corresponding to multiple image acquisition devices and the preset fusion results at the preset moment; when the target device space to which any one of the multiple objects belongs is one, the spatial position of any one of the objects in the preset fusion result at the preset moment is converted into the spatial position of any one of the objects in the target device space; when the number of target device spaces to which any one of the multiple objects belongs is multiple, the spatial position of any one of the objects in the preset fusion result at the preset moment is split into the spatial positions of any one of the objects in multiple target device spaces.

[0202] In the above embodiment of the present application, the generation module is also used to determine a second offset parameter corresponding to multiple target device spaces, wherein the second offset parameter is used to characterize the offset between the spatial position of any object in the corresponding target device space and the spatial position of any object in the preset space; the spatial position of any object is superimposed with the second offset parameters corresponding to the multiple target device spaces respectively to obtain the spatial position of any object in the multiple target device spaces.

[0203] In the above embodiment of the present application, the generation module is also used to determine the mean of the random distribution based on the number of multiple objects within a preset range; and to perform sampling in the random distribution corresponding to the mean of the random distribution to obtain an offset.

[0204] In the above embodiment of the present application, the supervision module is also used to input the synthesized data into the initial fusion model to obtain the predicted fusion result of any moment output by the initial fusion model, the predicted recognition result set at any moment, and the predicted fusion result at historical moments; based on the predicted recognition result set at any moment and the preset recognition result set, a spatial consistency loss function is constructed; based on the predicted fusion result at the previous moment and the preset fusion result, a temporal consistency loss function is constructed; based on the spatial consistency loss function and the temporal consistency loss function, the model parameters of the initial fusion model are adjusted to obtain the target fusion model.

[0205] In the above embodiment of the present application, the supervision module is also used to input the synthesized data into the initial fusion model to obtain the predicted fusion result at any moment output by the initial fusion model, a set of spatial offsets at any moment, and a time domain offset at any moment, wherein the spatial offset set is used to characterize the offset of the predicted fusion result at any moment relative to the predicted recognition result set at any moment, and the time domain offset is used to characterize the offset of the predicted fusion result at any moment relative to the predicted fusion result at a historical moment; based on the spatial offset set at any moment, the predicted fusion result at any moment is distorted and transformed to obtain a set of predicted recognition results at any moment; based on the time domain offset at any moment, the predicted fusion result at any moment is distorted and transformed to obtain a predicted fusion result at a historical moment.

[0206] In the above embodiments of the present application, the device further includes: an output module, a receiving module, and an updating module.

[0207] Among them, the output module is used to output the target fusion result; the receiving module is used to receive the modified fusion result, wherein the modified fusion result is the result obtained by modifying the target fusion result; the updating module is used to update the target fusion model based on the modified fusion result and multiple images.

[0208] In the above-mentioned embodiment of the present application, the output module is used to output the preset fusion results of multiple moments and the preset recognition result sets of multiple moments; the receiving module is used to receive the modified fusion results of multiple moments and the modified recognition result sets of multiple moments, wherein the modified fusion results of multiple moments are the results obtained by modifying the preset fusion results of multiple moments, and the modified recognition result sets of multiple moments are the results obtained by modifying the preset recognition result sets of multiple moments; the generating module is used to generate synthetic data based on the modified recognition result set of any moment and the modified fusion results of historical moments.

[0209] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0210] According to an embodiment of the present application, a vehicle tracking device for implementing the above vehicle tracking method is also provided. Fig.10 is a schematic diagram of a vehicle tracking device according to Example 6 of the present application, such as Fig.10 As shown, the device 1000 includes: an acquisition module 1002 , a recognition module 1004 , a fusion module 1006 , and a tracking module 1008 .

[0211] Among them, the acquisition module is used to acquire multiple monitoring images, wherein different monitoring images are collected by different monitoring devices on at least one vehicle in the traffic area, and there are overlapping areas in the monitoring ranges of different monitoring devices; the recognition module is used to identify vehicles in multiple monitoring images and obtain recognition results corresponding to multiple monitoring images, wherein the recognition result corresponding to each monitoring image is used to characterize the spatial position of the vehicle contained in the corresponding monitoring image in the corresponding device space; the fusion module is used to use the vehicle recognition model to fuse the recognition results corresponding to the multiple monitoring images to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one vehicle in the bird's-eye view space, and the vehicle recognition model is a model obtained by self-supervised learning using synthetic data; the tracking module is used to track at least one vehicle based on the target fusion result to generate a driving trajectory of at least one vehicle.

[0212] It should be noted that the acquisition module 1002, the identification module 1004, the fusion module 1006, and the tracking module 1008 correspond to steps S602 to S608 in Example 2, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and may be run in the AR / VR device provided in Example 1.

[0213] In the above embodiments of the present application, the fusion module is also used to obtain historical fusion results, wherein the historical fusion results are used to characterize the historical spatial position of at least one vehicle in the bird's-eye view space; the backbone module is used to perform feature extraction on the recognition results corresponding to multiple monitoring images to obtain image features of multiple monitoring images, and the backbone module is used to perform feature extraction on the historical fusion results to obtain historical image features; the fusion module is used to fuse the image features of multiple monitoring images and the historical image features to obtain the target fusion result.

[0214] In the above embodiments of the present application, the device also includes: a generation module and a supervision module.

[0215] Among them, the acquisition module is used to obtain the preset fusion result of the preset moment, wherein the preset fusion result is used to characterize the spatial position of multiple test vehicles in the preset area in the bird's-eye view space at the preset moment; the generation module is used to generate the preset fusion results of multiple moments based on the preset fusion result of the preset moment, wherein the preset moment is the first moment of the multiple moments; the generation module is also used to generate a set of preset recognition results for multiple moments based on the preset fusion result of the preset moment, wherein different preset recognition results in the set of preset recognition results at each moment are used to characterize the spatial positions of multiple test vehicles in different equipment spaces at the corresponding moment; the generation module is also used to generate synthetic data based on the set of preset recognition results at any moment in the multiple moments, and the preset fusion results of historical moments before any moment; the supervision module is also used to use the synthetic data to perform self-supervised learning on the initial fusion model to obtain a vehicle recognition model.

[0216] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0217] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.11 is a schematic diagram of an image processing device according to Embodiment 7 of the present application, such as Fig.11 As shown, the device 1100 includes: a first display module 1102 and a second display module 1104 .

[0218] Among them, the first display module is used to respond to the input instructions acting on the operation interface, and display multiple images on the operation interface, wherein different images are obtained by multiple image acquisition devices acquiring at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; the second display module is used to respond to the processing instructions acting on the operation interface, and display the target fusion result on the operation interface, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion result is the result obtained by fusing the recognition results corresponding to multiple images using the target fusion model, and the target fusion model is a model obtained by self-supervised learning using synthetic data, and the recognition result corresponding to each image is the result obtained by identifying the target in the corresponding image, and the recognition result is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space.

[0219] It should be noted that the first display module 1102 and the second display module 1104 correspond to steps S702 to S704 in Example 3, and the two modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be part of a device and run in the AR / VR device provided in Example 1.

[0220] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0221] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.12 is a schematic diagram of an image processing device according to Embodiment 8 of the present application, such as Fig.12 As shown, the device 1200 includes: an acquisition module 1202 , a recognition module 1204 , a fusion module 1206 , and an output module 1208 .

[0222] Among them, the acquisition module is used to acquire multiple images by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter includes multiple images, different images are obtained by different image acquisition devices acquiring at least one target in a preset area, and there is an overlapping area in the image acquisition range of different image acquisition devices; the recognition module is used to identify the targets in the multiple images and obtain the recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the fusion module is used to fuse the recognition results corresponding to the multiple images using the target fusion model to obtain the target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data; the output module is used to output the target fusion result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target fusion result.

[0223] It should be noted that the acquisition module 1202, the identification module 1204, the fusion module 1206, and the output module 1208 correspond to steps S802 to S808 in Example 4, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and run in the AR / VR device provided in Example 1.

[0224] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0225] An embodiment of the present application may provide a computer terminal, which may include a server and a client. The server may be any server in a server device group or a cloud server.

[0226] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0227] In this embodiment, the computer terminal can execute the program code in the method.

[0228] Optionally, Fig.13 is a structural block diagram of a computer terminal according to an embodiment of the present application. Fig.13 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 102, a memory 104, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0229] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0230] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: acquiring multiple images, wherein different images are obtained by different image acquisition devices capturing at least one target in a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; identifying the targets in the multiple images to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; using a target fusion model to fuse the recognition results corresponding to the multiple images to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0231] By using the embodiment of the present application, multiple images are acquired, wherein different images are acquired by different image acquisition devices capturing at least one target within a preset area, and there is an overlapping area between the image acquisition ranges of the different image acquisition devices; the targets in the multiple images are identified to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data, thereby achieving improved accuracy in target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0232] It can be understood by those skilled in the art that Fig.13 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android or iOS phone), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Fig.13The structure of the electronic device is not limited. For example, the computer terminal A may also include Fig.13 More or fewer components (such as network interfaces, display devices, etc.) as shown in the figure, or Fig.13 Different configurations shown.

[0233] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. Example

[0234] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0235] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in the computer terminal cluster, or in any one of the mobile terminals in the mobile terminal group.

[0236] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: acquiring multiple images, wherein different images are acquired by different image acquisition devices capturing at least one target within a preset area, and there is an overlapping area between the image acquisition ranges of different image acquisition devices; identifying targets in the multiple images to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; and fusing the recognition results corresponding to the multiple images using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data.

[0237] By using the embodiment of the present application, multiple images are acquired, wherein different images are acquired by different image acquisition devices capturing at least one target within a preset area, and there is an overlapping area between the image acquisition ranges of the different image acquisition devices; the targets in the multiple images are identified to obtain recognition results corresponding to the multiple images, wherein the recognition result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; the recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of at least one target in the preset space, and the target fusion model is a model obtained by self-supervised learning using synthetic data, thereby achieving improved accuracy in target recognition. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies. Example

[0238] According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0239] Fig.14 is a flowchart of an image processing method according to Embodiment 11 of the present application, such as Fig.14 As shown, the method includes: step S1402, obtaining a preset fusion result at a preset time.

[0240] The preset fusion result is used to represent the spatial positions of multiple objects in a preset area in a preset space at a preset time.

[0241] Step S1404: generating preset fusion results at multiple moments based on the preset fusion result at the preset moment.

[0242] The preset moment is the first moment among the multiple moments.

[0243] Step S1406, generating a set of preset recognition results at multiple moments based on the preset fusion result at the preset moment.

[0244] The different preset recognition results in the preset recognition result set at each moment are used to represent the spatial positions of multiple objects in different device spaces at the corresponding moment.

[0245] Step S1408, generating synthetic data based on a preset recognition result set of any one of the multiple moments and a preset fusion result of a historical moment before any one of the moments.

[0246] Step S1410, using the synthetic data to perform self-supervised learning on the initial fusion model to obtain a target fusion model.

[0247] The target fusion model is used to execute any one of the methods in the above embodiments.

[0248] By collecting preset recognition result sets at multiple moments and preset fusion results at historical moments, a more comprehensive and accurate data set can be obtained. Using these data sets to generate synthetic data can help the model better learn the associations and patterns between data. By performing self-supervised learning on the initial fusion model with synthetic data, the model parameters can be continuously adjusted and updated by comparing the model prediction results with the actual results of the synthetic data, thereby obtaining a more accurate target fusion model. Synthetic data is generated by combining data from multiple moments and historical information, and then the model is continuously improved through self-supervised learning, thereby improving the performance of the model in fusion tasks.

[0249] Through the above steps, a preset fusion result of a preset moment is obtained, wherein the preset fusion result is used to characterize the spatial positions of multiple objects in a preset area in a preset space at a preset moment; based on the preset fusion result of the preset moment, preset fusion results of multiple moments are generated, wherein the preset moment is the first moment of the multiple moments; based on the preset fusion result of the preset moment, a set of preset recognition results of multiple moments is generated, wherein different preset recognition results in the set of preset recognition results of each moment are used to characterize the spatial positions of multiple objects in different device spaces at the corresponding moment; based on the set of preset recognition results of any moment in the multiple moments, and the preset fusion results of historical moments before any moment, synthetic data is generated; and the synthetic data is used to perform self-supervised learning on the initial fusion model to obtain the target fusion model. It is easy to notice that by identifying targets in multiple images acquired by multiple image acquisition devices, the spatial positions of targets contained in the multiple images in the corresponding device space can be obtained. The spatial positions of targets in multiple images in different device spaces can be fused through the target fusion model to obtain the spatial positions of targets in the same preset space, thereby achieving the purpose of fusing the spatial positions of targets across devices. By performing self-supervised learning on the target fusion model with synthetic data, the accuracy of model fusion can be improved while reducing the cost of model learning, thereby improving the accuracy of fusing the recognition results corresponding to multiple images, and obtaining the spatial position of the target with higher accuracy in the preset space, so as to improve the accuracy of target recognition, thereby solving the technical problem of low target recognition accuracy in related technologies.

[0250] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0251] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.15 is a schematic diagram of an image processing device according to Embodiment 12 of the present application, such as Fig.15 As shown, the device 1500 includes: an acquisition module 1502 , a first generation module 1504 , a second generation module 1506 , a third generation module 1508 , and a supervision module 1510 .

[0252] Among them, the acquisition module is used to obtain the preset fusion result of the preset moment, wherein the preset fusion result is used to characterize the spatial positions of multiple objects in the preset area in the preset space at the preset moment; the first generation module is used to generate the preset fusion results of multiple moments based on the preset fusion result of the preset moment, wherein the preset moment is the first moment of the multiple moments; the second generation module is used to generate a set of preset recognition results of multiple moments based on the preset fusion result of the preset moment, wherein different preset recognition results in the set of preset recognition results of each moment are used to characterize the spatial positions of multiple objects in different device spaces at the corresponding moment; the third generation module is used to generate synthetic data based on the set of preset recognition results of any moment among the multiple moments, and the preset fusion results of the historical moments before any moment; the supervision module is used to use the synthetic data to perform self-supervised learning on the initial fusion model to obtain the target fusion model, wherein the target fusion model is used to execute any one of the methods in the above embodiments.

[0253] It should be noted that the acquisition module 1502, the first generation module 1504, the second generation module 1506, the third generation module 1508, and the supervision module 1510 correspond to steps S1402 to S1410 in Example 11, and the five modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and run in the AR / VR device provided in Example 1.

[0254] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1. Example

[0255] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor. Example

[0256] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented. Example

[0257] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0258] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0259] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0260] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0261] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0262] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0263] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0264] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire multiple images, wherein different images are acquired by different image acquisition devices capturing at least one target within a preset area, and image acquisition ranges of the different image acquisition devices have overlapping areas; Identify the targets in the multiple images to obtain identification results corresponding to the multiple images, wherein the identification result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; The recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of the at least one target in a preset space, the target fusion model is a model obtained by self-supervised learning using synthetic data, the synthetic data is generated based on a preset recognition result set at any one of multiple moments, and a preset fusion result at a historical moment before the any one moment, the preset fusion result is used to characterize the spatial positions of multiple objects in the preset area in the preset space, and different preset recognition results in the preset recognition result set are used to characterize the spatial positions of the multiple objects in different device spaces at corresponding moments; The target fusion model is used to fuse the recognition results corresponding to the multiple images to obtain the target fusion result, including: Using the backbone module to extract features from the recognition results and historical fusion results corresponding to the multiple images, to obtain image features and historical image features of the multiple images; The image features of the multiple images and the historical image features are fused using a fusion module to obtain the target fusion result.

2. The method according to claim 1, characterized in that: The method further comprises: Obtain the preset fusion result at the preset time; Based on the preset fusion result at the preset moment, generating preset fusion results at multiple moments, wherein the preset moment is the first moment among the multiple moments; Based on the preset fusion result of the preset moment, a set of preset recognition results of the multiple moments is generated, wherein different preset recognition results in the set of preset recognition results of each moment are used to characterize the spatial positions of the multiple objects in different device spaces at the corresponding moment, and the set of preset recognition results of the multiple moments is obtained by superimposing the offset of the next moment after the preset moment and the preset fusion result of the preset moment.

3. The method according to claim 2, characterized in that The generating of preset fusion results at multiple moments based on the preset fusion result at the preset moment includes: Determine an offset corresponding to a first moment, wherein the first moment is used to represent a next moment after the preset moment among the multiple moments, and the offset corresponding to the first moment is used to represent a movement speed of the multiple objects at the first moment; Superimposing the offset corresponding to the first moment with the preset fusion result at the preset moment to obtain the preset fusion result at the first moment; Determining an offset corresponding to a second moment after the first moment; Superimposing the offset corresponding to the second moment with the preset fusion result at the first moment to obtain the preset fusion result at the second moment; The second moment is taken as the new first moment, and the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with the preset fusion result of the first moment are repeated until the second moment is the last moment of the multiple moments.

4. The method according to claim 2, characterized in that: The generating a set of preset recognition results at multiple moments based on the preset fusion result at the preset moment includes: Determine the target device space to which the multiple objects belong based on the device spaces corresponding to the multiple image acquisition devices and the preset fusion results at the preset time; Splitting the preset fusion result at the preset time according to the number of the target device spaces to obtain a preset recognition result set at the preset time; Determine an offset corresponding to a first moment, wherein the first moment is used to represent a next moment after the preset moment among the multiple moments, and the offset corresponding to the first moment is used to represent a movement speed of the multiple objects at the first moment; Superimposing the offset corresponding to the first moment with different preset recognition results in the preset recognition result set at the preset moment to obtain the preset recognition result set at the first moment; Determining an offset corresponding to a second moment after the first moment; Superimposing the offset corresponding to the second moment with different preset recognition results in the preset recognition result set at the first moment to obtain a preset recognition result set corresponding to the second moment; The second moment is taken as the new first moment, and the steps of determining the offset corresponding to the second moment and superimposing the offset corresponding to the second moment with different preset recognition results in the preset recognition result set of the first moment are repeated until the second moment is the last moment of the multiple moments.

5. The method according to claim 4, characterized in that The step of splitting the preset fusion result at the preset moment based on the device space corresponding to the multiple image acquisition devices to obtain the preset recognition result set at the preset moment includes: Determine the target device space to which the multiple objects belong based on the device spaces corresponding to the multiple image acquisition devices and the preset fusion results at the preset time; In a case where the target device space to which any one of the multiple objects belongs is one, converting the spatial position of the any one of the objects in the preset fusion result at the preset time into the spatial position of the any one of the objects in the target device space; In the case that any one of the multiple objects belongs to multiple target device spaces, the spatial position of the any one object in the preset fusion result at the preset moment is split into the spatial positions of the any one object in the multiple target device spaces.

6. The method according to claim 5, characterized in that The step of splitting the spatial position of any one of the objects in the preset fusion result at the preset time into the spatial positions of the any one of the objects in the spaces of multiple target devices includes: Determine second offset parameters corresponding to the multiple target device spaces, wherein the second offset parameter is used to represent an offset between a spatial position of any one of the objects in the corresponding target device space and a spatial position of any one of the objects in the preset space; The spatial position of the arbitrary object is superimposed on the second offset parameters corresponding to the multiple target device spaces respectively to obtain the spatial position of the arbitrary object in the multiple target device spaces.

7. The method according to claim 3 or 4, characterized in that: The determining the offset corresponding to the first moment includes: Determining a random distribution mean based on the number of the plurality of objects within a preset range; Sampling is performed in the random distribution corresponding to the mean of the random distribution to obtain the offset.

8. The method according to claim 2, characterized in that: The using the synthetic data to perform self-supervised learning on the initial fusion model to obtain the target fusion model includes: Inputting the synthesized data into the initial fusion model, obtaining the predicted fusion result at any moment output by the initial fusion model, the predicted recognition result set at any moment, and the predicted fusion result at the historical moment; Constructing a spatial consistency loss function based on the predicted recognition result set and the preset recognition result set at any moment; Based on the predicted fusion result of the previous moment and the preset fusion result, a temporal consistency loss function is constructed; Based on the spatial domain consistency loss function and the temporal domain consistency loss function, the model parameters of the initial fusion model are adjusted to obtain the target fusion model.

9. The method according to claim 8, characterized in that The step of inputting the synthesized data into the initial fusion model to obtain the prediction fusion result at any moment output by the initial fusion model, the prediction recognition result set at any moment, and the prediction fusion result at the historical moment includes: Input the synthesized data into the initial fusion model to obtain the predicted fusion result at any moment output by the initial fusion model, the spatial offset set at any moment, and the time domain offset at any moment, wherein the spatial offset set is used to characterize the offset of the predicted fusion result at any moment relative to the predicted recognition result set at any moment, and the time domain offset is used to characterize the offset of the predicted fusion result at any moment relative to the predicted fusion result at the historical moment; Based on the spatial offset set at any moment, a prediction fusion result at any moment is distorted and transformed to obtain a prediction recognition result set at any moment; Based on the time domain offset of any one moment, the prediction fusion result of any one moment is distorted and transformed to obtain the prediction fusion result of the historical moment.

10. The method according to claim 1, characterized in that The method further comprises: Outputting the target fusion result; receiving a modified fusion result, wherein the modified fusion result is a result obtained by modifying the target fusion result; Based on the modified fusion result and the multiple images, the target fusion model is updated.

11. The method according to claim 2, characterized in that The method further comprises: Outputting the preset fusion results of the multiple moments and the preset recognition result sets of the multiple moments; Receiving the modified fusion results of the multiple moments and the modified recognition result sets of the multiple moments, wherein the modified fusion results of the multiple moments are the results obtained by modifying the preset fusion results of the multiple moments, and the modified recognition result sets of the multiple moments are the results obtained by modifying the preset recognition result sets of the multiple moments; The synthetic data is generated based on the set of modification recognition results at any one moment and the modification fusion results at the historical moments.

12. A vehicle tracking method, characterized in that: include: Acquire multiple monitoring images, wherein different monitoring images are acquired by different monitoring devices from at least one vehicle in a traffic area, and the monitoring ranges of the different monitoring devices have overlapping areas; Identify the vehicles in the multiple monitoring images to obtain identification results corresponding to the multiple monitoring images, wherein the identification result corresponding to each monitoring image is used to characterize the spatial position of the vehicle contained in the corresponding monitoring image in the corresponding device space; The recognition results corresponding to the multiple monitoring images are fused using a vehicle recognition model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of the at least one vehicle in the bird's-eye view space, the vehicle recognition model is a model obtained by self-supervised learning using synthetic data, the synthetic data is generated based on a preset recognition result set at any one of multiple moments, and a preset fusion result at a historical moment before the any one moment, the preset fusion result is used to characterize the spatial positions of multiple test vehicles in the bird's-eye view space in the bird's-eye view space, and different preset recognition results in the preset recognition result set are used to characterize the spatial positions of the multiple test vehicles in different device spaces at corresponding moments; Based on the target fusion result, tracking the at least one vehicle to generate a driving trajectory of the at least one vehicle; The vehicle recognition model at least includes a backbone module and a fusion module; the vehicle recognition model is used to fuse the recognition results corresponding to the multiple monitoring images to obtain a target fusion result, including: Using the backbone module to extract features from the recognition results and historical fusion results corresponding to the multiple monitoring images, to obtain image features and historical image features of the multiple monitoring images; The fusion module is used to fuse the image features of the multiple monitoring images and the historical image features to obtain the target fusion result.

13. The method according to claim 12, characterized in that The method further comprises: Obtain the preset fusion result at the preset time; Based on the preset fusion result at the preset moment, generating preset fusion results at multiple moments, wherein the preset moment is the first moment among the multiple moments; Based on the preset fusion result at the preset moment, generating a preset recognition result set at the multiple moments, wherein different preset recognition results in the preset recognition result set at each moment are used to characterize the spatial positions of the multiple test vehicles in different equipment spaces at the corresponding moment, and the preset recognition result set at the multiple moments is obtained by superimposing the offset at the next moment after the preset moment and the preset fusion result at the preset moment; Generate the synthetic data based on a preset recognition result set of any one of the multiple moments and a preset fusion result of a historical moment before the any one moment; The synthetic data is used to perform self-supervised learning on the initial fusion model to obtain the vehicle recognition model.

14. An image processing method, characterized in that: include: Acquire multiple images by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes the multiple images, different images are acquired by different image acquisition devices acquiring at least one target within a preset area, and image acquisition ranges of the different image acquisition devices have overlapping areas; Identify the targets in the multiple images to obtain identification results corresponding to the multiple images, wherein the identification result corresponding to each image is used to characterize the spatial position of the target contained in the corresponding image in the corresponding device space; The recognition results corresponding to the multiple images are fused using a target fusion model to obtain a target fusion result, wherein the target fusion result is used to characterize the spatial position of the at least one target in a preset space, the target fusion model is a model obtained by self-supervised learning using synthetic data, the synthetic data is generated based on a preset recognition result set at any one of multiple moments, and a preset fusion result at a historical moment before the any one moment, the preset fusion result is used to characterize the spatial positions of multiple objects in the preset area in the preset space, and different preset recognition results in the preset recognition result set are used to characterize the spatial positions of the multiple objects in different device spaces at corresponding moments; Outputting the target fusion result by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the target fusion result; The target fusion model is used to fuse the recognition results corresponding to the multiple images to obtain the target fusion result, including: Using the backbone module to extract features from the recognition results and historical fusion results corresponding to the multiple images, to obtain image features and historical image features of the multiple images; The image features of the multiple images and the historical image features are fused using a fusion module to obtain the target fusion result.

15. A method for training a target fusion model, characterized in that: include: Get multiple images; Identify the objects in the multiple images to obtain identification results corresponding to the multiple images, wherein the identification result corresponding to each image is used to represent the spatial position of the object contained in the corresponding image in the corresponding device space; Determining a preset fusion result of multiple objects in the multiple images at a preset moment based on the recognition results corresponding to the multiple images, wherein the preset fusion result is used to characterize the spatial positions of the multiple objects in the preset area in the preset space at the preset moment; Based on the preset fusion result at the preset moment, generating preset fusion results at multiple moments, wherein the preset moment is the first moment among the multiple moments; Based on the preset fusion result at the preset moment and the offsets at different moments, generating a set of preset recognition results at the multiple moments, wherein the different preset recognition results in the set of preset recognition results at each moment are used to characterize the spatial positions of the multiple objects in different device spaces reflected in the image at the corresponding moment, and the offsets at different moments are used to represent the movement speeds of the multiple objects at different moments; Generate synthetic data based on a preset recognition result set of any one of the multiple moments and a preset fusion result of a historical moment before the any one of the moments; The synthetic data is used to perform self-supervised learning on the initial fusion model to obtain a target fusion model, wherein the target fusion model includes a backbone module and a fusion module, the backbone module is used to extract features of the recognition results and historical fusion results corresponding to the multiple images to obtain image features and historical image features of the multiple images; the fusion module fuses the image features of the multiple images with the historical image features to obtain a target fusion result.

16. A computer terminal, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 15 when running.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 15.

18. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.

Images