Data processing method and device

Abstract

The invention provides a data processing method and device which can be applied to the technical field of data processing. The data processing method comprises the following steps: acquiring initial radar data and initial image data about a target object; fusing the initial radar data and the initial image data to obtain fused data; inputting the fusion data into an intelligent model to obtain target data about the target object, the target data representing target object monitoring information; wherein the monitoring effect of the monitoring information on the target object is better than the monitoring effect of the initial radar data or the initial image data on the target object.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and more particularly to a data processing method and apparatus. Background Art

[0002] Video tracking technology can capture the visual features of a target in detail, but performs poorly in scenarios such as insufficient light, line-of-sight occlusion, or fast target movement. On the other hand, radar tracking technology performs excellently in measuring distance, especially in poor visibility conditions, but its performance in target recognition and tracking is often limited because it usually provides a low spatial resolution and lacks the capture of detailed visual features of the target. Summary of the Invention

[0003] In view of the above problems, the present disclosure provides a data processing method and apparatus.

[0004] According to a first aspect of the present disclosure, there is provided a data processing method, including: obtaining initial radar data and initial image data about a target object; fusing the initial radar data and the initial image data to obtain fused data; inputting the fused data into an intelligent model to obtain target data about the target object, where the target data represents monitoring information of the target object; wherein the monitoring effect of the target object through the monitoring information is better than the monitoring effect of the initial radar data or the initial image data on the target object.

[0005] According to an embodiment of the present disclosure, fusing the initial radar data and the initial image data to obtain fused data includes: constructing a radar display feature according to the initial radar data, where the radar display feature represents displacement information of the target object; inputting the initial image data into a multi-modal image detection model to obtain an image feature, where the image feature represents attribute information of the target object; determining the fused data according to the image feature and the radar display feature.

[0006] According to an embodiment of the present disclosure, constructing a radar display feature according to the initial radar data includes: obtaining an image placeholder about the initial image data, where the ratio of the image placeholder to the initial image data is the same; preprocessing the initial radar data according to the image placeholder to obtain transitional radar data; splicing the transitional radar data and the image placeholder to obtain spliced data; mapping the spliced data to obtain the radar display feature.

[0007] According to an embodiment of the present disclosure, preprocessing the initial radar data according to the image placeholder to obtain transitional radar data includes: converting the initial radar data into transitional radar data, where the data types and sizes of the transitional radar data and the image placeholder are the same.

[0008] According to an embodiment of the present disclosure, the method further includes: supplementing the transitional radar data when the data volume of the initial radar data is less than the image placeholder.

[0009] According to an embodiment of the present disclosure, determining the fusion data according to the image feature and the radar display feature includes: replacing the image placeholder in the radar display feature with the image feature to obtain the fusion data.

[0010] According to an embodiment of the present disclosure, the intelligent model can be obtained through the following operations: acquiring the historical radar data, historical image data, and historical target data of multiple historical target objects; fusing the historical radar data and historical image data of each historical target object to obtain the historical fusion data corresponding to the historical target object; inputting the multiple historical fusion data into the intelligent model to be trained to obtain the predicted target data; and adjusting the parameters of the intelligent model to be trained according to the predicted target data and the historical target data to obtain the intelligent model.

[0011] According to an embodiment of the present disclosure, fusing the historical radar data and historical image data of each historical target object to obtain the historical fusion data corresponding to the historical target object includes: constructing a historical radar display feature according to the historical radar data, where the historical radar display feature represents the displacement information of the historical target object; inputting the historical image data into a multi-modal image detection model to obtain historical image features, where the historical image features represent the attribute information of the target object; and determining the historical fusion data according to the historical image features and the historical radar features.

[0012] According to an embodiment of the present disclosure, constructing a historical radar display feature according to the historical radar data includes: acquiring a historical image placeholder for the historical image data; preprocessing the historical radar data according to the historical image placeholder to obtain intermediate radar data; splicing the intermediate radar data with the historical image placeholder to obtain historical spliced data; and mapping the historical spliced data to obtain the historical radar display feature; wherein the historical radar data is converted into intermediate radar data with the same data type and size as the historical image placeholder data.

[0013] According to an embodiment of the present disclosure, determining the historical fusion data according to the historical image features and the historical radar features includes: replacing the historical image placeholder in the historical radar display feature with the historical image features to obtain the historical fusion data.

[0014] The second aspect of the present disclosure provides a data processing device, including: an acquisition module configured to acquire initial radar data and initial image data regarding a target object; a fusion module configured to fuse the initial radar data and the initial image data to obtain fused data; a processing module configured to input the fused data into an intelligent model to obtain target data regarding the target object, where the target data represents monitoring information of the target object; wherein the monitoring effect of the target object through the monitoring information is better than that of the initial radar data or the initial image data on the target object.

[0015] The third aspect of the present disclosure provides an electronic device, including: one or more processors; a memory configured to store one or more computer programs, wherein the above-mentioned one or more processors execute the above-mentioned one or more computer programs to implement the steps of the above-mentioned method.

[0016] The fourth aspect of the present disclosure further provides a computer-readable storage medium, on which computer programs or instructions are stored, and when the computer programs or instructions are executed by a processor, the steps of the above-mentioned method are implemented.

[0017] The fifth aspect of the present disclosure further provides a computer program product, including computer programs or instructions, and when the computer programs or instructions are executed by a processor, the steps of the above-mentioned method are implemented. Description of the Drawings

[0018] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above-mentioned content and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0019] Figure 1 Schematically shows one of the flowcharts of the data processing method according to an embodiment of the present disclosure;

[0020] Figure 2 Schematically shows another flowchart of the data processing method according to an embodiment of the present disclosure;

[0021] Figure 3 Schematically shows yet another flowchart of the data processing method according to an embodiment of the present disclosure;

[0022] Figure 4 Schematically shows one of the flowcharts of the intelligent model training method according to an embodiment of the present disclosure;

[0023] Figure 5 Schematically shows another flowchart of the intelligent model training method according to an embodiment of the present disclosure;

[0024] Figure 6 Schematically shows yet another flowchart of the intelligent model training method according to an embodiment of the present disclosure;

[0025] Figure 7 Schematically shows the schematic diagram of the data processing method according to an embodiment of the present disclosure;

[0026] Figure 8 Schematically shows the structural block diagram of the data processing apparatus according to an embodiment of the present disclosure; and

[0027] Figure 9 Schematically shows the block diagram of the electronic device suitable for implementing the data processing method according to an embodiment of the present disclosure. Detailed implementation manners

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0029] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0032] Embodiments of the present disclosure provide a data processing method and apparatus. Before introducing the technical solutions provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure will be described first.

[0033] Video tracking technology can capture the visual features of targets in detail, but it performs poorly in scenarios such as insufficient light, line-of-sight occlusion, or fast target movement. On the other hand, radar tracking technology excels in measuring distance, especially in poor visibility conditions. However, due to its generally low spatial resolution and lack of capture of detailed visual features of targets, its performance in target recognition and tracking is often limited.

[0034] Exemplarily, in adverse weather conditions such as at night, in rain or fog, the images captured by a video camera may be affected by insufficient lighting, raindrop or fog occlusion, etc., resulting in a decrease in image quality, which in turn affects the target recognition and detection effects. At the same time, although radar has the ability to work all-weather, it is vulnerable to rain clutter and fog.

[0035] The data collected by radar and cameras differ in format, sampling rate, timestamp, etc., making it difficult to achieve precise data fusion. In addition, there may be compatibility issues between devices produced by different manufacturers, further increasing the difficulty of data fusion.

[0036] Due to reasons such as occlusion and detection errors, the radar and video fusion system may have false alarms and missed detections. For example, when a ship is occluded by other objects, the system may not be able to accurately detect the presence of the ship; or when the system misidentifies other targets as ship targets, false alarms will occur.

[0037] Embodiments of the present disclosure provide a data processing method, which includes: obtaining initial radar data and initial image data about a target object; fusing the initial radar data and the initial image data to obtain fused data; inputting the fused data into an intelligent model to obtain target data about the target object, where the target data represents monitoring information of the target object; wherein the monitoring effect on the target object through the monitoring information is better than the monitoring effect of the initial radar data or the initial image data on the target object.

[0038] The following will be through Figures 1 to 6 to describe in detail the data processing method of the embodiments of the present disclosure.

[0039] Figure 1 Schematically shows one of the flowcharts of the data processing method according to an embodiment of the present disclosure.

[0040] As Figure 1 shown, the data processing method of this embodiment includes operation S210 to operation S230.

[0041] In operation S210, obtain initial radar data and initial image data about a target object.

[0042] Exemplarily, the initial radar data can be collected by a radar sensor and usually includes information such as the azimuth, distance, speed, and motion trajectory of the target object. The initial radar data includes, but is not limited to, the distance, speed, azimuth angle, echo intensity, etc. of the target object.

[0043] The initial image data can be collected by an image sensor, such as a camera or an infrared sensor. The image data contains the visual information of the target object and can provide the appearance features, position, and details of the target. The initial image data can include a two-dimensional image or three-dimensional point cloud data of the target object.

[0044] In operation S220, the initial radar data and the initial image data are fused to obtain fused data.

[0045] Exemplarily, the features in the radar data and the image data are extracted and jointly processed. For example, the azimuth of the radar target is combined with the target in the image data to form a more complete target representation (i.e., fused data).

[0046] In the process of fusing the data, a multi-dimensional target data representation can be obtained, which synthesizes the azimuth information of the radar and the appearance information of the image to form more accurate data. These fused data will contain information such as the motion trajectory, appearance features, position, and speed of the target.

[0047] In operation S230, the fused data is input into the intelligent model to obtain target data about the target object, and the target data represents the monitoring information of the target object.

[0048] Among them, the monitoring effect of the target object through the monitoring information is better than that of the initial radar data or the initial image data on the target object.

[0049] Exemplarily, the fusion of radar and image data makes information such as the position, speed, and type of the target more accurate. The intelligent model can accurately identify and classify the target according to the appearance features and motion laws of the target. In complex scenarios, the fused data helps the intelligent model distinguish multiple targets and perform independent tracking.

[0050] For example, in the field of autonomous driving, radar and image data are fused for vehicle and pedestrian detection, tracking, and classification to improve the safety and reliability of the autonomous driving system. In the field of drone monitoring, by fusing data from different sensors for flight path planning and target recognition, the autonomous navigation ability of the aircraft in complex environments is improved.

[0051] It is understandable that radar data can provide all-weather target monitoring capabilities. Especially in low visibility, adverse weather, or complex environments, radar can effectively detect the presence of targets. Image data, on the other hand, can provide high-precision appearance information of targets, such as color, shape, and texture, enhancing the ability of target detection and recognition. Fusing radar data and image data can enhance the ability of target detection and recognition.

[0052] Figure 2 Schematically shows the second flowchart of the data processing method according to an embodiment of the present disclosure.

[0053] As mentioned above, in operation S220, the initial radar data and the initial image data are fused to obtain fused data. In one implementable way, as Figure 2 shown, this operation may further include operations S310 to S330.

[0054] In operation S310, a radar display feature is constructed based on the initial radar data, and the radar display feature characterizes the displacement information of the target object.

[0055] In operation S320, the initial image data is input into a multi-modal image detection model to obtain image features, and the image data characterizes the attribute information of the target object.

[0056] In operation S330, the fused data is determined based on the image features and the radar display features.

[0057] Exemplarily, the radar display feature may be the displacement information of an unknown target object. For example, in the case where the target object is a ship, the radar display feature may be the position, sailing speed, sailing direction, etc. of a certain ship. The radar display feature has an association relationship with the target object.

[0058] The initial image data contains specific target objects. The initial image data is input into the resample module of the multi-modal image detection model to extract the image features of the target objects.

[0059] Fusing the image features and the radar display features can determine the object corresponding to the target detected by the radar, as well as the motion information and position information, etc. of the object. Image data can identify different types of targets (such as vehicles, pedestrians, animals, etc.). By combining the distance and motion information provided by the radar, image data can not only be used to confirm the type of the target, but also provide the position, speed, and its dynamic behavior (such as whether it is moving, accelerating, etc.) of the target. This comprehensive identification helps to improve the accuracy and robustness of target recognition.

[0060] Figure 3 Schematically shows the third flowchart of the data processing method according to an embodiment of the present disclosure.

[0061] As described above, in operation S310, a radar display feature is constructed based on the initial radar data. In one implementable manner, as Figure 3 shown, this operation may further include operations S410 to S440.

[0062] In operation S410, an image placeholder for the initial image data is obtained, and the image placeholder has the same ratio as the initial image data.

[0063] In operation S420, the initial radar data is preprocessed according to the image placeholder to obtain transitional radar data.

[0064] In operation S430, the transitional radar data is spliced with the image placeholder to obtain spliced data.

[0065] In operation S440, the spliced data is mapped to obtain a radar display feature.

[0066] Exemplarily, an image placeholder is a temporary image used to replace the position of the actual image during the page loading process until the actual image is loaded.

[0067] The image placeholder is usually determined according to the aspect ratio of the actual image (initial image data). This ensures that the placeholder can maintain the same display ratio as the actual image, thereby avoiding changes in the display page layout or content jitter. For example, if the original size of the image is 600x400 pixels, the aspect ratio of the placeholder should be 3:2. When using CSS or other means for placeholder, the consistency between the placeholder and the actual image can be ensured by setting a fixed ratio or a ratio container.

[0068] It can be understood that by splicing the radar data with the image placeholder, a visual smooth transition during the subsequent image loading process can be ensured, thereby enhancing the overall user experience. The image placeholder is used to fill the preliminary framework of the radar display image and provide a visual guide during the data loading process until the real radar image or target image is loaded. This can not only keep the interface beautiful but also avoid layout jitter and enhance the user's viewing experience.

[0069] As described above, in operation S420, the initial radar data is preprocessed according to the image placeholder to obtain transitional radar data. In one implementable manner, this operation may further include: converting the initial radar data into transitional radar data, and the transitional radar data has the same data type and size as the image placeholder data.

[0070] Exemplarily, data such as azimuth information, velocity information, and position information in the initial radar data are converted into strings or matrices that are the same as the image placeholder. The length of the transitional radar data string after the conversion of the initial radar data is consistent with the length of the string representing the image placeholder.

[0071] As described above, the data processing method of this embodiment further includes: when the data volume of the initial radar data is less than the image placeholder, supplementing the transitional radar data.

[0072] For example, if the initial radar data is converted into a string with the same length as the image placeholder. When the actual string length of the initial radar data is less than the string length of the image placeholder, spaces can be added at the end (or the front) of the string until the predetermined length is reached.

[0073] In other embodiments, in addition to spaces, specific characters can also be selected to supplement the string length. For example, characters such as zero (0), asterisk (*), underscore (_), etc. can be selected for supplementation.

[0074] As described above, in operation S330, according to the image features and radar display features, the fusion data is determined. In one implementable manner, this operation may further include: replacing the image placeholder in the radar display features with the image features to obtain the fusion data.

[0075] Exemplarily, the image features of the target object in the initial image data replace the image placeholder in the radar display features. The category and motion information of the target object are obtained.

[0076] Figure 4 Schematically shows one of the flowcharts of the intelligent model training method according to an embodiment of the present disclosure.

[0077] As described above, for the above-mentioned intelligent model. In one implementable manner, as Figure 4 shown, the intelligent model can perform the following operations S510 to operation S540.

[0078] In operation S510, historical radar data, historical image data, and historical target data of multiple historical target objects are obtained.

[0079] In operation S520, the historical radar data and historical image data of each historical target object are fused to obtain historical fusion data corresponding to the historical target object.

[0080] In operation S530, multiple historical fusion data are input into the intelligent model to be trained to obtain predicted target data.

[0081] In operation S540, the parameters of the intelligent model to be trained are adjusted according to the predicted target data and the historical target data to obtain the intelligent model.

[0082] For example, during the training of the intelligent model, training data can be selected first. The training data can include ship image data, radar data corresponding to the ship, and motion data (historical target data) corresponding to the ship; it can also include vehicle image data, radar data corresponding to the vehicle, and motion data (historical target data) corresponding to the vehicle. The training set can include target objects of multiple categories. The embodiments of the present disclosure do not limit this.

[0083] The historical radar data and historical image data are fused to obtain fusion data including information such as the motion trajectory, appearance features, position, and speed of the object. The fusion data is input into the intelligent model, and the appearance features and motion laws of the object are predicted (predicted target data). The historical target data is compared with the actual historical target data, and the parameters of the intelligent model are continuously adjusted until the matching degree between the predicted target data output by the intelligent model and the historical target data reaches a preset threshold, thereby obtaining a trained intelligent model.

[0084] For example, the intelligent model can adopt the llama3 model.

[0085] Figure 5 Schematically shows the second flowchart of the intelligent model training method according to the embodiments of the present disclosure.

[0086] As described above, in operation S520, the historical radar data and historical image data of each historical target object are fused to obtain historical fusion data corresponding to the historical target object. In one implementable manner, as Figure 5 shown, this operation can further include operations S610 to S630.

[0087] In operation S610, a historical radar display feature is constructed based on the historical radar data, and the historical radar display feature characterizes the displacement information of the historical target object.

[0088] In operation S620, the historical image data is input into the multi-modal image detection model to obtain historical image features, and the historical image data characterizes the attribute information of the target object.

[0089] In operation S630, the historical fusion data is determined according to the historical image features and historical radar features.

[0090] Regarding the operation of fusing the historical radar data and historical image data to obtain the historical fusion data during the training of the intelligent model, reference can be made to the above Figure 2 description and will not be elaborated here.

[0091] Figure 6 Schematically shows the third flowchart of the intelligent model training method according to an embodiment of the present disclosure.

[0092] As described above, in operation S610, historical radar display features are constructed based on historical radar data. In one implementable manner, as Figure 6 shown, this operation may further include operations S710 to S740.

[0093] In operation S710, obtain historical image placeholders for historical image data.

[0094] In operation S720, preprocess the historical radar data according to the historical image placeholders to obtain intermediate radar data.

[0095] In operation S730, splice the intermediate radar data with the historical image placeholders to obtain historical spliced data.

[0096] In operation S740, map the historical spliced data to obtain historical radar display features.

[0097] Among them, the historical radar data is converted into intermediate radar data with the same data type and size as the historical image placeholder data.

[0098] During the intelligent model training process, for the operation of constructing historical radar display features, reference may be made to the above Figure 3 description and will not be elaborated here.

[0099] Here, in the process of obtaining historical fusion data, the historical image features of the historical image data are used to replace the historical image placeholders in the historical radar display features.

[0100] For the convenience of understanding the embodiments of the present disclosure, the data processing method of the embodiments of the present disclosure will be described in detail in combination with Figure 7 the following.

[0101] Figure 7 Schematically shows the principle diagram of the data processing method according to an embodiment of the present disclosure.

[0102] Extract image features in the initial image data of the target object through a multi-modal image detection model. And determine image placeholders according to the initial image data. Splice the initial radar data of the target object with the image placeholders and input them into the embedding layer of the multi-modal image detection model to obtain radar display features. Replace the image placeholders in the radar display features with image features to obtain fusion data, and input the fusion data into the intelligent model to output target data about the target object.

[0103] Based on the above data processing method, the present disclosure also provides a data processing apparatus. The following will be combined with Figure 8 to describe the apparatus in detail.

[0104] Figure 8 The structure block diagram of the data processing apparatus according to an embodiment of the present disclosure is schematically shown.

[0105] As Figure 8 shown, the data processing apparatus 800 of this embodiment includes an acquisition module 810, a fusion module 820, and a processing module 830.

[0106] The acquisition module 810 is configured to acquire initial radar data and initial image data about a target object. In one embodiment, the acquisition module 810 may be configured to perform the operation S210 described above, which will not be elaborated here.

[0107] The fusion module 820 is configured to fuse the initial radar data and the initial image data to obtain fused data. In one embodiment, the fusion module 820 may be configured to perform the operation S220 described above, which will not be elaborated here.

[0108] The processing module 830 is configured to input the fused data into an intelligent model to obtain target data about the target object, where the target data represents the monitoring information of the target object. In one embodiment, the processing module 830 may be configured to perform the operation S230 described above, which will not be elaborated here.

[0109] Among them, the monitoring effect of the target object through the monitoring information is better than the monitoring effect of the initial radar data or the initial image data on the target object.

[0110] According to an embodiment of the present disclosure, any one or more of the acquisition module 810, the fusion module 820, and the processing module 830 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the fusion module 820, and the processing module 830 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable means such as integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the acquisition module 810, the fusion module 820, and the processing module 830 may be at least partially implemented as a computer program module, and when the computer program module is run, it can perform corresponding functions.

[0111] Figure 9 Schematically shows a block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present disclosure.

[0112] As Figure 9 shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 901 may also include on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0113] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. The processor 901 performs various operations of the method flow according to an embodiment of the present disclosure by executing the program in the ROM 902 and / or the RAM 903. It should be noted that the program may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to an embodiment of the present disclosure by executing the program stored in one or more memories.

[0114] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, and the input / output (I / O) interface 905 is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. The drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0115] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0116] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903.

[0117] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program includes program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to cause the computer system to implement the data processing method provided by the embodiments of the present disclosure.

[0118] When the computer program is executed by the processor 901, the above functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to the embodiments of the present disclosure, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0119] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and be downloaded and installed through the communication part 909, and / or be installed from the removable medium 911. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0120] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or be installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiments of the present disclosure are executed. According to the embodiments of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0121] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include but are not limited to, such as Java, C++, python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0123] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0124] The embodiments of the present disclosure have been described above. However, these embodiments are merely for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A data processing method, characterized in that, The method includes: Obtaining initial radar data and initial image data regarding a target object; Fusing the initial radar data and the initial image data to obtain fused data; Inputting the fused data into an intelligent model to obtain target data regarding the target object, where the target data represents the monitoring information of the target object; Wherein, the monitoring effect of the target object through the monitoring information is better than the monitoring effect of the initial radar data or the initial image data on the target object.

2. The method according to claim 1, wherein Fusing the initial radar data and the initial image data to obtain fused data, including: Constructing a radar display feature according to the initial radar data, where the radar display feature represents the displacement information of the target object; Inputting the initial image data into a multi-modal image detection model to obtain image features, where the image features represent the attribute information of the target object; Determining the fused data according to the image features and the radar display features.

3. The method according to claim 2, wherein Constructing a radar display feature according to the initial radar data, including: Obtaining an image placeholder regarding the initial image data, where the image placeholder has the same ratio as the initial image data; Preprocessing the initial radar data according to the image placeholder to obtain transitional radar data; Splicing the transitional radar data and the image placeholder to obtain spliced data; Mapping the spliced data to obtain the radar display feature.

4. The method according to claim 3, wherein Preprocessing the initial radar data according to the image placeholder to obtain transitional radar data, including: Converting the initial radar data into the transitional radar data, where the transitional radar data has the same data type and size as the image placeholder data.

5. The method according to claim 4, characterized in that, The method further includes: When the data volume of the initial radar data is less than the image placeholder, supplementing the transitional radar data.

6. The method according to claim 3, wherein Determining the fused data according to the image features and the radar display features, including: Replacing the image placeholder in the radar display feature with the image features to obtain the fused data.

7. The method according to claim 1, characterized in that, The intelligent model can be obtained through the following operations: Obtaining historical radar data, historical image data, and historical target data of multiple historical target objects; Fusing the historical radar data and historical image data of each historical target object to obtain historical fused data corresponding to the historical target object; Inputting multiple pieces of the historical fused data into a to-be-trained intelligent model to obtain predicted target data; Adjusting the parameters of the to-be-trained intelligent model according to the predicted target data and the historical target data to obtain the intelligent model.

8. The method according to claim 7, characterized in that, Fusing the historical radar data and historical image data of each historical target object to obtain historical fused data corresponding to the historical target object, including: Constructing a historical radar display feature according to the historical radar data, where the historical radar display feature represents the displacement information of the historical target object; Inputting the historical image data into a multi-modal image detection model to obtain historical image features, where the historical image features represent the attribute information of the target object; Determine historical fusion data according to the historical image features and the historical radar features.

9. The method according to claim 8, characterized in that, Construct historical radar display features according to the historical radar data, including: Obtain a historical image placeholder for the historical image data; Preprocess the historical radar data according to the historical image placeholder to obtain intermediate radar data; Stitch the intermediate radar data and the historical image placeholder to obtain historical stitched data; Map the historical stitched data to obtain the historical radar display features; Wherein, the historical radar data is converted into the intermediate radar data with the same data type and size as the historical image placeholder data.

10. The method according to claim 9, wherein Determine historical fusion data according to the historical image features and the historical radar features, including: Replace the historical image placeholder in the historical radar display features with the historical image features to obtain the historical fusion data.

11. A data processing device, characterized in that, The device includes: An acquisition module, configured to acquire initial radar data and initial image data of a target object; A fusion module, configured to fuse the initial radar data and the initial image data to obtain fusion data; A processing module, configured to input the fusion data into an intelligent model to obtain target data about the target object, where the target data characterizes the monitoring information of the target object; Wherein, the monitoring effect of the target object through the monitoring information is better than the monitoring effect of the initial radar data or the initial image data on the target object.

Images