Data processing methods, apparatus, electronic devices and storage media

Abstract

This disclosure provides a data processing method, apparatus, electronic device, and storage medium, relating to the field of data processing technology, and particularly to the fields of artificial intelligence, intelligent transportation, and intelligent highway maintenance. The implementation scheme includes: acquiring multiple consecutive frames of images; performing target object detection on each frame to obtain target objects in each frame; determining a second coordinate for each target object based on a first coordinate corresponding to each frame; merging the second coordinates from multiple frames into the same coordinate system to obtain multiple second coordinates; for a third coordinate among the multiple second coordinates, determining a preset number of adjacent coordinates closest to the third coordinate; determining a first distance threshold based on the distance between the third coordinate and each of the preset number of adjacent coordinates; determining a deduplication range at the third coordinate; and merging the target objects corresponding to the coordinates within the deduplication range to obtain a deduplication result.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to the fields of artificial intelligence, smart transportation, and smart highway maintenance. Specifically, it relates to a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] In intelligent highway maintenance and inspection systems, a primary method involves using vehicle-mounted cameras to take photos at fixed distances, followed by deep learning to identify and inventory roadside infrastructure. Traditional methods often result in duplicate images between adjacent pictures, causing the same roadside facility (such as a light pole or gantry) to appear repeatedly in the results. This negatively impacts data analysis and the final inventory summary of roadside infrastructure.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] This disclosure provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] According to one aspect of this disclosure, a data processing method is provided, comprising: acquiring a series of consecutive frames of images, the frames being acquired by an acquisition device while moving along a road, and each frame of the images including a first coordinate of the acquisition device at the time of acquisition; for each frame of the images, performing the following operations: performing target object detection on the frame to obtain at least one target object in the frame; and determining at least one second coordinate corresponding to the at least one target object based on the first coordinate corresponding to the frame; merging the at least one second coordinate corresponding to each frame of the images into the same coordinate system to obtain a plurality of second coordinates; for a third coordinate among the plurality of second coordinates, determining a preset number of adjacent coordinates closest to the third coordinate among the plurality of second coordinates; determining a first distance threshold based on the distance between the third coordinate and each of the preset number of adjacent coordinates; determining a deduplication range at the third coordinate based on the third coordinate and the first distance threshold, so as to determine a plurality of fourth coordinates among the plurality of second coordinates based on the deduplication range; and merging the plurality of target objects corresponding to the plurality of fourth coordinates to obtain a deduplication result, the deduplication result including the coordinates of the merged objects.

[0007] According to another aspect of this disclosure, a data processing apparatus is provided, comprising: a first acquisition unit configured to acquire a series of multiple frames of images, the multiple frames of images being acquired by an acquisition device while moving along a road, and each frame of the multiple frames of images including a first coordinate of the acquisition device at the time of acquisition; an execution unit configured to perform operations of the following subunits for each frame of the multiple frames of images, the execution unit including: a detection subunit configured to perform target object detection on the frame of images to acquire at least one target object in the frame of images; and a first determination subunit configured to determine at least one second coordinate corresponding to the at least one target object based on the first coordinate corresponding to the frame of images; and a second acquisition unit configured to acquire the corresponding coordinates of each frame of images in the multiple frames of images. At least one second coordinate is merged into the same coordinate system to obtain multiple second coordinates; a first determining unit is configured to determine a preset number of adjacent coordinates that are closest to the third coordinate among the multiple second coordinates; a second determining unit is configured to determine a first distance threshold based on the distance between the third coordinate and each of the preset number of adjacent coordinates; a third determining unit is configured to determine a deduplication range at the third coordinate based on the third coordinate and the first distance threshold, so as to determine multiple fourth coordinates among the multiple second coordinates based on the deduplication range; and a deduplication unit is configured to merge multiple target objects corresponding to the multiple fourth coordinates to obtain a deduplication result, the deduplication result including the coordinates of the merged object.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method described above.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described data processing method.

[0010] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-described data processing method when executed by a processor.

[0011] According to one or more embodiments of this disclosure, it is possible to dynamically determine the distance threshold of the deduplication range, thereby improving the accuracy of deduplication of highway object recognition results.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown;

[0015] Figure 2 A flowchart of a data processing method according to an embodiment of the present disclosure is shown;

[0016] Figure 3 A second coordinate distribution diagram of a multi-frame image is shown as an exemplary embodiment of the present disclosure;

[0017] Figure 4 A structural block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown;

[0018] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0021] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0022] In related technologies, deep learning-based target tracking schemes can identify already recognized objects using neural networks and predict which are duplicate recognition results. However, this method often fails to accurately identify multiple road facilities (such as multiple streetlights) because they typically share similar characteristics.

[0023] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes a motor vehicle 110, a server 120, and one or more communication networks 130 that couple the motor vehicle 110 to the server 120.

[0025] In embodiments of this disclosure, the motor vehicle 110 may include a computing device according to embodiments of this disclosure and / or be configured to perform a method according to embodiments of this disclosure.

[0026] Server 120 may run one or more services or software applications that enable the execution of the data processing methods described above. In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. A user of motor vehicle 110 may sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0027] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0028] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0029] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of vehicle 110.

[0030] Network 130 can be any type of network well known to those skilled in the art, and can support data communication using any of a variety of available protocols (including, but not limited to, TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be satellite communication networks, local area networks (LANs), Ethernet-based networks, token ring networks, wide area networks (WANs), the Internet, virtual networks, virtual private networks (VPNs), intranets, extranets, blockchain networks, public switched telephone networks (PSTNs), infrared networks, wireless networks (including, for example, Bluetooth, WiFi), and / or any combination of these with other networks.

[0031] System 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. The data repository 150 may reside in various locations. For example, a data repository used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. The data repository 150 may be of different types. In some embodiments, the data repository used by server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data from and from the database in response to commands.

[0032] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0033] Motor vehicle 110 may include sensors 111 for sensing the surrounding environment. Sensors 111 may include one or more of the following sensors: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter-wave radar, and a lidar (LiDAR). Different sensors can provide different detection accuracy and range. Cameras may be mounted in front of, behind, or at other locations on the vehicle. Visual cameras can capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual cameras, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained. Infrared cameras can capture objects in night vision conditions. Ultrasonic sensors may be mounted around the vehicle to measure the distance of objects outside the vehicle using the strong directionality of ultrasound. Millimeter-wave radar may be mounted in front of, behind, or at other locations on the vehicle to measure the distance of objects outside the vehicle using the characteristics of electromagnetic waves. LiDAR may be mounted in front of, behind, or at other locations on the vehicle to detect the edges and shape information of objects, thereby performing object recognition and tracking. Due to the Doppler effect, the radar device can also measure the speed changes of the vehicle and moving objects.

[0034] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., BeiDou, GPS, GLONASS, and GALILEO) from satellite 141 and generating coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142. The mobile communication network can implement any suitable communication technology, such as current or emerging wireless communication technologies (e.g., 5G technology) like GSM / GPRS, CDMA, and LTE. The communication device 112 may also have a vehicle-to-everything (V2X) module, configured to enable vehicle-to-the-world communication, for example, vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. Furthermore, the communication device 112 may also have a module configured to communicate with a user terminal 145 (including but not limited to smartphones, tablets, or wearable devices such as watches) via, for example, a wireless local area network conforming to the IEEE 502.11 standard or Bluetooth. Using the communication device 112, the motor vehicle 110 can also access the server 120 via the network 130.

[0035] The motor vehicle 110 may also include a control unit 113. The control unit 113 may include a processor, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors, that communicates with various types of computer-readable storage devices or media. The control unit 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking respectively, without human intervention or with limited human intervention. Some processing functions of the control unit 113 can be implemented via cloud computing. For example, some processing can be performed using an onboard processor while other processing can be performed using cloud computing resources. The control unit 113 may be configured to perform methods according to this disclosure. Furthermore, the control unit 113 may be implemented as an example of a computing device on the motor vehicle side (client) according to this disclosure.

[0036] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0037] According to embodiments of this disclosure, such as Figure 2 As shown, a data processing method 200 is provided, including: step S201, acquiring multiple consecutive frames of images, wherein the multiple frames of images are acquired by an acquisition device while moving along a road, and each frame of the multiple frames of images includes a first coordinate of the acquisition device when the image is acquired; for each frame of the multiple frames of images, performing the following operations: step S202, performing target object detection on the frame of images to obtain at least one target object in the frame of images; and step S203, determining at least one second coordinate corresponding to the at least one target object based on the first coordinate corresponding to the frame of images; step S204, merging the at least one second coordinate corresponding to each frame of images in the multiple frames of images into the same frame of images. In a coordinate system, multiple second coordinates are obtained; step S205: for the third coordinate among the multiple second coordinates, a preset number of adjacent coordinates that are closest to the third coordinate among the multiple second coordinates are determined; step S206: based on the distance between the third coordinate and each adjacent coordinate among the preset number of adjacent coordinates, a first distance threshold is determined; step S207: based on the third coordinate and the first distance threshold, a deduplication range at the third coordinate is determined, so as to determine multiple fourth coordinates among the multiple second coordinates based on the deduplication range; and step S208: multiple target objects corresponding to the multiple fourth coordinates are merged to obtain a deduplication result, the deduplication result including the coordinates of the merged object.

[0038] This enables dynamic determination of the distance threshold for deduplication range, improving the accuracy of deduplication in highway object recognition results.

[0039] In some embodiments, the acquisition device can be, for example, a monocular camera, a binocular camera, or a lidar device. This acquisition device can be mounted on a vehicle (such as a data acquisition vehicle or a drone), and as the vehicle moves along the road, it triggers the acquisition of a road image frame at a certain acquisition frequency or in response to detecting that the vehicle has moved a certain distance, thereby obtaining multiple consecutive image frames. The road image can be an RGB image or a point cloud image.

[0040] In some embodiments, when image acquisition is triggered each time, the location information of the vehicle at the time of acquisition can be recorded accordingly, such as the coordinate information of the vehicle in the world coordinate system (i.e., the first coordinate). In some embodiments, the above coordinate information can be obtained, for example, based on a Global Navigation Satellite System (GNSS) or a variant rotary encoder.

[0041] In some embodiments, after acquiring multiple consecutive frames of images, target detection can be performed on the target object in each frame of the image first.

[0042] In some embodiments, the target object may be various facilities along the road, such as streetlights, gantries, signs, etc.

[0043] In some embodiments, a trained object detection model can be applied to detect target objects in each frame of the image and output the detection bounding box of the target object in each frame. The object detection model can be trained using sample images labeled with the target object.

[0044] In some embodiments, after obtaining the target object detection box in each frame image, the coordinate information (i.e., the second coordinate) of each target object in the world coordinate system can be determined based on the position of the detection box in the image and the first coordinate corresponding to the frame image.

[0045] In some embodiments, the coordinates of the target object in the image coordinate system can be transformed to the world coordinate system based on the respective calibration information of the monocular or binocular camera and the position information (i.e., the first coordinates) of the camera when acquiring the image, thereby obtaining the second coordinates of the target object.

[0046] In some embodiments, the coordinates of the target object in the image coordinate system can be transformed to the world coordinate system based on the coordinate information corresponding to each point in the point cloud data and the position information of the lidar when acquiring the image (i.e., the first coordinates), thereby obtaining the second coordinates of the target object.

[0047] In some embodiments, the coordinates of the target object in the image coordinate system can be determined based on a reference point at the detection box of the target object (e.g., the center of the detection box, a point on the bottom edge of the detection box, etc.).

[0048] In some embodiments, after determining the second coordinates of each target object in each frame of an image, all the second coordinates in multiple frames of images can be integrated into the same coordinate system.

[0049] Figure 3 A second coordinate distribution diagram of a multi-frame image of an exemplary embodiment of the present disclosure is shown.

[0050] like Figure 3 As shown, it integrates all the second coordinates from each of the three consecutive frames of images. For example, coordinate points 301, 302, and 303 come from different images.

[0051] In some embodiments, for all the integrated second coordinates, one coordinate point (i.e., the third coordinate) can be selected and other coordinate points can be traversed to determine a preset number of neighboring coordinate points that are closest to the third coordinate.

[0052] According to some embodiments, the preset number can be N-1, where N is the number of images in the multi-frame image. Therefore, by controlling the number of adjacent coordinates, unreasonable deduplication ranges can be avoided, thereby improving the accuracy of deduplication.

[0053] In some exemplary embodiments, reference is made to Figure 3 We can use coordinate point 301 as the third coordinate, and determine two adjacent coordinate points as coordinate point 302 and coordinate point 303 based on the above method.

[0054] Subsequently, a first distance threshold can be determined based on the distance between the third coordinate and each adjacent coordinate to determine the radius of the deduplication range. In some embodiments, the average distance between the third coordinate and each adjacent coordinate can be used as the first distance threshold.

[0055] According to some embodiments, determining a first distance threshold based on the distance between each adjacent coordinate in a third coordinate and a preset number of adjacent coordinates may include: determining a mean distance and a standard deviation distance based on the distance between each adjacent coordinate in a third coordinate and a preset number of adjacent coordinates; and determining a first distance threshold based on the mean distance and the standard deviation distance.

[0056] Therefore, by using the mean and standard deviation of the distance to determine the distance threshold for deduplication, the accuracy of the distance threshold can be improved while dynamically determining the distance threshold, so as to avoid including the coordinates of other target objects in the deduplication range.

[0057] In some embodiments, the mean μ and standard deviation σ of the distance between the third coordinate and each adjacent coordinate can be calculated separately, and a first distance threshold can be determined based on the mean μ and standard deviation σ. For example, the first distance threshold can be determined by calculating the sum of the mean μ and standard deviation σ.

[0058] According to some embodiments, determining a first distance threshold based on the distance mean and the distance standard deviation includes: determining the influence factor of the distance standard deviation; and determining the first distance threshold based on the distance mean, the distance standard deviation, and the influence factor.

[0059] In some embodiments, the first preset distance S can be calculated using the following formula:

[0060] S=μ+factor*σ

[0061] Here, factor represents the influence factor of distance standard deviation, and its value range can be (0,1).

[0062] The first distance threshold calculated in the above manner conforms to a Gaussian distribution. The deduplication range determined based on this first distance threshold can effectively avoid excluding some outliers (such as coordinates that may be the identification results of other target objects) from the deduplication range, thereby avoiding false deduplication and improving the accuracy of the deduplication results. At the same time, by setting an influencing factor, the sensitivity to outlier coordinates can be adjusted, thereby improving the adaptability to different scenarios.

[0063] In some embodiments, after determining the first distance threshold, the deduplication range can be determined based on the first distance threshold and the third coordinate, for example... Figure 3 As shown, the deduplication range 304 can be determined using coordinate point 301 and its corresponding first distance threshold.

[0064] In some embodiments, the coordinate points within the deduplication range can be directly merged into a single merged object, and the location information of the merged object can be determined based on the average value of the coordinate points within the deduplication range, and recorded in the merging result. In some embodiments, the coordinate points within the deduplication range can also be further evaluated to determine the merging method.

[0065] According to some embodiments, merging multiple target objects corresponding to multiple fourth coordinates to obtain deduplication results may include: in response to the multiple fourth coordinates originating from different images in multiple frames, determining the average of the multiple coordinates as the coordinates of the merged object.

[0066] Therefore, by determining whether the target objects corresponding to the coordinates within the deduplication range come from different images, erroneous deduplication of multiple target objects that are close to each other can be avoided.

[0067] In some embodiments, when it is determined that each coordinate point within a deduplication range comes from a different image, it can be determined that the target objects corresponding to multiple coordinate points within the deduplication range are the same target object. In this case, the coordinate points within the deduplication range can be directly merged into a merged object, and the position information of the merged object can be determined based on the average value of the coordinate points within the deduplication range and recorded in the merging result.

[0068] In some embodiments, the relationship between the number of coordinate points included in the deduplication range and the number of images in the multi-frame images can be used to determine whether each coordinate point in the deduplication range comes from a different image. When the number of coordinate points included in the deduplication range is less than or equal to the number of images in the multi-frame images, it can be determined that each coordinate point in the deduplication range comes from a different image. This simplifies the calculations in the deduplication step while maintaining accuracy.

[0069] According to some embodiments, the deduplication result may further include a first number of merged objects. Merging multiple target objects corresponding to multiple fourth coordinates to obtain the deduplication result further includes: in response to the multiple fourth coordinates including at least two coordinates from the same frame image in multiple frames, determining a first number based on the number of at least two coordinates; and determining the coordinates of the merged objects based on the average of the multiple fourth coordinates.

[0070] In real-world scenarios, there are situations where multiple target objects are very close together, such as when light poles and traffic sign poles are placed next to each other. Therefore, the deduplication range determined by the above method will include at least one coordinate point corresponding to each of the multiple target objects.

[0071] In some embodiments, when it is determined that multiple coordinate points within a deduplication range originate from the same image, it can be determined that the target objects corresponding to the multiple coordinate points within the deduplication range correspond to multiple target objects in the actual scene. In this case, the number of coordinate points originating from the same image can be determined first, and the number of target objects included within the deduplication range can be determined based on the number of coordinate points. For example, refer to... Figure 3Another deduplication range 305 contains 5 coordinate points. Among them, coordinate points 306 and 307 come from the same image, coordinate points 308 and 309 come from the same image, and coordinate points 310 and 311 come from the same image. Therefore, it can be determined that multiple coordinate points in the deduplication range 305 correspond to two target objects respectively, and thus it can be determined that the number of merged objects in the deduplication range is 2.

[0072] In some embodiments, the location information of the merged object can be determined directly based on the average value of each coordinate point in the deduplication range, and the number of merged objects can be determined in the manner described above. The two pieces of information are then recorded in the merging result, thereby completing the deduplication of the identification result at that location.

[0073] Therefore, by determining the number of coordinates from the same image within the deduplication range, the number of multiple target objects that are relatively close is determined, and the number of target objects and their shared merged coordinates are recorded in the deduplication result, it is possible to avoid erroneous deduplication while ensuring the accuracy of deduplication and simplifying the calculation in the deduplication process by representing relatively close target objects with the same coordinates.

[0074] In some embodiments, the relationship between the number of coordinate points included in the deduplication range and the number of images in the multi-frame images can be used to determine whether each coordinate point in the deduplication range comes from a different image. When the number of coordinate points included in the deduplication range is greater than the number of images in the multi-frame images, it can be determined that multiple coordinate points in the deduplication range correspond to multiple different target objects. Furthermore, the number of target objects can be determined based on the relationship between the number of coordinate points and the number of images. For example, if there are 5 coordinate points and 3 images in a deduplication range, it can be determined that the deduplication range contains 2 target objects. Thus, while ensuring accuracy, the calculation in the deduplication step can be further simplified.

[0075] In some embodiments, when a deduplication range may contain multiple target objects, a new coordinate point can be randomly selected within this deduplication range, and operations similar to the data processing method described above can be performed. This allows for the re-determination of a more precise deduplication range, ensuring that each coordinate point within a deduplication range corresponds to only one target object. The deduplication result is then obtained based on the re-determined deduplication range. This further improves the accuracy of the deduplication result.

[0076] According to some embodiments, such as Figure 4As shown, a data processing apparatus 400 is also provided, including: a first acquisition unit 410 configured to acquire a series of multiple frames of images, the multiple frames of images being acquired by an acquisition device while moving along a road, and each frame of the multiple frames of images including a first coordinate of the acquisition device when the image is acquired; an execution unit 420 configured to perform the operations of the following sub-units for each frame of the multiple frames of images, the execution unit 420 including: a detection sub-unit 421 configured to perform target object detection on the frame of images to acquire at least one target object in the frame of images; and a first determination sub-unit 422 configured to determine at least one second coordinate corresponding to at least one target object based on the first coordinate corresponding to the frame of images; and a second acquisition unit 430 configured to phase each frame of the multiple frames of images At least one second coordinate is merged into the same coordinate system to obtain multiple second coordinates; a first determining unit 440 is configured to determine a preset number of adjacent coordinates that are closest to the third coordinate among the multiple second coordinates; a second determining unit 450 is configured to determine a first distance threshold based on the distance between the third coordinate and each of the preset number of adjacent coordinates; a third determining unit 460 is configured to determine a deduplication range at the third coordinate based on the third coordinate and the first distance threshold, so as to determine multiple fourth coordinates among the multiple second coordinates based on the deduplication range; and a deduplication unit 470 is configured to merge multiple target objects corresponding to the multiple fourth coordinates to obtain a deduplication result, the deduplication result including the coordinates of the merged object.

[0077] The operations of units 410-470 and subunits 421 and 422 in the data processing device 400 are similar to the operations of steps S201-S208 in the method 200 described above, and will not be repeated here.

[0078] According to some embodiments, the second determining unit may include: a second determining subunit configured to determine a distance mean and a distance standard deviation based on a third coordinate and the distance between each adjacent coordinate in a preset number of adjacent coordinates; and a third determining subunit configured to determine a first distance threshold based on the distance mean and the distance standard deviation.

[0079] According to some embodiments, the third determining subunit may include: a first determining module configured to determine the influence factor of the distance standard deviation; and a second determining module configured to determine a first distance threshold based on the distance mean, the distance standard deviation, and the influence factor.

[0080] According to some embodiments, the deduplication unit may include: a fourth determining subunit configured to determine the average of the multiple coordinates as the coordinates of the merged object in response to multiple fourth coordinates being derived from different images in multiple frames.

[0081] According to some embodiments, the deduplication result may further include a first number of merged objects, and the deduplication unit may further include: a fifth determining subunit configured to determine a first number based on the number of at least two coordinates in response to at least two coordinates from the same frame image in a plurality of fourth coordinates; and a sixth determining subunit configured to determine the coordinates of the merged objects based on the average of the plurality of fourth coordinates.

[0082] According to some embodiments, the preset number can be N-1, where N is the number of images in the multi-frame image.

[0083] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0084] refer to Figure 5 The present invention describes a structural block diagram of an electronic device 500 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0085] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0086] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 can be any type of device capable of inputting information to electronic device 500. Input unit 506 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 507 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 508 may include, but is not limited to, hard disk and optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 502.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0087] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).

[0088] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0092] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0093] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0094] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0095] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A data processing method, comprising: Acquire a series of multiple frames of images, wherein the multiple frames of images are acquired by the acquisition device while moving along the road, and each frame of the multiple frames of images includes the first coordinates of the acquisition device when the image is acquired; For each frame of the multi-frame image, perform the following operations: Perform target object detection on the frame image to obtain at least one target object in the frame image; as well as Based on the first coordinates corresponding to the frame image, determine at least one second coordinate in the world coordinate system corresponding to the at least one target object; At least one second coordinate corresponding to each frame of the multi-frame images is merged into the same coordinate system to obtain multiple second coordinates; For any third coordinate selected from the plurality of second coordinates, determine a preset number of adjacent coordinates that are closest to the third coordinate from the plurality of second coordinates; A first distance threshold is determined based on the distance between each adjacent coordinate in the third coordinate and the preset number of adjacent coordinates; Based on the third coordinate and the first distance threshold, the deduplication range at the third coordinate is determined, so as to determine a plurality of fourth coordinates located within the deduplication range among the plurality of second coordinates; as well as Multiple target objects corresponding to the multiple fourth coordinates are merged to obtain a deduplication result, which includes the coordinates of the merged objects.

2. The method according to claim 1, wherein, Determining the first distance threshold based on the distance between each adjacent coordinate in the third coordinate and the preset number of adjacent coordinates includes: Based on the distance between each adjacent coordinate in the third coordinate and the preset number of adjacent coordinates, determine the mean distance and the standard deviation of the distance; and The first distance threshold is determined based on the mean distance and the standard deviation of the distance.

3. The method according to claim 2, wherein, Determining the first distance threshold based on the mean distance and the standard deviation distance includes: Determine the influence factor of the distance standard deviation; and The first distance threshold is determined based on the mean distance, the standard deviation of the distance, and the influencing factor.

4. The method according to any one of claims 1-3, wherein, The step of merging the multiple target objects corresponding to the multiple fourth coordinates to obtain the deduplication result includes: In response to the fact that the plurality of fourth coordinates are respectively derived from different images in the plurality of frames, the average value of the plurality of coordinates is determined as the coordinates of the merged object.

5. The method according to any one of claims 1-3, wherein, The deduplication result also includes the first number of merged objects, and the process of merging the multiple target objects corresponding to the multiple fourth coordinates to obtain the deduplication result further includes: In response to the plurality of fourth coordinates including at least two coordinates from the same frame image in the plurality of frames, the first quantity is determined based on the number of the at least two coordinates; and The coordinates of the merged object are determined based on the average of the multiple fourth coordinates.

6. The method according to any one of claims 1-3, wherein, The preset quantity is N-1, where N is the number of images in the multi-frame image.

7. A data processing apparatus, comprising: The first acquisition unit is configured to acquire a series of multiple frames of images, which are acquired by the acquisition device while moving along the road, and each frame of the multiple frames includes the first coordinates of the acquisition device when the image is acquired; An execution unit is configured to perform the operations of the following subunits for each of the multiple frames of images, the execution unit comprising: A detection subunit is configured to perform target object detection on the frame image to obtain at least one target object in the frame image; and The first determining subunit is configured to determine at least one second coordinate in the world coordinate system corresponding to the at least one target object based on the first coordinate corresponding to the frame image; The second acquisition unit is configured to merge at least one second coordinate corresponding to each frame of the multi-frame images into the same coordinate system to acquire multiple second coordinates; The first determining unit is configured to, for a third coordinate arbitrarily selected from the plurality of second coordinates, determine a preset number of adjacent coordinates that are closest to the third coordinate among the plurality of second coordinates; The second determining unit is configured to determine a first distance threshold based on the distance between each adjacent coordinate in the third coordinate and the preset number of adjacent coordinates; The third determining unit is configured to determine a deduplication range at the third coordinate based on the third coordinate and the first distance threshold, so as to determine a plurality of fourth coordinates located within the deduplication range among the plurality of second coordinates; and The deduplication unit is configured to merge multiple target objects corresponding to the plurality of fourth coordinates to obtain a deduplication result, the deduplication result including the coordinates of the merged objects.

8. The apparatus according to claim 7, wherein, The second determining unit includes: The second determining subunit is configured to determine the mean distance and the standard deviation of the distance based on the distance between each adjacent coordinate in the third coordinate and the preset number of adjacent coordinates; and The third determining subunit is configured to determine the first distance threshold based on the mean distance and the standard deviation of the distance.

9. The apparatus according to claim 8, wherein, The third determining subunit includes: The first determining module is configured to determine the influence factor of the distance standard deviation; and The second determining module is configured to determine the first distance threshold based on the distance mean, the distance standard deviation, and the influence factor.

10. The apparatus according to any one of claims 7-9, wherein, The deduplication unit includes: The fourth determining subunit is configured to determine the average of the plurality of coordinates as the coordinates of the merged object in response to the plurality of fourth coordinates being derived from different images in the plurality of frames.

11. The apparatus according to any one of claims 7-9, wherein, The deduplication result also includes a first number of merged objects, and the deduplication unit further includes: The fifth determining subunit is configured to, in response to at least two coordinates from the same frame image in the plurality of fourth coordinates, determine the first quantity based on the number of the at least two coordinates; and The sixth determining subunit is configured to determine the coordinates of the merged object based on the average of the plurality of fourth coordinates.

12. The apparatus according to any one of claims 7-9, wherein, The preset quantity is N-1, where N is the number of images in the multi-frame image.

13. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images