Information processing method, model training method, and related device

Abstract

An information processing method, a model training method, and a related device, which are applicable to the field of intelligent driving. The information processing method is used to sequentially process, by means of a model, environmental information of a plurality of time points spaced by a first duration, and the method comprises: inputting, into the model, environmental information of a first time point and first information corresponding to a first historical time point, so as to obtain a first feature corresponding to the first time point, wherein an interval between the first time point and the first historical time point is a second duration, and the second duration is greater than the first duration; generating, by means of the model, a second identifier for each sub-feature of the first feature, wherein among features that correspond to different time points, sub-features corresponding to a same second identifier represent sub-features corresponding to a same object; and, on the basis of the first feature and the second identifier, obtaining, by means of a machine learning model, position information and first identifiers of objects in the environment of the first time point, wherein objects corresponding to a same first identifier represent a same object. Thus, accurate position information and first identifiers of objects in the environment can be obtained.

Description

An information processing method, a model training method, and related equipment. Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to an information processing method, a model training method, and related equipment. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0003] One common application of artificial intelligence (AI) technology is object tracking. For example, sensors can acquire environmental information, and objects can be tracked based on this information. Specifically, machine learning models can use environmental information from a given moment and historical moments to generate the position information and identifier (ID) of each object within at least one object in the environment at the given moment. Objects with the same identifier in different moments represent the same object, thus enabling object tracking via ID. However, since the desired position information for each object often requires its accurate bounding box, orientation, and / or size, the annotation cost for each object's desired position information is high. Manual annotation of desired position information often employs frame extraction, meaning the time interval between two adjacent moments corresponding to two adjacent desired position information is often large, resulting in a significant time interval between the first moment and historical moments.

[0004] The vehicle needs to process environmental information frequently through machine learning models. In other words, the time interval between adjacent moments in the environmental information that the vehicle needs to process through machine learning models is often small. However, if the time interval between the first moment and the historical moment is directly adjusted from a large interval to a small interval for the vehicle to adapt to the vehicle's need for processing high-frequency environmental information, this will change the input of the machine learning model. According to experimental data, this will result in poor accuracy of the location information and ID obtained by the machine learning model. Summary of the Invention

[0005] This application provides an information processing method, a model training method, and related equipment for obtaining more accurate location information and identification of objects in the environment through machine learning models.

[0006] This application provides the following technical solution:

[0007] Firstly, this application provides an information processing method that applies artificial intelligence technology to the field of intelligent driving. In this method, an execution device processes environmental information from multiple moments sequentially at a first frequency using a machine learning model to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple moments. Objects corresponding to the same first identifier in the environment at different moments represent the same object. The first frequency represents the time interval between two adjacent moments as a first duration. The specific process by which the execution device obtains the location information and the first identifier corresponding to each object in the environment at the first moment using a machine learning model may include:

[0008] The execution device inputs the environmental information of the first moment and the first information corresponding to each of the M historical moments (i.e., the M first pieces of information) into the machine learning model. The third module of the machine learning model performs feature extraction on the environmental information of the first moment and the M first pieces of information to generate the first feature corresponding to the first moment. M is an integer greater than or equal to 1. The M historical moments include the first historical moment. The first information includes: the environmental information of the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information of the first historical moment. The time interval between the first moment and the first historical moment is the second duration. The second duration is greater than the first duration. The first feature includes the sub-feature corresponding to each object in at least one object in the environment of the first moment.

[0009] The execution device generates a second identifier for each sub-feature included in the first feature through a machine learning model. Sub-features corresponding to the same second identifier in the features at different times represent sub-features corresponding to the same object. Based on the first feature and the second identifier of each sub-feature included in the first feature, the machine learning model obtains the location information and the first identifier corresponding to each object in the environment at the first time.

[0010] Optionally, the M historical moments may also include other historical moments prior to the first historical moment. The time intervals between adjacent historical moments in the M historical moments may be the same or different. The time intervals between adjacent historical moments in the M historical moments may all be greater than or equal to the second duration. For example, if the time interval between some adjacent historical moments in the M historical moments is greater than the second duration, then the time interval between those adjacent historical moments may be an integer multiple of the second duration.

[0011] For example, the environmental information at the first moment may include at least one image and / or point cloud data corresponding to the environment at the first moment. Optionally, the environmental information at the first moment may also include map information.

[0012] For example, the position information of an object may include at least one of the following: the position of the object's bounding box, the outline of the object's bounding box, the position coordinates of the object, the orientation of the object or the bounding box, the size of the object or the bounding box, or other information.

[0013] Optionally, if the execution device is a vehicle or a processor integrated into a vehicle, the environment around the execution device can be understood as the traffic environment around the vehicle. For example, the vehicle can be a car, truck, motorcycle, bus, boat, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, airplane and helicopter, etc. This application embodiment does not make any special limitation.

[0014] In this implementation, a machine learning model processes environmental information from multiple time points sequentially according to a first frequency. This yields the location information and a first identifier for each object in at least one object within the environment at each of the multiple time points. Objects corresponding to the same first identifier in the environment at different time points represent the same object. The first frequency represents the time interval between two adjacent time points as a first duration. When processing the environmental information at each time point using the machine learning model, the environmental information from the first time point and the first information corresponding to the first historical time point are input into the model. Since the time interval between the first time point and the first historical time point is a second duration, which is longer than the first duration, this allows the machine learning model to use the same input during both the training and application phases. This improves the accuracy of the location information and first identifier obtained by the machine learning model. By calling the machine learning model at the first frequency, it is possible to adapt to the vehicle's need for processing high-frequency environmental information. In addition, after generating the first feature corresponding to the first moment through the machine learning model, the first feature includes the sub-features corresponding to each object in the environment at the first moment. The machine learning model also generates a second identifier for each sub-feature included in the first feature. Among the features corresponding to different moments, the sub-features corresponding to the same second identifier represent the sub-features corresponding to the same object. That is, the machine learning model also predicts the correlation between the sub-features of each object. Based on the first feature and the second identifiers of the sub-features in the first feature, the machine learning model finally obtains the location information and first identifier corresponding to each object in the environment at the first moment. This is beneficial to obtain more accurate location information and first identifiers through the machine learning model.

[0015] In one possible implementation, the execution device generates a second identifier for each sub-feature included in the first feature through a machine learning model. This includes: the execution device generating a second identifier for each sub-feature included in the first feature based on the first feature and second information, wherein the second information is obtained based on the second feature corresponding to each historical moment in N historical moments (i.e., N second features), where N is an integer greater than or equal to 1, the N historical moments include the second historical moment, and the time interval between the second historical moment and the first moment is a first duration. Optionally, when N is greater than or equal to 2, the N historical moments may also include other historical moments before the second historical moment, and the time interval between adjacent historical moments in the N historical moments may be the same or different; the time interval between adjacent historical moments in the N historical moments may all be greater than or equal to the first duration. Optionally, the time interval between adjacent historical moments in the N historical moments may all be less than the second duration. For example, if the time interval between some adjacent historical moments in the N historical moments is greater than the first duration, then the time interval between those adjacent historical moments may be a multiple of the first duration.

[0016] For example, each second identifier can be an integer value, a string, or other data form, such as an integer value that can be 1, 2, 3, 4, 5, or other numbers; for example, when the second identifier is a string, the second identifier can be represented in the form of one-hot encoding, or the second identifier can also be composed of numbers and letters, etc.

[0017] In this implementation, since the machine learning model processes environmental information at multiple times sequentially according to a first frequency, where the first frequency represents the time interval between two adjacent times as a first duration, the second feature corresponding to the second historical time can be obtained. Based on the first feature and the second information, the machine learning model generates a second identifier for each sub-feature included in the first feature. The second information is obtained based on the second feature corresponding to the second historical time. That is, when generating the second identifier for each sub-feature in the first feature corresponding to the first time, not only is each sub-feature in the first feature itself referenced, but also the sub-features in the second feature corresponding to the second historical time are referenced. This enables the determination of the second identifier for each sub-feature in the first feature based on the relationship between each sub-feature in the first feature and the sub-features in the second historical time, which helps to improve the accuracy of the generated second identifier.

[0018] In one possible implementation, the second feature corresponding to the second historical moment includes the sub-feature corresponding to each object in at least one object in the environment of the second historical moment, and the second information is obtained based on the features of the N second features corresponding to N historical moments (including the second historical moment) and the features of the second identifier of each sub-feature in each of the N second features.

[0019] For example, the "features of the second identifier of the sub-features included in the second feature corresponding to each of the N historical moments" are obtained through the neural network layer in the first module of the first machine learning model. Optionally, the features of the second identifier can be the features generated by the Pth-th neural network layer from the end in the first module of the machine learning model, where P is an integer greater than 1. In other words, in the process of generating the second identifier through the first module of the machine learning model, the features of the second identifier are generated first, and then the second identifier is generated based on the features of the second identifier. For example, the value of P can be 2, 3, 4, or other values.

[0020] When N is greater than or equal to 2, optionally, the execution device can fuse all sub-features corresponding to the same object among the N second features based on the second identifier of each sub-feature included in each of the N second features, to obtain the fused feature corresponding to the same object. The historical feature corresponding to the object includes the fused feature of the object and the feature of the second identifier corresponding to the object. The execution device can obtain the historical feature of each object in the environment at least one time in N historical moments based on the N second features and the feature of the second identifier of each sub-feature included in each of the N second features, that is, obtain the second information. In other words, the second information includes at least one historical feature that corresponds one-to-one with at least one object in the environment at N historical moments. The historical feature corresponding to each object includes the fused feature of the object and the feature of the second identifier corresponding to the object.

[0021] In this implementation, the second information is obtained based on the second feature corresponding to the second historical moment and the features of the second identifier of the sub-features included in the second feature. Therefore, when generating the second identifier of each sub-feature in the first feature corresponding to the first moment, not only the features of each sub-feature in the first feature are referenced, but also the features of the sub-features in the second feature corresponding to the second historical moment and the features of the second identifier of each sub-feature are referenced. Since the features of the second identifier can carry richer information that the machine learning model can understand, it is beneficial to further improve the accuracy of the generated second identifier.

[0022] In one possible implementation, the second identifier is generated by the first module in the machine learning model. When the execution device generates a second identifier for each sub-feature included in the first feature using the machine learning model based on the first feature and the second information, the execution device obtains the location information and the first identifier corresponding to each object in the environment at the first moment through the machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature. This includes: the execution device generating the location information corresponding to each object in the environment at the first moment through the second module in the machine learning model based on the sub-features corresponding to each object in the environment at the first moment, and the second identifier of the sub-feature corresponding to each object in at least one object in the environment at the first moment is used as the first identifier corresponding to each object.

[0023] In this implementation, since the sub-features corresponding to the same second identifier in the features at different times represent the sub-features corresponding to the same object, that is, a second identifier can also uniquely point to an object, it is feasible to directly use the second identifier as the first identifier. In addition, by using two different modules (i.e., the first module and the second module) in the machine learning model to predict the location information and the first identifier respectively, the process of generating prediction information by the machine learning model is divided in a more detailed way, which helps to reduce the difficulty of the first module and the second module respectively, thereby improving the accuracy of the generated location information and the first identifier.

[0024] In one possible implementation, the execution device generates position information and a first identifier for each object in the environment at a first moment through a machine learning model, including: the execution device generates prediction information for each object in the environment at a first moment through a machine learning model, the prediction information including position information and a first identifier, and the prediction information further including at least one of the following: velocity, acceleration or angular velocity.

[0025] In this implementation, since the machine learning model generates the position information and first identifier of each object, the position of the same object at different times can be tracked with the help of the first identifier. This allows for a more accurate inference of the object's velocity, acceleration, or angular velocity. Not only does the machine learning model provide more information, but it also helps ensure the accuracy of the obtained information.

[0026] Secondly, this application provides a model training method that can apply artificial intelligence technology to the field of intelligent driving. In this method, the training device inputs the environmental information at a first moment and the first information corresponding to a first historical moment into a machine learning model to obtain a first feature corresponding to the first moment. The first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is a second duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. A second identifier for each sub-feature included in the first feature is generated by a first module of the machine learning model. Sub-features corresponding to the same second identifier in features corresponding to different moments represent sub-features corresponding to the same object. Based on the first feature and the second identifier of each sub-feature included in the first feature, the machine learning model obtains the location information and first identifier corresponding to each object in the environment at the first moment. Objects corresponding to the same first identifier in the environment at different moments represent the same object. A first loss function is used to train the machine learning model. The first loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

[0027] In one possible implementation, the training phase of the machine learning model includes a first training phase and a second training phase. In the first training phase, the machine learning model processes environmental information at multiple times according to a second frequency, where the second frequency represents the time interval between two adjacent times as a second duration. The training device trains the machine learning model using a first loss function. The training phase includes: in the first training phase, the training device trains the machine learning model using the first loss function.

[0028] The method further includes: in the second training phase, the training device processes the environmental information of multiple time moments sequentially according to the first frequency through the machine learning model to obtain the location information and first identifier of each object in at least one object in the environment at each of the multiple time moments. The first frequency represents the time interval between two adjacent time moments as the first duration, and the first duration is less than the second duration. Keeping the weight parameters of the third module and the second module unchanged, the weight parameters of the first module are updated using the second loss function. The second loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature.

[0029] For example, the first frequency refers to the frequency used by the training device in the second training phase. When the training device performs training operations within the same batch, it uses the same frequency to process the environmental information at multiple time points in sequence. In other words, the time interval between adjacent time points corresponding to multiple training samples within the same batch is the same. When the training device performs training operations in different batches, it may use the same or different frequencies.

[0030] In this implementation, the training phase of the machine learning model is divided into a first training phase and a second training phase. In the first training phase, environmental information at multiple time points is processed sequentially according to a second frequency, where the second frequency represents the time interval between two adjacent time points (the second duration). The loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information. This allows the machine learning model to learn how to generate accurate location information and the second identifier. The second duration is relatively long, thus allowing compatibility with obtaining the expected location information using frame extraction and avoiding the problem of excessively high annotation costs. In the second training phase, a higher first frequency is used to process environmental information at multiple time points sequentially. The loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature. Accurate expected location information is no longer needed. Since the annotation cost of the expected second identifier is low, the two-stage training method still avoids the problem of excessively high annotation costs. Furthermore, in the second training phase, the weight parameters of the second and third modules remain unchanged, and only the weight parameters of the first module used to generate the second identifier are updated. This not only preserves the machine learning model's ability to generate accurate location information but also helps the second module learn to generate more accurate second identifiers through the second training phase.

[0031] In the second aspect of this application, the training device is also used to perform the steps of the first aspect and various possible implementations of the first aspect. The specific implementations of the steps in the second aspect, the meanings of the terms, and the beneficial effects are all described in the first aspect and will not be repeated here.

[0032] Thirdly, this application provides an information processing device that can apply artificial intelligence technology to the field of intelligent driving. The device is used to process environmental information at multiple times sequentially according to a first frequency using a machine learning model, to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple times. Objects corresponding to the same first identifier in the environment at different times represent the same object. The first frequency represents the time interval between two adjacent times as a first duration. The device includes: an input module, used to input the environmental information at the first time and the first information corresponding to the first historical time into the machine learning model to obtain a first feature corresponding to the first time, wherein the first information includes: environmental information at the first historical time and / or the first historical time. The first historical moment has a second feature corresponding to it, which is obtained based on the environmental information of the first historical moment. The time interval between the first historical moment and the first historical moment is a second duration, which is longer than the first duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. The generation module is used to generate a second identifier for each sub-feature included in the first feature through a machine learning model. In this module, sub-features corresponding to the same second identifier in features at different times represent sub-features corresponding to the same object. The processing module is used to obtain the location information and the first identifier corresponding to each object in the environment at the first moment through a machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature.

[0033] In the third aspect of this application, the information processing apparatus is also used to perform the steps of the execution device in the first aspect and various possible implementations of the first aspect. The specific implementations of the steps in the third aspect, the meanings of the terms, and the beneficial effects thereon can all be found in the first aspect, and will not be repeated here.

[0034] Fourthly, this application provides a model training device that can apply artificial intelligence technology to the field of intelligent driving. The device includes: an input module, used to input environmental information at a first moment and first information corresponding to a first historical moment into a machine learning model to obtain a first feature corresponding to the first moment, wherein the first information includes: environmental information at the first historical moment and / or a second feature corresponding to the first historical moment, the second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment, the time interval between the first moment and the first historical moment is a second duration, and the first feature includes sub-features corresponding to each object among at least one object in the environment at the first moment; and a generation module, used to generate the model through the machine learning model. The first module generates a second identifier for each sub-feature included in the first feature, wherein sub-features corresponding to the same second identifier in features at different times represent sub-features corresponding to the same object; the processing module is used to obtain the location information and first identifier corresponding to each object in the environment at the first time through a machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature, wherein objects corresponding to the same first identifier in the environment at different times represent the same object; the training module is used to train the machine learning model using a first loss function, wherein the first loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

[0035] In the fourth aspect of this application, the training device for the model is also used to perform the steps executed by the training device in the second aspect and various possible implementations of the second aspect. The specific implementation methods, the meanings of the terms, and the beneficial effects of the steps in the fourth aspect can be found in the first aspect, and will not be repeated here.

[0036] Fifthly, this application provides an apparatus including a processor and a memory, the processor being coupled to the memory, the memory storing program instructions, which, when executed by the processor, implement the methods described in the first or second aspect.

[0037] In a sixth aspect, this application provides a vehicle including a processor and a memory, the processor being coupled to the memory, the memory storing program instructions, and the method described in the first aspect being implemented when the program instructions stored in the memory are executed by the processor.

[0038] In a seventh aspect, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the first or second aspect.

[0039] Eighthly, this application provides a computer program product comprising a program that, when run on a computer, causes the computer to perform the methods described in the first or second aspect.

[0040] Ninthly, this application provides a chip system including a processor for supporting the implementation of the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the terminal device or communication device. This chip system may be composed of chips or may include chips and other discrete devices.

[0041] The second to ninth aspects of this application correspond to the first aspect or multiple possible ways of the first aspect, and have corresponding beneficial effects. Attached Figure Description

[0042] Figure 1 is a structural diagram of an artificial intelligence main framework provided in this application;

[0043] Figure 2 is a system architecture diagram of an information processing system provided in an embodiment of this application;

[0044] Figure 3 is a flowchart illustrating an information processing method provided in an embodiment of this application.

[0045] Figure 4 is a schematic diagram of the second information provided in an embodiment of this application. In Figure 4, the second identifier is taken as an example of using an integer value.

[0046] Figure 5 is a schematic diagram of a method for generating a second identifier for each sub-feature in a first feature using a first module of a machine learning model, according to an embodiment of this application.

[0047] Figure 6 is another schematic diagram of the generation of the second identifier by the first module of the machine learning model according to an embodiment of this application;

[0048] Figure 7 is a schematic diagram of generating location information and a first identifier through a machine learning model according to an embodiment of this application;

[0049] Figure 8 is a flowchart illustrating a training method for a model provided in an embodiment of this application;

[0050] Figure 9 is a schematic diagram of an information processing device provided in an embodiment of this application;

[0051] Figure 10 is a schematic diagram of a training device for a model provided in an embodiment of this application;

[0052] Figure 11 is a schematic diagram of a device provided in an embodiment of this application;

[0053] Figure 12 is a structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0054] The embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, and not all, of the embodiments of this application. Those skilled in the art will recognize that, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0055] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0056] In the embodiments of this application, "instruction" can include direct and indirect instructions, as well as explicit and implicit instructions. The information indicated by a certain piece of information (hereinafter referred to as instruction information) is called the information to be instructed. In specific implementation, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is an association between the other information and the information to be instructed; or it can indicate only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction can be implemented by using a pre-agreed (e.g., protocol predefined) arrangement of various information, thereby reducing the instruction overhead to a certain extent. This application does not limit the specific method of instruction. It is understood that for the sender of the instruction information, the instruction information can be used to indicate the information to be instructed; for the receiver of the instruction information, the instruction information can be used to determine the information to be instructed.

[0057] First, the overall workflow of the artificial intelligence system is described, as shown in Figure 1. Figure 1 is a structural diagram of one aspect of the artificial intelligence framework provided in this application. The framework is then elaborated on from two dimensions: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed by technology) to the industrial ecosystem of the system.

[0058] (1) Infrastructure

[0059] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), tensor processing units (TPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0060] (2) Data

[0061] The data at the next layer of infrastructure represents the data sources in the field of intelligent driving. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0062] (3) Data processing

[0063] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0064] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0065] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0066] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0067] (4) General ability

[0068] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0069] (5) Smart Products and Industry Applications

[0070] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, intelligent driving, and smart cities.

[0071] The method provided in this application can be applied to scenarios where machine learning models are used to track the positions of objects in an environment at multiple different times. These machine learning models can be understood as tracking models. Optionally, the method provided in this application can be used in the field of intelligent driving to track the positions of objects in the traffic environment surrounding a vehicle. Alternatively, the method provided in this application can also be applied to the field of intelligent transportation, for example, to track the position of a person riding an electric bicycle without a helmet in a traffic environment, or to track the position of a pedestrian running a red light in a traffic environment. The application scenarios of the method provided in this application are not exhaustively listed here.

[0072] In related technologies, sensors can acquire environmental information about the vehicle's surroundings at multiple different times. A machine learning model, based on this environmental information and information from at least one historical time, obtains the location information of objects in the traffic environment surrounding the vehicle at each time, along with a first identifier (ID). Objects corresponding to the same first identifier in the environment at different times represent the same object, thus enabling the tracking of the same object's location at multiple different times. Since the desired location information for each object often requires frame-sampling during manual annotation, the time interval between two adjacent moments corresponding to two adjacent desired location information is often large. Consequently, the time interval between the first moment and adjacent moments in at least one historical time is often large. The vehicle-side needs to process environmental information frequently through the machine learning model; that is, the time interval between adjacent moments in the multiple environmental information moments processed by the vehicle-side through the machine learning model is often small. However, directly adjusting the time interval between the first moment and adjacent moments in at least one historical time from a large interval to a smaller interval on the vehicle-side to adapt to the vehicle-side's high-frequency environmental information processing requirements would change the input to the machine learning model. Experimental data shows that this leads to poor accuracy in the location information and first identifier obtained through the machine learning model.

[0073] To address the aforementioned issues, this application discloses: A machine learning model processes environmental information from multiple time points sequentially at a first frequency to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple time points. Objects corresponding to the same first identifier in the environment at different time points represent the same object. The first frequency represents the time interval between two adjacent time points as a first duration. The method includes: an execution device inputting the environmental information of the first time point and the first information corresponding to a first historical time point into a machine learning model; and generating a first feature corresponding to the first time point through the machine learning model. The first information includes: environmental information of the first historical time point and / or a second feature corresponding to the first historical time point. The corresponding second feature is obtained based on the environmental information of the first historical moment. The time interval between the first moment and the first historical moment is the second duration, which is longer than the first duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. Then, the execution device can generate a second identifier for each sub-feature included in the first feature through a machine learning model. Among the features corresponding to different moments, the sub-features corresponding to the same second identifier represent the sub-features corresponding to the same object. The execution device can obtain the location information and the first identifier corresponding to each object in the environment at the first moment through a machine learning model based on the first feature and the second identifier of the sub-features corresponding to each object in at least one object in the environment at the first moment.

[0074] In this embodiment, since the time interval between the first moment and the first historical moment is the second duration, it is beneficial to enable the machine learning model to use the same input in the training and application phases, which is beneficial to improving the accuracy of the location information and the first identifier obtained by the machine learning model. Furthermore, by using the first frequency to call the machine learning model, it is possible to adapt to the vehicle's processing needs for high-frequency environmental information. In addition, after the first feature corresponding to the first moment is generated by the machine learning model, the first feature includes the sub-features corresponding to each object in the environment at the first moment. The machine learning model also generates a second identifier for each sub-feature included in the first feature. Among the features corresponding to different moments, the sub-features corresponding to the same second identifier represent the sub-features corresponding to the same object. That is, the machine learning model also predicts the correlation between the sub-features of each object. Based on the first feature and the second identifiers of the sub-features in the first feature, the machine learning model finally obtains the location information and the first identifier corresponding to each object in the environment at the first moment, which is beneficial to obtain more accurate location information and the first identifier by the machine learning model.

[0075] Before detailing the method provided in this application, the architecture of the information processing system provided in this application will be described first. Please refer to Figure 2, which is a system architecture diagram of the information processing system provided in an embodiment of this application. In Figure 2, the information processing system 200 includes a training device 210, a database 220, an execution device 230, and a data storage system 240. The execution device 230 includes a computing module 231. The database 220 stores a training dataset. During the training phase of the machine learning model 201, the training device 210 can use the training dataset to perform training operations to obtain the trained machine learning model 201.

[0076] During the application phase of the machine learning model 201, the inference process of the machine learning model 201 can be executed by the computing module 231 of the execution device 230. Optionally, the execution device 230 can be integrated into the vehicle, allowing users to directly interact with the vehicle where the execution device 230 is deployed. For example, the execution device 230 can be a module in the vehicle's host CPU that uses the machine learning model for data processing. The execution device 230 can also be a graphics processing unit (GPU), neural network processing unit (NPU), or tensor processing unit (TPU) in the vehicle, etc. The aforementioned GPU, NPU, or TPU is mounted on the vehicle's host CPU as a coprocessor, and the vehicle's host CPU allocates tasks, etc.

[0077] The execution device 230 can access data, code, etc., in the data storage system 240, and can also store data, instructions, etc., in the data storage system 240. The data storage system 240 can be located within the execution device 230, or it can be an external memory relative to the execution device 230.

[0078] It should be noted that Figure 2 is merely a schematic diagram of one architecture of the information processing system provided in this application embodiment, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in some other embodiments of this application, the execution device 230 and the client device can be separate and independent devices. The execution device 230 is configured with an input / output (I / O) interface, through which the execution device 230 can interact with the client device for data exchange, etc. The specific architecture of the information processing system can be determined according to the actual application scenario. The following only describes the specific implementation process of the application stage and the training stage of the machine learning model 201.

[0079] I. Application Phase

[0080] Please refer to Figure 3, which is a flowchart illustrating an information processing method provided in this application. In the method provided in this application, the execution device can process environmental information at multiple times sequentially according to a first frequency using a machine learning model. In other words, the machine learning model can be invoked according to the first frequency to obtain the location information and first identifier corresponding to each object in at least one object in the environment at each of the multiple times. Objects corresponding to the same first identifier in the environment at different times represent the same object. The first frequency represents the time interval between two adjacent times in the aforementioned multiple times as a first duration. For example, the first duration can be 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.25 seconds, or other lengths. The specific process by which the execution device obtains the location information and first identifier corresponding to each object in the environment at the first time using this machine learning model may include:

[0081] 301. Input the environmental information at the first moment and the first information corresponding to the first historical moment into the machine learning model to obtain the first feature corresponding to the first moment. The first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is the second duration, which is longer than the first duration. The first feature includes the sub-features corresponding to each object in at least one object in the environment at the first moment.

[0082] Here, the first moment can also be understood as the current moment. The time interval between the first moment and the first historical moment is the second duration. The second duration is longer than the first duration. For example, the second duration can be P times the first duration, where P is an integer greater than 1. For example, the value of P can be 2, 3, 4, 5, 6, 7 or other values, which can be set according to the actual application scenario.

[0083] For example, step 301 may include: the execution device acquiring environmental information at a first moment, inputting the environmental information at the first moment and the first information corresponding to each of the M historical moments (i.e., the M first pieces of information) into a machine learning model, and performing feature extraction on the environmental information at the first moment and the M first pieces of information through the third module of the machine learning model to generate a first feature corresponding to the first moment, wherein the first feature includes the sub-features corresponding to each object in at least one object in the environment at the first moment.

[0084] Where M is an integer greater than or equal to 1. For example, the value of M can be 1, 2, 3, 4, 5, or other values, which can be determined according to the actual application scenario. The M historical moments include the first historical moment. Optionally, the M historical moments may also include other historical moments before the first historical moment. The time interval between the first historical moment is the second duration. The time interval between adjacent historical moments in the M historical moments can be the same or different. The time interval between adjacent historical moments in the M historical moments can all be greater than or equal to the second duration. For example, if the time interval between some adjacent historical moments in the M historical moments is greater than the second duration, then the time interval between those adjacent historical moments can be an integer multiple of the second duration.

[0085] For example, let the first moment be referred to as moment t, the first duration can be 0.1 seconds, and the second duration can be 0.5 seconds. Then, the execution device can sequentially obtain the position information of objects in the environment at moments t, t+0.1 seconds, t+0.2 seconds, etc., and the first identifier through a machine learning model. In order to obtain the position information of objects in the environment at moment t and the first identifier, the execution device can input the environmental information at moment t, the first information 1 corresponding to t-0.5 seconds, and the first information 2 corresponding to t-1.5 seconds into the machine learning model. The aforementioned t-0.5 seconds and t-1.5 seconds are two first historical moments. It should be understood that the example here is only for the convenience of understanding this solution and is not intended to limit this solution.

[0086] For example, the third module of a machine learning model may include at least one of the following: a convolutional neural network (CNN) layer, a fully connected neural network layer, an attention-based neural network layer, or other types of neural network layers, which can be determined based on the actual application scenario.

[0087] Optionally, the third module of the machine learning model can be the third module of the object detection model. The object detection model refers to the machine learning model used to detect objects in the traffic environment at the current moment. In other words, the object detection model can be understood as a machine learning model that detects objects in the traffic environment at the current moment that are moving at low speed.

[0088] For example, the third module of the machine learning model may include: a convolutional neural network layer + a region proposal network (RPN) layer. The third feature can be obtained through the convolutional neural network layer. The third feature includes environmental information at the first time step and features of M first information. The aforementioned features can be represented in the form of a feature map. The RPN determines the target region of each object in the aforementioned feature map of at least one object in the environment at the first time step. Thus, the sub-features corresponding to each object in the environment at the first time step can be obtained from the third feature, that is, the first feature is obtained.

[0089] Alternatively, the third module of the machine learning model may include a convolutional neural network layer plus a transformer neural network layer. The third feature can be obtained through the convolutional neural network layer, and then the sub-features corresponding to each object in at least one object in the environment at the first time step can be obtained through the transformer neural network layer based on the aforementioned third feature, which is to say, the first feature is obtained.

[0090] Alternatively, the third module of the machine learning model can also be composed of other types of neural network layers. This example is only to demonstrate the feasibility of this solution, and the specific implementation can be determined based on the actual application scenario.

[0091] For example, the M historical moments refer to the M historical moments preceding the first moment. The first information corresponding to each of the M historical moments (including the first historical moment) includes the environmental information of that historical moment and / or the second feature corresponding to that historical moment. The second feature corresponding to each historical moment is obtained based on the environmental information of that historical moment. For example, the second feature corresponding to the first historical moment may include the sub-features corresponding to each object in at least one object in the environment of the first historical moment. The meaning and acquisition method of "the second feature corresponding to the first historical moment" are similar to those of "the first feature corresponding to the first moment," the only difference being that the second feature corresponds to a historical moment, and the first feature corresponds to the first moment. The acquisition method of the second feature will not be described in detail here.

[0092] For example, the environmental information at the first moment may include at least one image and / or point cloud data corresponding to the environment at the first moment. Optionally, the environmental information at the first moment may also include map information.

[0093] For example, the execution device can acquire at least one image of the environment surrounding the execution device via a first sensor at a first moment. Optionally, the acquired at least one image can be an image from a perspective view (PV); and / or, the execution device can acquire point cloud data of the environment surrounding the execution device via a second sensor at a first moment. It should be understood that in this application, the execution subject is referred to as "execution device," but the actual execution subject can be the intelligent driving system within the execution device.

[0094] For example, "the environment around the execution device" can be understood as the environment within the field of view of the sensors deployed on the execution device (such as the aforementioned first or second sensor). Exemplarily, the first sensor can be a photoelectric sensor, such as a camera or event camera; for example, the second sensor can be an ultrasonic sensor, a lidar sensor, a millimeter-wave radar sensor, or other sensors capable of measuring and obtaining point cloud data, etc., which are not exhaustively listed in the embodiments of this application.

[0095] The environmental information at the first historical moment may include at least one image and / or point cloud data corresponding to the environment at the first historical moment. The acquisition method of "environmental information at the first historical moment" is similar to that of "environmental information at the first moment", and will not be described in detail here.

[0096] Optionally, if the executing device is a vehicle or a processor integrated into a vehicle, the environment surrounding the executing device can be understood as the traffic environment surrounding the vehicle. For example, the vehicle can be a car, truck, motorcycle, bus, boat, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, airplane, or helicopter, etc., and this application embodiment does not impose any particular limitation. Alternatively, the executing device can also be a mobile phone, laptop, desktop computer, monitoring device, or other types of equipment, etc.

[0097] 302. Generate a second identifier for each sub-feature included in the first feature using a machine learning model, wherein the sub-features corresponding to the same second identifier in the features at different times represent the sub-features corresponding to the same object.

[0098] Optionally, step 302 may include: the execution device generating a second identifier for each sub-feature included in the first feature based on the first feature and the second information through a first module of a machine learning model; wherein the second information is obtained based on the second feature (i.e., N second features) corresponding to each historical moment in N historical moments, where N is an integer greater than or equal to 1, the N historical moments include the second historical moment, and the time interval between the second historical moment and the first moment is the first duration. Optionally, when N is greater than or equal to 2, the N historical moments may also include other historical moments before the second historical moment, and the time interval between adjacent historical moments in the N historical moments may be the same or different; the time interval between adjacent historical moments in the N historical moments may all be greater than or equal to the first duration. Optionally, the time interval between adjacent historical moments in the N historical moments may all be less than the second duration. For example, if the time interval between some adjacent historical moments in the N historical moments is greater than the first duration, then the time interval between the adjacent historical moments may be a multiple of the first duration.

[0099] The values ​​of N and M can be the same or different. For example, the value of N can be 1, 2, 3, 4, 5 or other values. Alternatively, the N historical moments can include each moment after the machine learning model starts performing a tracking task.

[0100] For example, the first duration is 0.1 seconds, N is 3, the first moment is moment t, and the N historical moments can include moment t-0.1 seconds, moment t-0.3 seconds, and moment t-0.7 seconds; or, for another example, the first duration is 0.15 seconds, N is 3, the first moment is moment t, and the N historical moments can include moment t-0.15 seconds, moment t-0.3 seconds, and moment t-0.6 seconds, etc. The specific settings can be combined with the actual application scenario.

[0101] For example, each second identifier can be an integer value, a string, or other data form. For example, an integer value can be 1, 2, 3, 4, 5, or other numbers. For example, when the second identifier is a string, the second identifier can be represented in the form of one-hot encoding, or the second identifier can also be composed of numbers and letters, etc. The specifics can be determined in combination with the actual application scenario.

[0102] For example, both the first historical moment and the second historical moment are historical moments prior to the first moment. The second feature corresponding to the second historical moment may include the sub-features corresponding to each object in at least one object in the environment of the second historical moment. The meaning and acquisition method of "the second feature corresponding to the second historical moment" are similar to those of "the first feature corresponding to the first moment". The only difference is that the second feature corresponds to the historical moment and the first feature corresponds to the first moment. The acquisition method of the second feature will not be described in detail here.

[0103] First, we will introduce how the second information is obtained. In one case, the second feature corresponding to the second historical moment includes the sub-features corresponding to each object in at least one object in the environment of the second historical moment. The second information is obtained based on the features of the second feature corresponding to each historical moment in N historical moments (that is, N second features) and the second identifier of the sub-features included in the second feature corresponding to each historical moment.

[0104] For example, the execution device generates a second identifier for each sub-feature in the first feature corresponding to the first time step through the first module of the machine learning model. The execution device also generates a second identifier for each sub-feature in the second feature corresponding to each historical time step in N historical time steps through the first module of the machine learning model. The "feature of the second identifier of the sub-feature included in the second feature corresponding to each historical time step in N historical time steps" is obtained through the neural network layer in the first module of the first machine learning model.

[0105] Optionally, the features of the second identifier can be the features generated by the Pth-th neural network layer from the end in the first module of the machine learning model, where P is an integer greater than 1. In other words, in the process of generating the second identifier through the first module of the machine learning model, the features of the second identifier are generated first, and then the second identifier is generated based on the features of the second identifier. For example, the value of P can be 2, 3, 4 or other values, which can be determined in combination with the actual application scenario.

[0106] For example, the second feature corresponding to the second historical moment may include sub-features corresponding to each object in at least one object in the environment of the second historical moment. Sub-features corresponding to the same second identifier in features corresponding to different moments represent sub-features corresponding to the same object. That is, sub-features corresponding to the same second identifier in the second features corresponding to different historical moments in N historical moments represent sub-features corresponding to the same object. The execution device may fuse all sub-features corresponding to the same object in the N second features based on the second identifier of each sub-feature included in each of the N second features to obtain the fused feature corresponding to the same object. The historical feature corresponding to the object includes the fused feature of the object and the feature of the second identifier corresponding to the object. The execution device may obtain the historical feature of each object in at least one object in the environment of N historical moments based on the N second features corresponding to the N historical moments and the feature of the second identifier of each sub-feature included in each of the N second features. That is, the second information is obtained. In other words, the second information includes at least one historical feature corresponding one-to-one with at least one object in the environment of N historical moments. The historical feature corresponding to each object includes the fused feature of the object and the feature of the second identifier corresponding to the object.

[0107] For example, the execution device may fuse all sub-features corresponding to the same object among N second features to obtain the fused feature corresponding to the same object. This may include: concatenating all sub-features corresponding to the same object to obtain the fused feature; or, taking the latest sub-feature among all sub-features corresponding to the same object as the fused feature; or, obtaining the fused feature by averaging all sub-features corresponding to the same object, etc. The specific fusion method can be determined in combination with the actual application scenario. To understand this solution more intuitively, please refer to Figure 4. Figure 4 is a schematic diagram of the second information provided in an embodiment of this application. In Figure 4, taking an integer value as an example, the first row of Figure 4 shows that the boxed 1 represents the feature of the second identifier 1, and the other box with the boxed 1 represents the historical features of the object corresponding to the second identifier 1. Next, the second row of Figure 4 shows that the boxed 3 represents the feature of the second identifier 3, and the other box with the boxed 3 represents the historical features of the object corresponding to the second identifier 3. Next, the third row of Figure 4 shows that the boxed 7 represents the feature of the second identifier 7, and the other box with the boxed 7 represents the historical features of the object corresponding to the second identifier 7. The second information may also include the historical features corresponding to other objects that appeared in N historical moments, which will not be described in detail here. It should be understood that the example in Figure 4 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0108] In another scenario, the second information is obtained based on the second feature (i.e., N second features) corresponding to each historical moment in the N historical moments and the second identifier of the sub-features included in each of the N historical moments. The second information includes at least one historical feature corresponding one-to-one with at least one object in the environment of the N historical moments. The historical feature corresponding to each object includes the fused feature of the object and the second identifier corresponding to the object. It should be noted that the specific implementation of the execution device obtaining the second information based on the N second features corresponding to the N historical moments and the second identifier of each sub-feature included in the N second features is similar to the specific implementation of obtaining the second information based on the features of the N second features corresponding to the N historical moments and the second identifier of each sub-feature included in the N second features. The difference is that the "feature of the second identifier" in the above description is replaced with "second identifier," which will not be described again here.

[0109] In another scenario, the second information is obtained solely based on N second features corresponding to N historical moments. For example, the execution device can directly fuse the N second features to obtain the second information.

[0110] For example, the execution device processes the second information and the first feature through a first module of a machine learning model to generate a second identifier for each sub-feature included in the first feature.

[0111] Optionally, the first module of the machine learning model may include a neural network module based on a cross attention mechanism and a task head; for example, the neural network module based on the attention mechanism may be a transformer module, and the task head may include at least one of the following: a fully connected neural network layer, a residual neural network layer, a multilayer perceptron (MLP), a support vector machine, or other types of neural network layers, which can be determined in conjunction with the actual application scenario.

[0112] The execution device can process the second information and the first feature through the transformer module to obtain the updated first feature; based on the updated first feature, it generates the feature of the second identifier of each sub-feature in the first feature through the task header, and then generates the second identifier of each sub-feature in the first feature through the task header based on the feature of each second identifier.

[0113] If any two adjacent historical moments in N historical moments are of the first duration, optionally, the execution device can also update the second information based on the features of the second identifiers of the sub-features in the first feature corresponding to each new moment, after generating the second identifier of each sub-feature in the first feature. For example, in one case, any one of the at least one sub-features included in the first feature (or the updated first feature) is called the target sub-feature. In other words, the target sub-feature can be any one of the sub-features in the first feature or any one of the sub-features in the updated first feature; the specific choice can be determined based on the actual application scenario. Since the sub-features corresponding to the same second identifier in the second feature corresponding to different historical moments represent the sub-features corresponding to the same object, the execution device can determine which historical feature (hereinafter referred to as the "target historical feature") in the second information corresponds to the target sub-feature based on the second identifier of the target sub-feature, and then fuse the target sub-feature with the fused feature in the target historical feature again to obtain the re-fused feature. The execution device can perform the aforementioned operation on each sub-feature specifically included in the first feature to update the second information. Optionally, the execution device may also use the features of the second identifier of each sub-feature in the first feature to update the features of at least one second identifier included in the second information, so as to replace the features of the second identifier with the latest features, thereby realizing the update of the second information.

[0114] For example, the execution device may fuse the target sub-feature with the fused features in the target historical features again to obtain the refused features. This may include: the execution device concatenating the target sub-feature with the fused features in the target historical features to obtain the refused features; or, replacing the fused features in the target historical features with the target sub-feature to obtain the refused features; or, obtaining the fused features by averaging the target sub-feature with the fused features in the target historical features, etc. The specific fusion method can be determined in combination with the actual application scenario.

[0115] In another scenario, the execution device may also directly fuse the first feature with the second information to update the second information.

[0116] To understand this solution more intuitively, please refer to Figure 5. Figure 5 is a schematic diagram of a method for generating a second identifier for each sub-feature in a first feature using a first module of a machine learning model, as provided in an embodiment of this application. In Figure 5, the N marked with a box represents that the second identifier has not yet been generated. As shown in Figure 5, after generating the first feature corresponding to the first time, the execution device processes the first feature and the second information through the transformer module to obtain the updated first feature. The updated first feature is then processed through the task header to obtain the second identifier for each sub-feature in the updated first feature. Since the sub-features in the first feature correspond one-to-one with the sub-features in the updated first feature, the second identifier for each sub-feature in the updated first feature can be regarded as the second identifier for each sub-feature in the first feature, that is, the second identifier for the sub-feature corresponding to each object in at least one object in the environment at the first time is obtained.

[0117] Figure 5 also illustrates the updating process of the second information by using the updated first feature as an example. As shown in Figure 5, the execution device also updates the second information according to the second identifier of each sub-feature in the updated first feature. Specifically, it may include determining which historical feature in the second information each sub-feature in the updated first feature is fused with based on the second identifier of each sub-feature in the updated first feature, and then fusion each sub-feature in the updated first feature with the fused feature in the corresponding historical feature again. Alternatively, it may use the features of the second identifier of each sub-feature in the updated first feature to update the features of at least one second identifier included in the second information to achieve the updating of the second information. It should be understood that the example in Figure 5 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0118] Please refer to Figure 6. Figure 6 is another schematic diagram of the generation of the second identifier by the first module of the machine learning model provided in the embodiment of this application. In Figure 6, N with a box represents the feature that has not yet generated the second identifier. In Figure 6, the generated second identifier is taken as an integer value. As shown in Figure 6, at time 1, since the second feature of the historical time has not yet been generated, the second information can be randomly generated information. The first module processes the first feature and the second information corresponding to time 1 to obtain the three second identifiers (i.e., 3, 8 and 6 in Figure 6) of the three sub-features corresponding to the three objects in the environment at time 1. These can be regarded as IDs randomly assigned to the three sub-features. At time 2, the second information is obtained based on the second feature corresponding to time 1 and the second identifier of each sub-feature of the second feature corresponding to time 1 (i.e., 3, 8, and 6 in Figure 6). The second information includes: the features of the three second identifiers 3, 8, and 6, and the features of the three objects corresponding to the three identifiers 3, 8, and 6 after fusion. After obtaining the first feature corresponding to time 2, the first feature and the second information corresponding to time 2 are processed by the first module to generate four second identifiers (i.e., 8, 3, 2, and 4 in Figure 6). Since 2 and 4 have not appeared in the historical time, they can be regarded as randomly assigned IDs. 6 disappears in the environment at time 2. At time 3, the second information is obtained based on the second feature corresponding to time 1, the second identifier of each sub-feature of the second feature corresponding to time 1 (i.e., 3, 8, and 6 in Figure 6), the second feature corresponding to time 2, and the second identifier of each sub-feature of the second feature corresponding to time 2 (i.e., 8, 3, 2, and 4 in Figure 6). The second information includes: the features of the five second identifiers 3, 8, 6, 2, and 4, and the features of the five objects corresponding to the five identifiers 3, 8, 6, 2, and 4 after fusion, and so on. The processing of subsequent time points will not be described here. It should be understood that the example in Figure 6 is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0119] Alternatively, the transformer module mentioned above can be replaced with a convolutional neural network module, a fully connected neural network module, a residual neural network module, or other types of neural network modules, depending on the specific application scenario.

[0120] In this embodiment, since the machine learning model processes environmental information at multiple times sequentially according to a first frequency, where the first frequency represents the time interval between two adjacent times as a first duration, the second feature corresponding to the second historical time can be obtained. Based on the first feature and the second information, the machine learning model generates a second identifier for each sub-feature included in the first feature. The second information is obtained based on the second feature corresponding to the second historical time. That is, when generating the second identifier for each sub-feature in the first feature corresponding to the first time, not only is each sub-feature in the first feature itself referenced, but also the sub-features in the second feature corresponding to the second historical time are referenced. This enables the determination of the second identifier for each sub-feature in the first feature based on the relationship between each sub-feature in the first feature and the sub-features in the second historical time, which is beneficial to improving the accuracy of the generated second identifier.

[0121] Optionally, the second information is obtained based on the features of the second feature corresponding to the second historical moment and the features of the second identifier of the sub-features included in the second feature. Then, when generating the second identifier of each sub-feature in the first feature corresponding to the first moment, not only the features of each sub-feature in the first feature itself are referenced, but also the features of the sub-features in the second feature corresponding to the second historical moment and the features of the second identifier of each sub-feature are referenced. Since the features of the second identifier can carry richer information that the machine learning model can understand, it is beneficial to further improve the accuracy of the generated second identifier.

[0122] Alternatively, step 302 may include: the execution device may fuse the first feature and the N second features corresponding to the N historical moments through the first module of the machine learning model to obtain the updated first feature; and then, based on the updated first feature, generate the second identifier of each sub-feature included in the first feature through the first module of the machine learning model.

[0123] Alternatively, step 302 may include: the execution device may generate a second identifier for each sub-feature included in the first feature based solely on the first feature through the first module of the machine learning model. That is, it may not be necessary to use the second information obtained based on the second feature corresponding to each historical moment in N historical moments, but only based on the first feature to generate the second identifier for each sub-feature.

[0124] 303. Based on the first feature and the second identifier of each sub-feature included in the first feature, the location information and the first identifier corresponding to each object in the environment at the first time are obtained through a machine learning model. Objects corresponding to the same first identifier in the environment at different times represent the same object.

[0125] For example, if the execution device is a vehicle or the execution device is integrated into a vehicle, at least one object can include objects in the traffic environment, such as vehicles, electric vehicles, pedestrians, or other objects in the traffic environment; or if the execution device is a smart home or a self-driving device integrated into a smart home, at least one object can include obstacles in the environment at the first moment, such as tables, chairs, boxes, or other objects in the environment. The specific types of objects included in at least one object can be determined based on the actual application scenario. The examples here are only for the convenience of understanding this solution.

[0126] Optionally, in one case, if the execution device generates a second identifier for each sub-feature included in the first feature based on the first feature and the second information through a machine learning model, step 303 may include: the execution device generating location information corresponding to each object in at least one object in the environment at the first moment based on the sub-features corresponding to each object in the environment at the first moment included in the first feature through a second module of the machine learning model; determining the second identifier of the sub-features corresponding to each object in the environment at least one object in the first moment as the first identifier corresponding to each object, thereby obtaining the location information and the first identifier corresponding to each object in the environment at the first moment.

[0127] For example, the second module of the machine learning model may include at least one of the following: a fully connected neural network layer, a residual neural network layer, an MLP, a support vector machine, or other types of neural network layers, etc., which are not exhaustively listed in this embodiment. The first identifier can be an integer value, a string, or other data format, etc., and can be determined according to the actual application scenario.

[0128] For example, the position information of an object may include at least one of the following: the position of the object's bounding box, the outline of the object's bounding box, the position coordinates of the object, the orientation of the object or the bounding box, the size of the object or the bounding box, or other information, which can be determined in combination with the actual application scenario.

[0129] For example, the second module of the machine learning model generates the position information of three objects in the environment at the first moment, which are, in order, position information 1 of object 1, position information 2 of object 2, and position information 3 of object 3. The first module of the machine learning model generates three second identifiers, which are, in order, identifier 1, identifier 2, and identifier 3. Then, the position information and first identifier of object 1 are position information 1 and identifier 1, the position information and first identifier of object 2 are position information 2 and identifier 2, and the position information and first identifier of object 3 are position information 3 and identifier 3. It should be understood that this example is only for the convenience of understanding this solution.

[0130] To more intuitively understand this solution, please refer to Figure 7. Figure 7 is a schematic diagram of generating location information and a first identifier through a machine learning model according to an embodiment of this application. Figure 7 can be understood in conjunction with the above description of Figure 5. Repeated parts will not be described again. As shown in Figure 7, the execution device can input the environmental information at the first moment and the first information corresponding to the first historical moment into the machine learning model to obtain the first feature corresponding to the first moment; based on the first feature corresponding to the first moment, the second identifier of each sub-feature in the first feature is generated through the first module of the machine learning model; based on the first feature corresponding to the first moment, the location information of each object in at least one object in the environment at the first moment is generated through the first module of the machine learning model. The second identifier of the sub-feature corresponding to each object in at least one object in the environment at the first moment is used as the first identifier corresponding to each object. It should be understood that the example in Figure 7 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0131] In this embodiment, since the sub-features corresponding to the same second identifier in the features corresponding to different times represent the sub-features corresponding to the same object, that is, a second identifier can also uniquely point to an object, it is feasible to directly use the second identifier as the first identifier. In addition, by using two different modules (i.e., the first module and the second module) in the machine learning model to predict the location information and the first identifier respectively, the process of generating prediction information by the machine learning model is divided in a more detailed way, which helps to reduce the difficulty of the first module and the second module respectively, thereby helping to improve the accuracy of the generated location information and the first identifier.

[0132] Alternatively, in another scenario, step 303 may include: the execution device generating, through the second module of the machine learning model, the location information and the first identifier of each object in the environment at the first moment, based on the sub-features corresponding to each object in the environment at the first moment and the second identifier of each sub-feature, including the first feature; in other words, the execution device processes the sub-features corresponding to each object in the environment at the first moment and the second identifier of each sub-feature through the second module of the machine learning model to generate the location information and the first identifier of each object in the environment at the first moment.

[0133] Optionally, the machine learning model generates predicted information for each object in the environment at the first moment. This predicted information includes not only position information and a first identifier, but also at least one of the following: velocity, acceleration, angular velocity, or other information about each object, which can be determined based on the actual application scenario. In this embodiment, since the machine learning model generates position information and a first identifier for each object, the first identifier allows for tracking of the same object's position at different times. This also enables more accurate inference of the object's velocity, acceleration, or angular velocity. Not only does the machine learning model provide more information, but it also helps ensure the accuracy of the obtained information.

[0134] For example, in one case, if the execution device generates a second identifier for each sub-feature included in the first feature based on the first feature and the second information through a machine learning model, step 303 may include: the execution device generating position information corresponding to each object in the environment at the first moment based on the sub-features corresponding to each object in the environment at the first moment included in the first feature through a second module of the machine learning model, and also generating at least one of the following for each object: velocity, acceleration, angular velocity or other information for each object; and determining the second identifier of the sub-feature corresponding to each object in the environment at the first moment as the first identifier corresponding to each object.

[0135] In another scenario, step 303 may include: the execution device generating prediction information corresponding to each object in the environment at the first moment based on the sub-features corresponding to each object in the environment at the first moment and the second identifier of each sub-feature, using the second module of the machine learning model; in other words, the execution device processes the sub-features corresponding to each object in the environment at the first moment and the second identifier of each sub-feature using the second module of the machine learning model to generate prediction information corresponding to each object in the environment at the first moment.

[0136] In this embodiment, a machine learning model processes environmental information from multiple time points sequentially at a first frequency to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple time points. Objects corresponding to the same first identifier in the environment at different time points represent the same object. The first frequency represents the time interval between two adjacent time points as a first duration. When processing the environmental information at each time point using the machine learning model, the environmental information of the first time point and the first information corresponding to the first historical time point are input into the machine learning model. Since the time interval between the first time point and the first historical time point is a second duration, which is longer than the first duration, it is beneficial to ensure that the machine learning model uses the same input during the training and application phases, thereby improving the accuracy of the location information and first identifier obtained by the machine learning model. By employing a first-frequency call to the machine learning model, it is possible to adapt to the vehicle's need for processing high-frequency environmental information. Furthermore, after generating a first feature corresponding to the first moment through the machine learning model, this first feature includes sub-features corresponding to each object in the environment at the first moment. The machine learning model also generates a second identifier for each sub-feature included in the first feature. Sub-features corresponding to the same second identifier in features at different moments represent sub-features corresponding to the same object. That is, the machine learning model also predicts the correlation between the sub-features of each object. Based on the first feature and the second identifiers of the sub-features in the first feature, the machine learning model finally obtains the location information and first identifier corresponding to each object in the environment at the first moment. This is beneficial for obtaining more accurate location information and first identifiers through the machine learning model.

[0137] II. Training Phase

[0138] Please refer to Figure 8, which is a flowchart illustrating a model training method provided in an embodiment of this application. The model training method provided in an embodiment of this application may include:

[0139] 801. Input the environmental information at the first moment and the first information corresponding to the first historical moment into the machine learning model to obtain the first feature corresponding to the first moment, wherein the first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment, the second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment, the time interval between the first moment and the first historical moment is the second duration, and the first feature includes the sub-features corresponding to each object in at least one object in the environment at the first moment.

[0140] 802. The first module of the machine learning model generates a second identifier for each sub-feature included in the first feature, wherein the sub-features corresponding to the same second identifier in the features corresponding to different times represent the sub-features corresponding to the same object.

[0141] 803. Based on the first feature and the second identifier of each sub-feature included in the first feature, the location information and the first identifier corresponding to each object in the environment at the first moment are obtained through a machine learning model.

[0142] For example, the training device may be equipped with a training dataset, which includes multiple training data sets. Each training data set includes a training sample and expected information corresponding to the training sample. The training sample includes environmental information at a first moment, and the expected information includes expected position information and expected second identifier for each object in at least one object in the environment at the first moment. The expected information can also be referred to as the truth value, or the expected information can also be understood as the correct information. The expected position information and expected second identifier for each object can also be referred to as the correct position information and correct second identifier for each object.

[0143] Optionally, the expected information may also include at least one of the following: the correct velocity, correct acceleration, correct angular velocity, or other information about each object in the environment at the first moment.

[0144] The specific meanings of the terms and the specific implementation methods of the steps in steps 801 to 803 of the training device can be found in the description of the corresponding embodiment in Figure 3 above, and will not be repeated here. Optionally, if step 802 includes: the training device generating a second identifier for each sub-feature included in the first feature through the first module of the machine learning model based on the first feature and the second information, then the training stage of the machine learning model may include a first training stage and a second training stage, steps 801 to 804 are the first training stage of the machine learning model, and subsequent steps 805 and 806 are the second training stage of the machine learning model.

[0145] For example, in the first training stage of the machine learning model, the training device processes the environmental information of multiple moments sequentially according to the second frequency through the machine learning model to obtain the location information and first identifier of each object in at least one object in the environment at each of the multiple moments. The second frequency represents the time interval between two adjacent moments in the aforementioned multiple moments as the second duration. Since the second feature corresponding to each moment in the multiple moments is generated sequentially by the machine learning model in a low-frequency manner, and the second information is obtained based on the second feature corresponding to the second historical moment, the time interval between the second historical moment and the first moment is the first duration. If the first duration is less than the second duration, the second feature corresponding to the second historical moment may not be obtained. For example, the training device can fill in the unobtainable second feature by filling in preset information.

[0146] 804. The machine learning model is trained using a first loss function, which indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

[0147] For example, after the training device generates prediction information through the machine learning model, which includes the second identifier of the sub-feature corresponding to each object in the environment at the first moment and the position information corresponding to each object, it can generate the function value of the first loss function; based on the function value of the first loss function, the weight parameters in the machine learning model are updated using the backpropagation algorithm to achieve the training of the machine learning model.

[0148] The first loss function indicates the similarity between predicted and expected information. The goal of training using the first loss function includes improving the similarity between predicted and expected information. For example, the first loss function indicates the similarity between the second identifier of a sub-feature and the expected second identifier of the sub-feature, and the similarity between the position information corresponding to each object and the expected position information. Optionally, the first loss function also indicates at least one of the following: the similarity between the velocity corresponding to each object and the correct velocity, the similarity between the angular velocity corresponding to each object and the correct angular velocity, or the similarity between the acceleration corresponding to each object and the correct acceleration, etc.

[0149] The training device repeatedly executes steps 801 to 804 multiple times until the first convergence condition is met, thereby obtaining a machine learning model that has undergone training operations. If steps 801 to 804 represent the first training phase, it can also be referred to as obtaining a machine learning model that has undergone training operations in the first training phase. For example, the first convergence condition may include: satisfying the convergence condition of the first loss function, and / or, the number of times the machine learning model is trained reaches a first preset number.

[0150] 805. In the second training phase, the machine learning model processes the environmental information of multiple time moments sequentially according to the first frequency to obtain the location information and first identifier of each object in at least one object in the environment at each of the multiple time moments. The first frequency represents the time interval between two adjacent time moments as the first duration, and the first duration is less than the second duration.

[0151] Steps 805 and 806 are optional. The specific meanings of the terms in step 805 and the specific implementation methods of the steps can be found in the description of the corresponding embodiment in Figure 3 above, and will not be repeated here.

[0152] 806. Keep the weight parameters of the third module and the second module unchanged, and update the weight parameters of the first module using the second loss function. The second loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature.

[0153] For example, in the second training stage of the machine learning model, after the training device generates prediction information through the machine learning model, it can generate the function value of the first loss function; keeping the weight parameters of the third module and the second module in the machine learning model unchanged, the weight parameters of the first module in the machine learning model are updated using the backpropagation algorithm based on the function value of the first loss function, so as to realize the training of the first module of the machine learning model.

[0154] The second loss function indicates the similarity between the second identifier of a sub-feature and its expected second identifier. The purpose of training using the loss function includes improving the similarity between the second identifier of a sub-feature and its expected second identifier. Optionally, the first loss function may also indicate at least one of the following: the similarity between the velocity corresponding to each object and the correct velocity, the similarity between the angular velocity corresponding to each object and the correct angular velocity, or the similarity between the acceleration corresponding to each object and the correct acceleration, etc.

[0155] During the second training phase, the training device may repeat steps 805 and 806 multiple times to train multiple batches of the first module of the machine learning model until the second convergence condition is met. For example, the second convergence condition may include: meeting the convergence condition of the second loss function, and / or, the number of times the machine learning model has been trained reaches a second preset number.

[0156] For example, the first frequency refers to the frequency used by the training device in the second training phase. When the training device performs training operations within the same batch, it uses the same frequency to process the environmental information at multiple times in sequence. In other words, the time interval between adjacent times is the same for multiple training samples within the same batch.

[0157] The training device can use the same or different frequencies when performing training operations for different batches. For example, when performing training operations for batch 1, a first frequency 1 is used to process environmental information at multiple times. The first frequency 1 represents the time interval between two adjacent times as a first duration 1. When performing training operations for batch 2, a first frequency 2 is used to process environmental information at multiple times. The first frequency 1 represents the time interval between two adjacent times as a first duration 2. The first duration 1 and the first duration 2 can be the same or different. If the first duration 1 and the first duration 2 are different, then both the first duration 1 and the first duration 2 are less than the second duration.

[0158] In this embodiment, the training phase of the machine learning model is divided into a first training phase and a second training phase. In the first training phase, environmental information at multiple time points is processed sequentially according to a second frequency, where the second frequency represents the time interval between two adjacent time points as a second duration. The loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information. This allows the machine learning model to learn how to generate accurate location information and the second identifier. The second duration is relatively long, thus enabling compatibility with obtaining the expected location information using frame extraction and avoiding the problem of excessively high annotation costs. In the second training phase, a higher first frequency is used to process environmental information at multiple time points sequentially. The loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature. Accurate expected location information is no longer required. Since the annotation cost of the expected second identifier is low, the two-stage training method still avoids the problem of excessively high annotation costs. Furthermore, in the second training phase, the weight parameters of the second and third modules remain unchanged, and only the weight parameters of the first module used to generate the second identifier are updated. This not only preserves the machine learning model's ability to generate accurate location information but also helps the second module learn to generate more accurate second identifiers through the second training phase.

[0159] Based on the embodiments corresponding to Figures 1 to 8, in order to better implement the above-mentioned solutions of the embodiments of this application, related equipment for implementing the above-mentioned solutions is also provided below. Specifically, refer to Figure 9, which is a structural schematic diagram of an information processing device provided in an embodiment of this application. The information processing device 900 is used to process environmental information at multiple times sequentially according to a first frequency using a machine learning model, to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple times. Objects corresponding to the same first identifier in the environment at different times represent the same object. The first frequency represents the time interval between two adjacent times as a first duration. The information processing device 900 includes: an input module 901, used to input the environmental information at the first time and the first information corresponding to the first historical time into the machine learning model to obtain a first feature corresponding to the first time, wherein the first information includes: environmental information at the first historical time and / or The second feature corresponding to the first historical moment is obtained based on the environmental information of the first historical moment. The time interval between the first moment and the first historical moment is the second duration, which is longer than the first duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. The generation module 902 is used to generate a second identifier for each sub-feature included in the first feature through a machine learning model. Among the features corresponding to different moments, the sub-features corresponding to the same second identifier represent the sub-features corresponding to the same object. The processing module 903 is used to obtain the location information and the first identifier corresponding to each object in the environment at the first moment through a machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature.

[0160] Optionally, the generation module 902 is specifically used to generate a second identifier for each sub-feature included in the first feature through a machine learning model based on the first feature and the second information, wherein the second information is obtained based on the second feature corresponding to the second historical moment, and the time interval between the first moment and the second historical moment is a first duration.

[0161] Optionally, the second feature corresponding to the second historical moment includes sub-features corresponding to each object in at least one object in the environment of the second historical moment, and the second information is obtained based on the features of the second feature corresponding to the second historical moment and the features of the sub-features included in the second feature corresponding to the second historical moment.

[0162] Optionally, the second identifier is generated by the first module in the machine learning model. The processing module 903 is specifically used to generate the location information corresponding to each object in the environment at the first moment based on the sub-features corresponding to each object in the environment at the first moment, which are included in the first feature. The second identifier of the sub-features corresponding to each object in at least one object in the environment at the first moment is used as the first identifier corresponding to each object.

[0163] Optionally, the processing module 903 is specifically used to generate prediction information for each object in the environment at the first moment through a machine learning model. The prediction information includes position information and a first identifier, and the prediction information also includes at least one of the following: velocity, acceleration or angular velocity.

[0164] It should be noted that the information interaction and execution process between the modules / units in the information processing device 900 are based on the same concept as the various method embodiments corresponding to Figures 1 to 8 in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0165] Please refer to Figure 10, which is a schematic diagram of a model training device provided in an embodiment of this application. The model training device 1000 includes: an input module 1001, used to input environmental information at a first moment and first information corresponding to a first historical moment into a machine learning model to obtain a first feature corresponding to the first moment, wherein the first information includes: environmental information at the first historical moment and / or a second feature corresponding to the first historical moment, the second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment, the time interval between the first moment and the first historical moment is a second duration, and the first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment; and a generation module 1002, used to generate the model through machine learning. The first module of the model generates a second identifier for each sub-feature included in the first feature, wherein sub-features corresponding to the same second identifier in features at different times represent sub-features corresponding to the same object; the processing module 1003 is used to obtain the location information and first identifier corresponding to each object in the environment at the first time through a machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature, wherein objects corresponding to the same first identifier in the environment at different times represent the same object; the training module 1004 is used to train the machine learning model using a first loss function, wherein the first loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

[0166] Optionally, the training phase of the machine learning model includes a first training phase and a second training phase. In the first training phase, the machine learning model processes environmental information at multiple time points sequentially according to a second frequency, where the second frequency represents the time interval between two adjacent time points as a second duration. The training module 1004 is specifically used in the first training phase to train the machine learning model using a first loss function. The processing module 1003 is also used in the second training phase to process environmental information at multiple time points sequentially according to the first frequency using the machine learning model, obtaining the location information and first identifier corresponding to each object in at least one object in the environment at each of the multiple time points. The first frequency represents the time interval between two adjacent time points as a first duration, and the first duration is less than the second duration. The training module 1004 is also used to keep the weight parameters of the third module and the second module unchanged, and update the weight parameters of the first module using a second loss function. The second loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature.

[0167] It should be noted that the information interaction and execution process between the modules / units in the model training device 1000 are based on the same concept as the various method embodiments corresponding to Figures 1 to 8 in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0168] The following describes a device provided in an embodiment of this application. Please refer to Figure 11, which is a schematic diagram of the structure of a device provided in an embodiment of this application. Optionally, the device 1100 performs the functions of the execution device or training device in the various method embodiments corresponding to Figures 1 to 8.

[0169] Device 1100 includes a memory 1102 and at least one processor 1101. Optionally, device 1100 further includes at least one accelerator 1103. Optionally, processor 1101 implements the method in the above embodiments by reading program instructions stored in memory 1102; or, processor 1101 reads program instructions stored in memory 1102 and implements the steps executed by the machine learning model in the method in the above embodiments through accelerator 1103; or, processor 1101 may also implement the method in the above embodiments by reading program instructions stored internally; or, processor 1101 may also read program instructions stored internally and implement the steps executed by the machine learning model in the method in the above embodiments through accelerator 1103.

[0170] When the processor 1101 reads the program instructions stored in the memory 1102 to implement the method in the above embodiments, the memory 1102 stores the program instructions that implement the method provided in the above embodiments of this application.

[0171] Optionally, at least one processor 1101 is one or more CPUs, either a single-core CPU or a multi-core CPU. For example, memory 1102 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, or optical memory. Memory 1102 stores program instructions for the operating system. For example, at least one accelerator 1103 may include at least one of the following: GPU, NPU, TPU, ASIC, FPGA, or other types of accelerators. After the program instructions stored in memory 1102 are read by the at least one processor 1101, device 1100 executes the corresponding operations in the foregoing embodiments.

[0172] Optionally, the device 1100 also includes a network interface 1104, which can be a wired interface or a wireless interface. The network interface 1104 is used to perform data transmission and reception in the various method embodiments corresponding to Figures 1 to 8.

[0173] It should be understood that network interface 1104 has the functions of receiving and sending data. The functions of "receiving data" and "sending data" can be integrated into the same transceiver interface, or the functions of "receiving data" and "sending data" can be implemented in different interfaces, which is not limited here. In other words, network interface 1104 may include one or more interfaces for implementing the functions of "receiving data" and "sending data".

[0174] After the processor 1101 reads the program instructions from the memory 1102, other functions that the device 1100 can perform are described in the preceding method embodiments.

[0175] Optionally, the device 1100 also includes a bus 1105, through which the processor 1101 and memory 1102 are typically interconnected, or in other ways.

[0176] The device 1100 provided in this application embodiment is used to execute the methods executed by the execution device or training device in the above-described method embodiments, and to achieve the corresponding beneficial effects. The specific implementation of the device 1100 shown in FIG11 can be referred to the descriptions in the foregoing method embodiments, and will not be repeated here.

[0177] This application also provides a vehicle, as shown in Figure 12. Figure 12 is a structural schematic diagram of a vehicle provided in this application embodiment. The vehicle 100 is configured for fully or partially automated driving mode. For example, the vehicle 100 can control itself while in automated driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of other vehicles performing possible behaviors, and control the vehicle 100 based on the determined information. When the vehicle 100 is in automated driving mode, the vehicle 100 can also be set to operate without human interaction.

[0178] Vehicle 100 may include various subsystems, such as a mobility system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0179] The mobility system 102 may include components that provide powered motion to the vehicle 100. In one embodiment, the mobility system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121.

[0180] Engine 118 can be an internal combustion engine, an electric motor, an air-compressed engine, or other combinations of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. Engine 118 converts energy source 119 into mechanical energy. Examples of energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 can also provide energy to other systems of vehicle 100. Transmission 120 transmits mechanical power from engine 118 to wheels 121. Transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0181] Sensor system 104 may include several sensors for sensing information about the environment surrounding vehicle 100. For example, sensor system 104 may include a positioning system 122 (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors for the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensing data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0182] The positioning system 122 can be used to estimate the geographical location of the vehicle 100. An IMU 124 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope. A radar 126 can use radio signals to sense objects in the surrounding environment of the vehicle 100, specifically millimeter-wave radar or lidar. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or direction of travel of objects. A laser rangefinder 128 can use lasers to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. A camera 130 can be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 130 can be a still camera or a video camera.

[0183] The control system 106 controls the operation of the vehicle 100 and its components. The control system 106 may include various components, including a steering system 132, a throttle 134, a braking unit 136, a computer vision system 140, a trajectory control system 142, and an obstacle avoidance system 144.

[0184] The steering system 132 is operable to adjust the forward direction of the vehicle 100. For example, in one embodiment, it may be a steering wheel system. The throttle 134 controls the operating speed of the engine 118 and thus the speed of the vehicle 100. The braking unit 136 controls the deceleration of the vehicle 100. The braking unit 136 may use friction to slow down the wheels 121. In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 may also take other forms to slow down the rotational speed of the wheels 121 to control the speed of the vehicle 100. The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the vehicle 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, etc. The route control system 142 is used to determine the driving route and speed of the vehicle 100. In some embodiments, the route control system 142 may include a lateral planning module 1421 and a longitudinal planning module 1422, which are respectively used to combine data from the obstacle avoidance system 144, GPS 122, and one or more predetermined maps to determine the driving route and speed for the vehicle 100. The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise traverse obstacles in the environment of the vehicle 100, which may specifically be physical obstacles and virtual moving bodies that may collide with the vehicle 100. In one example, the control system 106 may add or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0185] Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users via peripheral device 108. Peripheral device 108 may include wireless communication system 146, on-board computer 148, microphone 150, and / or speaker 152. In some embodiments, peripheral device 108 provides a means for a user of vehicle 100 to interact with user interface 116. For example, on-board computer 148 may provide information to a user of vehicle 100. User interface 116 may also operate on-board computer 148 to receive user input. On-board computer 148 may be operated via a touchscreen. In other cases, peripheral device 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100. Similarly, speaker 152 may output audio to a user of vehicle 100. Wireless communication system 146 may communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 146 may utilize a wireless local area network (WLAN) for communication. In some embodiments, the wireless communication system 146 may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with the device. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices that can enable public and / or private data communication between the vehicle and / or a roadside station.

[0186] Power source 110 can provide power to various components of vehicle 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of vehicle 100. In some embodiments, power source 110 and energy source 119 can be implemented together, as is the case in some fully electric vehicles.

[0187] Some or all of the functions of vehicle 100 are controlled by computer system 112. Computer system 112 may include at least one processor 113, which executes program instructions 115 stored in a non-transitory computer-readable medium such as memory 114. Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner. Processor 113 may include any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, processor 113 may also include a dedicated device such as a GPU, NPU, TPU, ASIC, FPGA, or other hardware-based processor. Although FIG12 functionally illustrates the processor, memory, and other components of computer system 112 in the same block, those skilled in the art will understand that the processor or memory may actually include multiple processors or memories not stored in the same physical housing. For example, memory 114 may be a hard disk drive or other storage medium located in a housing different from that of computer system 112. Therefore, references to processor 113 or memory 114 will be understood to include references to a collection of processors or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, can each have their own processor that performs only calculations related to the component's specific function.

[0188] In all the aspects described herein, processor 113 may be located remotely from vehicle 100 and may communicate wirelessly with vehicle 100. In other aspects, some of the processes described herein are executed on processor 113 located within vehicle 100, while others are executed by remote processor 113, including taking the necessary steps to perform a single operation.

[0189] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that can be executed by processor 113 to perform various functions of vehicle 100, including those described above. Memory 114 may also contain additional program instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the mobility system 102, sensor system 104, control system 106, and peripheral devices 108. In addition to instructions 115, memory 114 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information may be used by vehicle 100 and computer system 112 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes. A user interface 116 is provided to or receives information from a user of vehicle 100. Optionally, user interface 116 may include one or more input / output devices within the set of peripheral devices 108, such as wireless communication system 146, on-board computer 148, microphone 150, and speaker 152.

[0190] Computer system 112 can control the functions of vehicle 100 based on input received from various subsystems (e.g., driving system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering system 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0191] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 114 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0192] Optionally, the above components are merely examples. In practical applications, components in each of the above modules may be added or removed according to actual needs. Figure 12 should not be construed as a limitation on the embodiments of this application. A vehicle traveling on a road, such as vehicle 100 above, can identify objects in its surrounding environment to determine adjustments to its current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the vehicle can be determined.

[0193] Optionally, vehicle 100 or computing devices associated with vehicle 100, such as computer system 112, computer vision system 140, and memory 114 as shown in Figure 12, can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can also be considered together to predict the behavior of a single identified object. Vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, vehicle 100 can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of vehicle 100, such as the lateral position of vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc. In addition to providing program instructions to adjust the speed of the vehicle, the computing device may also provide program instructions to modify the steering angle of the vehicle 100 so that the vehicle 100 follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the vehicle 100 (e.g., cars in adjacent lanes on the road).

[0194] In this embodiment, the processor 113 in the vehicle 100 is used to execute the method executed by the execution device in the embodiments corresponding to Figures 1 to 8. It should be noted that the specific manner in which the processor 113 executes the aforementioned steps is based on the same concept as the method embodiments corresponding to Figures 1 to 8 in this application, and the resulting technical effects are the same as those in the method embodiments corresponding to Figures 1 to 8 in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0195] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the steps executed by the execution device or training device in the methods described in the embodiments shown in Figures 1 to 8.

[0196] This application also provides a computer program product, which includes a program that, when run on a computer, causes the computer to perform the steps performed by the execution device or training device in the methods described in the embodiments shown in Figures 1 to 8 above.

[0197] This application also provides a circuit system including a processing circuit configured to perform the steps executed by the execution device or training device in the methods described in the embodiments shown in Figures 1 to 8 above.

[0198] The execution device or information processing apparatus provided in this application embodiment can specifically be a chip. The chip includes a processing unit, such as a processor. Optionally, the chip also includes a communication unit, such as an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to execute the methods described in the embodiments shown in Figures 1 to 8. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. The storage unit can also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0199] The processor mentioned above can be a general-purpose central processing unit, microprocessor, GPU, NPU, TPU, ASIC, FPGA, or one or more integrated circuits used to control the execution of the program in the first aspect of the above method.

[0200] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0201] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CLUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0202] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0203] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. An information processing method, characterized in that, The method is used to process environmental information at multiple times sequentially according to a first frequency using a machine learning model, to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple times. Objects corresponding to the same first identifier in the environment at different times represent the same object. The first frequency represents the time interval between two adjacent times as a first duration. The method includes: The environmental information at the first moment and the first information corresponding to the first historical moment are input into the machine learning model to obtain the first feature corresponding to the first moment. The first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is a second duration, which is longer than the first duration. The first feature includes the sub-features corresponding to each object in at least one object in the environment at the first moment. The machine learning model generates a second identifier for each sub-feature included in the first feature, wherein the sub-features corresponding to the same second identifier in the features corresponding to different times represent the sub-features corresponding to the same object; Based on the first feature and the second identifier of each sub-feature included in the first feature, the machine learning model obtains the location information and the first identifier corresponding to each object in the environment at the first moment.

2. The method according to claim 1, characterized in that, The step of generating a second identifier for each sub-feature included in the first feature through the machine learning model includes: generating a second identifier for each sub-feature included in the first feature through the machine learning model based on the first feature and second information, wherein the second information is obtained based on the second feature corresponding to the second historical moment, and the time interval between the first moment and the second historical moment is the first duration.

3. The method according to claim 2, characterized in that, The second feature corresponding to the second historical moment includes sub-features corresponding to each object in at least one object in the environment of the second historical moment, and the second information is obtained based on the second feature corresponding to the second historical moment and the features of the second identifier of the sub-features included in the second feature corresponding to the second historical moment.

4. The method according to claim 2, characterized in that, The second identifier is generated by the first module in the machine learning model. The second identifier, based on the first feature and each sub-feature included in the first feature, is used by the machine learning model to obtain the location information and the first identifier corresponding to each object in the environment at the first time moment, including: Based on the sub-features corresponding to each object in the environment at the first moment, the second module in the machine learning model generates the location information corresponding to each object in the environment at the first moment. The second identifier of the sub-features corresponding to each object in at least one object in the environment at the first moment is used as the first identifier corresponding to each object.

5. The method according to any one of claims 1 to 4, characterized in that, The step of generating the location information and the first identifier corresponding to each object in the environment at the first moment through the machine learning model includes: The machine learning model generates prediction information for each object in the environment at the first moment. The prediction information includes the location information and the first identifier. The prediction information also includes at least one of the following: velocity, acceleration, or angular velocity.

6. A method for training a model, characterized in that, The method includes: The environmental information at the first moment and the first information corresponding to the first historical moment are input into the machine learning model to obtain the first feature corresponding to the first moment. The first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is the second duration. The first feature includes the sub-features corresponding to each object in at least one object in the environment at the first moment. The first module of the machine learning model generates a second identifier for each sub-feature included in the first feature, wherein sub-features corresponding to the same second identifier in the features at different times represent sub-features corresponding to the same object. Based on the first feature and the second identifier of each sub-feature included in the first feature, the location information and the first identifier corresponding to each object in the environment at the first time are obtained through the machine learning model. Objects corresponding to the same first identifier in the environment at different times represent the same object. The machine learning model is trained using a first loss function, which indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

7. The method according to claim 6, characterized in that, The training phase of the machine learning model includes a first training phase and a second training phase. In the first training phase, the machine learning model processes environmental information at multiple times according to a second frequency. The second frequency represents the time interval between two adjacent times as a second duration. The step of training the machine learning model using a first loss function includes: in the first training phase, training the machine learning model using the first loss function. The method further includes: In the second training phase, the environmental information at multiple times is processed sequentially by the machine learning model according to the first frequency to obtain the location information and the first identifier corresponding to each object in at least one object in the environment at each of the multiple times. The first frequency represents the time interval between two adjacent times as the first duration, and the first duration is less than the second duration. Keeping the weight parameters of the third module and the second module unchanged, the weight parameters of the first module are updated using a second loss function, which indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature.

8. An information processing device, characterized in that, The device is used to process environmental information at multiple times sequentially according to a first frequency using a machine learning model, to obtain the location information and a first identifier corresponding to each object in at least one object in the environment at each of the multiple times. Objects corresponding to the same first identifier in the environment at different times represent the same object. The first frequency represents the time interval between two adjacent times as a first duration. The device includes: An input module is used to input environmental information at a first moment and first information corresponding to a first historical moment into a machine learning model to obtain a first feature corresponding to the first moment. The first information includes: environmental information at the first historical moment and / or a second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is a second duration, which is longer than the first duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. The generation module is used to generate a second identifier for each sub-feature included in the first feature through the machine learning model, wherein the sub-features corresponding to the same second identifier in the features corresponding to different times represent the sub-features corresponding to the same object; The processing module is used to obtain the location information and the first identifier corresponding to each object in the environment at the first time moment through the machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature.

9. The apparatus according to claim 8, characterized in that, The generation module is specifically used to generate a second identifier for each sub-feature included in the first feature based on the first feature and the second information through the machine learning model, wherein the second information is obtained based on the second feature corresponding to the second historical moment, and the time interval between the first moment and the second historical moment is the first duration.

10. The apparatus according to claim 9, characterized in that, The second feature corresponding to the second historical moment includes sub-features corresponding to each object in at least one object in the environment of the second historical moment, and the second information is obtained based on the second feature corresponding to the second historical moment and the features of the second identifier of the sub-features included in the second feature corresponding to the second historical moment.

11. The apparatus according to claim 9, characterized in that, The second identifier is generated by the first module in the machine learning model. The processing module is specifically used to generate the location information corresponding to each object in the environment at the first moment based on the sub-features corresponding to each object in the environment at the first moment, which are included in the first feature. The second identifier of the sub-features corresponding to each object in at least one object in the environment at the first moment is used as the first identifier corresponding to each object.

12. The apparatus according to any one of claims 8 to 11, characterized in that, The processing module is specifically used to generate prediction information for each object in the environment at the first moment through the machine learning model. The prediction information includes the position information and the first identifier, and the prediction information also includes at least one of the following: velocity, acceleration, or angular velocity.

13. A training device for a model, characterized in that, The device includes: The input module is used to input the environmental information at a first moment and the first information corresponding to a first historical moment into the machine learning model to obtain a first feature corresponding to the first moment. The first information includes: the environmental information at the first historical moment and / or the second feature corresponding to the first historical moment. The second feature corresponding to the first historical moment is obtained based on the environmental information at the first historical moment. The time interval between the first moment and the first historical moment is a second duration. The first feature includes sub-features corresponding to each object in at least one object in the environment at the first moment. The generation module is configured to generate a second identifier for each sub-feature included in the first feature through the first module of the machine learning model, wherein sub-features corresponding to the same second identifier at different times represent sub-features corresponding to the same object. The processing module is used to obtain the location information and the first identifier corresponding to each object in the environment at the first time moment through the machine learning model based on the first feature and the second identifier of each sub-feature included in the first feature. Objects corresponding to the same first identifier in the environment at different times represent the same object. The training module is used to train the machine learning model using a first loss function, the first loss function indicating the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature, as well as the similarity between the location information and the expected location information.

14. The apparatus according to claim 13, characterized in that, The training phase of the machine learning model includes a first training phase and a second training phase. In the first training phase, the machine learning model processes environmental information at multiple times according to a second frequency. The second frequency represents the time interval between two adjacent times as a second duration. Specifically, the training module is used to train the machine learning model using the first loss function in the first training phase. The processing module is further configured to process the environmental information of multiple time moments sequentially according to a first frequency through a machine learning model during the second training phase, thereby obtaining the location information and the first identifier corresponding to each object in at least one object in the environment at each of the multiple time moments, wherein the first frequency represents the time interval between two adjacent time moments as a first duration, and the first duration is less than the second duration. The training module is further configured to keep the weight parameters of the third module and the second module unchanged, and update the weight parameters of the first module using a second loss function, wherein the second loss function indicates the similarity between the second identifier of the sub-feature and the expected second identifier of the sub-feature.

15. A device, characterized in that, The method includes a processor coupled to a memory storing program instructions that, when executed by the processor, implement the method of any one of claims 1 to 7.

16. A vehicle, characterized in that, The method includes a processor coupled to a memory storing program instructions, which, when executed by the processor, implement the method of any one of claims 1 to 5.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 7.

18. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 7.

19. A chip, characterized in that, The chip includes a processor for performing the steps of the method according to any one of claims 1 to 7.

Images