Methods, devices, equipment and media for detecting the motion state of a target object

Abstract

This disclosure provides a method, apparatus, device, and storage medium for detecting the motion state of a target object, which can be applied to the fields of computer technology and autonomous driving technology. The method includes: responding to receiving multiple frames of images of a target object, detecting the multiple frames to obtain temporal information of the target object's position, wherein the temporal information represents the target object's position in each frame and the acquisition time information of each frame; obtaining first motion state prediction information of the target object based on the temporal information; obtaining first reference state information, second reference state information, and second motion state prediction information based on the acquisition time information and the temporal information; and obtaining target motion state information based on the first reference state information, second reference state information, first motion state prediction information, and second motion state prediction information.

Description

Technical Field

[0001] This disclosure relates to the fields of computer technology and autonomous driving technology, and in particular to methods, apparatus, devices, media and program products for detecting the motion state of a target object. Background Technology

[0002] With the development of technology and the improvement of people's living standards, autonomous driving technology, as an emerging technology, has led to an increasing demand for driverless vehicles (Vehicles). For example, Vehicles can be used to transport goods in industrial parks, airports, and ports, reducing transportation costs.

[0003] In implementing this disclosure, it was discovered that autonomous vehicles need to perceive the motion state of moving objects on the road in order to automatically plan their paths. Existing technologies use an extended Kalman filter (EKF) to predict the motion state of the perceived objects. However, when a moving object undergoes a sudden change in speed, the EKF cannot accurately predict the object's next motion state. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, apparatus, device, medium and program product for detecting the motion state of a target object.

[0005] According to a first aspect of this disclosure, a method for detecting the motion state of a target object is provided, comprising:

[0006] In response to receiving multiple frames of images of the target object, the multiple frames of images are detected to obtain the position and timing information of the target object. The position and timing information represents the position information of the target object in each frame of the image and the acquisition time information of each frame of the image.

[0007] Based on the location and time sequence information, the first motion state prediction information of the target object is obtained;

[0008] Based on the collected time information and location time sequence information, first reference state information, second reference state information, and second motion state prediction information are obtained. The first reference state information represents the average velocity of the target object within a preset distance, and the second reference state information represents the average velocity of the target object within a preset time period.

[0009] The target motion state information is obtained based on the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information.

[0010] According to embodiments of this disclosure, based on the acquisition time information and location timing information, first reference state information, second reference state information, and second motion state prediction information are obtained, including:

[0011] Based on the preset distance and preset duration, the location information of the first time, the location information of the second time, and the location information of the third time are determined from the collected time information and location time sequence information. The first time represents the current time, the second time represents a historical time that is a first time interval from the current time, and the third time represents a historical time that is a second time interval from the current time. The second time interval is a preset multiple of the first time interval.

[0012] Based on the location information at the first moment, the location information at the third moment, and the second duration, the first reference state information is obtained; and

[0013] Based on the position information at the first moment, the position information at the second moment, the first preset duration information, and the preset multiple, the second reference state information and the second motion state prediction information are obtained.

[0014] According to embodiments of this disclosure, first reference state information is obtained based on the location information at a first time point, the location information at a third time point, and a second duration, including:

[0015] Based on the location information at the first and third moments, the first distance information is obtained; and

[0016] Based on the first distance information and the second duration, the first reference state information is obtained.

[0017] According to embodiments of this disclosure, second reference state information and second motion state prediction information are obtained based on position information at a first time moment, position information at a second time moment, first preset duration information, and a preset multiple, including:

[0018] Based on the location information at the first moment and the location information at the second moment, the second distance information is obtained;

[0019] Based on the second distance information and the first preset duration information, the second motion state prediction information is obtained; and

[0020] Based on the preset multiple and the second motion state prediction information, the second reference state information is obtained.

[0021] According to embodiments of this disclosure, it further includes:

[0022] The number of target images is determined based on the image acquisition parameters of the target object;

[0023] Based on the number of target images, multiple target images are determined from multiple frames of images;

[0024] Multiple target images are stored sequentially in a message queue according to their acquisition time.

[0025] According to embodiments of this disclosure, target motion state information is obtained based on first reference state information, second reference state information, first motion state prediction information, and second motion state prediction information, including:

[0026] If it is determined that the first reference state information is less than the first motion state prediction information, or if it is determined that the second reference state information is greater than the first motion state prediction information, then the second motion state prediction information is determined as the target motion state information.

[0027] According to embodiments of this disclosure, obtaining target motion state information based on first reference state information, second reference state information, first motion state prediction information, and second motion state prediction information further includes:

[0028] If it is determined that the first reference state information is greater than the first motion state prediction information, and the second reference state information is less than the first motion state prediction information, then the first motion state prediction information is determined to be the target motion state information.

[0029] According to embodiments of this disclosure, detection is performed on multiple frames of images to obtain temporal information about the location of a target object, including:

[0030] Detection is performed on multiple frames of images to obtain multiple target detection boxes, including the target object; and

[0031] Based on the position coordinates of the upper left corner of multiple target detection boxes and the acquisition time information of each frame image, the position temporal information of the target object is obtained.

[0032] A second aspect of this disclosure provides a motion state detection device for a target object, comprising:

[0033] The detection module is used to detect the multiple frames of images of the target object received, and obtain the position and timing information of the target object. The position and timing information represents the position information of the target object in each frame of the image and the acquisition time information of each frame of the image.

[0034] The first prediction module is used to obtain the first motion state prediction information of the target object based on the position and time sequence information using the extended Kalman filter algorithm.

[0035] The second prediction module is used to obtain first reference state information, second reference state information, and second motion state prediction information based on the collected time information and location time sequence information. The first reference state information represents the average velocity of the target object within a preset distance, and the second reference state information represents the average velocity of the target object within a preset time period.

[0036] The acquisition module is used to obtain the target motion state information based on the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information.

[0037] A third aspect of this disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the motion state detection method for the target object described above.

[0038] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the aforementioned motion state detection method for a target object.

[0039] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the motion state detection method for the target object described above.

[0040] According to embodiments of this disclosure, after processing multiple frames of images of the target object, the average speed of the target object within a preset distance and the average speed within a preset time are calculated. The motion state prediction information obtained by the extended Kalman filter algorithm can be corrected based on the average speed of the target object within the preset distance and the average speed within the preset time. Utilizing the movement of the target object across multiple frames not only overcomes the susceptibility of single-frame movement to noise interference but also reflects the accuracy of the motion state prediction information obtained by the extended Kalman filter algorithm in a timely manner. When the moving target object undergoes a rapid change in speed, the inaccurate motion state prediction information obtained by the extended Kalman filter algorithm can be corrected, accurately predicting the next motion state of the moving target object. This solves the problem that when using the extended Kalman filter to predict the motion state of the sensing results, the extended Kalman filter cannot accurately predict the next motion state of the moving target object when it undergoes a rapid change in speed. Attached Figure Description

[0041] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0042] Figure 1 The illustration schematically depicts an application scenario of a motion state detection method, apparatus, device, medium, and program product for a target object according to embodiments of the present disclosure.

[0043] Figure 2 A flowchart illustrating a motion state detection method for a target object according to an embodiment of the present disclosure is shown schematically.

[0044] Figure 3 This schematically illustrates a flowchart of a method for obtaining first reference state information, second reference state information, and second motion state prediction information based on acquisition time information and location timing information according to an embodiment of the present disclosure.

[0045] Figure 4 This schematically illustrates a flowchart of a method for obtaining target motion state information based on first reference state information, second reference state information, first motion state prediction information, and second motion state prediction information according to an embodiment of the present disclosure.

[0046] Figure 5 A flowchart illustrating a method for detecting the motion state of a target object according to another embodiment of the present disclosure is shown schematically.

[0047] Figure 6 A schematic diagram illustrating the structure of a motion state detection device for a target object according to an embodiment of the present disclosure is shown; and

[0048] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a motion state detection method for a target object according to an embodiment of the present disclosure. Detailed Implementation

[0049] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0050] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0051] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0052] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0053] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0054] In the technical solutions of this disclosure, user authorization or consent is obtained before acquiring or collecting user personal information.

[0055] In related technologies, autonomous vehicles need to predict the speed and direction of moving objects on the road during operation in order to achieve automatic path planning. Generally, due to the influence of sensing devices and sensing algorithms, the sensed moving objects on the road are a nonlinear system, and are also affected by noise, resulting in unstable results between frames. An extended Kalman filter (EKF) is typically used to predict the operational status of these sensing results. Although the EKF can effectively solve this instability, its prediction performance is not ideal in some highly nonlinear situations. Existing technologies have not effectively solved this problem and have abandoned the use of the EKF, opting for other algorithms for prediction.

[0056] In implementing this disclosure, it was discovered that in nonlinear cases, the EKF linearizes the nonlinear function around the average value of the current estimated state. This linearization is performed at each minute time step, and the resulting Jacobian matrix is ​​then used to predict and update the state of the Kalman filter algorithm. However, when the target data to be predicted belongs to a highly nonlinear state, the system may be nondifferentiable, the Jacobian matrix may not exist, and the prediction performance will be affected, leading to inaccurate predictions. In autonomous vehicle scenarios, for highly nonlinear states formed by sudden speed changes on the road, autonomous vehicles struggle to accurately predict future states, potentially causing collision risks. For example, if the speed prediction is inaccurate and a vehicle ahead brakes suddenly, the autonomous vehicle may make an incorrect judgment, potentially leading to rear-end collisions or other accidents.

[0057] The embodiments of this disclosure provide a method for detecting the motion state of a target object, comprising: in response to receiving multiple frames of images of the target object, detecting the multiple frames of images to obtain positional temporal information of the target object, wherein the positional temporal information represents the position information of the target object in each frame of the image and the acquisition time information of each frame of the image; using an extended Kalman filter algorithm, obtaining first motion state prediction information of the target object based on the positional temporal information; obtaining first reference state information, second reference state information, and second motion state prediction information based on the acquisition time information and the positional temporal information, wherein the first reference state information represents the average velocity of the target object within a preset distance, and the second reference state information represents the average velocity of the target object within a preset time period; and obtaining target motion state information based on the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information.

[0058] Figure 1 The illustration schematically depicts an application scenario of a motion state detection method, apparatus, device, medium, and program product for a target object according to embodiments of the present disclosure.

[0059] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0060] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0061] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0062] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0063] It should be noted that the motion state detection method for the target object provided in this embodiment can generally be executed by server 105. Correspondingly, the motion state detection device for the target object provided in this embodiment can generally be located in server 105. The motion state detection method for the target object provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the motion state detection device for the target object provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0064] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0065] The following will be based on Figure 1 The described scene, through Figures 2-5 The motion state detection method for the target object according to the disclosed embodiments is described in detail.

[0066] Figure 2 A flowchart illustrating a motion state detection method for a target object according to an embodiment of the present disclosure is shown schematically.

[0067] like Figure 2 As shown, the motion state detection method 200 for the target object in this embodiment includes operations S210 to S240.

[0068] In operation S210, in response to receiving multiple frames of images of the target object, the multiple frames of images are detected to obtain the position and timing information of the target object. The position and timing information represents the position information of the target object in each frame of the image and the acquisition time information of each frame of the image.

[0069] According to embodiments of this disclosure, after an autonomous vehicle senses a target object through a sensing device or sensing algorithm, it acquires multiple frames of images of the target object using an image acquisition device or a laser acquisition device. These multiple frames of images can be sent to a server via a network. Upon receiving the multiple frames of images, the server performs detection on the images to obtain the target object's location and timing information.

[0070] The target objects can be pedestrians, vehicles, and other mobile devices.

[0071] For example, after an autonomous vehicle detects a pedestrian in front of it using its sensing devices, it uses an image acquisition device to capture multiple frames of images of the pedestrian. These multiple frames of images can be sent to a server via a network. After receiving the multiple frames of images, the server performs detection on the images to obtain the pedestrian's position information in each frame and the acquisition time information for that frame.

[0072] In operation S220, the first motion state prediction information of the target object is obtained based on the position and timing information.

[0073] For example, the extended Kalman filter algorithm can be used to obtain the first motion state prediction information of the target object based on the position and time sequence information.

[0074] According to embodiments of this disclosure, the extended Kalman filter algorithm can be implemented by performing a Taylor expansion of the nonlinear function, omitting higher-order terms and retaining the first-order terms of the expansion, thereby linearizing the nonlinear function. Finally, the first motion state estimate and variance estimate of the target object are approximately calculated using the Kalman filter algorithm.

[0075] According to embodiments of this disclosure, the first motion state prediction information may be the predicted velocity information and the predicted direction information of the first motion state.

[0076] In operation S230, based on the acquisition time information and position timing information, the first reference state information, the second reference state information, and the second motion state prediction information are obtained. The first reference state information represents the average speed of the target object within a preset distance, and the second reference state information represents the average speed of the target object within a preset time period.

[0077] According to an embodiment of this disclosure, location information collected at the current time can be obtained based on location timing information; a second time is determined based on the current time and a first time interval from the current time; a second time interval is determined by a preset multiple and the first time interval; a third time is determined based on the current time and the second time interval from the current time; a first distance is obtained based on the location information collected at the current time and the location information at the third time; and a first reference state is obtained based on the first distance information and the second time interval.

[0078] The first reference state information can be the velocity information of the first reference state. The preset multiplier can be obtained empirically and is not specifically limited here.

[0079] According to an embodiment of this disclosure, second distance information is obtained based on the current location information and the second location information; second motion state prediction information is obtained based on the second distance information and the first preset duration information.

[0080] The second motion state prediction information can be the predicted velocity information of the second motion state. The first preset duration information can represent the preset duration between the current time and the preset time.

[0081] According to embodiments of this disclosure, a second reference state information can be obtained based on a preset multiple and second motion state prediction information.

[0082] The second reference state information can be the velocity information of the second reference state. The preset multiplier can be obtained empirically and is not specifically limited here.

[0083] In operation S240, the target motion state information is obtained based on the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information.

[0084] According to the embodiments of this disclosure, the first motion state prediction information can be compared with the first reference state information and the second reference state information respectively, and the target motion state information can be determined based on the comparison results.

[0085] For example, if the first motion state prediction information is greater than the first reference state information or less than the second reference state information, the second motion state prediction information is determined to be the target motion state information. If the first motion state prediction information is less than the first reference state information or greater than the second reference state information, the first motion state prediction information is determined to be the target motion state information.

[0086] According to embodiments of this disclosure, after processing multiple frames of images of the target object, the average speed of the target object within a preset distance and the average speed within a preset time period are calculated. The motion state prediction information obtained by the extended Kalman filter algorithm can be corrected based on the average speed of the target object within the preset distance and the average speed within the preset time period. Utilizing the multiple frames of the target object's movement not only overcomes the susceptibility of single-frame movement to noise interference but also reflects the accuracy of the motion state prediction information obtained by the extended Kalman filter algorithm in a timely manner. When the moving target object undergoes a rapid change in speed, the inaccurate motion state prediction information obtained by the extended Kalman filter algorithm can be corrected, accurately predicting the next motion state of the moving target object. This solves the problem that when using the extended Kalman filter to predict the motion state of the sensing results, the extended Kalman filter cannot accurately predict the next motion state of the moving target object when it undergoes a rapid change in speed.

[0087] Figure 3 The flowchart illustrates a method for obtaining first reference state information, second reference state information, and second motion state prediction information based on acquisition time information and location timing information according to an embodiment of the present disclosure.

[0088] like Figure 3 As shown, the method 330 of this embodiment for obtaining first reference state information, second reference state information and second motion state prediction information based on acquisition time information and location timing information includes operations S331 to S333.

[0089] In operation S331, based on the preset distance and preset duration, the location information of the first moment, the location information of the second moment, and the location information of the third moment are determined from the collected time information and location time sequence information. The first moment represents the current moment, the second moment represents a historical moment with an interval of the first duration from the current moment, and the third moment represents a historical moment with an interval of the second duration from the current moment. The second duration is a preset multiple of the first duration.

[0090] According to embodiments of this disclosure, the preset multiplier can be determined based on the sensor frame rate, the maximum number of dropped frames, and practical experience. The preset distance can be determined based on the single-frame interval distance and the preset multiplier. The preset duration can be determined based on the single-frame interval time and the preset multiplier.

[0091] According to embodiments of this disclosure, location information at a first moment can be obtained based on the acquisition time information and location timing information. A second moment is determined based on the first moment and a first duration interval thereof; the location information at the second moment is obtained based on the second moment, the acquisition time information, and the location timing information. A second duration is determined based on the first duration and a preset multiple; a third moment is determined based on the second duration and the first moment; the location information at the third moment is obtained based on the third moment, the acquisition time information, and the location timing information.

[0092] In operation S332, the first reference state information is obtained based on the position information at the first moment, the position information at the third moment, and the second duration.

[0093] According to embodiments of this disclosure, position difference information can be obtained based on the position information at the first time and the position information at the third time; and first reference state information can be obtained based on the position difference information and the second duration.

[0094] According to an embodiment of this disclosure, the first reference state information can represent the speed information calculated by dividing the distance obtained by the single frame interval time by a preset multiple.

[0095] For example, the position information at the first moment can be subtracted from the position information at the third moment to obtain the position difference information; the position difference information can be divided by the second duration to obtain the first reference state information.

[0096] In operation S333, based on the position information at the first moment, the position information at the second moment, the first preset duration information, and the preset multiple, the second reference state information and the second motion state prediction information are obtained.

[0097] According to embodiments of this disclosure, the first preset duration information can characterize the single-frame interval time magnified by a preset factor. Second distance information can be obtained based on the position information at the first moment and the position information at the second moment; second motion state prediction information can be obtained based on the second distance information and the first preset duration information. Dividing the second motion state prediction information by a preset factor yields second reference state information.

[0098] According to an embodiment of this disclosure, the second reference state information can represent a single frame interval time amplified by a preset multiple, and the speed information obtained by dividing the single frame interval distance by the single frame interval time.

[0099] According to the embodiments of this disclosure, by amplifying the duration and distance by a preset factor, the motion state prediction information obtained by the extended Kalman filter algorithm can be corrected based on the average speed of the target object within a preset distance and the average speed within a preset duration, so that more accurate motion state prediction information can be obtained in a timely manner when the target object changes speed rapidly.

[0100] According to embodiments of this disclosure, obtaining first reference state information based on location information at a first time point, location information at a third time point, and a second duration may include:

[0101] Based on the location information at the first moment and the location information at the third moment, the first distance information is obtained; and based on the first distance information and the second duration, the first reference state information is obtained.

[0102] According to embodiments of this disclosure, the difference between the position information at a first time point and the position information at a third time point can be calculated to obtain first distance information. Dividing the first distance information by a second time duration yields first reference state information.

[0103] According to embodiments of this disclosure, the first reference state information utilizes multiple frames to encompass the effects of noise, accurately reflecting the EKF-predicted velocity state. Under normal circumstances, the first reference state information should be much larger than the first motion state prediction information. However, during rapid deceleration, because the EKF-predicted first motion state prediction information is not updated in a timely manner, the first reference state information has a sufficiently fast response change, resulting in the first reference state information being smaller than the first motion state prediction information. In this case, the EKF-predicted velocity is inaccurate, i.e., the predicted velocity is distorted.

[0104] According to embodiments of this disclosure, obtaining second reference state information and second motion state prediction information based on position information at a first time moment, position information at a second time moment, first preset duration information, and a preset multiple may include:

[0105] Based on the position information at the first moment and the position information at the second moment, second distance information is obtained; based on the second distance information and the first preset duration information, second motion state prediction information is obtained; and based on the preset multiple and the second motion state prediction information, second reference state information is obtained.

[0106] According to embodiments of this disclosure, the difference between the position information at a first time point and the position information at a second time point can be calculated to obtain second distance information. Dividing the second distance information by a first preset duration information yields second motion state prediction information. Dividing the second motion state prediction information by a preset multiple yields second reference state information.

[0107] According to embodiments of this disclosure, the second reference state information utilizes multiple frames to encompass the effects of noise, accurately reflecting the EKF-predicted velocity state. Normally, the second reference state information is much smaller than the first motion state prediction information. However, during rapid acceleration, because the EKF-predicted first motion state prediction information is not updated in a timely manner, the second reference state information responds and changes sufficiently quickly. Therefore, the second reference state information may become larger than the first motion state prediction information. In this case, the EKF-predicted velocity is inaccurate, i.e., the predicted velocity is distorted.

[0108] According to embodiments of this disclosure, the motion state detection method for a target object may further include:

[0109] The number of target images is determined based on the image acquisition parameters of the target object; multiple target images are identified from multiple frames based on the number of target images; and the multiple target images are stored in a message queue in the order of acquisition time.

[0110] According to embodiments of this disclosure, image acquisition parameters of a target object can be obtained using an image acquisition device.

[0111] According to embodiments of this disclosure, storing multiple target images sequentially in a message queue according to their acquisition time order facilitates the acquisition of the target object's location and timing information. The message queue, acting as middleware, enables decoupling between different systems in multi-system data interaction scenarios, reducing the complexity of program design for each system. This allows users to focus on their core business without having to repeat work on porting programs to different system software, thus significantly reducing the technical burden.

[0112] Figure 4 The flowchart illustrates a method for obtaining target motion state information based on first reference state information, second reference state information, first motion state prediction information, and second motion state prediction information according to an embodiment of the present disclosure.

[0113] like Figure 4 As shown, the method 440 for obtaining target motion state information based on first reference state information, second reference state information, first motion state prediction information and second motion state prediction information in this embodiment includes operations S441 to S442.

[0114] In operation S441, if it is determined that the first reference state information is less than the first motion state prediction information, or if it is determined that the second reference state information is greater than the first motion state prediction information, the second motion state prediction information is determined to be the target motion state information.

[0115] According to embodiments of this disclosure, during rapid acceleration, the second reference state information may be greater than the first motion state prediction information, while during rapid deceleration, the first reference state information may be less than the first motion state prediction information. By comparing the first motion state prediction information with both the first and second reference state information, when the first reference state information is less than the first motion state prediction information, or the second reference state information is greater than the first motion state prediction information, the EKF's predicted velocity is inaccurate. By determining the second motion state prediction information as the target motion state information, the problem of inaccurate EKF prediction can be corrected in a timely manner.

[0116] In operation S442, if it is determined that the first reference state information is greater than the first motion state prediction information and the second reference state information is less than the first motion state prediction information, the first motion state prediction information is determined to be the target motion state information.

[0117] According to embodiments of this disclosure, under normal conditions of non-rapid acceleration or deceleration, there may be situations where the first reference state information is greater than the first motion state prediction information and where the second reference state information is less than the first motion state prediction information. The first motion state prediction information is compared with both the first and second reference state information. When the first reference state information is greater than the first motion state prediction information, or the second reference state information is less than the first motion state prediction information, the EKF's predicted speed is accurate, and the first motion state prediction information is determined as the target motion state information.

[0118] According to embodiments of this disclosure, the first motion state prediction information is compared with the first reference state information and the second reference state information to determine the accuracy of the motion state prediction by the extended Kalman filter algorithm. This solves the problem that when using the extended Kalman filter to predict the motion state of the sensing results, it cannot accurately predict the next motion state of the moving target when the moving target undergoes a sudden change in speed. When the moving target undergoes a sudden change in speed, the inaccurate motion state prediction information obtained by the extended Kalman filter algorithm can be corrected in a timely manner, accurately predicting the next motion state of the moving target. This mitigates the drawback of the extended Kalman filter's poor performance in highly nonlinear scenarios such as sudden changes in speed of the target object.

[0119] According to embodiments of this disclosure, detecting multiple frames of images to obtain the temporal information of the target object's location may include:

[0120] Detection is performed on multiple frames of images to obtain multiple target detection boxes including the target object; and the positional temporal information of the target object is obtained based on the position coordinates of the upper left corner of the multiple target detection boxes and the acquisition time information of each frame of the image.

[0121] According to embodiments of this disclosure, the temporal information of the target object can characterize the position information of the target object in each frame of the image and the acquisition time information of each frame of the image.

[0122] For example, after an autonomous vehicle detects a target object using sensing devices or algorithms, it acquires multiple frames of images of the target object using an image acquisition device or a laser acquisition device. These multiple frames can be sent to a server via a network. Upon receiving the images, the server performs detection on them, obtaining multiple target detection boxes that include the target object. Based on the top-left corner coordinates of these detection boxes and the acquisition time information of each frame, the server obtains the temporal information of the target object's location. The target object can be a pedestrian, vehicle, or other mobile device.

[0123] According to the embodiments of this disclosure, by obtaining the position and time sequence information of the target object and then predicting the motion state information of the target object, it is beneficial to effectively solve the problem that the EKF prediction effect is not ideal in some highly nonlinear situations.

[0124] According to embodiments of this disclosure, the motion state information of a target object can also be predicted using a deep neural network.

[0125] Figure 5 A flowchart illustrating a motion state detection method for a target object according to another embodiment of the present disclosure is shown schematically.

[0126] like Figure 5 As shown, the motion state detection method 500 for the target object in this embodiment includes operations S501 to S503.

[0127] In operation S501, the queue length of the message queue is determined based on the target object detection algorithm.

[0128] According to embodiments of this disclosure, the target object detection algorithm can be a conventional detection algorithm, and no specific limitation is made in these embodiments. The queue length of the message queue can be determined based on the maximum number of consecutive frame drops for the target object by the target object detection algorithm and the sensor frame rate.

[0129] For example, assuming the maximum number of consecutive frames lost by the target object detection algorithm is N, a number greater than N can be set to ensure that the queue contains two valid sensing results in most cases. However, given real-time requirements, assuming the sensor frame rate is 10 frames per second and the drone needs to make a judgment within 1 second, a queue length less than 10 can be chosen to guarantee real-time performance. For instance, the sensor frame rate could be 10, the maximum number of lost frames could be 4, and the queue length could be set to 5.

[0130] When operating S502, the coordinates of the perceived target object in each frame are stored in the message queue in real time according to the first-in-first-out method.

[0131] According to embodiments of this disclosure, each target object is perceived and identified as having a queue of the aforementioned queue length. The number of target images can be determined based on the image acquisition parameters of the target object; multiple target images are determined from multiple frames of images based on the number of target images; the multiple target images are stored sequentially in a message queue according to the acquisition time order. Detection is performed on the multiple frames of images to obtain multiple target detection boxes including the target object; and the positional time sequence information of the target object is obtained based on the position coordinates of the upper left corner of the multiple target detection boxes and the acquisition time information of each frame of image.

[0132] When operating S503, the message queue corresponding to the target object is updated based on the real-time movement of the target object.

[0133] According to embodiments of this disclosure, an updating device can be used to update the message queue corresponding to the target object in real time based on the real-time movement of the target object.

[0134] In operation S504, the first reference state information v1 is determined based on the values ​​of the head and tail of the message queue and the single frame interval time.

[0135] According to an embodiment of this disclosure, for example, based on the values ​​of the message queue head and tail, two coordinate points can be denoted as A(x1,y1) and B(x2,y2) on the X-axis and Y-axis of the coordinate system, and the distance between the two coordinates can be calculated as shown in the following formula (1), which is the target object movement distance and denoted as s1.

[0136]

[0137] Based on the target object's moving distance and the single-frame interval time, the first reference state information v1 is determined, and the calculation formula can be shown in the following formula (2):

[0138] v1=s1 / t (2)

[0139] Where t represents the single-frame interval time.

[0140] In operation S505, the second motion state prediction information v2 is determined based on the message queue tail and the value of the previous element at the message queue tail, and the single frame interval time.

[0141] According to the embodiments of this disclosure, the distance denoted as s2 is calculated based on the tail of the message queue and the value of the element above the tail of the message queue. The second motion state prediction information v2 can be obtained by dividing s2 by the single frame interval time t. The calculation formula is as follows (3):

[0142] v2=s2 / t (3)

[0143] In operation S506, the second reference state information v3 is determined based on the second motion state prediction information v2 and the message queue length.

[0144] According to embodiments of this disclosure, the second reference state information v3 can be determined based on the value of the second motion state prediction information v2 divided by the queue length minus 1.

[0145] For example, if the queue length is 5 and the value of the queue length minus 1 is 4, then the preset multiple can be 4.

[0146] In operation of S507, the extended Kalman filter algorithm is used to obtain the first motion state prediction information v of the target object.

[0147] According to embodiments of this disclosure, the extended Kalman filter algorithm can be implemented by performing a Taylor expansion of the nonlinear function, omitting higher-order terms and retaining the first-order terms of the expansion, thereby linearizing the nonlinear function. Finally, the first motion state estimate and variance estimate of the target object are approximately calculated using the Kalman filter algorithm.

[0148] In operation S508, is the first reference state information v1 less than the first motion state prediction information v?

[0149] According to embodiments of this disclosure, the magnitudes of the first reference state information v1 and the first motion state prediction information v can be determined. If the first reference state information v1 is less than the first motion state prediction information v, operation S510 is executed; if the first reference state information v1 is not less than the first motion state prediction information v, operation S511 is executed.

[0150] In operation S509, is the second reference state information v3 greater than the first motion state prediction information v?

[0151] According to embodiments of this disclosure, the magnitudes of the second reference state information v3 and the first motion state prediction information v can be determined. If the second reference state information v3 is greater than the first motion state prediction information v, operation S510 is executed; if the second reference state information v3 is not greater than the first motion state prediction information v, operation S511 is executed.

[0152] In operation S510, the second motion state prediction information v2 is determined as the target motion state information of the target object, and the first motion state prediction information v is updated using the second motion state prediction information v2.

[0153] In operation S511, the first motion state prediction information v is determined as the target motion state information of the target object.

[0154] According to embodiments of this disclosure, the first motion state prediction information v is the predicted velocity of EKF. When the moving target object is in a linearly stable state, the first motion state prediction information v is closer to the reality, but when the velocity changes rapidly, the velocity is inaccurate. The second motion state prediction information v2 can represent the velocity of the moving target object at the current moment and the previous moment. When the moving target object is instable in a linearly stable state, the velocity is inaccurate, but when the velocity changes rapidly, the predicted velocity of EKF is closer to the reality.

[0155] According to embodiments of this disclosure, for example, when the queue length is 5, the first reference state information v1 represents the speed calculated by dividing the distance obtained by amplifying the single-frame interval time by 4 times. Under linear stationary conditions, the EKF predicted speed of the moving target object is closer to reality, and v1 is much greater than v. The second reference state information v3 represents the speed obtained by dividing the single-frame interval distance by 4 times the single-frame interval time. Under linear stationary conditions, the EKF predicted speed of the moving target object is closer to reality, and v3 is much less than v.

[0156] According to embodiments of this disclosure, considering that the second motion state prediction information v2 represents the speed of a single frame and is unstable due to perception and noise, the accuracy of the EKF predicted speed cannot be determined by comparing v2 and v. However, v1 and v3 utilize multiple frames to encompass the influence of noise, and through unfair comparison conditions, they can accurately reflect the accuracy of the EKF predicted speed. Normally, v1 is much larger than v. However, during rapid deceleration, because the first motion state prediction information v predicted by the EKF is not updated in a timely manner, v1 has a sufficiently fast response change, resulting in v1 being smaller than v. In this case, the EKF predicted speed is inaccurate, i.e., the predicted speed is distorted, and v2 needs to be used for updating. Normally, v3 is much smaller than v. However, during rapid acceleration, because the first motion state prediction information v predicted by the EKF is not updated in a timely manner, v3 has a sufficiently fast response change, resulting in v3 being larger than v. In this case, the EKF predicted speed is inaccurate, i.e., the predicted speed is distorted, and v2 needs to be used for updating.

[0157] According to embodiments of this disclosure, after processing multiple frames of images of the target object, the average speed of the target object within a preset distance and the average speed within a preset time are calculated. The motion state prediction information obtained by the extended Kalman filter algorithm can be corrected based on the average speed of the target object within the preset distance and the average speed within the preset time. Utilizing the movement of the target object across multiple frames not only overcomes the susceptibility of single-frame movement to noise interference but also reflects the accuracy of the motion state prediction information obtained by the extended Kalman filter algorithm in a timely manner. When the moving target object undergoes a rapid change in speed, the inaccurate motion state prediction information obtained by the extended Kalman filter algorithm can be corrected, accurately predicting the next motion state of the moving target object. This solves the problem that when using the extended Kalman filter to predict the motion state of the sensing results, the extended Kalman filter cannot accurately predict the next motion state of the moving target object when it undergoes a rapid change in speed.

[0158] Based on the above-described method for detecting the motion state of a target object, this disclosure also provides a device for detecting the motion state of a target object. The following will be combined with... Figure 6 The device is described in detail.

[0159] Figure 6 A schematic block diagram of a motion state detection device for a target object according to an embodiment of the present disclosure is shown.

[0160] like Figure 6 As shown, the motion state detection device 600 for the target object in this embodiment includes a detection module 610, a first prediction module 620, a second prediction module 630, and an acquisition module 640.

[0161] The detection module 610 is used to detect the multiple frames of images of the target object received, and obtain the position and timing information of the target object. The position and timing information represents the position information of the target object in each frame of the image and the acquisition time information of each frame of the image. In one embodiment, the detection module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0162] The first prediction module 620 is used to obtain the first motion state prediction information of the target object based on the position and time sequence information using the extended Kalman filter algorithm. In one embodiment, the first prediction module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0163] The second prediction module 630 is used to obtain first reference state information, second reference state information, and second motion state prediction information based on the acquisition time information and location timing information. The first reference state information represents the average velocity of the target object within a preset distance, and the second reference state information represents the average velocity of the target object within a preset time period. In one embodiment, the second prediction module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0164] The obtaining module 640 is used to obtain target motion state information based on the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information. In one embodiment, the obtaining module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0165] According to embodiments of this disclosure, the second prediction module 630 includes a first determining unit, a second determining unit, and a third determining unit.

[0166] The first determining unit is used to determine the location information of the first moment, the location information of the second moment, and the location information of the third moment from the collected time information and location time sequence information according to the preset distance and preset duration. The first moment represents the current moment, the second moment represents a historical moment with an interval of the first duration from the current moment, and the third moment represents a historical moment with an interval of the second duration from the current moment. The second duration is a preset multiple of the first duration.

[0167] The second determining unit is used to obtain the first reference state information based on the position information at the first time, the position information at the third time, and the second duration.

[0168] The third determining unit is used to obtain the second reference state information and the second motion state prediction information based on the position information at the first moment, the position information at the second moment, the first preset duration information, and the preset multiple.

[0169] According to embodiments of this disclosure, the obtaining module 640 includes a fourth determining unit.

[0170] The fourth determining unit is used to determine the second motion state prediction information as the target motion state information when it is determined that the first reference state information is less than the first motion state prediction information, or when it is determined that the second reference state information is greater than the first motion state prediction information.

[0171] According to embodiments of this disclosure, the obtaining module 640 includes a fifth determining unit.

[0172] The fifth determining unit is used to determine the first motion state prediction information as the target motion state information when it is determined that the first reference state information is greater than the first motion state prediction information and the second reference state information is less than the first motion state prediction information.

[0173] According to embodiments of this disclosure, any plurality of modules among the detection module 610, the first prediction module 620, the second prediction module 630, and the acquisition module 640 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the detection module 610, the first prediction module 620, the second prediction module 630, and the acquisition module 640 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the detection module 610, the first prediction module 620, the second prediction module 630, and the acquisition module 640 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0174] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a motion state detection method for a target object according to an embodiment of the present disclosure.

[0175] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0176] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0177] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0178] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0179] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0180] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0181] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0182] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0183] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0184] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0186] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0187] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for detecting a motion state of a target object, comprising: in response to receiving a plurality of images of the target object, detecting the plurality of images to obtain position time sequence information of the target object, the position time sequence information representing position information of the target object in each image and acquisition time information of each image; using an extended Kalman filtering algorithm, obtaining first motion state prediction information of the target object according to the position time sequence information, wherein the first motion state prediction information comprises predicted speed information of a first motion state; obtaining first reference state information, second reference state information and second motion state prediction information according to the position time sequence information, wherein the first reference state information represents an average speed of the target object within a preset distance, the second reference state information represents an average speed of the target object within a preset time length, and the second motion state information comprises predicted speed information of a second motion state; and obtaining target motion state information according to the first reference state information, the second reference state information, the first motion state prediction information and the second motion state prediction information, comprising: in a case where the first reference state information is determined to be less than the first motion state prediction information or the second reference state information is determined to be greater than the first motion state prediction information, determining the second motion state prediction information as the target motion state information.

2. The method of claim 1, wherein, The obtaining of the first reference state information, the second reference state information and the second motion state prediction information according to the position time sequence information comprises: determining, according to the preset distance and the preset time length, position information at a first time, position information at a second time and position information at a third time from the position time sequence information, wherein the first time represents a current time, the second time represents a historical time spaced apart from the current time by a first time length, the third time represents a historical time spaced apart from the current time by a second time length, and the second time length is a preset multiple of the first time length; obtaining the first reference state information according to the position information at the first time, the position information at the third time and the second time length; and obtaining the second reference state information and the second motion state prediction information according to the position information at the first time, the position information at the second time, first preset time length information and the preset multiple.

3. The method of claim 2, wherein, The obtaining of the first reference state information according to the position information at the first time, the position information at the third time and the second time length comprises: obtaining first distance information according to the position information at the first time and the position information at the third time; and obtaining the first reference state information according to the first distance information and the second time length.

4. The method of claim 2, wherein, The obtaining of the second reference state information and the second motion state prediction information according to the position information at the first time, the position information at the second time, first preset time length information and the preset multiple comprises: obtaining second distance information according to the position information at the first time and the position information at the second time; and obtain the second motion state prediction information according to the second distance information and the first preset time length information; and obtain the second reference state information according to the preset multiple and the second motion state prediction information.

5. The method of claim 2, further comprising: determining a target image quantity according to an image acquisition parameter of the target object; determining a plurality of target images from the plurality of images according to the target image quantity; storing the plurality of target images in a message queue in a sequence of acquisition time.

6. The method of claim 1, wherein the obtaining the target motion state information according to the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information further comprises: determining the first motion state prediction information as the target motion state information in a case where it is determined that the first reference state information is greater than the first motion state prediction information and it is determined that the second reference state information is less than the first motion state prediction information.

7. The method of claim 1, wherein, The detecting the plurality of images to obtain the position time sequence information of the target object comprises: detecting the plurality of images to obtain a plurality of target detection boxes including the target object; and obtaining the position time sequence information of the target object according to position coordinates of upper left corners of the plurality of target detection boxes and acquisition time information of each frame of image.

8. A device for detecting a motion state of a target object, comprising: a detection module configured to, in response to receiving a plurality of images of a target object, detect the plurality of images to obtain position time sequence information of the target object, the position time sequence information representing position information of the target object in each frame of image and acquisition time information of each frame of image; a first prediction module configured to, using an extended Kalman filtering algorithm, obtain first motion state prediction information of the target object according to the position time sequence information, wherein the first motion state prediction information comprises predicted speed information of a first motion state; a second prediction module configured to obtain first reference state information, second reference state information, and second motion state prediction information according to the position time sequence information, wherein the first reference state information represents an average speed of the target object within a preset distance, the second reference state information represents an average speed of the target object within a preset time length, and the second motion state prediction information comprises predicted speed information of a second motion state; and an obtaining module configured to obtain target motion state information according to the first reference state information, the second reference state information, the first motion state prediction information, and the second motion state prediction information. The obtaining module comprises: a fourth determination unit configured to, in a case where it is determined that the first reference state information is less than the first motion state prediction information or it is determined that the second reference state information is greater than the first motion state prediction information, determine the second motion state prediction information as the target motion state information.

9. An electronic device, comprising: one or more processors; a memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to carry out the method of any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, carries out the method according to any one of claims 1-7.

Images