A method, device, computer equipment and storage medium for identifying an operation scene

Abstract

The present invention discloses a method, device, electronic device and storage medium for identifying a running scene. The method obtains multi-frame perception data collected from an environment during vehicle driving, uses a detection frame to mark an object moving in the environment in the multi-frame perception data to form an original motion trajectory of the object, associates the detection frame with a motion parameter, and filters out fluctuating detection frames according to the object motion law to obtain a target motion trajectory. The state value of the perception data is calculated from the target motion trajectory according to the motion parameters, and the trend of the object's motion state switching is represented by the change of the state value of the multi-frame perception data. The target time range is selected according to the state value on the time axis of the target motion trajectory to represent the scene where the object's motion state switches. The present invention improves recognition efficiency by automatically identifying the scene where the object's motion state switches from the perception data collected by the vehicle, and filters out fluctuating detection frames during the recognition process to improve the accuracy of scene recognition.

Description

Technical Field

[0001] The present invention relates to the field of scene recognition technology, and in particular to a method, device, computer equipment and related devices for recognizing an operating scene. Background Art

[0002] A driving scenario refers to a user scenario formed by the interaction of a vehicle with various objects in the environment, such as pedestrians, other vehicles, road facilities, and weather, during driving. When developing the vehicle's autonomous driving technology, by acquiring driving scenarios throughout the vehicle's entire use cycle and building a database of driving scenarios, the test and evaluation results of the autonomous driving technology are closer to reality, and the applicability of the autonomous driving technology is improved.

[0003] In the vehicle driving scene, movable objects are one of the important perception objects. Unlike static road facilities, the motion states of movable objects in driving scenes include static and moving. When the motion state of an object switches, how to adjust the vehicle's decision and behavior is the key point studied by autonomous driving technology to ensure vehicle safety and road traffic safety. Based on the data-driven research method, through the input of a large number of scenes of the motion state switching of movable objects, the recognition rate of the motion state switching of movable objects during driving will be improved when the vehicle uses autonomous driving technology, which will help to ensure road traffic safety.

[0004] However, the distribution rate of the above-mentioned motion state switching scenarios in the real world is not high. At present, the methods for identifying and obtaining the motion state switching scenarios of movable objects mainly include manual recognition collection and model prediction analysis. Among them, manual recognition of the motion state switching scenarios of movable objects is usually based on the naked eye judgment of the motion state of movable objects in image data or video data, and the recognition efficiency is low. In addition, due to limited manpower and material resources, the number of scenes collected is limited, and it is difficult to achieve the data-driven effect of research on autonomous driving technology; the motion state switching scenarios of movable objects are classified through the predictive analysis of the model, which is usually manifested as finding the start time and end time of the motion state switching scenarios of movable objects according to the prediction results of the model and custom rules, and determining the motion state switching scenarios of movable objects based on the start time and end time. However, the predictive analysis process of the model is easily affected by the instability of the input data, resulting in reduced accuracy of the predictive analysis results of the model. Summary of the invention

[0005] The present invention provides a method, device, computer equipment and storage medium for identifying a running scene, so as to solve the problem of low recognition efficiency and low accuracy when manually identifying a scene of a moving object switching its motion state.

[0006] According to one aspect of the present invention, a method for identifying an operation scenario is provided, comprising:

[0007] Obtain multi-frame perception data collected by the vehicle on the environment during driving;

[0008] Using detection frames in the plurality of frames of the perception data to mark objects moving in the environment to form original motion trajectories of the objects, the detection frames being associated with motion parameters;

[0009] According to the motion law of the object, the detection frame with fluctuation is filtered out in the original motion trajectory to obtain the target motion trajectory, and the detection frame with fluctuation is used to mark the object with jump;

[0010] From the target motion trajectory, a state value is calculated for each frame of the perception data according to the motion parameters to indicate a trend of switching of the motion state of the object;

[0011] On the time axis of the perception data, a target time range is selected according to the state value to identify a scene in which the motion state of the object switches.

[0012] According to another aspect of the present invention, there is provided a device for identifying an operation scenario, comprising:

[0013] A perception data acquisition module is used to acquire multiple frames of perception data collected by the vehicle on the environment during driving;

[0014] An original motion trajectory acquisition module is used to mark objects moving in the environment using detection frames in multiple frames of the perception data to form an original motion trajectory of the object, wherein the detection frame is associated with motion parameters, and the fluctuating detection frame is used to mark the object that has jumped;

[0015] A target motion trajectory acquisition module, used to filter out the detection frame with fluctuations in the original motion trajectory according to the motion law of the object, so as to obtain the target motion trajectory;

[0016] A state value calculation module, used to calculate a state value for each frame of the perception data from the target motion trajectory according to the motion parameters, so as to indicate a trend of switching of the motion state of the object;

[0017] A scene identification module is used to select a target time range according to the state value on the time axis of the perception data to identify the scene in which the motion state of the object switches.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for identifying the operating scenario described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for identifying an operating scenario described in any embodiment of the present invention when executed.

[0023] The technical solution provided by the present invention obtains multi-frame perception data collected by the vehicle on the environment during driving, and uses detection frames in the multi-frame perception data to identify objects moving in the environment to form the original motion trajectory of the object, wherein the detection frame is associated with the motion parameters of the object, and further obtains the target motion trajectory according to the motion law of the object for the detection frame with fluctuations in the original motion trajectory, thereby improving the accuracy of calculating the state value through the target motion trajectory, and further improving the accuracy of identifying the scene where the motion state of the object switches by selecting the target time range according to the state value on the time axis of the perception data. Compared with the scene where the motion state of a movable object switches manually, the method provided by the present invention automatically identifies the perception data collected by the vehicle, improves the efficiency of scene recognition in the face of a large amount of perception data, and improves the accuracy of scene recognition by screening the original motion trajectory.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flowchart of a method for identifying an operation scenario provided according to Embodiment 1 of the present invention;

[0027] Figure 2is a schematic diagram of the structure of a device for identifying an operation scene provided according to Embodiment 2 of the present invention;

[0028] Figure 3 It is a structural diagram of an electronic device for implementing the method for identifying an operation scenario of an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 A flowchart of a method for identifying an operation scene is provided for the first embodiment of the present invention. This embodiment can be applied to identifying the perception data collected by the vehicle during driving to identify the scene in which the motion state of the object switches. The method can be executed by an operation scene identification device, which can be implemented in the form of hardware and / or software, and can be configured in a computer device. Figure 1 As shown, the method includes:

[0033] S110, obtaining multiple frames of perception data collected by the vehicle on the environment during driving.

[0034] The method provided in this embodiment aims to identify the scene where the motion state of a movable object encountered during vehicle driving switches, that is, the objects mentioned in this embodiment are movable objects encountered during vehicle driving. Based on the movable nature of the object, this embodiment can capture the state of the object in motion, stillness, and switching between motion and stillness. The switching of motion state can be the object switching from a static motion state to a dynamic motion state, or the object switching from a dynamic motion state to a static motion state. The dynamic motion state of the object can also include a faster dynamic state and a slower dynamic state. Further, in this embodiment, the scene where the motion state of the object switches can also be the scene where the object switches between dynamic motion states at different speeds.

[0035] In this embodiment, the scene of identifying the switching of the motion state of an object is based on the perception data collected by the vehicle. The vehicle can collect perception data of the environment around the vehicle through various sensors during driving. When an object appears in the environment around the vehicle, the sensors on the vehicle can collect real and large amounts of perception data related to the object, including image data collected by the camera, or point cloud data collected by the laser radar. It can be known by those skilled in the art that each frame of perception data has a timestamp identifying the time when the perception data was collected, and the timestamp can be used to know the order and adjacent relationship of the collection time of different perception data.

[0036] S120, using detection boxes in multiple frames of perception data to mark objects moving in the environment to form original motion trajectories of the objects, and the detection boxes are associated with motion parameters.

[0037] In this embodiment, for the purpose of identifying the scenario where the motion state of an object switches, after obtaining multiple frames of perception data of the vehicle during driving, the perception data that can represent the characteristics of the object are first screened from the multiple frames of perception data. For example, when the perception data is image data, the perception data that can represent the characteristics of the object is the image data showing the object, and then the original motion trajectory of the object is identified from the multiple frames of perception data containing the object.

[0038] In this embodiment, in order to determine the motion state of the object in the perceived data, it is also necessary to obtain the motion parameters of the object in a specific frame, and determine the motion state of the object based on the motion parameters. For example, when the motion parameter is the speed of the object's movement, the motion state of the object can be determined based on the speed of the object's movement.

[0039] The acquisition of motion parameters is based on the recognition of objects. In this embodiment, if the perception data is image data, objects in the image data can be recognized through a global template method represented by Gavrila, a local template method represented by Broggi, or a classifier-based method or a statistical learning-based method. This embodiment is not limited to the method of recognizing objects in perception data.

[0040] Usually, after an object in the perception data is identified, a detection frame will be used to mark the object moving in the environment. When there are multiple different objects in the perception data, different detection frames will be used to mark different objects. It can be seen that different objects will have different identification codes during the marking process to distinguish different objects and different detection frames. The identification codes of the same object and the detection frames marking the same object in multiple frames of perception data should be the same. Therefore, in this embodiment, the detection frames of the same object in multiple frames of perception data and the motion parameters associated with the detection frames can be integrated according to the identification codes to identify the original motion trajectory of the object.

[0041] Furthermore, after identifying the object, when associating motion parameters with the detection frame of the object, if the motion parameter is the speed of the object's movement, then in this embodiment, the speed of the object's movement in the perception data can be obtained by differentially determining the positions of the detection frames in the perception data of adjacent frames, or in this embodiment, the speed of the object's movement in a specific frame of perception data can be obtained by using a laser radar or the like.

[0042] In the present embodiment, it is taken into consideration that frame skipping may occur in the process of collecting perception data. In the case of frame skipping, when the original motion trajectory of the object is identified based on the perception data with continuous collection time, the original motion trajectory will be discontinuous, which may easily lead to the erroneous identification of scenes in a period of time when the object's motion state has not switched as scenes where the motion state has switched, which is not conducive to the recognition efficiency of the object's motion state switching scenes. Therefore, in order to prevent this situation from occurring, in the present embodiment, before the perception data is marked with a detection frame, the perception data may be screened to see whether there is frame skipping during collection.

[0043] In this embodiment, the specific process of screening frame skipping conditions can be expressed as follows:

[0044] The time of collecting the perception data of two adjacent frames is queried and used as the first collection time and the second collection time respectively. From the above, it can be seen that the collection time of each frame of perception data can be identified by the timestamp of the perception data, and the collection time of the perception data can be obtained by reading the timestamp of the perception data.

[0045] The difference between the first acquisition time and the second acquisition time is calculated as the time difference. The time difference is then compared with the preset time interval. In this embodiment, when no frame skipping occurs, the acquisition of the perception data can be completed according to the preset, fixed time interval. For example, when the fixed time interval is 0.1 seconds, the acquisition time of two adjacent frames of perception data, that is, the difference between the first acquisition time and the second acquisition time should be 0.1 seconds.

[0046] If the time difference is equal to the time interval, it can be determined that the perception data of two adjacent frames are stable, and the perception data of two adjacent frames are retained. That is, when the difference in acquisition time between the perception data of two adjacent frames is equal to the preset time interval, it means that there is no perception data that is missed due to frame skipping between the perception data of two adjacent frames. When the original motion trajectory of the object is obtained by multi-frame perception data without frame skipping, the above-mentioned discontinuity of the original motion trajectory will not occur, and the multi-frame perception data without frame skipping can be retained.

[0047] If the time difference is greater than the time interval, it is determined that there is volatility in the perception data of two adjacent frames, and the perception data is deleted. It can be seen from the above that when the original motion trajectory of the object is obtained through multiple frames of perception data with frame skipping, the above-mentioned original motion trajectory will be discontinuous. Since the number of perception data obtained is usually large, in order to ensure the accuracy of identifying the scene where the motion state of the object switches, in this embodiment, the perception data of two adjacent frames with frame skipping can be deleted, thereby ensuring that the perception data used to obtain the original motion trajectory of the object does not have frame skipping.

[0048] S130, filtering out the detection frames with fluctuations in the original motion trajectory according to the motion law of the object to obtain the target motion trajectory, and the detection frames with fluctuations are used to mark the objects that have jumped.

[0049] In this embodiment, the function of the detection frame is to capture the object. After the object is captured, the detection frame is formed around the object. Therefore, when the object moves, the movement state of the object around the vehicle can be observed by observing the position change of the detection frame. When the object in the perception data is identified by the detection frame, due to the influence of instability when identifying the object, the position of the detection frame in the perception data will jump. For example, the position difference between the detection frames when identifying the object in two adjacent frames of perception data is too large. When such a detection frame is used to identify the movement state of the object and the switching of the movement state, errors will occur, and it is easy to mistakenly identify a scene where the movement state has not switched as a scene where the movement state has switched.

[0050] In this embodiment, the detection frame with fluctuation marks the object whose position has changed. Since the original motion trajectory of the object identified by the fluctuating detection frame is discontinuous, this will lead to the wrong identification of whether the object has switched its motion state. Therefore, in this embodiment, the detection frame with fluctuation can be filtered out according to the motion law of the object to obtain the target motion trajectory. The motion law of the object means that if the motion state of the object changes, this change process should be continuous and smooth without sudden changes. For example, the change of the position of the object identified by the detection frame should be continuous without jumping. The process of filtering out the detection frame with fluctuation in this embodiment can be specifically expressed as follows:

[0051] Query the detection frames used to mark the object in two adjacent frames of perception data respectively as the first target frame and the second target frame, and then calculate the overlap between the first target frame and the second target frame. The overlap between the first target frame and the second target frame can represent the difference between the positions of the first target frame and the second target frame.

[0052] Specifically, in this embodiment, the first area overlapped between the first target frame and the second target frame can be counted, and the second area occupied by the first target frame and the second target frame as a whole can be counted. For example, when the perception data is image data, the second area is the sum of the area occupied by the first target frame in the perception data where the first target frame is located and the area occupied by the second target frame in the perception data where the second target frame is located. Then, the ratio between the first area and the second area is calculated as the overlap between the first target frame and the second target frame.

[0053] After calculating the degree of overlap between the first target frame and the second target frame, the degree of overlap is compared with a preset overlap threshold. The overlap threshold is a pre-set reference value indicating that the position of the detection frame does not jump during the process from the first target frame to the second target frame. The specific value of this reference value can be obtained through experiments.

[0054] If after comparison, the overlap between the first target frame and the second target frame is less than or equal to the overlap threshold, it can be determined that there is stability in the two adjacent frame detection frames, and the two adjacent frame detection frames are retained. Stability means that there is no position jump in the two adjacent frame detection frames.

[0055] If after comparison, the overlap between the first target frame and the second target frame is greater than the overlap threshold, it can be determined that there is fluctuation in the detection frames of the two adjacent frames, and the detection frames of the two adjacent frames are deleted. Fluctuation means that there is a position jump between the detection frames of the two adjacent frames.

[0056] S140. Calculate a state value for each frame of perception data from the target motion trajectory according to motion parameters to indicate a trend of switching of the object's motion state.

[0057] In this embodiment, after excluding the influence of the jump of the collected perception data and the jump of the position of the detection frame marking the object, the target motion trajectory is filtered out from the original motion trajectory, and then the state value of the object in each frame of perception data can be calculated according to the motion parameters associated with the detection frame in the target motion trajectory to indicate the trend of the object's motion state switching, wherein the state value refers to the assignment of the object's motion state, such as static or dynamic, to facilitate the subsequent calibration of the object's motion state and identify the scene where the object's motion state switches according to the state value.

[0058] In this embodiment, the state value of each frame of perception data is calculated based on the motion parameters, which can be specifically expressed as follows: when the motion parameter is mainly speed, the reference speed is used to divide the motion state of the object in each frame of perception data. Specifically, it can be a comparison of the speed and a preset speed threshold. The speed threshold can also be calculated based on multiple experiments to obtain an empirical value, and the speed threshold can be used to divide the motion state of the object.

[0059] That is, if the speed obtained by comparison is greater than the speed threshold, it can be determined that the motion state of the object is dynamic, and if the speed obtained by comparison is less than or equal to the speed threshold, it can be determined that the motion state of the object is static.

[0060] Furthermore, the present embodiment also takes into account the instability in the calculation of motion parameters determined based on the perception data of a single frame. For example, when the motion parameter is speed, the instability in the calculation will lead to a reduction in the accuracy of the speed. Furthermore, when the motion state of the object in the perception data is determined based on the speed, errors may also occur in the judgment of the motion state of the object. Therefore, the present embodiment can also calibrate the speed through a speed smoothing operation to improve the accuracy of the calculated speed.

[0061] Specifically, the speed calibration process can be expressed as:

[0062] Traverse multiple frame speeds, read the current frame speed and the calibrated previous frame speed in turn, that is, traverse the motion parameter-speed associated with the detection frame in the multi-frame perception data, obtain the speed for calibration, that is, the current frame speed and the calibrated speed corresponding to the perception data of the previous frame of the current frame, and then multiply the speed of the current frame by the preset first weight to obtain the first weight adjustment value, multiply the speed of the previous frame after calibration by the preset second weight to obtain the second weight adjustment value, and finally add the first weight adjustment value to the second weight adjustment value as the calibrated speed of the current frame. This is a process of weighted averaging the speed of the current frame by introducing the historical state, that is, using the calibrated previous frame speed to perform exponential moving average on the speed of the current frame to achieve the effect of speed smoothing, avoiding the influence of instability when calculating the speed through single-frame perception data, thereby improving the accuracy of speed calculation. Among them, the second weight is actually the attenuation coefficient of the exponential moving average calculation process, and the sum of the first weight and the second weight is 1.

[0063] In this embodiment, after the state of the object in each frame of perception data is divided, windows can be added in sequence with the motion state of each frame as the center. The purpose of adding windows in this embodiment is to smooth and calibrate the motion state of the center according to the motion state of the object in the perception data within the window, so as to improve the accuracy of the calculation of the state value of the motion state. The window refers to a preset time interval with the collection time of the perception data corresponding to the motion state of the center as the center. The perception data collected within the time interval and the perception data corresponding to the motion state of the center together constitute a window.

[0064] According to the distribution information of all motion states in the window, the state value is calculated for the perception data used to mark the motion state as the center to indicate the tendency of the object's motion state to switch in the window.

[0065] Specifically, it includes: configuring a flag value for each motion state, the flag value is positively correlated with the speed corresponding to the motion state, for example, configuring a flag value of 1 for a dynamic motion state, and configuring a flag value of 0 for a static motion state.

[0066] The average value of the mark values ​​of all motion states in the window is calculated. The average value is the state value of the perception data used to mark the motion state as the center, so as to indicate the trend of the object's motion state switching in the window. This process calculates the motion state of all the perception data in the window to obtain an average value, and uses this average value as the actual state value of the motion state of the center. By moving the center motion state, that is, changing the motion state as the center, the window will slide accordingly, but the length of the window remains unchanged. During the sliding process, the actual state value of the motion state of the object in each frame of perception data is calculated respectively. Finally, according to the actual state value of the motion state of each frame, the trend of the object's motion state switching in the window can be obtained. For example, when the actual state value of the motion state in the window becomes larger and larger, it means that the object's motion state may switch from static to dynamic. If the actual state value of the motion state in the window becomes larger and larger, it means that the object's motion state may switch from static to dynamic.

[0067] S150. On the time axis of the perception data, a target time range is selected according to the state value to identify a scene where the motion state of the object switches.

[0068] In this embodiment, after calculating the average value of the mark values ​​of all motion states in the window, the actual state value of the motion state in the window can be obtained as described above. According to the changes in the actual state values ​​of each motion state in the window, the trend of the object's motion state switching can be known. Specifically, a first extreme value and a second extreme value can be selected. The first extreme value is the critical value of the state value when the object's motion state switches from static to dynamic, and the second extreme value is the critical value of the state value when the object's motion state switches from dynamic to static.

[0069] When the actual state value of the motion state, that is, the average value, falls within the interval formed by the first extreme value and the second extreme value, it means that the perception data corresponding to this motion state can represent the scenario where the motion state of the object switches. Based on this, the corresponding perception data can be screened on the time axis of the collected multi-frame perception data to identify the scenario where the motion state of the object switches. The time axis is composed of the collection time of each frame of perception data.

[0070] Specifically, the process of identifying the scene can be expressed as follows:

[0071] The state value within the interval formed by the first extreme value and the second extreme value is selected as the target state value, that is, the actual state values ​​of all motion states are selected, and if the state value is within the above interval, the state value is used as the target state.

[0072] The perception data pointed to by the target state value is marked on the time axis of the perception data to obtain the target time point, that is, the collection time of the perception data used to obtain the target state value is marked on the time axis, and the collection time is used as the target time point. The target time point is used to identify the scene where the motion state of the object switches.

[0073] In this embodiment, if the target time points are continuous, multiple target time points can be connected into a target time range to identify the scenario where the object's motion state switches. The perception data within the target time range can identify the scenario of the object's motion state switching from dynamic to static, or from static to dynamic.

[0074] Furthermore, after determining the target time range, in order to avoid missing scenes at both ends of the motion state switching process due to calculation errors, the target time range can be extended, including extending a preset first time period at the starting end of the target time range, and / or extending a preset second time period at the ending end of the target time range.

[0075] If at least two target time ranges overlap on the time axis after the target time range is extended, the at least two target time ranges may be merged into a new target time range, and the scene where the motion state of the object switches may be identified according to the new target time range.

[0076] The technical solution provided by the present invention obtains multi-frame perception data collected by the vehicle on the environment during driving, and uses detection frames in the multi-frame perception data to identify objects moving in the environment to form the original motion trajectory of the object, wherein the detection frame is associated with the motion parameters of the object, and further obtains the target motion trajectory according to the motion law of the object for the detection frame with fluctuations in the original motion trajectory, thereby improving the accuracy of calculating the state value through the target motion trajectory, and further improving the accuracy of identifying the scene where the object's motion state switches by selecting the target time range according to the state value on the time axis where the perception data is located. Compared with the scene where the object's motion state switches manually, the method provided by the present invention automatically identifies the perception data collected by the vehicle, improves the efficiency of scene recognition in the face of a large amount of perception data, and improves the accuracy of scene recognition by screening the original motion trajectory.

[0077] Embodiment 2

[0078] Figure 2 A schematic diagram of the structure of an operating scenario recognition device provided in Embodiment 2 of the present invention.

[0079] like Figure 2 As shown, the device comprises:

[0080] The perception data acquisition module 210 is used to acquire multiple frames of perception data collected by the vehicle on the environment during driving;

[0081] An original motion trajectory acquisition module 220, configured to mark objects moving in the environment using detection frames in multiple frames of the perception data to form an original motion trajectory of the object, wherein the detection frames are associated with motion parameters;

[0082] A target motion trajectory acquisition module 230 is used to filter out the detection frame with fluctuations in the original motion trajectory according to the motion law of the object to obtain the target motion trajectory, wherein the detection frame with fluctuations is used to mark the object that has jumped;

[0083] A state value calculation module 240, configured to calculate a state value for each frame of the perception data from the target motion trajectory according to the motion parameters, so as to indicate a trend of switching of the motion state of the object;

[0084] The scene identification module 250 is used to select a target time range according to the state value on the time axis of the perception data to identify the scene in which the motion state of the object switches.

[0085] Optionally, the operating scenario recognition device further includes:

[0086] A collection time query module, used to query the time of collecting the perception data of two adjacent frames, which are used as the first collection time and the second collection time respectively;

[0087] A time difference calculation module, used to calculate the difference between the first acquisition time and the second acquisition time as the time difference;

[0088] A time difference comparison module, used for comparing the time difference with a preset time interval;

[0089] a first stability determination module, configured to determine that the perception data of two adjacent frames are stable if the time difference is less than or equal to the time interval, and retain the perception data of the two adjacent frames;

[0090] The first volatility determination module is used to determine that there is volatility in the perception data of two adjacent frames if the time difference is greater than the time interval, and delete the perception data.

[0091] Optionally, the target motion trajectory acquisition module 230 includes:

[0092] A detection frame marking module, used to query the detection frames used to mark the object in two adjacent frames of the perception data as the first target frame and the second target frame;

[0093] A coincidence calculation module, used for calculating the coincidence between the first target frame and the second target frame;

[0094] A coincidence comparison module, used for comparing the coincidence with a preset coincidence threshold;

[0095] A second stability determination module, configured to determine that the detection frames of two adjacent frames are stable if the overlap is greater than or equal to the overlap threshold, and retain the detection frames of the two adjacent frames;

[0096] The second fluctuation determination module is used to determine that there is fluctuation in the detection frames of two adjacent frames if the overlap is less than the overlap threshold, and delete the detection frames of the two adjacent frames.

[0097] Optionally, the overlap calculation module includes:

[0098] A first area counting module, used for counting a first area overlapped between the first target frame and the second target frame;

[0099] A second area counting module, used for counting a second area occupied by the first target frame and the second target frame as a whole;

[0100] The area ratio calculation module is used to calculate the ratio between the first area and the second area as the overlap between the first target frame and the second target frame.

[0101] Optionally, the state value calculation module 240 includes:

[0102] A motion state classification module, used for classifying the motion state of the object in each frame of the perception data with reference to the speed;

[0103] A window adding module, used for sequentially adding windows with the motion state of each frame as the center;

[0104] The central state value calculation module is used to calculate the state value of the perception data used to mark the motion state as the center according to the distribution information of all the motion states in the window, so as to indicate the trend of the motion state of the object switching in the window.

[0105] Optionally, the motion state division module includes:

[0106] A speed comparison module, used for comparing the speed with a preset speed threshold;

[0107] a dynamic determination module, configured to determine that the motion state of the object is dynamic if the speed is greater than the speed threshold;

[0108] The static determination module is used to determine that the motion state of the object is static if the speed is less than or equal to the speed threshold.

[0109] Optionally, the state value calculation module 240 further includes:

[0110] A speed reading module, used for traversing the speeds of multiple frames, and sequentially reading the speed of the current frame and the speed of the previous frame after calibration;

[0111] A first weight value acquisition module, used for multiplying the speed of the current frame by a preset first weight to obtain a first weight value;

[0112] A second weight value acquisition module, used for multiplying the calibrated speed of the previous frame by a preset second weight to obtain a second weight value;

[0113] A calibration speed acquisition module, used for adding the first adjustment weight value and the second adjustment weight value as the speed after calibration of the current frame;

[0114] The sum of the first weight and the second weight is 1.

[0115] Optionally, the central state value calculation module includes:

[0116] A flag value configuration module, configured to configure a flag value for each motion state, wherein the flag value is positively correlated with the speed corresponding to the motion state;

[0117] An average value calculation module is used to calculate an average value of the mark values ​​of all the motion states in the window, and the average value is used to mark the state value of the perception data of the motion state as the center to indicate the trend of the motion state of the object switching in the window.

[0118] Optionally, the scene identification module 250 includes:

[0119] An extreme value selection module, used for selecting a first extreme value and a second extreme value, wherein the first extreme value is a critical value of the state value when the motion state of the object switches from static to dynamic, and the second extreme value is a critical value of the state value when the motion state of the object switches from dynamic to static;

[0120] a target state value screening module, used for screening the state value located in the interval formed by the first extreme value and the second extreme value as the target state value;

[0121] A target time point acquisition module, used to mark the perception data pointed to by the target state value on the time axis where the perception data is located, and obtain the target time point;

[0122] The target time range acquisition module is used to connect multiple target time points into a target time range if the target time points are continuous, so as to identify the scene where the motion state of the object switches.

[0123] Optionally, the scene identification module 250 further includes:

[0124] A target time range extension module, used to extend a preset first time period at the start end of the target time range, and / or to extend a preset second time period at the end end of the target time range;

[0125] The target time range merging module is used to merge at least two target time ranges into a new target time range if at least two target time ranges overlap.

[0126] The operating scenario recognition device provided in the embodiment of the present invention can execute the operating scenario recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0127] Embodiment 3

[0128] Figure 3 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0129] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0131] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the recognition method of the running scene.

[0132] In some embodiments, the method for identifying the operation scenario can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for identifying the operation scenario described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the method for identifying the operation scenario in any other appropriate manner (for example, by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying an operation scene, characterized in that: include: Obtain multi-frame perception data collected by the vehicle on the environment during driving; Using detection frames in the plurality of frames of the perception data to mark objects moving in the environment to form original motion trajectories of the objects, the detection frames being associated with motion parameters; According to the motion law of the object, the detection frame with fluctuation is filtered out in the original motion trajectory to obtain the target motion trajectory, and the detection frame with fluctuation is used to mark the object that has jumped; From the target motion trajectory, a state value is calculated for each frame of the perception data according to the motion parameters to indicate a trend of switching of the motion state of the object; On the time axis of the perception data, a target time range is selected according to the state value to identify a scene in which the motion state of the object switches; The filtering out the detection frame with fluctuation in the original motion trajectory according to the motion law of the object to obtain the target motion trajectory includes: Respectively querying the detection frames used to mark the object in two adjacent frames of the perception data as the first target frame and the second target frame; Calculating the overlap between the first target frame and the second target frame; comparing the degree of overlap with a preset degree of overlap threshold; If the overlap is less than or equal to the overlap threshold, it is determined that the detection frames of the two adjacent frames are stable, and the detection frames of the two adjacent frames are retained; If the overlap degree is greater than the overlap degree threshold, it is determined that there is fluctuation in the detection frames of the two adjacent frames, and the detection frames of the two adjacent frames are deleted.

2. The method according to claim 1, characterized in that Before filtering out the detection frame with fluctuations in the original motion trajectory according to the motion law of the object to obtain the target motion trajectory, the method further includes: Querying the time of collecting the perception data of two adjacent frames, and using them as the first collection time and the second collection time respectively; Calculate the difference between the first acquisition time and the second acquisition time as a time difference; Comparing the time difference with a preset time interval; If the time difference is equal to the time interval, it is determined that the perception data of the two adjacent frames are stable, and the perception data of the two adjacent frames are retained; If the time difference is greater than the time interval, it is determined that there is fluctuation in the perception data of two adjacent frames, and the perception data is deleted.

3. The method according to claim 1, characterized in that The calculating the overlap between the first target frame and the second target frame includes: Counting a first area of ​​overlap between the first target frame and the second target frame; Counting a second area occupied by the first target frame and the second target frame as a whole; A ratio between the first area and the second area is calculated as a degree of overlap between the first target frame and the second target frame.

4. The method according to any one of claims 1 to 3, characterized in that: The motion parameter includes speed, and the state value of each frame of the perception data is calculated according to the motion parameter from the target motion trajectory to indicate the trend of the motion state of the object switching, including: classifying a motion state of the object in each frame of the perception data with reference to the speed; Add windows in sequence with the motion state of each frame as the center; According to the distribution information of all the motion states within the window, a state value is calculated for the perception data used to mark the motion state as the center to indicate a tendency of the motion state of the object to switch within the window.

5. The method according to claim 4, characterized in that The step of dividing the motion state of the object in each frame of the perception data with reference to the speed includes: comparing the speed with a preset speed threshold; If the speed is greater than the speed threshold, determining that the motion state of the object is dynamic; If the speed is less than or equal to the speed threshold, it is determined that the motion state of the object is static.

6. The method according to claim 4, characterized in that The step of calculating a state value for each frame of the perception data according to the motion parameters from the target motion trajectory to indicate a trend of switching of the motion state of the object further includes: Traversing the speeds of multiple frames, and sequentially reading the speed of the current frame and the speed of the previous frame after calibration; Multiplying the speed of the current frame by a preset first weight to obtain a first weight adjustment value; Multiplying the calibrated speed of the previous frame by a preset second weight to obtain a second weighting value; Adding the first weight adjustment value and the second weight adjustment value as the calibrated speed of the current frame; The sum of the first weight and the second weight is 1.

7. The method according to claim 4, characterized in that The calculating of the state value of the perception data for marking the motion state as the center according to the distribution information of all the motion states in the window to indicate the tendency of the motion state of the object to switch in the window includes: configuring a flag value for each of the motion states, wherein the flag value is positively correlated with the speed corresponding to the motion state; An average value is calculated for the mark values ​​of all the motion states within the window, and the average value is used to mark the state value of the perception data of the motion state as the center to indicate a trend of switching of the motion state of the object within the window.

8. The method according to any one of claims 1-3, 5-7, characterized in that: The step of selecting a target time range according to the state value on the time axis of the perception data to identify a scene in which the motion state of the object switches includes: Selecting a first extreme value and a second extreme value, wherein the first extreme value is a critical value of the state value when the motion state of the object switches from static to dynamic, and the second extreme value is a critical value of the state value when the motion state of the object switches from dynamic to static; Selecting the state value within the interval formed by the first extreme value and the second extreme value as the target state value; Marking the perception data pointed to by the target state value on the time axis where the perception data is located, to obtain a target time point; If the target time points are continuous, multiple target time points are connected to form a target time range to identify a scene where the motion state of the object switches.

9. The method according to claim 8, characterized in that The method further comprises: selecting a target time range according to the state value on the time axis of the perception data to identify a scene in which the motion state of the object switches. Extending a preset first time period at the starting end of the target time range, and / or extending a preset second time period at the ending end of the target time range; If at least two of the target time ranges overlap, the at least two target time ranges are merged into a new target time range.

10. A device for identifying an operation scene, characterized in that: include: A perception data acquisition module is used to acquire multiple frames of perception data collected by the vehicle on the environment during driving; An original motion trajectory acquisition module, used to mark objects moving in the environment using detection frames in multiple frames of the perception data to form an original motion trajectory of the object, wherein the detection frames are associated with motion parameters; A target motion trajectory acquisition module is used to filter out the detection frame with fluctuations in the original motion trajectory according to the motion law of the object to obtain the target motion trajectory, and the detection frame with fluctuations is used to mark the object that has jumped; A state value calculation module, used to calculate a state value for each frame of the perception data from the target motion trajectory according to the motion parameters, so as to indicate a trend of switching of the motion state of the object; A scene identification module, used to select a target time range according to the state value on the time axis of the perception data to identify a scene in which the motion state of the object switches; The target motion trajectory acquisition module includes: A detection frame marking module, used to query the detection frames used to mark the object in two adjacent frames of the perception data as the first target frame and the second target frame; A coincidence calculation module, used for calculating the coincidence between the first target frame and the second target frame; A coincidence comparison module, used for comparing the coincidence with a preset coincidence threshold; A second stability determination module, configured to determine that the detection frames of two adjacent frames are stable if the overlap is greater than or equal to the overlap threshold, and retain the detection frames of the two adjacent frames; The second fluctuation determination module is used to determine that there is fluctuation in the detection frames of two adjacent frames if the overlap is less than the overlap threshold, and delete the detection frames of the two adjacent frames.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for identifying the operating scenario according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for identifying an operating scenario according to any one of claims 1 to 9 when executed.

Images