A method, device, network device and storage medium for determining spatial characteristics
By obtaining images during vehicle loading and unloading, extracting and processing spatial feature maps of multi-frame images, and using convolutional neural network to calculate spatial difference information, the problems of low efficiency and low accuracy of vehicle space feature recognition in the prior art are solved, and the problem of low efficiency and accuracy of vehicle space feature recognition in the prior art is realized. More efficient identification effect.
Patent Information
- Application Number
- CN201910738887.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-08-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-08-12
AI Technical Summary
In the statistics of vehicle loading and unloading volumes in the express delivery and warehousing industries, the existing technology relies on manual measurement efficiency and the single-frame image recognition method is greatly affected by noise and has a low accuracy rate.
By obtaining the images at the beginning and end of the vehicle loading and unloading process, the spatial feature map of the multi-frame image is extracted, and the feature cross-learning is used to calculate spatial differences information to determine the spatial characteristics of the vehicle.
The accuracy of vehicle space feature recognition is improved, and the spatial and temporal dimensional feature information of multi-frame images is used for judgment, which enhances the recognition accuracy.
Smart Images

Figure CN112395903B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of mobile terminal technology, and in particular to a method, device, network equipment and storage medium for determining spatial features. Background Art
[0002] Vehicles are needed for loading and unloading in the express delivery and warehousing industries, and the loading and unloading volume of the vehicles often needs to be counted. Generally, the loading and unloading volume of the vehicle is judged based on the spatial characteristics corresponding to the start of loading and unloading (such as the loading rate of the carriage) and the spatial characteristics corresponding to the end of loading and unloading.
[0003] The existing method is generally to judge the spatial characteristics of vehicles manually, for example, the volume of the cargo in the vehicle is measured one by one to calculate the total volume of the cargo in the compartment, and then the corresponding spatial characteristics of the vehicle are determined based on the volume of the compartment and the total volume of the cargo. However, this method is time-consuming, laborious and inefficient.
[0004] In order to improve the efficiency of identifying vehicle spatial features, a method of using convolutional neural network to identify single-frame images of vehicles is proposed to obtain the spatial features of the vehicle. However, this method is greatly affected by the noise of single-frame images and its recognition accuracy is low. Summary of the invention
[0005] The embodiments of the present application provide a method, apparatus, network device and storage medium for determining spatial features, which can improve the accuracy of identifying spatial features.
[0006] In a first aspect, an embodiment of the present application provides a method for determining a spatial feature, comprising:
[0007] Acquire images at the beginning of loading and unloading, and images at the end of loading and unloading, to obtain an image set;
[0008] Respectively obtain the spatial feature map corresponding to each frame image in the image set;
[0009] Obtaining spatial difference information corresponding to the spatial feature map at different times;
[0010] The spatial characteristics of the vehicle are determined according to the spatial difference information and the spatial characteristic map.
[0011] In some embodiments, the step of acquiring images at the beginning of loading and unloading, and images at the end of loading and unloading, to obtain an image set includes:
[0012] Obtaining a video corresponding to the vehicle loading and unloading process;
[0013] The image when the loading and unloading starts and the image when the loading and unloading ends are extracted from the video to obtain the image set.
[0014] In some implementations, respectively acquiring a spatial feature map corresponding to each frame image in the image set includes:
[0015] Each frame image in the image set is input into the trained two-dimensional convolutional neural network in the spatial feature determination network to perform spatial feature learning, so as to obtain the spatial feature map corresponding to each frame image.
[0016] In some implementations, obtaining spatial difference information corresponding to the spatial feature map at different times includes:
[0017] Performing a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map;
[0018] The superimposed spatial feature map is input into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map.
[0019] In some embodiments, the step of inputting the superimposed spatial feature map into a three-dimensional convolutional neural network in a trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map comprises:
[0020] Respectively obtain the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map;
[0021] Based on the three-dimensional convolutional neural network, the spatial feature maps at different moments in the superimposed spatial feature map are connected according to the moment corresponding to the spatial feature map of each frame to obtain a plurality of connected spatial feature maps;
[0022] Feature cross-learning is performed on the multiple connected spatial feature maps respectively to obtain the spatial difference information.
[0023] In some implementations, determining the spatial feature of the vehicle according to the spatial difference information and the spatial feature map includes:
[0024] Determine the first loss value of the spatial difference information and the spatial feature map according to the cross entropy loss function in the classifier in the trained spatial feature determination network; and determine the second loss value of the spatial difference information and the spatial feature map according to the focal loss function in the classifier;
[0025] Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value;
[0026] A spatial feature of the vehicle is determined according to the target loss value.
[0027] In some implementations, before respectively acquiring the spatial feature maps corresponding to each frame image in the image set, the method further includes:
[0028] Get video samples;
[0029] Extracting image samples of the start of loading and unloading and image samples of the end of loading and unloading from the video samples to obtain an image sample set;
[0030] A preset spatial feature determination network is trained according to the image sample set to obtain the trained spatial feature determination network.
[0031] In some implementations, the training of a preset spatial feature determination network according to the image sample set to obtain the trained spatial feature determination network includes:
[0032] Determine the two-dimensional convolutional neural network and the three-dimensional convolutional neural network in the network based on the preset spatial features, and obtain spatial feature sample graphs of image samples in the image sample set, and spatial sample difference information corresponding to the spatial feature sample graphs at different times;
[0033] Based on the preset spatial feature determination network, a cross entropy loss function and a focal loss function in a classifier are determined, and according to the determined spatial feature sample graph and the spatial sample difference information, a spatial feature prediction value of an image sample in the image sample set is determined;
[0034] Obtaining true values of spatial features of image samples in the image sample set;
[0035] The preset spatial feature determination network is converged according to the spatial feature prediction value and the spatial feature true value to obtain the trained spatial feature determination network.
[0036] In some embodiments, after extracting the image samples of the start of loading and unloading and the image samples of the end of loading and unloading from the video samples to obtain the image sample set, the method further includes:
[0037] Performing augmentation processing on the image samples in the image sample set to obtain an augmented image sample set;
[0038] The step of training a preset spatial feature determination network according to the image sample set to obtain the trained spatial feature determination network includes:
[0039] The preset spatial feature determination network is trained according to the augmented image sample set to obtain the trained spatial feature determination network.
[0040] In some implementations, the augmenting the image samples in the image sample set includes:
[0041] Performing compression processing on the image samples in the image sample set according to different compression samples; or,
[0042] Performing random cropping processing on the image samples in the image sample set according to a preset cropping area; or,
[0043] Randomly adjust the brightness, contrast, chroma and saturation of the image samples in the image sample set; or,
[0044] The order of the images corresponding to the start and end of loading and unloading of the image samples in the image sample set is randomly swapped.
[0045] In some embodiments, after acquiring the image when loading and unloading of the vehicle starts and the image when loading and unloading of the vehicle ends to obtain the image set, the method further includes:
[0046] Compressing each frame of the image in the image set to a preset size to obtain a compressed image set;
[0047] Performing cropping processing on each frame image in the compressed image set according to a preset cropping area to obtain a cropped image set;
[0048] At this time, the spatial feature maps corresponding to each frame image in the image set are obtained respectively, including:
[0049] The spatial feature maps corresponding to each frame image in the cropped image set are respectively obtained.
[0050] In a second aspect, an embodiment of the present application further provides a device for determining a spatial feature, including:
[0051] The first acquisition unit is used to acquire images when loading and unloading of the vehicle starts, and images when loading and unloading of the vehicle ends, to obtain an image set;
[0052] A second acquisition unit is used to respectively acquire a spatial feature map corresponding to each frame image in the image set;
[0053] A third acquisition unit is used to acquire spatial difference information corresponding to the spatial feature map at different times;
[0054] A determination unit is used to determine the spatial characteristics of the vehicle based on the spatial difference information and the spatial characteristic map.
[0055] In some implementations, the first acquisition unit is specifically configured to:
[0056] Obtaining a video corresponding to the vehicle loading and unloading process;
[0057] The image when the loading and unloading starts and the image when the loading and unloading ends are extracted from the video to obtain the image set.
[0058] In some implementations, the second acquiring unit is specifically configured to:
[0059] Each frame image in the image set is input into the trained two-dimensional convolutional neural network in the spatial feature determination network to perform spatial feature learning, so as to obtain the spatial feature map corresponding to each frame image.
[0060] In some implementations, the third acquisition unit is specifically configured to:
[0061] Performing a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map;
[0062] Inputting the superimposed spatial feature map into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning, so as to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map;
[0063] In some implementations, the third acquisition unit is further specifically configured to:
[0064] Respectively obtain the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map;
[0065] Based on the three-dimensional convolutional neural network, the spatial feature maps at different moments in the superimposed spatial feature map are connected according to the moment corresponding to the spatial feature map of each frame to obtain a plurality of connected spatial feature maps;
[0066] Feature cross-learning is performed on the multiple connected spatial feature maps respectively to obtain the spatial difference information.
[0067] In some implementations, the determining unit is specifically configured to:
[0068] Determine the first loss value of the spatial difference information and the spatial feature map according to the cross entropy loss function in the classifier in the trained spatial feature determination network; and determine the second loss value of the spatial difference information and the spatial feature map according to the focal loss function in the classifier;
[0069] Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value;
[0070] A spatial feature of the vehicle is determined according to the target loss value.
[0071] In some embodiments, the device further comprises:
[0072] A fourth acquisition unit, used to acquire a video sample;
[0073] An extraction unit, used to extract image samples of the start of loading and unloading, and image samples of the end of loading and unloading from the video samples, to obtain an image sample set;
[0074] A training unit is used to train a preset spatial feature determination network according to the image sample set to obtain the trained spatial feature determination network.
[0075] In some embodiments, the training unit is specifically used to:
[0076] Determine the two-dimensional convolutional neural network and the three-dimensional convolutional neural network in the network based on the preset spatial features, and obtain spatial feature sample graphs of image samples in the image sample set, and spatial sample difference information corresponding to the spatial feature sample graphs at different times;
[0077] Based on the preset spatial feature determination network, a cross entropy loss function and a focal loss function in a classifier are determined, and according to the determined spatial feature sample graph and the spatial sample difference information, a spatial feature prediction value of an image sample in the image sample set is determined;
[0078] Obtaining true values of spatial features of image samples in the image sample set;
[0079] The preset spatial feature determination network is converged according to the spatial feature prediction value and the spatial feature true value to obtain the trained spatial feature determination network.
[0080] In some embodiments, the device further comprises:
[0081] An augmentation unit, used for performing augmentation processing on the image samples in the image sample set to obtain an augmented image sample set;
[0082] At this time, the training unit is specifically used for:
[0083] The preset spatial feature determination network is trained according to the augmented image sample set to obtain the trained spatial feature determination network.
[0084] In some embodiments, the augmentation unit is specifically used for:
[0085] Performing compression processing on the image samples in the image sample set according to different compression samples; or,
[0086] Performing random cropping processing on the image samples in the image sample set according to a preset cropping area; or,
[0087] Randomly adjust the brightness, contrast, chroma and saturation of the image samples in the image sample set; or,
[0088] The order of the images corresponding to the start and end of loading and unloading of the image samples in the image sample set is randomly swapped.
[0089] In some embodiments, the device further comprises:
[0090] A compression unit, used for compressing each frame image in the image set to a preset size to obtain a compressed image set;
[0091] A cropping unit, used for cropping each frame image in the compressed image set according to a preset cropping area to obtain a cropped image set;
[0092] At this time, the second acquisition unit is specifically used for:
[0093] The spatial feature maps corresponding to each frame image in the cropped image set are respectively obtained.
[0094] In a third aspect, an embodiment of the present application further provides a network device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes the steps in any one of the spatial feature determination methods provided in the embodiments of the present application.
[0095] In a fourth aspect, an embodiment of the present application further provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for a processor to load to execute the steps in any one of the spatial feature determination methods provided in the embodiments of the present application.
[0096] In the embodiment of the present application, the spatial feature determination device first obtains images when loading and unloading of the vehicle starts, and images when loading and unloading ends, to obtain an image set; then obtains the spatial feature map corresponding to each frame image in the image set; then obtains the spatial difference information corresponding to the spatial feature map at different times; finally, the spatial feature of the vehicle is determined based on the spatial difference information and the spatial feature map. This solution extracts the spatial feature maps of multiple frames of images and the spatial difference information of the spatial feature maps of each frame at different times, and uses the feature information in the spatial and temporal dimensions of multiple frames of images to determine the spatial features, which can improve the accuracy of identifying spatial features. BRIEF DESCRIPTION OF THE DRAWINGS
[0097] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0098] Figure 1 This is a schematic diagram of an application scenario of the method for determining spatial features provided in an embodiment of the present application;
[0099] Figure 2 It is a flowchart of a method for determining spatial features provided in an embodiment of the present application;
[0100] Figure 3 It is a flowchart of a network training process for determining spatial features provided in an embodiment of the present application;
[0101] Figure 4 is another flowchart of the method for determining spatial features provided in an embodiment of the present application;
[0102] Figure 5 is a schematic diagram of a device for determining spatial features provided in an embodiment of the present application;
[0103] Figure 6 is another schematic diagram of a device for determining spatial features provided in an embodiment of the present application;
[0104] Figure 7 It is a structural diagram of the network device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0105] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0106] In the following description, the specific embodiments of the present application will be described with reference to the steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and the computer execution referred to herein includes the operation of a computer processing unit by an electronic signal representing data in a structured form. This operation converts the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure maintained by the data is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.
[0107] The principles of the present application may be operated using many other general or specific purpose computing, communication environments or configurations. Examples of well-known computing systems, environments and configurations suitable for use with the present application may include, but are not limited to, handheld phones, personal computers, servers, multiprocessor systems, microcomputer-based systems, mainframe computers, and distributed computing environments, including any of the above systems or devices.
[0108] The terms "first", "second", and "third" in this application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "include", "have", and any variations thereof are intended to cover non-exclusive inclusions.
[0109] The embodiments of the present application provide a method, apparatus, network device and storage medium for determining spatial features, wherein the apparatus for determining spatial features can be integrated in a network device, which can be a server, a terminal or other device.
[0110] In some embodiments, in order to improve the accuracy of identifying spatial features, a computer model may be used to implement the spatial feature determination method provided in the embodiments of the present application. For example, in some embodiments, a spatial feature determination network may be used to identify spatial features.
[0111] See also Figure 1 , Figure 1This is a schematic diagram of an application scenario of a method for determining spatial features provided by an embodiment of the present application. In the schematic diagram, the method for determining spatial features in the present application is implemented by a spatial feature determination network, and the spatial feature determination network is composed of features, networks, and classifiers, wherein the feature extraction network is composed of a two-dimensional (2D) convolutional neural network and a three-dimensional (3D) convolutional neural network. In this application scenario, 2 frames (the specific number of frames is not limited in this application) of images when loading and unloading are started, and 2 frames (the specific number of frames is not limited in this application) of images when loading and unloading are ended, a total of 4 frames of images, and then the 4 frames of images are respectively input into 4 weight-shared 2D convolutional neural networks to obtain spatial feature maps corresponding to the 4 frames of images, and then the obtained spatial feature maps are superimposed, and then the superimposed spatial feature maps are input into the 3D convolutional neural network to obtain the spatial difference information corresponding to the spatial feature map at different times, and then the spatial difference information and the spatial feature map are input into the classifier to obtain the recognition result, that is, the spatial feature of the vehicle.
[0112] The spatial characteristics of the vehicle are the relationship between the volume occupied by the cargo in the vehicle compartment and the total volume of the compartment, for example, the loading rate or remaining space rate of the vehicle compartment.
[0113] See also Figure 2 , Figure 2 : is a flowchart of a method for determining a spatial feature provided by an embodiment of the present application. The execution subject of the method for determining a spatial feature may be a device for determining a spatial feature provided by an embodiment of the present application. The method for determining a spatial feature may include:
[0114] 201. Acquire images when loading and unloading of a vehicle starts, and images when loading and unloading of the vehicle ends, to obtain an image set.
[0115] In some embodiments, the process of obtaining images of the start and end of loading and unloading of a vehicle to obtain an image set includes: obtaining a video corresponding to the vehicle loading and unloading process; then extracting images of the start and end of loading and unloading from the video to obtain an image set, wherein the images in the image set also include the time corresponding to the image. The video is a video corresponding to the loading and unloading port of the vehicle compartment.
[0116] The video includes the process of the vehicle starting to load and unload and ending to load and unload. In some embodiments, a certain time at the beginning of the video can be determined as the process of the vehicle starting to load and unload, and a certain time at the end of the video can be determined as the process of the vehicle ending to load and unload.
[0117] In some embodiments, images of when loading and unloading starts at 2 frames, and images of when loading and unloading ends at 2 frames can be extracted from the video, wherein the specific number of frames extracted from the video is not limited, that is, in this embodiment, images of when loading and unloading starts at 1 frame, and images of when loading and unloading ends at 1 frame can be extracted from the video, or images of when loading and unloading starts at 3 frames, and images of when loading and unloading ends at 3 frames, etc. can be extracted.
[0118] In some embodiments, in order to improve the calculation speed, after obtaining the images when the loading and unloading process of the vehicle starts and the images when the loading and unloading process ends to obtain the image set, it is also necessary to compress each frame image in the image set to a preset size, for example, compressing the original 1280×960 to 256×256 size to obtain an image set of compressed images;
[0119] In addition, each frame image in the image set of the compressed image may be cropped according to a preset cropping area to obtain a cropped image set, for example, by cropping the 224×224 center of the image so that the proportion of the image occupied by the vehicle becomes higher.
[0120] 202. Obtain a spatial feature map corresponding to each frame image in the image set respectively.
[0121] In some embodiments, specifically, after the trained spatial feature determination network is loaded, each frame image in the image set can be input into multiple 2D convolutional neural networks in the trained spatial feature determination network for spatial feature learning to obtain a spatial feature map corresponding to each frame image, wherein the weights of the multiple 2D convolutional neural networks are shared.
[0122] Among them, in some embodiments, when each frame image in the image set is compressed to a preset size to obtain a compressed image set; and each frame image in the compressed image set is cropped according to a preset cropping area to obtain a cropped image set, at this time, the spatial feature map corresponding to each frame image in the image set is obtained respectively, including: respectively obtaining the spatial feature map corresponding to each frame image in the cropped image set.
[0123] 203. Obtain spatial difference information corresponding to the spatial feature map at different times.
[0124] Specifically, in some embodiments, the spatial difference information corresponding to the spatial feature map at different times is obtained, including: superimposing the spatial feature maps corresponding to each frame image to obtain a superimposed spatial feature map; then inputting the superimposed spatial feature map into the trained 3D convolutional neural network in the spatial feature determination network to perform feature cross-learning to obtain the spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map, which can speed up the feature extraction efficiency.
[0125] Among them, in some embodiments, the superimposed spatial feature map is input into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map, which can specifically include: respectively obtaining the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map; based on the three-dimensional convolutional neural network, connecting the spatial feature maps at different times in the superimposed spatial feature map according to the time corresponding to each frame of the spatial feature map to obtain multiple connected spatial feature maps; and performing feature cross-learning on the multiple connected spatial feature maps to obtain spatial difference information.
[0126] That is, this embodiment can obtain the moments corresponding to two different spatial feature maps, and then generate the spatial difference information corresponding to the different moments according to the different moments and the two different spatial feature maps.
[0127] The spatial difference information includes spatial difference information between two different spatial feature maps, for example, the difference in loading rates between two spatial feature maps at different moments.
[0128] 204. Determine the spatial characteristics of the vehicle according to the spatial difference information and the spatial characteristic map.
[0129] Specifically, in some embodiments, the extracted spatial difference information and spatial feature map can be passed to the cross-entropy loss function (cross-entropy loss) in the classifier in the spatial feature determination network to determine the first loss value of the spatial difference information and the spatial feature map; and the second loss value of the spatial difference information and the spatial feature map is determined according to the focal loss function (focal loss) in the classifier; then the target loss value is determined according to the first loss value, the second loss value, the weight corresponding to the first loss value, and the weight corresponding to the second loss value; finally, the spatial characteristics of the vehicle are determined according to the target loss value, for example, the category with the largest predicted probability in the target loss value is determined as the spatial characteristic of the vehicle.
[0130] More specifically, in some embodiments, the difference in loading rates between two spatial feature maps can be determined based on spatial difference information, and then the initial loading rates corresponding to the two spatial feature maps can be determined respectively based on the two spatial feature maps. Finally, the target loading rates corresponding to the two spatial feature maps can be determined based on the difference in loading rates corresponding to the two spatial feature maps and the initial loading rates corresponding to the two spatial feature maps.
[0131] Among them, in some embodiments, when the spatial characteristic of the vehicle is the loading rate of the vehicle, the final spatial characteristic of the vehicle includes the loading rate corresponding to the start of loading and unloading of the vehicle and the loading rate corresponding to the end of loading and unloading. Among them, in some embodiments, when the accuracy of the loading rate is 10%, that is, the loading rate is labeled with 11 situations of 0%, 10%, 20%...100%, at this time, the loading rate categories corresponding to the start of loading and unloading and the end of loading and unloading can be: 11×11=121 categories (each category includes the loading rate corresponding to the start of loading and unloading and the loading rate corresponding to the end of loading and unloading, a total of two loading rates).
[0132] The accuracy of the loading rate may also be other values, such as 1%, 5%, etc. The specific accuracy to be used depends on the actual situation.
[0133] The embodiment of the present application determines a target category from these 121 categories, and the two loading rates corresponding to the category are the loading rates corresponding to the start and end of loading and unloading, respectively.
[0134] In the embodiment of the present application, the spatial feature determination device first obtains images when the vehicle loading and unloading process starts, and images when the loading and unloading ends, to obtain an image set; then obtains the spatial feature map corresponding to each frame image in the image set; then obtains the spatial difference information corresponding to the spatial feature map at different times; finally, according to the spatial difference information and the spatial feature map, the spatial feature of the vehicle is determined. This solution extracts the spatial feature map of multiple frames of images and the spatial difference information of the spatial feature map of each frame at different times, and uses the feature information in the spatial and temporal dimensions of multiple frames of images to judge the spatial features, which can improve the accuracy of identifying spatial features.
[0135] The method for determining the spatial features described in the above embodiment will be further described in detail below.
[0136] The method for determining spatial features in the embodiment of the present application includes two parts: network training and network application:
[0137] 1. Network training:
[0138] In the embodiment of the present application, before applying the spatial feature determination network, it is first necessary to train the preset spatial feature determination network to obtain the trained spatial feature determination network, wherein the preset spatial feature determination network is composed of a feature extraction network and a classifier, wherein the feature extraction network is composed of a 2D convolutional neural network and a 3D convolutional neural network, see Figure 3 In the embodiment of the present application, the training process of the spatial feature determination network includes the following steps:
[0139] 301. Obtain a video sample.
[0140] Among them, the video sample in the embodiment of the present application is a video corresponding to the vehicle loading and unloading process with a known loading rate.
[0141] The embodiment of the present application can obtain multiple video samples.
[0142] 302. Extract image samples of the start of loading and unloading and image samples of the end of loading and unloading from the video sample to obtain an image sample set.
[0143] In some embodiments, the video sample is first analyzed into frames of images, and then image samples of the start of loading and unloading and image samples of the end of loading and unloading are selected from the frames of images. For example, 2 frames of images of the start of loading and unloading and 2 frames of images of the end of loading and unloading are selected, and a total of 4 frames of images are selected as an image sample set.
[0144] In some embodiments, after acquiring the image sample set, the image samples in the image sample set need to be marked. Specifically, each frame of the image sample can be marked with a corresponding label. When the accuracy of the loading rate is 10%, the label can be one of 0, 1, 2...10, representing 0%-100% respectively, wherein the label reflects the actual loading rate of the image sample. Since an image sample set includes the loading rate at the beginning of loading and unloading and the loading rate at the end of loading and unloading, there may be 11×11=121 categories in an image sample set.
[0145] 303. Train a preset spatial feature determination network according to the image sample set to obtain a trained spatial feature determination network.
[0146] In some embodiments, in order to prevent the network from overfitting, a data augmentation method needs to be added to enrich the samples, wherein the augmentation method may include the following methods:
[0147] (1) The image samples in the image sample set are compressed according to different compression samples (resize functions) so that the network is robust to different resize modes.
[0148] (2) Randomly cropping the image samples in the image sample set according to a preset cropping area.
[0149] (3) Randomly adjust the brightness, contrast, hue and saturation of the image samples in the image sample set.
[0150] (4) Randomly swapping the order of the images corresponding to the start and end of loading and unloading in the image sample set.
[0151] In this embodiment, the augmentation method described in any one or more of (1)-(4) may be executed, and the execution order is not limited at this time.
[0152] In addition, the augmentation method in the present application may also include:
[0153] Generate a random number, and flip the cropped sample image according to the random number.
[0154] After the image samples in the image sample set are augmented, the preset spatial feature determination network will be trained according to the augmented image sample set to obtain the trained spatial feature determination network.
[0155] This can effectively increase the network's demand for data volume and maintain good performance even with relatively few sample trainings.
[0156] The training of the preset spatial feature determination network according to the image sample set (the augmented image sample set) may specifically include:
[0157] The image samples in the image sample set are passed into the 2D convolutional neural network and the 3D convolutional neural network in the feature extraction network, and then the features extracted from the feature extraction network (the spatial feature sample map of the image samples in the image sample set, and the spatial sample difference information corresponding to the spatial feature sample map at different times) are passed to the classifier by the joint loss composed of cross-entropy loss and focal loss to obtain the spatial feature prediction value of the image samples in the image sample set (i.e., the loss value for various labels). After obtaining the spatial feature prediction value, back propagation is performed based on the spatial feature prediction value and the true value of the spatial features of the image samples in the image sample set, and the network parameters are trained using the gradient descent algorithm; when the difference between the spatial feature prediction value and the true value of the spatial feature is less than 0.1 (the specific numerical value is not limited), it is determined that the preset spatial feature determination network converges, the model is fitted, the training is completed, and the trained spatial feature determination network is obtained.
[0158] 2. Network Application:
[0159] Please refer to Figure 4 , Figure 4 Another flowchart of the method for determining spatial features provided in an embodiment of the present application. The method for determining spatial features can be applied to a network device having a trained spatial feature determination network built therein. In this embodiment, the spatial feature is described by taking the loading rate as an example. Figure 4 As shown, the process of the method for determining the spatial feature can be as follows:
[0160] 401. Obtain a video corresponding to the vehicle loading and unloading process.
[0161] Among them, the video is the video corresponding to the loading and unloading port of the vehicle.
[0162] In some embodiments, in addition to capturing the vehicle's compartment, the video may also capture information such as the vehicle's doors and license plates.
[0163] 402. Extract the image when the loading and unloading process starts and the image when the loading and unloading process ends from the video to obtain the image set.
[0164] In some embodiments, images of when loading and unloading starts at 2 frames, and images of when loading and unloading ends at 2 frames can be extracted from the video, wherein the specific number of frames extracted from the video is not limited, that is, in this embodiment, images of when loading and unloading starts at 1 frame, and images of when loading and unloading ends at 1 frame can be extracted from the video, or images of when loading and unloading starts at 3 frames, and images of when loading and unloading ends at 3 frames, etc. can be extracted.
[0165] In some embodiments, in order to improve the calculation speed, after obtaining the images when the loading and unloading process of the vehicle starts and the images when the loading and unloading process ends to obtain the image set, it is also necessary to compress each frame image in the image set to a preset size, for example, compressing the original 1280×960 to 256×256 size to obtain an image set of compressed images;
[0166] In addition, each frame image in the image set of the compressed image may be cropped according to a preset cropping area to obtain a cropped image set, for example, by cropping the 224×224 center of the image so that the proportion of the image occupied by the vehicle becomes higher.
[0167] 403. Input each frame image in the image set into the trained 2D convolutional neural network in the spatial feature determination network to perform spatial feature learning, and obtain the spatial feature map corresponding to each frame image.
[0168] After the trained spatial feature determination network is loaded, each frame image in the image set can be input into multiple 2D convolutional neural networks in the trained spatial feature determination network for spatial feature learning to obtain a spatial feature map corresponding to each frame image, wherein the weights of the multiple 2D convolutional neural networks are shared.
[0169] Among them, in some embodiments, when each frame image in the image set is compressed to a preset size to obtain a compressed image set; and each frame image in the compressed image set is cropped according to a preset cropping area to obtain a cropped image set, at this time, the spatial feature map corresponding to each frame image in the image set is obtained respectively, including: respectively obtaining the spatial feature map corresponding to each frame image in the cropped image set.
[0170] 404. Perform a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map.
[0171] Before inputting the spatial feature map into the 3D convolutional neural network, it is necessary to perform superposition processing on the spatial feature maps corresponding to each frame image, that is, stacking the spatial feature maps obtained by each 2D convolutional neural network to obtain a superimposed spatial feature map.
[0172] 405. Input the superimposed spatial feature map into the 3D convolutional neural network in the trained spatial feature determination network to perform feature cross-learning, so as to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map.
[0173] Among them, in some embodiments, the superimposed spatial feature map is input into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map, which can specifically include: respectively obtaining the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map; then based on the three-dimensional convolutional neural network, the spatial feature maps at different times in the superimposed spatial feature map are connected according to the time corresponding to each frame of the spatial feature map to obtain multiple connected spatial feature maps; and then feature cross-learning is performed on the multiple connected spatial feature maps to obtain the above-mentioned spatial difference information.
[0174] That is, this embodiment can obtain the moments corresponding to two different spatial feature maps, and then the 3D convolutional neural network generates the spatial difference information corresponding to the different moments according to the different moments and the two different spatial feature maps.
[0175] The spatial difference information includes the difference in loading rates between two different spatial feature maps.
[0176] 406. Determine a first loss value of the spatial difference information and the spatial feature map according to the cross-entropy loss in the classifier in the network based on the trained spatial features; and determine a second loss value of the spatial difference information and the spatial feature map according to the focal loss in the classifier.
[0177] Among them, the combination of cross-entropy loss and focal loss can encourage the network to mine difficult-to-learn samples and improve the recognition accuracy of the algorithm.
[0178] 407. Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value.
[0179] Specifically, the embodiment of the present application pre-assigns a weight to the cross-entropy loss (i.e., the weight corresponding to the first loss value), and pre-assigns a weight to the focal loss (i.e., the weight corresponding to the second loss value). In some embodiments, the weight corresponding to the first loss value may be 0.5, and the weight corresponding to the second loss value may be 0.5.
[0180] In some embodiments, the target loss value = first loss value × weight corresponding to the first loss value + second loss value × weight corresponding to the second loss value.
[0181] 408. Determine, according to the target loss value, a loading rate corresponding to when the vehicle starts loading and unloading and a loading rate corresponding to when the vehicle ends loading and unloading.
[0182] Specifically, the category (label) with the largest predicted probability in the target loss value can be determined as the corresponding category of the vehicle, which includes the loading rate corresponding to when the vehicle starts loading and unloading and the loading rate corresponding to when the vehicle ends loading and unloading.
[0183] In the embodiment of the present application, the spatial feature determination device first obtains images when loading and unloading of the vehicle starts, and images when loading and unloading ends, to obtain an image set; then obtains the spatial feature map corresponding to each frame image in the image set; then obtains the spatial difference information corresponding to the spatial feature map at different times; finally, the spatial feature of the vehicle is determined based on the spatial difference information and the spatial feature map. This solution extracts the spatial feature maps of multiple frames of images and the spatial difference information of the spatial feature maps of each frame at different times, and uses the feature information in the spatial and temporal dimensions of multiple frames of images to determine the spatial features, which can improve the accuracy of the vehicle loading rate.
[0184] That is, the present application uses two types of convolutional neural networks (2D convolutional neural networks and 3D convolutional neural networks) to extract spatial and temporal feature information respectively, so as to obtain features of different dimensions and realize accurate identification of different truck loading rates; specifically, the present application uses 2D convolutional neural networks to extract spatial information and features in a single frame image; and uses 3D convolutional neural networks to extract temporal dimension information between multiple frame images, learns their temporal feature changes, and integrates the information between frames, which can improve the accuracy of vehicle loading rates.
[0185] In order to facilitate better implementation of the method for determining spatial features provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the method for determining spatial features. The meanings of the terms are the same as those in the method for determining spatial features, and the specific implementation details can refer to the description in the method embodiment.
[0186] See also Figure 5 , Figure 5 A schematic diagram of the structure of a device for determining spatial features provided in an embodiment of the present application, wherein the device 500 for determining spatial features may include a first acquisition unit 501, a second acquisition unit 502, a third acquisition unit 503, and a determination unit 504, etc., wherein:
[0187] The first acquisition unit 501 is used to acquire images when loading and unloading of a vehicle starts, and images when loading and unloading of a vehicle ends, to obtain an image set;
[0188] A second acquisition unit 502 is used to respectively acquire a spatial feature map corresponding to each frame image in the image set;
[0189] The third acquisition unit 503 is used to acquire the spatial difference information corresponding to the spatial feature map at different times;
[0190] The determination unit 504 is used to determine the spatial characteristics of the vehicle according to the spatial difference information and the spatial characteristic map.
[0191] In some implementations, the first acquiring unit 501 is specifically configured to:
[0192] Obtaining a video corresponding to the vehicle loading and unloading process;
[0193] The image when the loading and unloading starts and the image when the loading and unloading ends are extracted from the video to obtain the image set.
[0194] In some implementations, the second acquiring unit 502 is specifically configured to:
[0195] Each frame image in the image set is input into the trained two-dimensional convolutional neural network in the spatial feature determination network to perform spatial feature learning, so as to obtain the spatial feature map corresponding to each frame image.
[0196] In some implementations, the third obtaining unit 503 is specifically configured to:
[0197] Performing a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map;
[0198] Inputting the superimposed spatial feature map into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning, so as to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map;
[0199] In some implementations, the third obtaining unit 503 is further specifically configured to:
[0200] Respectively obtain the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map;
[0201] Based on the three-dimensional convolutional neural network, the spatial feature maps at different moments in the superimposed spatial feature map are connected according to the moment corresponding to the spatial feature map of each frame to obtain a plurality of connected spatial feature maps;
[0202] Feature cross-learning is performed on the multiple connected spatial feature maps respectively to obtain the spatial difference information.
[0203] In some implementations, the determining unit 504 is specifically configured to:
[0204] Determine the first loss value of the spatial difference information and the spatial feature map according to the cross entropy loss function in the classifier in the trained spatial feature determination network; and determine the second loss value of the spatial difference information and the spatial feature map according to the focal loss function in the classifier;
[0205] Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value;
[0206] A spatial feature of the vehicle is determined according to the target loss value.
[0207] See also Figure 6 In some embodiments, the device 500 further includes:
[0208] The fourth acquisition unit 505 is used to acquire a video sample;
[0209] An extraction unit 506 is used to extract image samples of the start of loading and unloading, and image samples of the end of loading and unloading from the video samples to obtain an image sample set;
[0210] The training unit 507 is used to train a preset spatial feature determination network according to the image sample set to obtain the trained spatial feature determination network.
[0211] In some implementations, the training unit 507 is specifically configured to:
[0212] Determine the two-dimensional convolutional neural network and the three-dimensional convolutional neural network in the network based on the preset spatial features, and obtain spatial feature sample graphs of image samples in the image sample set, and spatial sample difference information corresponding to the spatial feature sample graphs at different times;
[0213] Based on the preset spatial feature determination network, a cross entropy loss function and a focal loss function in a classifier are determined, and according to the determined spatial feature sample graph and the spatial sample difference information, a spatial feature prediction value of an image sample in the image sample set is determined;
[0214] Obtaining true values of spatial features of image samples in the image sample set;
[0215] The preset spatial feature determination network is converged according to the spatial feature prediction value and the spatial feature true value to obtain the trained spatial feature determination network.
[0216] In some embodiments, the apparatus 500 further comprises:
[0217] An augmentation unit 508, configured to perform augmentation processing on the image samples in the image sample set to obtain an augmented image sample set;
[0218] At this time, the training unit 507 is specifically used for:
[0219] The preset spatial feature determination network is trained according to the augmented image sample set to obtain the trained spatial feature determination network.
[0220] In some implementations, the augmentation unit 508 is specifically configured to:
[0221] Performing compression processing on the image samples in the image sample set according to different compression samples; or,
[0222] Performing random cropping processing on the image samples in the image sample set according to a preset cropping area; or,
[0223] Randomly adjust the brightness, contrast, chroma and saturation of the image samples in the image sample set; or,
[0224] The order of the images corresponding to the start and end of loading and unloading of the image samples in the image sample set is randomly swapped.
[0225] In some embodiments, the apparatus 500 further comprises:
[0226] A compression unit 509, used to compress each frame image in the image set to a preset size to obtain a compressed image set;
[0227] A cropping unit 510, configured to crop each frame of the compressed image set according to a preset cropping region to obtain a cropped image set;
[0228] At this time, the second acquiring unit 502 is specifically used for:
[0229] The spatial feature maps corresponding to each frame image in the cropped image set are respectively obtained.
[0230] In the embodiment of the present application, first, the first acquisition unit 501 acquires images when loading and unloading of the vehicle starts, and images when loading and unloading ends, to obtain an image set; then the second acquisition unit 502 respectively acquires the spatial feature map corresponding to each frame image in the image set; the third acquisition unit 503 then acquires the spatial difference information corresponding to the spatial feature map at different times; finally, the determination unit 504 determines the spatial features of the vehicle based on the spatial difference information and the spatial feature map. This solution extracts the spatial feature maps of multiple frames of images and the spatial difference information of the spatial feature maps of each frame at different times, and uses the feature information in the spatial and temporal dimensions of the multiple frames of images to judge the spatial features, which can improve the accuracy of the vehicle loading rate.
[0231] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0232] refer to Figure 7 The embodiment of the present application provides a network device 700, which may include one or more processors 701 of processing cores, one or more computer-readable storage media memories 702, radio frequency (RF) circuits 703, power supplies 704, input units 705, and display units 706. Those skilled in the art will appreciate that Figure 7 The network device structure shown in the figure does not constitute a limitation on the network device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0233] The processor 701 is the control center of the network device. It uses various interfaces and lines to connect various parts of the entire network device. By running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, it executes various functions of the network device and processes data, thereby monitoring the network device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.
[0234] The memory 702 may be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702 .
[0235] The RF circuit 703 may be used for receiving and sending signals during the process of sending and receiving information.
[0236] The network device also includes a power supply 704 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 701 through a power management system, so that the power management system can manage charging, discharging, and power consumption.
[0237] The network device may further include an input unit 705, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0238] The network device may also include a display unit 706, which may be used to display information input by a user or information provided to a user and various graphical user interfaces of the network device, which may be composed of graphics, text, icons, videos, and any combination thereof. Specifically in this embodiment, the processor 701 in the network device will load the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 will run the application stored in the memory 702, thereby realizing various functions, as follows:
[0239] Acquire images at the beginning of loading and unloading, and images at the end of loading and unloading, to obtain an image set;
[0240] Respectively obtain the spatial feature map corresponding to each frame image in the image set;
[0241] Obtaining spatial difference information corresponding to the spatial feature map at different times;
[0242] The spatial characteristics of the vehicle are determined according to the spatial difference information and the spatial characteristic map.
[0243] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the method for determining the spatial features above, and will not be repeated here.
[0244] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0245] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the spatial feature determination methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:
[0246] Acquire images at the beginning of loading and unloading, and images at the end of loading and unloading, to obtain an image set;
[0247] Respectively obtain the spatial feature map corresponding to each frame image in the image set;
[0248] Obtaining spatial difference information corresponding to the spatial feature map at different times;
[0249] The spatial characteristics of the vehicle are determined according to the spatial difference information and the spatial characteristic map.
[0250] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0251] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0252] Since the instructions stored in the storage medium can execute the steps in any method for determining spatial features provided in the embodiments of the present application, the beneficial effects that can be achieved by any method for determining spatial features provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0253] The above is a detailed introduction to a method, device, network device and storage medium for determining spatial features provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for determining spatial features, It is characterized in that include: Acquire images at the beginning of loading and unloading, and images at the end of loading and unloading, to obtain an image set; Respectively obtain the spatial feature map corresponding to each frame image in the image set; Obtaining spatial difference information corresponding to the spatial feature map at different times; Determine the first loss value of the spatial difference information and the spatial feature map according to the cross entropy loss function in the classifier in the trained spatial feature determination network; and determine the second loss value of the spatial difference information and the spatial feature map according to the focal loss function in the classifier; Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value; determining a spatial characteristic of the vehicle according to the target loss value; The obtaining of spatial difference information corresponding to the spatial feature map at different times includes: Performing a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map; The superimposed spatial feature map is input into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map.
2. The method according to claim 1, It is characterized in that The separately acquiring the spatial feature map corresponding to each frame image in the image set includes: Each frame image in the image set is input into the trained two-dimensional convolutional neural network in the spatial feature determination network to perform spatial feature learning, so as to obtain the spatial feature map corresponding to each frame image.
3. The method according to claim 1, It is characterized in that The step of inputting the superimposed spatial feature graph into the trained three-dimensional convolutional neural network in the spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature graphs at different times in the superimposed spatial feature graph comprises: Respectively obtain the time corresponding to each frame of the spatial feature map in the superimposed spatial feature map; Based on the three-dimensional convolutional neural network, the spatial feature maps at different moments in the superimposed spatial feature map are connected according to the moment corresponding to the spatial feature map of each frame to obtain a plurality of connected spatial feature maps; Feature cross-learning is performed on the multiple connected spatial feature maps respectively to obtain the spatial difference information.
4. The method according to any one of claims 1 to 3, It is characterized in that Before respectively acquiring the spatial feature maps corresponding to each frame image in the image set, the method further includes: Get video samples; Extracting image samples of the start of loading and unloading and image samples of the end of loading and unloading from the video samples to obtain an image sample set; A preset spatial feature determination network is trained according to the image sample set to obtain the trained spatial feature determination network.
5. The method according to claim 4, It is characterized in that The step of training a preset spatial feature determination network according to the image sample set to obtain the trained spatial feature determination network includes: Determine the two-dimensional convolutional neural network and the three-dimensional convolutional neural network in the network based on the preset spatial features, and obtain spatial feature sample graphs of image samples in the image sample set, and spatial sample difference information corresponding to the spatial feature sample graphs at different times; Determine a cross entropy loss function and a focal loss function in a classifier in the network based on the preset spatial feature, and determine a spatial feature prediction value of an image sample in the image sample set according to the spatial feature sample graph and the spatial sample difference information; Obtaining true values of spatial features of image samples in the image sample set; The preset spatial feature determination network is converged according to the spatial feature prediction value and the spatial feature true value to obtain the trained spatial feature determination network.
6. A device for determining spatial characteristics, It is characterized in that include: The first acquisition unit is used to acquire images when loading and unloading of the vehicle starts, and images when loading and unloading of the vehicle ends, to obtain an image set; A second acquisition unit is used to respectively acquire a spatial feature map corresponding to each frame image in the image set; A third acquisition unit is used to acquire spatial difference information corresponding to the spatial feature map at different times; A determination unit, configured to determine a first loss value of the spatial difference information and the spatial feature map according to a cross entropy loss function in a classifier in a trained spatial feature determination network; and determine a second loss value of the spatial difference information and the spatial feature map according to a focal loss function in the classifier; Determine a target loss value according to the first loss value, the second loss value, a weight corresponding to the first loss value, and a weight corresponding to the second loss value; Determining a spatial characteristic of the vehicle according to the target loss value; The obtaining of spatial difference information corresponding to the spatial feature map at different times includes: Performing a superposition process on the spatial feature maps corresponding to the frames of image to obtain a superimposed spatial feature map; The superimposed spatial feature map is input into the three-dimensional convolutional neural network in the trained spatial feature determination network for feature cross-learning to obtain spatial difference information corresponding to the spatial feature maps at different times in the superimposed spatial feature map.
7. A network device, It is characterized in that It comprises a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the method for determining spatial features as described in any one of claims 1 to 5 is executed.
8. A storage medium, It is characterized in that The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method for determining spatial features as described in any one of claims 1 to 5.
