Video processing method and apparatus, display cabinet, and storage medium

Abstract

The present disclosure provides a video processing method, device, display cabinet and storage medium, wherein the method comprises: acquiring a video stream collected by a camera in a display cabinet; the video stream is a video collected by the camera from the period when the cabinet door of the display cabinet is opened to closed; extracting multi-scale features of a plurality of target video frames in the video stream; wherein the multi-scale features contain a plurality of image features with feature sizes decreasing in turn; determining a target semantic feature of each image feature; wherein the target semantic feature is a feature with a semantic intensity meeting the requirements in the image feature in the spatial dimension; performing deblurring processing on each target video frame based on the target semantic feature to obtain the deblurred video stream.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video processing, and in particular, to a video processing method and device, a display cabinet, and a storage medium. BACKGROUND

[0002] In recent years, display cabinets have been widely used in convenience stores and large supermarkets, such as self-service display cabinets, refrigerated display cabinets for displaying refrigerated items, and the like. In the existing display cabinets, one or more cameras are usually installed in the display cabinet, and the items in the display cabinet are identified through the one or more cameras. Therefore, the clarity of the video captured by each camera will affect the identification result of the identification of the items in the display cabinet. When the clarity of the video captured by each camera is poor, the accuracy of the identification result of the identification of the items in the display cabinet may be reduced, thereby affecting the user experience of using the display cabinet. SUMMARY

[0003] The embodiments of the present disclosure at least provide a video processing method and device, a display cabinet, and a storage medium. By determining the target semantic feature of each image feature, the embodiments of the present disclosure can improve the deblurring effect of the video stream and obtain a clearer video stream.

[0004] In a first aspect, the embodiments of the present disclosure provide a video processing method, including: acquiring a video stream captured by a camera in a display cabinet; the video stream is a video captured by the camera from a period when a cabinet door of the display cabinet is opened to a period when the cabinet door is closed; extracting multi-scale features of a plurality of target video frames in the video stream; wherein the multi-scale features include a plurality of image features with feature sizes decreasing in turn; determining a target semantic feature of each of the image features; wherein the target semantic feature is a feature with a semantic intensity meeting a requirement in the image feature in a spatial dimension; and performing deblurring processing on each of the target video frames based on the target semantic feature to obtain the video stream after deblurring.

[0005] In a second aspect, the embodiments of the present disclosure provide a video processing device, including: an acquisition unit configured to acquire a video stream captured by a camera in a display cabinet; the video stream is a video captured by the camera from a period when a cabinet door of the display cabinet is opened to a period when the cabinet door is closed; an extraction unit configured to extract multi-scale features of a plurality of target video frames in the video stream; wherein the multi-scale features include a plurality of image features with feature sizes decreasing in turn; a determination unit configured to determine a target semantic feature of each of the image features; wherein the target semantic feature is a feature with a semantic intensity meeting a requirement in the image feature in a spatial dimension; and a deblurring unit configured to perform deblurring processing on each of the target video frames based on the target semantic feature to obtain the video stream after deblurring.

[0006] In a third aspect, the embodiments of the present disclosure provide a display cabinet, comprising: a display cabinet body, a camera and a processor; wherein the camera is installed on the display cabinet body, and the camera is in communication connection with the processor; the camera is configured to capture a video stream of the inside of the display cabinet from when a cabinet door of the display cabinet is opened to when the cabinet door is closed; the processor is configured to acquire the video stream captured by the camera in the display cabinet; the video stream is a video captured by the camera from when the cabinet door of the display cabinet is opened to when the cabinet door is closed; and multi-scale features of a plurality of target video frames in the video stream are extracted; wherein the multi-scale features comprise a plurality of image features with feature sizes decreasing in turn; target semantic features of each of the image features are determined; wherein the target semantic features are features with semantic intensity meeting a requirement in the image features in a spatial dimension; and each of the target video frames is deblurred based on the target semantic features, to obtain the video stream after deblurring.

[0007] In a fourth aspect, the embodiments of the present disclosure also provide a display cabinet, comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the display cabinet is running, the processor and the memory are in communication through the bus, and the machine readable instructions are executed by the processor to perform the steps of the first aspect or any possible implementation manner of the first aspect.

[0008] In a fifth aspect, the embodiments of the present disclosure also provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps of the first aspect or any possible implementation manner of the first aspect.

[0009] The embodiments of the present disclosure provide a video processing method and device, a display cabinet and a storage medium. In the embodiments of the present disclosure, first, a video stream captured by a camera in a display cabinet is acquired, and multi-scale features of a plurality of target video frames in the video stream are extracted, and target semantic features of each image feature are determined, so that each target video frame is deblurred based on the target semantic features, to obtain the video stream after deblurring. In the embodiments of the present disclosure, the multi-scale features extracted based on the target video frames are features with sizes of image features decreasing in turn, and the process of extracting the multi-scale features of the target video frames is a down-sampling process of features. Since a large amount of feature data in the target video frames is lost in the down-sampling process of features, the deblurring effect of the target video frames may be poor. Based on this, the embodiments of the present disclosure can determine the target semantic features of each image feature in the multi-scale features, to compensate for the feature data lost in the down-sampling process by the target semantic features, to improve the deblurring effect of the video stream, and obtain a clearer video stream.

[0010] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the description and form a part of the description, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the description. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0012] Figure 1 A flow chart of a video processing method provided by an embodiment of the present disclosure is shown;

[0013] Figure 2 A flow chart of a specific method for determining the target semantic feature of each image feature in a video processing method provided by an embodiment of the present disclosure is shown;

[0014] Figure 3 A network structure diagram of an image deblurring network provided by an embodiment of the present disclosure is shown;

[0015] FIG. 4(a) shows a schematic diagram of the connection relationship between the convolution layer and the attention layer in an image deblurring network provided by an embodiment of the present disclosure;

[0016] FIG. 4(b) shows a schematic diagram of the connection relationship between the channel attention module and the residual spatial attention module in an attention layer provided by an embodiment of the present disclosure;

[0017] Figure 5 A schematic diagram of a video processing device provided by an embodiment of the present disclosure is shown;

[0018] Figure 6 A structural schematic diagram of a display cabinet provided by an embodiment of the present disclosure is shown;

[0019] Figure 7 A structural schematic diagram of another display cabinet provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0020] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings of the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, but not all the embodiments. The components of the embodiments of the present disclosure generally described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present disclosure.

[0021] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0022] The term "and / or" herein is only to describe an association relationship, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0023] It is found through research that in recent years, display cabinets have been widely used in convenience stores and large supermarkets, such as unmanned self-service display cabinets, refrigerated display cabinets for displaying refrigerated items, etc. In the existing display cabinets, one or more cameras are usually installed in the display cabinet, and the items in the display cabinet are identified through the one or more cameras. Therefore, the clarity of the video captured by each camera will affect the identification result of the identification of the items in the display cabinet. When the clarity of the video captured by each camera is poor, the accuracy of the identification result of the identification of the items in the display cabinet may be reduced, thereby affecting the user's experience of using the display cabinet.

[0024] Based on the above research, the present disclosure provides a video processing method, device, display cabinet and storage medium. In the embodiment of the present disclosure, first, the video stream captured by the camera in the display cabinet is obtained, and the multi-scale features of a plurality of target video frames in the video stream are extracted, and the target semantic features of each image feature are determined, so that each target video frame is deblurred based on the target semantic features, and the deblurred video stream is obtained. In the embodiment of the present disclosure, the multi-scale features extracted based on the target video frame are features with decreasing scales of image features, and the process of extracting the multi-scale features of the target video frame is the down-sampling process of the features. Because a large amount of feature data in the target video frame will be lost in the down-sampling process of the features, it may cause poor deblurring effect of the target video frame. Based on this, the embodiment of the present disclosure can determine the target semantic features of each image feature in the multi-scale features, so as to compensate for the lost feature data in the down-sampling process through the target semantic features, so as to improve the deblurring effect of the video stream and obtain a clearer video stream.

[0025] In the embodiment of the present disclosure, the display cabinet can be any cabinet body with an openable and closable cabinet door. For example, the display cabinet can be a display cabinet supporting unmanned self-service function. The display cabinet can also be a refrigeration cabinet for placing refrigerated goods in a supermarket. In addition, the display cabinet can also be a warming cabinet, a freezer for placing frozen goods, or any other cabinet body with an openable and closable cabinet door. The present disclosure does not specifically limit the type of display cabinet.

[0026] In order to facilitate the understanding of the present embodiment, first, a video processing method disclosed in the embodiment of the present disclosure will be described in detail. The execution subject of the video processing method provided in the embodiment of the present disclosure is generally a display cabinet with certain computing power.

[0027] Referring to Figure 1 The flowchart of the video processing method provided in the embodiment of the present disclosure is shown in FIG. 1. The method comprises steps S101 to S107, wherein:

[0028] S101: Obtain the video stream captured by the camera in the display cabinet; the video stream is the video captured by the camera from the period when the cabinet door of the display cabinet is opened to the period when the cabinet door is closed.

[0029] In the embodiment of the present disclosure, the installation position of the camera in the display cabinet can be on the cabinet door or in the interior of the display cabinet. The present disclosure does not specifically limit the installation position of the camera in the display cabinet.

[0030] Here, one or more cameras can be installed in the above display cabinet, for example, in the case where the number of cameras is multiple, the installation position of each camera in the display cabinet can be different, that is, the collection angle of each camera in the display cabinet is different. Through the above processing mode, the video of the user taking the goods in the display cabinet can be collected from different angles by the above multiple cameras respectively, so as to obtain the corresponding video stream. It should be understood that in the embodiments of the present disclosure, the number of cameras installed in the display cabinet is not limited, and the user can set it according to actual needs.

[0031] In the embodiments of the present disclosure, when the opening operation of the door of the display cabinet is detected, the video collection function of the camera in the display cabinet is started to collect video. When the closing operation of the door of the display cabinet is detected, the video collection is ended to obtain the video stream collected in the period from opening to closing of the door of the display cabinet.

[0032] A sensor is pre-installed in the display cabinet, wherein the sensor is used to listen to the opening / closing action of the door of the display cabinet, for example, the sensor can be a door magnetic switch or other device capable of realizing the opening / closing action of the door, and the present disclosure does not make specific limitation here, as long as it can be realized. The above display cabinet includes a processor, and the above multiple cameras can be controlled by the processor, and the processor and the sensor are in communication connection.

[0033] In specific implementation, when the sensor detects that the user opens the door, an opening signal can be generated and sent to the processor. After receiving the opening signal, the processor can control the camera to open. Similarly, when the sensor detects that the user closes the door, a closing signal can be generated and sent to the processor. After receiving the closing signal, the processor can control the camera to close.

[0034] S103: Extracting multi-scale features of multiple target video frames in the video stream; wherein the multi-scale features include multiple image features with feature sizes decreasing in turn.

[0035] In the embodiments of the present disclosure, after obtaining the video stream collected by the camera in the display cabinet, multiple target video frames can be determined in the video stream, and multi-scale features of the multiple target video frames can be extracted.

[0036] Here, the target video frame can be understood as a video frame containing a picture of interest in the video stream. For example, the target video frame can be a video frame containing a picture of the user taking something in the display cabinet in the video stream, and the present disclosure does not make specific limitation on the picture of interest contained in the target video frame, as long as it meets the actual needs.

[0037] S105: Determine the target semantic feature for each of the image features; wherein the target semantic feature is a feature in the image feature whose semantic strength meets the requirements in the spatial dimension.

[0038] In this embodiment of the disclosure, after extracting multiple image features for each target video frame, the target semantic features of each image feature can be determined. For example, the target semantic features in each image feature can be extracted using an attention network model.

[0039] Here, the target semantic feature is the image feature in the spatial dimension whose semantic strength meets the requirements. Meeting the semantic strength requirements can be understood as satisfying a pre-set threshold. For example, this semantic strength requirement can be used to indicate the threshold of the loss function corresponding to the attention network model, or it can be used to indicate the threshold of the number of iterations corresponding to the attention network model. This disclosure does not limit the semantic strength requirement; whatever is feasible is acceptable.

[0040] S107: Deblurring is performed on each of the target video frames based on the target semantic features to obtain the deblurred video stream.

[0041] In this embodiment of the disclosure, after acquiring the video stream captured by the camera in the display case, multi-scale features of multiple target video frames in the video stream can be extracted, and the target semantic features of each image feature in the multi-scale features can be determined. Then, each target video frame can be deblurred based on the target semantic features to obtain a deblurred video stream.

[0042] In the above embodiments, the information loss caused during the extraction of multi-scale features of the target video frame can be compensated by determining the target semantic features in each image feature. Simultaneously, deblurring the target video frame using the target semantic features can improve the accuracy of the deblurring process, resulting in a clearer video stream and thus enhancing the clarity and accuracy of the video stream acquired by the display case.

[0043] In one alternative implementation, such as Figure 2 As shown, determining the target semantic features for each image feature in step 105 specifically includes the following steps:

[0044] Step S21: Determine the channel attention feature for each of the image features; wherein the channel attention feature is used to indicate the correlation between the channel features in the image feature;

[0045] Step S22: Determine the target semantic features of the image features based on each image feature and the channel attention features of the image features.

[0046] In the embodiments of the present disclosure, the image feature can be input into a channel attention module (for example, the channel attention module can be a CAM (channel-wise attention module) model), and a channel attention feature of the image feature can be extracted based on the channel attention module.

[0047] In the embodiments of the present disclosure, after the channel attention feature of the image feature is determined, a target semantic feature of the image feature can be determined based on the image feature and the channel attention feature of the image feature.

[0048] In specific implementation, a spatial attention model can be acquired, a spatial attention feature in the channel attention feature can be determined based on the spatial attention model, and the target semantic feature can be determined based on the spatial attention feature and the image feature. The spatial attention model can be a residual spatial attention module (RSAM).

[0049] It should be noted that the channel attention module and the spatial attention model are not specifically limited in the present disclosure, and can be determined according to actual needs.

[0050] In the above embodiments, the dependence between channels in the image feature can be established by the channel attention module, and the correlation between channel features in the image feature can be enhanced. Meanwhile, the image feature and the channel attention feature can be input into the spatial attention model, more rich semantic information of the image feature in space can be extracted, and the target semantic information can be obtained, so that the obtained target semantic information can be rich.

[0051] In an optional embodiment, for the step S22 of determining a target semantic feature of each image feature based on the image feature and the channel attention feature of the image feature, the step S22 specifically includes the following steps:

[0052] Step S31: performing feature fusion on each image feature and the channel attention feature of the image feature to obtain a target fusion feature;

[0053] Step S32: determining a fusion semantic feature of the target fusion feature, wherein the fusion semantic feature is a feature in the target fusion feature whose semantic intensity meets a requirement.

[0054] Step S33: fusing the target fusion feature and the fusion semantic feature to obtain the target semantic feature.

[0055] In the embodiments of the present disclosure, after the channel attention feature of the image feature is determined in the manner described above, the image feature and the channel attention feature of the image feature can be fused by a preset fusion method to obtain a target fusion feature. For example, the preset fusion method can be multiplication calculation or addition calculation, and the present disclosure does not make specific limitation on the above-mentioned fusion method.

[0056] For example, each image feature can be input into the channel attention module RSAM to obtain a channel attention feature; then, the image feature and the channel attention feature are fused to obtain a target fusion feature. Here, the channel attention feature and the image feature can be multiplied to realize the fusion of the image feature and the channel attention feature, thereby obtaining the target fusion feature.

[0057] In the embodiments of the present disclosure, after the target fusion feature is determined, the target fusion feature can be input into the spatial attention model to extract a spatial attention feature, and the spatial attention feature is determined as a fusion semantic feature.

[0058] Then, the target fusion feature and the fusion semantic feature can be fused based on a preset fusion method to obtain a target semantic feature.

[0059] For example, each target fusion feature can be input into the residual spatial attention module CAM to obtain a fusion semantic feature of the target fusion feature; then, the target fusion feature and the fusion semantic feature of the target fusion feature are fused to obtain a target semantic feature. Here, the target fusion feature and the fusion semantic feature of the target fusion feature can be multiplied to realize the fusion of the target fusion feature and the fusion semantic feature of the target fusion feature, thereby obtaining the target semantic feature.

[0060] In the above-mentioned embodiments, by fusing the image feature and the channel attention feature of the image feature, the target fusion feature can contain more information, thereby making up for the problem of losing too much information of the image feature. At the same time, by determining the fusion semantic feature of the target fusion feature, the fusion semantic feature containing more semantic information can be obtained. After the target fusion feature and the fusion semantic feature are fused, the target semantic feature is obtained, and the feature containing more feature information and semantic information can be obtained.

[0061] In an optional implementation, the video processing method provided in this disclosure further includes an image deblurring network. Here, the image deblurring network includes: a first feature extraction network, a target attention network, and a second feature extraction network. The first feature extraction network comprises multiple max-pooling networks and multiple first convolutional networks, each first convolutional network comprising a convolutional layer and a downsampling layer. The target attention network comprises multiple attention layers, and each convolutional layer in the first convolutional network is connected to an attention layer.

[0062] In this embodiment of the disclosure, each network in the first feature extraction network can be defined as a first network, and each network in the second feature extraction network can be defined as a second network. That is, each first network in the first feature extraction network includes multiple max pooling networks and multiple first convolutional networks, and each second network in the second feature extraction network includes multiple second convolutional networks and a normalization layer, wherein each second convolutional network includes a convolutional layer and an upsampling layer.

[0063] For example, such as Figure 3 The diagram shows the network structure of an image deblurring network. Figure 3 As shown, the image deblurring network includes two max pooling networks and four first convolutional networks. Each first convolutional network's convolutional layer corresponds to an attention layer (the attention layer is located in...). Figure 3 (Not specifically shown in the text). The second feature extraction network contains six second convolutional networks, each of which contains one convolutional layer and one upsampling layer.

[0064] In this embodiment of the disclosure, after acquiring the video stream, the target video frame in the video stream can be deblurred using the image deblurring network described above, thereby obtaining the deblurred video stream. The target video frame can be all the video frames in the video stream or a portion of the video frames in the video stream; this disclosure does not specifically limit this.

[0065] As described above, the image deblurring network includes a first feature extraction network, a target attention network, and a second feature extraction network.

[0066] Based on this, step S103 above extracts multi-scale features of multiple video frames in the video stream, specifically including the following steps: extracting multi-scale features of each target video frame in the video stream through the first feature extraction network in the image deblurring network, wherein the first feature extraction network includes multiple first networks connected in sequence, the multiple first networks including: the multiple first networks include multiple first convolutional networks, each of the first convolutional networks includes a convolutional layer and a downsampling layer.

[0067] The step S105 determines the target semantic feature of each image feature, and specifically includes the following step: extracting the target semantic feature of each image feature through a target attention network in the image deblurring network, wherein the target attention network comprises a plurality of attention layers, and each convolution layer in the first feature extraction network is correspondingly provided with an attention layer.

[0068] Next, the target semantic feature and the image feature can be processed through the second feature extraction network, so that the target video frame is deblurred to obtain the deblurred video stream.

[0069] In the embodiment of the present disclosure, after obtaining the video stream collected by the camera in the display cabinet, the video stream can be input into the image deblurring network, and the multi-scale feature of each target video frame in the video stream can be extracted through the first feature extraction network in the image deblurring network. For example, as shown in Figure 3 The multi-scale feature of the target video frame can be extracted through the first convolution network in the first feature extraction network, and the target video frame can be processed in sequence through the max-pooling network in the first feature extraction network, so as to obtain the pooling processing result.

[0070] In the embodiment of the present disclosure, after the target video frame passes through the plurality of first networks, the matching image features can be obtained, wherein the number of image features is the same as the number of first networks. Then, the target semantic feature of the image feature output by each convolution layer in the first convolution network can be extracted through the attention layer in the target attention network of the image deblurring network.

[0071] For example, the target attention network can be a residual dual attention model (Residual Dual Attention Module, RDAM). At this time, each attention layer included in the target attention network comprises a CAM model and a RSAM model.

[0072] In the above embodiment, the multi-scale feature of the target video frame can be extracted through the first feature extraction network in the image deblurring network, and the target semantic feature of each image feature can be extracted through the target attention network in the image deblurring network, which can improve the speed of video processing, thereby meeting the real-time requirement of video processing.

[0073] In an optional embodiment, in the case where each attention layer comprises a channel attention module and a residual spatial attention module, the step of extracting the target semantic feature of each image feature through the target attention network in the image deblurring network specifically includes the following steps:

[0074] Step S41: extracting a channel attention feature of the image feature by the channel attention module; wherein the channel attention feature is used to indicate the correlation between the channel features in the image feature.

[0075] Step S42: determining the target semantic feature based on the channel attention feature and the image feature by the residual spatial attention module.

[0076] As shown in FIG. 4(a), it is the connection relationship between any one convolutional layer and attention layer in the first convolutional network. As shown in FIG. 4(b), each attention layer includes a channel attention module and a residual spatial attention module, wherein any one convolutional layer in the first convolutional network is connected with the channel attention module, and the channel attention module is connected with the residual spatial attention module. The above process will be described below by taking any one first convolutional network in the image deblurring network as an example.

[0077] In specific implementation, the convolutional layer can obtain input data, and perform convolutional processing on the input data through the convolutional layer, so as to obtain the image feature of the input data. Then, the channel attention feature of the image feature can be extracted by the channel attention module; then, the channel attention feature is multiplied with the image feature to obtain the target fusion feature; next, the target fusion feature is input into the residual spatial attention module, so that the residual spatial attention module extracts the fusion semantic feature of the target fusion feature; finally, the fusion semantic feature is multiplied with the target fusion feature to obtain the target semantic feature.

[0078] In the above embodiment, since a large amount of feature data in the target video frame is lost in the down-sampling process of the feature, the deblurring effect of the target video frame can be poor. Based on this, the disclosed embodiments can determine the target semantic feature of each image feature in the multi-scale feature, so as to compensate for the feature data lost in the sampling process by the target semantic feature, so as to improve the deblurring effect of the video stream and obtain a clearer video stream.

[0079] In an optional embodiment, the image deblurring network includes a second feature extraction network, and the second feature extraction network includes a plurality of second networks connected in sequence, each second network corresponding to a first network connected thereto. Based on this, the above step S107: deblurring each target video frame based on the target semantic feature to obtain the deblurred video stream, specifically includes the following steps:

[0080] Step S1071: input the target semantic features output by the last first convolutional network in the plurality of first networks into the second feature extraction network after processing, and input the target semantic features output by each max-pooling network and each first convolutional network in the first network into the corresponding second network;

[0081] Step S1072: process the input data by each second network to obtain a deblurred video frame after processing, so as to obtain the deblurred video stream.

[0082] As can be known from the above description, the image deblurring network further includes a second feature extraction network, as shown in Figure 3 . The second feature extraction network includes a plurality of second networks connected in sequence, and each second network is connected with a first network, for example, as shown in Figure 3 . Each first convolutional network in the first network is connected with a second convolutional network, and each max-pooling network in the first network is connected with a second convolutional network. The number of second convolutional networks is equal to the sum of the number of first convolutional networks and the number of max-pooling networks.

[0083] In the embodiments of the present disclosure, as shown in Figure 3 , after the target video frame is determined in the video stream, the target video frame can be input into the first convolutional network for processing. After a plurality of first convolutional processing, the corresponding target semantic features are output by the last first convolutional network. Then, the target semantic features output by the last first convolutional network (for example, the fourth first convolutional network in Figure 3 ) are input into the second feature extraction network for processing after feature expansion processing. After the second feature extraction network obtains the target semantic features output by the last first convolutional network, convolution calculation is performed by each second convolutional network in the second feature extraction network in sequence.

[0084] As can be known from the above description, each first network in the first feature extraction network is connected with a second network in the second feature extraction network, that is, each first network in the first feature extraction network is connected to each second network in the second feature extraction network in a skip connection manner. At this time, each second network can obtain the image features output by the first network connected in a skip connection manner.

[0085] For example, as shown in Figure 3As shown, the image features output by each max-pooling network in the first feature extraction network are input into the corresponding second network, i.e., the last two second convolution networks in the second feature extraction network, as input data; and the target semantic features output by each first convolution network in the first feature extraction network are input into the corresponding second network, i.e., the first four second convolution networks in the second feature extraction network, as input data.

[0086] In the embodiments of the present disclosure, after each second network in the second feature extraction network obtains the input data input by the corresponding first network and the output data of the previous network of the second network, the received data can be processed, so as to determine the output result of the last second convolution network as the deblurred target video frame, and thus the deblurred video stream can be obtained based on the deblurred target video frame.

[0087] Here, each second convolution network in the second feature extraction network includes a convolution layer and an up-sampling layer, based on which, the calculation process of the second convolution network in the second feature extraction network is described as follows:

[0088] Case one: the second convolution network is the first convolution network in the second feature extraction network.

[0089] The second convolution network obtains the target semantic features output by the last first convolution network in the first feature extraction network, and obtains the input data A after the target semantic features output by the last first convolution network in the first feature extraction network are processed; then, the up-sampling layer is used to perform up-sampling processing on the input data A to obtain a sampling result A; and then, the convolution layer is used to perform convolution processing on the sampling result A and the target semantic features to obtain a convolution processing result.

[0090] Case two: the second convolution network is a non-first convolution network in the second feature extraction network.

[0091] The second convolution network obtains the target semantic features output by the first convolution network (or the max-pooling network) in the first feature extraction network that is jump-connected to the second convolution network, and obtains the output data B of the previous second convolution network of the second convolution network; then, the up-sampling layer is used to perform up-sampling processing on the output data B to obtain a sampling result B; and then, the convolution layer is used to perform convolution processing on the sampling result B and the target semantic features to obtain a convolution processing result.

[0092] In an optional implementation, the image features extracted by each of the first convolutional networks in the plurality of first networks are features extracted based on target input data, the target input data of the first convolutional network in the plurality of first networks is the target video frame, and the target input data of the other first convolutional networks in the plurality of first networks is the target semantic features output by the first convolutional networks in the plurality of first networks located before the other first convolutional networks, the other first convolutional networks being the remaining first convolutional networks in the plurality of first networks except the first convolutional network.

[0093] Specifically, as shown in Figure 3 the image deblurring network includes four first convolutional networks, namely, the first convolutional network 1, the first convolutional network 2, the first convolutional network 3, and the first convolutional network 4. The first convolutional network 1 is the first convolutional network (i.e., the first convolutional network in the plurality of first networks), and the first convolutional network 2, the first convolutional network 3, and the first convolutional network 4 are the other first convolutional networks.

[0094] As shown in Figure 3 the target input data of the first convolutional network 1 is the target video frame. The target input data of the first convolutional network 2 is the target semantic features output by the first convolutional network 1, wherein the target semantic features are the sampling results obtained by performing down-sampling on the target semantic features output by the attention layer corresponding to the first convolutional network 1 through the down-sampling layer of the first convolutional network 1. The target input data of the first convolutional network 3 is the target semantic features output by the first convolutional network 2, wherein the target semantic features are the sampling results obtained by performing down-sampling on the target semantic features output by the attention layer corresponding to the first convolutional network 2 through the down-sampling layer of the first convolutional network 2. The target input data of the first convolutional network 4 is the target semantic features output by the first convolutional network 3, wherein the target semantic features are the sampling results obtained by performing down-sampling on the target semantic features output by the attention layer corresponding to the first convolutional network 3 through the down-sampling layer of the first convolutional network 3.

[0095] In an optional implementation, the step S103 of extracting the multi-scale features of the target video frames in the video stream further includes the following steps:

[0096] (1) determining a plurality of target video frames in the video stream, wherein the plurality of target video frames include video frames in which the user takes the goods from the display cabinet;

[0097] (2) extracting the multi-scale features of each of the target video frames.

[0098] In the embodiments of the present disclosure, when the number of video frames contained in the video stream is large, the requirement for the processing performance of the processor in the display cabinet is high, and for the scene with high real-time requirement, the existing processor cannot meet the real-time requirement of the user for a large number of video frames. If a processor with higher performance is selected, the research and development cost of the display cabinet will be increased. Based on this, in the embodiments of the present disclosure, a plurality of target video frames can be determined in the video stream, and a multi-scale feature of each target video frame is extracted. Here, the target video frame can be a video frame in the video stream for indicating that the user takes the corresponding article from the display cabinet.

[0099] In an optional implementation, the plurality of target video frames are determined in the video stream, specifically including the following steps:

[0100] Firstly, a body recognition model is used to perform body recognition on each video frame in the video stream to obtain a body recognition result.

[0101] Then, a plurality of video frames containing a target body part in the video stream are determined based on the body recognition result, and the plurality of video frames containing the target body part are determined as the plurality of target video frames, wherein the target body part is a body part of the user taking the corresponding article.

[0102] In the embodiments of the present disclosure, after the video stream is obtained, each video frame can be recognized by a body recognition model, so as to obtain a body recognition result of each video frame.

[0103] Here, the body recognition result can be used to indicate whether the body part of the user is contained in each video frame, and in the case of containing the body part of the user, the type of the body part. The body recognition result can also be used to indicate whether the target body part (for example, the hand of the user) of the user is contained in each video frame.

[0104] After the body recognition result of each video frame is determined, the target video frame containing the target body part can be recognized in the plurality of video frames based on the body recognition result, so that the target video frame is processed through the processes described in steps S103 to S107 to obtain the deblurred target video frame.

[0105] Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0106] Based on the same inventive concept, the embodiments of the present disclosure further provide a video processing device corresponding to the video processing method. Since the principle of the device in the embodiments of the present disclosure solves the problem similarly to the video processing method in the embodiments of the present disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0107] With reference to Figure 5 As shown in the figure, the video processing device provided by the embodiments of the present disclosure is intended to include: an acquisition unit 51, an extraction unit 52, a determination unit 53, and a deblurring unit 54.

[0108] The acquisition unit is configured to acquire a video stream captured by a camera in a display cabinet; the video stream is a video captured by the camera from the time when the cabinet door of the display cabinet is opened to the time when the cabinet door is closed.

[0109] The extraction unit is configured to extract multi-scale features of a plurality of target video frames in the video stream; wherein the multi-scale features include a plurality of image features with feature sizes decreasing in turn.

[0110] The determination unit is configured to determine a target semantic feature of each of the image features; wherein the target semantic feature is a feature with a semantic intensity satisfying a requirement in the image feature in a spatial dimension.

[0111] The deblurring unit is configured to perform deblurring processing on each of the target video frames based on the target semantic feature, to obtain a deblurred video stream.

[0112] In the embodiments of the present disclosure, first, a video stream captured by a camera in a display cabinet is acquired, and multi-scale features of a plurality of target video frames in the video stream are extracted, and a target semantic feature of each image feature is determined, so that each target video frame is deblurred based on the target semantic feature, to obtain a deblurred video stream. In the embodiments of the present disclosure, the multi-scale features extracted from the target video frames are features with sizes of image features decreasing in turn, and the process of extracting the multi-scale features of the target video frames is a down-sampling process of the features. Since a large amount of feature data in the target video frames is lost in the down-sampling process of the features, it may result in poor deblurring effect of the target video frames. Based on this, the embodiments of the present disclosure can determine the target semantic feature of each image feature in the multi-scale features, so as to compensate for the lost feature data in the down-sampling process through the target semantic feature, to improve the deblurring effect of the video stream, and obtain a clearer video stream.

[0113] In a possible implementation, the determination unit is further configured to determine a channel attention feature of each of the image features; wherein the channel attention feature is used to indicate the correlation between channel features in the image feature; and determine the target semantic feature of the image feature based on each of the image features and the channel attention feature of the image feature.

[0114] In a possible implementation, the determining unit is further configured to: perform feature fusion on each of the image features and the channel attention feature of the image feature to obtain a target fusion feature; determine a fusion semantic feature of the target fusion feature, where the fusion semantic feature is a feature in the target fusion feature that meets a requirement in terms of semantic intensity; and perform fusion on the target fusion feature and the fusion semantic feature to obtain the target semantic feature.

[0115] In a possible implementation, the extracting unit is further configured to: extract, by a first feature extraction network in the image deblurring network, a multi-scale feature of each target video frame in the video stream, where the first feature extraction network comprises a plurality of first networks connected in sequence, and the plurality of first networks comprise a plurality of first convolution networks, and each first convolution network comprises a convolution layer and a down-sampling layer; and the determining unit is further configured to: extract, by a target attention network in the image deblurring network, a target semantic feature of each of the image features, where the target attention network comprises a plurality of attention layers, and each of the convolution layers in the first feature extraction network is provided with an attention layer.

[0116] In a possible implementation, the extracting unit is further configured to: in a case where each of the attention layers comprises a channel attention module and a residual spatial attention module, extract, by the channel attention module, a channel attention feature of the image feature, where the channel attention feature is used to indicate a correlation between channel features in the image feature; and determine, by the residual spatial attention module, the target semantic feature based on the channel attention feature and the image feature.

[0117] In a possible implementation, the deblurring unit is further configured to: in a case where the image deblurring network comprises a second feature extraction network, the second feature extraction network comprises a plurality of second networks connected in sequence, the first network further comprises a plurality of max-pooling networks, and each of the second networks is connected to one of the first networks, input, to the second feature extraction network, a target semantic feature output by a last first convolution network in the plurality of first networks after processing, and input, to a corresponding second network, a target semantic feature output by each max-pooling network and each first convolution network in each of the first networks; and process, by each of the second networks, input data to obtain a deblurred video frame after processing, to obtain the deblurred video stream, where the input data of the second network comprises a target semantic feature output by the first network corresponding to the second network and output data of a network adjacent to the second network in the image deblurring network.

[0118] In one possible implementation, the image features extracted by each of the plurality of first convolutional networks are features extracted based on target input data. The target input data of the first first convolutional network in the plurality of first networks is the target video frame. The target input data of the other first convolutional networks in the plurality of first networks are the target semantic features output by the first convolutional network located before the other first convolutional network in the plurality of first networks. The other first convolutional networks are the remaining first convolutional networks in the plurality of first networks excluding the first first convolutional network.

[0119] In one possible implementation, the extraction unit is further configured to: determine a plurality of target video frames in the video stream, wherein the plurality of target video frames contain video footage of a user taking items from the display case; and extract multi-scale features of each of the target video frames.

[0120] In one possible implementation, the extraction unit is further configured to: perform limb recognition on each video frame in the video stream using a limb recognition model to obtain limb recognition results; determine multiple video frames in the video stream containing target limb parts based on the limb recognition results, and determine the multiple video frames containing target limb parts as the multiple target video frames, wherein the target limb part is the limb part of the user picking up the corresponding item.

[0121] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0122] Reference Figure 6 The diagram shown is a structural schematic of a display cabinet according to an embodiment of this disclosure. The display cabinet includes a cabinet body 111, a camera 112, and a processor 113. The camera is mounted on the cabinet body and is communicatively connected to the processor.

[0123] The camera is configured to capture a video stream of the interior of the display case from the time the case door is opened until it is closed.

[0124] The processor is configured to acquire a video stream captured by a camera in a display case; the video stream is the video captured by the camera from the opening to the closing of the display case door; extract multi-scale features of multiple target video frames in the video stream; wherein the multi-scale features include multiple image features with successively decreasing feature sizes; determine the target semantic features of each image feature; wherein the target semantic features are features whose semantic strength meets the requirements in the spatial dimension; and perform deblurring processing on each target video frame based on the target semantic features to obtain the deblurred video stream.

[0125] Since a large amount of feature data in the target video frame is lost in the down-sampling process of the features, the deblurring effect of the target video frame can be poor. Based on this, the embodiments of the present disclosure can determine the target semantic feature of each image feature in the multi-scale feature, so as to compensate for the feature data lost in the down-sampling process through the target semantic feature, to improve the deblurring effect of the video stream, and obtain a clearer video stream.

[0126] Corresponding to the video processing method in Figure 1 , the embodiments of the present disclosure also provide a display cabinet 700, as shown in Figure 7 , a structural schematic diagram of the display cabinet 700 provided by the embodiments of the present disclosure, comprising:

[0127] a processor 71, a memory 72, and a bus 73; the memory 72 is used to store execution instructions, including an internal memory 721 and an external memory 722; the internal memory 721 here is also called an internal memory, used to temporarily store operation data in the processor 71 and data exchanged with the external memory 722 such as a hard disk, the processor 71 exchanges data with the external memory 722 through the internal memory 721, when the display cabinet 700 is running, the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the following instructions:

[0128] acquire a video stream collected by a camera in the display cabinet; the video stream is collected by the camera from the period when the cabinet door of the display cabinet is opened to closed;

[0129] extract multi-scale features of a plurality of target video frames in the video stream; wherein the multi-scale features include a plurality of image features with feature sizes decreasing in turn;

[0130] determine a target semantic feature of each of the image features; wherein the target semantic feature is a feature with a semantic intensity meeting the requirements in the image feature in the spatial dimension;

[0131] perform deblurring processing on each of the target video frames based on the target semantic feature, to obtain the video stream after deblurring.

[0132] The embodiments of the present disclosure also provide a computer readable storage medium, which stores a computer program, and the computer program is run by a processor to execute the steps of the display cabinet item display angle adjustment method described in the above method embodiments. Wherein, the storage medium can be a volatile or non-volatile computer readable storage medium.

[0133] The embodiment of the present disclosure further provides a computer program product carrying program codes, the program codes comprising instructions for executing the steps of the method for adjusting the display angle of the displayed goods in the display cabinet described in the above method embodiments, and details can be referred to the above method embodiments, which will not be repeated here.

[0134] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) or the like.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the above method embodiments, which will not be repeated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented by other means. The above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0136] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0137] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0138] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0139] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art who is familiar with the technical field of the present disclosure can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A video processing method, characterized in that, include: Acquire video streams captured by cameras in display cases; the video streams are the videos captured by the cameras from the time the display case doors open to the time they close. Extract multi-scale features from multiple target video frames in the video stream; wherein, the multi-scale features include multiple image features with successively decreasing feature sizes; Determine the target semantic features for each of the image features; wherein, the target semantic features are the features in the image feature whose semantic strength meets the requirements in the spatial dimension; Based on the target semantic features, each target video frame is deblurred to obtain the deblurred video stream. The system also includes an image deblurring network, which comprises a first feature extraction network and a second feature extraction network. Each network in the first feature extraction network is defined as a first network, and each network in the second feature extraction network is defined as a second network. The first feature extraction network comprises multiple first networks connected in sequence, each containing multiple max pooling networks and multiple first convolutional networks. The second feature extraction network comprises multiple second networks connected in sequence, each containing multiple second convolutional networks. Each first convolutional network in the first network is connected to one second convolutional network, and each max pooling network in the first network is connected to one second convolutional network. The number of second convolutional networks is equal to the sum of the number of first convolutional networks and max pooling networks. Among the multiple second convolutional networks, the second convolutional network connected to the first convolutional network is located before the second convolutional network connected to the max pooling network. The target video frames are processed sequentially by the max pooling network in the first feature extraction network to obtain the pooling processing result. The step of extracting multi-scale features of multiple target video frames in the video stream includes: extracting multi-scale features of each target video frame in the video stream through a first feature extraction network in the image deblurring network, wherein each of the first convolutional networks includes a convolutional layer and a downsampling layer; The step of determining the target semantic features of each image feature includes: extracting the target semantic features of each image feature through the target attention network in the image deblurring network, wherein the target attention network contains multiple attention layers, and each convolutional layer in the first feature extraction network is provided with one attention layer. The step of deblurring each target video frame based on the target semantic features to obtain the deblurred video stream includes: The target semantic features output by the last first convolutional network in the plurality of first networks are processed and then input into the second feature extraction network. The target semantic features output by each max pooling network and each first convolutional network in each first network are input into the corresponding second network. The input data is processed by each of the second networks to obtain deblurred video frames, thereby obtaining the deblurred video stream. The input data of the second network includes: the target semantic features output by the first network corresponding to the second network and the output data of the upper layer network adjacent to the second network in the image deblurring network.

2. The method according to claim 1, characterized in that, Determining the target semantic features for each of the image features includes: Determine the channel attention feature for each of the image features; wherein the channel attention feature is used to indicate the correlation between the channel features in the image feature; The target semantic features of an image feature are determined based on each image feature and its channel attention features.

3. The method according to claim 2, characterized in that, The determination of the target semantic features of an image feature based on each image feature and its channel attention features includes: For each of the image features and the channel attention features of that image feature, feature fusion is performed to obtain the target fused feature; Determine the fusion semantic features of the target fusion features, wherein the fusion semantic features are the features in the target fusion features that meet the semantic strength requirements; The target fusion feature and the fusion semantic feature are fused to obtain the target semantic feature.

4. The method according to claim 3, characterized in that, Each attention layer includes a channel attention module and a residual space attention module. The extraction of target semantic features for each image feature through the target attention network in the image deblurring network includes: The channel attention module extracts channel attention features of the image features; wherein, the channel attention features are used to indicate the correlation between various channel features in the image features; The residual space attention module determines the target semantic features based on the channel attention features and the image features.

5. The method according to claim 1, characterized in that, The image features extracted by each of the plurality of first convolutional networks are features extracted based on target input data. The target input data of the first first convolutional network in the plurality of first networks is the target video frame. The target input data of the other first convolutional networks in the plurality of first networks are the target semantic features output by the first convolutional network located before the other first convolutional network in the plurality of first networks. The other first convolutional networks are the remaining first convolutional networks in the plurality of first networks excluding the first first convolutional network.

6. The method according to claim 1, characterized in that, The extraction of multi-scale features from multiple target video frames in the video stream includes: Multiple target video frames are identified in the video stream, wherein the multiple target video frames contain video footage of a user taking an item from the display case; Extract multi-scale features for each of the target video frames.

7. The method according to claim 6, characterized in that, The step of determining multiple target video frames in the video stream includes: The body recognition model is used to perform body recognition on each video frame in the video stream to obtain the body recognition result. Based on the limb recognition results, multiple video frames containing target limb parts are determined in the video stream, and these multiple video frames containing target limb parts are identified as the multiple target video frames, wherein the target limb part is the limb part of the user who picks up the corresponding item.

8. A video processing apparatus, characterized in that, include: The acquisition unit is used to acquire the video stream captured by the camera in the display case. The video stream is the video captured by the camera from the time the display case door opens to the time it closes; An extraction unit is used to extract multi-scale features from multiple target video frames in the video stream; wherein the multi-scale features include multiple image features with successively decreasing feature sizes; A determining unit is used to determine the target semantic features of each of the image features; wherein, the target semantic features are features in the image feature whose semantic strength meets the requirements in the spatial dimension; A deblurring unit is used to perform deblurring processing on each of the target video frames based on the target semantic features to obtain the deblurred video stream; It also includes an image deblurring network, which comprises a first feature extraction network and a second feature extraction network. Each network in the first feature extraction network is defined as a first network, and each network in the second feature extraction network is defined as a second network. The first feature extraction network comprises multiple first networks connected in sequence, and each first network includes multiple max pooling networks and multiple first convolutional networks. The second feature extraction network comprises multiple second networks connected in sequence, and each second network includes multiple second convolutional networks. Each first convolutional network in the first network is connected to a corresponding second convolutional network, and each max pooling network in the first network is connected to a corresponding second convolutional network. The number of second convolutional networks is equal to the sum of the number of first convolutional networks and max pooling networks. Among the multiple second convolutional networks, the second convolutional network connected to the first convolutional network is located before the second convolutional network connected to the max pooling network. The target video frames are processed sequentially by the max pooling network in the first feature extraction network to obtain the pooling processing result. The extraction unit is further configured to: extract multi-scale features of each target video frame in the video stream through the first feature extraction network in the image deblurring network, wherein each of the first convolutional networks includes a convolutional layer and a downsampling layer; The determining unit is further configured to: extract target semantic features of each image feature through the target attention network in the image deblurring network, wherein the target attention network contains multiple attention layers, and each convolutional layer in the first feature extraction network is provided with one attention layer. The deblurring unit is further configured to: process the target semantic features output by the last first convolutional network in the plurality of first networks and input them into the second feature extraction network, and input the target semantic features output by each max pooling network and each first convolutional network in each of the first networks into the corresponding second network; The input data is processed by each of the second networks to obtain deblurred video frames, thereby obtaining the deblurred video stream. The input data of the second network includes: the target semantic features output by the first network corresponding to the second network and the output data of the upper layer network adjacent to the second network in the image deblurring network.

9. A display case, characterized in that, include: The display case includes a cabinet body, a camera, and a processor; wherein the camera is mounted on the display case body and is communicatively connected to the processor. The camera is configured to capture a video stream of the interior of the display case from the time the case door is opened until it is closed. The processor is configured to acquire a video stream captured by a camera in a display case; the video stream is the video captured by the camera from the opening to the closing of the display case door; extract multi-scale features of multiple target video frames in the video stream; wherein the multi-scale features include multiple image features with successively decreasing feature sizes; determine the target semantic features of each image feature; wherein the target semantic features are features whose semantic strength meets requirements in the spatial dimension; perform deblurring processing on each target video frame based on the target semantic features to obtain the deblurred video stream; and further includes an image deblurring network, the network including a first feature The system comprises a first feature extraction network and a second feature extraction network. Each network in the first feature extraction network is defined as a first network, and each network in the second feature extraction network is defined as a second network. The first feature extraction network includes multiple first networks connected in sequence, and each first network contains multiple max pooling networks and multiple first convolutional networks. The second feature extraction network includes multiple second networks connected in sequence, and each second network contains multiple second convolutional networks. Each first convolutional network in the first network is connected to one second convolutional network, and each max pooling network in the first network is connected to one second convolutional network. The number of second convolutional networks is equal to the number of first convolutional networks and max pooling networks. The number of max pooling networks; among multiple second convolutional networks, the second convolutional network connecting the first convolutional network is located before the second convolutional network connecting the max pooling network; the target video frames are processed sequentially through the max pooling network in the first feature extraction network to obtain the pooling processing result; the extraction of multi-scale features of multiple target video frames in the video stream includes: extracting multi-scale features of each target video frame in the video stream through the first feature extraction network in the image deblurring network, wherein each first convolutional network contains a convolutional layer and a downsampling layer; the determination of the target semantic features of each image feature includes: through the image deblurring network The target attention network in the first feature extraction network extracts target semantic features for each image feature. The target attention network contains multiple attention layers, and each convolutional layer in the first feature extraction network corresponds to one attention layer. The step of deblurring each target video frame based on the target semantic features to obtain the deblurred video stream includes: processing the target semantic features output by the last first convolutional network in the multiple first networks and then inputting them into the second feature extraction network; and inputting the target semantic features output by each max pooling network and each first convolutional network in each of the first networks into the corresponding second network.The input data is processed by each of the second networks to obtain deblurred video frames, thus obtaining the deblurred video stream. The input data of the second network includes: the target semantic features output by the first network corresponding to that second network and the output data of the upper-layer network adjacent to the second network in the image deblurring network.

10. A display case, characterized in that, include: The processor, memory, and bus are provided, wherein the memory stores machine-readable instructions executable by the processor, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the video processing method as described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the video processing method as described in any one of claims 1 to 7.

Images