Image processing method, device, equipment and storage medium

Abstract

The present invention discloses an image processing method, device, equipment and storage medium. The image processing method is executed by a camera, and the method comprises: obtaining a threshold parameter and an unlocking signal; obtaining a first image set according to the unlocking signal, and filtering the first image set according to the threshold parameter to obtain a target image set. Through the technical solution of the present invention, the first image set is filtered using the threshold parameter to obtain the target image set, which can reduce invalid video clips, reduce the size of video files, and thus reduce the traffic consumption of transmitting video files.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] In recent years, with the development of digital economy such as mobile payment and mobile Internet, self-service vending machines can be seen everywhere in subway stations, supermarkets, office buildings and other places.

[0003] The workflow of the vending machine is as follows: the user scans the QR code on the vending machine to unlock the cabinet, and the camera of the vending machine records the user's shopping behavior. When the user closes the door and locks the cabinet after shopping, the camera stops recording. During this recording process, a large number of invalid video clips are generated because the user fails to open the door in time when unlocking the door, fails to shop effectively after opening the door, and fails to close the door in time after shopping, which leads to large video files and excessive traffic consumption. Summary of the invention

[0004] The embodiments of the present invention provide an image processing method, apparatus, device and storage medium, which can reduce invalid video clips, reduce the size of video files, and further reduce the traffic consumption of transmitting video files.

[0005] According to one aspect of the present invention, there is provided an image processing method, which is performed by a camera, and the method comprises:

[0006] Get threshold parameters and unlocking signal;

[0007] A first image set is acquired according to the unlocking signal, and the first image set is screened according to a threshold parameter to obtain a target image set.

[0008] According to another aspect of the present invention, there is provided an image processing method, which is executed by a controller, and the method includes:

[0009] Get threshold parameters and unlocking signal;

[0010] The threshold parameter and the unlocking signal are sent to the camera, so that the camera acquires a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set.

[0011] According to another aspect of the present invention, there is provided an image processing device, the image processing device comprising:

[0012] A first acquisition module, used to acquire a threshold parameter and an unlocking signal;

[0013] The obtaining module is used to obtain a first image set according to the unlocking signal, and filter the first image set according to a threshold parameter to obtain a target image set.

[0014] According to another aspect of the present invention, there is provided an image processing device, the image processing device comprising:

[0015] A second acquisition module, used to acquire a threshold parameter and an unlocking signal;

[0016] The first sending module is used to send the threshold parameter and the unlocking signal to the camera, so that the camera obtains a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

[0022] The embodiment of the present invention obtains a threshold parameter and an unlocking signal; obtains a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set, thereby solving the problem of a large number of invalid video clips caused by the user's failure to open the door in time when unlocking, failure to shop in time and effectively after opening the door, and failure to close the door in time after shopping, which in turn leads to large video files and excessive traffic consumption. Using the threshold parameter to filter the first image set to obtain the target image set can reduce invalid video clips, reduce the size of video files, and thus reduce the traffic consumption of transmitting video files.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of an image processing method in Embodiment 1 of the present invention;

[0026] Figure 2 is a flow chart of an image processing method in Embodiment 2 of the present invention;

[0027] Figure 3 This is a flow chart of obtaining a video file in Embodiment 2 of the present invention;

[0028] Figure 4 is a structural schematic diagram of an image processing device in Embodiment 3 of the present invention;

[0029] Figure 5 is a structural schematic diagram of an image processing device in Embodiment 4 of the present invention;

[0030] Figure 6 It is a structural schematic diagram of an electronic device in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] Embodiment 1

[0035] Figure 1 is a flow chart of an image processing method in the first embodiment of the present invention. This embodiment is applicable to the case where a user records a video while shopping at a self-service vending machine. The method can be executed by the image processing device in the third embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method is executed by a camera and specifically includes the following steps:

[0036] S110, obtaining a threshold parameter and an unlocking signal.

[0037] The threshold parameter is a preset threshold parameter, which may include at least one of a door opening confidence threshold, a shopping behavior confidence threshold, and a time threshold. The threshold parameter may be stored in a controller and sent to the camera by the controller. The unlocking signal is a signal for unlocking the cabinet after the user scans the QR code at the vending machine.

[0038] Specifically, the method for obtaining the threshold parameters and the unlocking signal can be: preset the threshold parameters in the controller, the controller sends the threshold parameters to the camera, the user scans the QR code to open the cabinet lock, the controller recognizes the unlocking signal, and sends the unlocking signal to the camera, then the camera can obtain the threshold parameters and the unlocking signal.

[0039] S120, acquiring a first image set according to the unlocking signal, and screening the first image set according to a threshold parameter to obtain a target image set.

[0040] The first image set is a set of all images from the time when the user unlocks the door to the time when the user exceeds a preset time threshold and forcibly ends shopping, or a set of all images from the time when the user unlocks the door to the time when the user closes the door after shopping, or a set of all images from the time when the user unlocks the door to the time when the door opening confidence is detected to be less than the door opening confidence threshold. The target image set may be all images in the first image set that satisfy both the preset first condition and the preset second condition, wherein the preset first condition may be a door opening valid image, and the preset second condition may be a coding valid image.

[0041] Specifically, the first image set is obtained according to the unlocking signal, and the first image set is filtered according to the threshold parameters, and the method for obtaining the target image set can be: after the camera obtains the unlocking signal sent by the controller, it starts image capture, and the first image set can be obtained according to the unlocking signal and the door closing signal of the user's shopping end, and according to the preset threshold parameters, such as the door opening confidence threshold and the shopping behavior confidence threshold, the preset conditions are set, and the first image set is filtered according to the preset conditions, and all images that meet the preset conditions are determined as the target image set. The first image set is obtained according to the unlocking signal, and the first image set is filtered according to the threshold parameters, and the method for obtaining the target image set can also be: the first image set is obtained according to the unlocking signal and the preset time threshold, and the first image set is filtered according to the preset conditions, and all images that meet the preset conditions are determined as the target image set. A first image set is obtained according to an unlocking signal, and the first image set is filtered according to a threshold parameter. A method for obtaining a target image set can also be: after the camera obtains the unlocking signal, image capture is turned on. If an image with a door opening confidence greater than or equal to the door opening confidence threshold is captured, it means that the door has been unlocked and opened. If an image with a door opening confidence less than the door opening confidence threshold is captured after the door is unlocked and opened, it means that the door has been closed. In this way, a first image set can be obtained according to the unlocking signal and an image with a door opening confidence less than the door opening confidence threshold is captured after the door is unlocked and opened, and the first image set is filtered according to preset conditions, and all images that meet the preset conditions are determined as the target image set.

[0042] It should be noted that the image capture can be performed using YUV image capture. The YUV image is the original image stream before encoding and compression on the camera side, and the image algorithm processing is performed on the original YUV image before compression.

[0043] Optionally, acquiring a first image set according to the unlocking signal includes:

[0044] Get the heartbeat status;

[0045] If the heartbeat state is normal, a door closing signal is acquired, and a first image set is acquired according to the unlocking signal and the door closing signal.

[0046] The heartbeat status can be determined based on the heartbeat monitoring threads in the camera and the controller, and both the camera and the controller can monitor the heartbeat status. A cycle time is set. If the controller and the camera cannot communicate within the cycle time, the heartbeat status is determined to be abnormal. If the controller and the camera can communicate normally, the heartbeat status is determined to be normal. It should be noted that the heartbeat monitoring thread is carried out in real time, that is, the heartbeat status may change during the user's shopping process.

[0047] Specifically, the heartbeat status may be obtained by determining the heartbeat status according to the communication status of the heartbeat monitoring thread set between the camera and the controller.

[0048] Specifically, if the heartbeat status is normal, a door closing signal is obtained, and the method for obtaining the first image set according to the unlocking signal and the door closing signal can be: if the user's heartbeat status remains normal during the shopping process, the controller can send a door closing signal to the camera after obtaining the door closing information. When the camera obtains the door closing signal, it stops capturing images, and the camera obtains the first image set from the unlocking signal to the door closing signal according to the unlocking signal and the door closing signal.

[0049] By acquiring the heartbeat status and acquiring the door closing signal when the heartbeat status is normal, the accuracy of the door closing signal can be improved, and invalid signals can be avoided, thereby improving the accuracy of the first image set acquired according to the unlocking signal and the door closing signal.

[0050] Optionally, the threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold;

[0051] The first image set is screened according to a threshold parameter to obtain a target image set, including:

[0052] Obtaining a door opening confidence corresponding to each frame of the first image set;

[0053] The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set;

[0054] Obtaining a shopping behavior confidence corresponding to each frame of the second image set;

[0055] The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

[0056] Among them, the door opening confidence threshold and the shopping behavior confidence threshold are both preset threshold parameters. If the door opening confidence of the image is greater than or equal to the door opening confidence threshold, it means that the door is opened. If the door opening confidence of the image is less than the door opening confidence threshold, it means that the door is not opened or has been closed. If the shopping behavior confidence of the image is greater than or equal to the shopping behavior confidence threshold, it means that effective shopping behavior has occurred. If the shopping behavior confidence of the image is less than the shopping behavior confidence threshold, it means that no effective shopping behavior has occurred.

[0057] The second image set is a set generated from all valid door opening images in the first image set.

[0058] Specifically, the method for obtaining the door opening confidence corresponding to each frame of the first image set can be: performing a door opening and closing algorithm detection on each frame of the first image set to obtain the door opening confidence of each frame of the first image set. The door opening and closing algorithm detection is an important part of the image elapse technology, which is used to accurately determine whether the door is open.

[0059] Specifically, the first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set, and the second image set is obtained by obtaining the door opening confidence of each frame of the image in the first image set, and generating the second image set with images whose door opening confidence is greater than or equal to the door opening confidence threshold.

[0060] Specifically, the shopping behavior confidence corresponding to each frame in the second image set can be obtained by performing shopping behavior algorithm detection on each frame in the second image set to obtain the shopping behavior confidence of each frame in the second image set. Shopping behavior algorithm detection is also an important part of image elapse technology, which mainly judges whether effective shopping behavior occurs based on image features such as human skin, arms, and motion trajectories and a deep learning framework.

[0061] Specifically, the second image set is screened according to the shopping behavior confidence and the shopping behavior confidence threshold corresponding to each frame image in the second image set, and the target image set is obtained by obtaining the shopping behavior confidence of each frame image in the second image set, and generating the target image set with images having a shopping behavior confidence greater than or equal to the shopping behavior confidence threshold.

[0062] Optionally, the first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain the second image set, including:

[0063] Determine an image in the first image set whose door opening confidence is greater than or equal to the door opening confidence threshold as a door opening valid image;

[0064] An image whose door opening confidence is less than a door opening confidence threshold is determined as an invalid door opening image;

[0065] If there are a preset number of consecutive invalid door opening images, the first invalid door opening image is determined as a valid door opening image;

[0066] A second image set is generated according to the door opening valid image.

[0067] The valid door opening image is a valid image after the door is opened in the first image set, and the invalid door opening image is an invalid image without the door being opened in the first image set. The preset number is the number of consecutive invalid door opening images preset according to actual conditions, for example, the preset number can be 3.

[0068] Specifically, the method for determining images in the first image set whose door opening confidence is greater than or equal to the door opening confidence threshold as valid door opening images can be: if the door opening confidence of the image is greater than or equal to the door opening confidence threshold, it means that the door has been opened after the user unlocks the door. At this time, the image captured by the camera is a valid door opening image, so the image in the first image set whose door opening confidence is greater than or equal to the door opening confidence threshold is determined as a valid door opening image.

[0069] Specifically, the method of determining an image whose door opening confidence is less than the door opening confidence threshold as an invalid door opening image may be: determining an image whose door opening confidence is less than the door opening confidence threshold in the first image set as an invalid door opening image.

[0070] Specifically, if there are a preset number of consecutive invalid door opening images, the first invalid door opening image is determined as the valid door opening image in the following manner: if there are a preset number of consecutive invalid door opening images in the first image set, the first invalid door opening image among the preset number of consecutive invalid door opening images is determined as the valid door opening image; if there are invalid door opening images and there are no preset number of consecutive invalid door opening images, the invalid door opening images are discarded. For example, the preset number may be 3, and if there are 3 consecutive invalid door opening images, the first invalid door opening image among the three invalid door opening images is determined as the valid door opening image.

[0071] Specifically, the method of generating the second image set according to the door-opening valid images may be: generating the second image set from all the door-opening valid images in the first image set.

[0072] It should be noted that in the actual shopping process, after scanning the code to unlock the door, the user may not open the door to shop immediately, that is, the actual door opening action will be delayed. At the same time, there is an inherent delay in the camera in the traditional recording method. In order to capture the complete shopping action image, the recording is generally started 2 to 3 seconds before unlocking. The above situation will cause the camera to always record invalid images, resulting in a larger video file. By judging whether the image captured by the camera is a valid door opening image through the door opening confidence threshold in the embodiment of the present invention, invalid door opening images can be effectively reduced, thereby reducing the size of the video file.

[0073] Optionally, the second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set, including:

[0074] Determine an image in the second image set whose shopping behavior confidence is greater than or equal to a shopping behavior confidence threshold as a coding valid image;

[0075] Determine an image whose shopping behavior confidence is less than a shopping behavior confidence threshold as an encoding invalid image;

[0076] If there are a preset number of consecutive coding invalid images, the first coding invalid image is determined as a coding valid image;

[0077] Generate a set of target images based on the encoded valid images.

[0078] The coded valid image is a valid shopping behavior image in the second image set, and the coded invalid image is an invalid shopping behavior image in the second image set.

[0079] Specifically, the method for determining an image in the second image set whose shopping behavior confidence is greater than or equal to the shopping behavior confidence threshold as a valid encoding image can be: if the shopping behavior confidence of the image is greater than or equal to the shopping behavior confidence threshold, it means that the user is performing a valid shopping behavior. At this time, the door opening valid image is also a valid shopping behavior image, that is, a valid encoding image. Therefore, the image in the second image set whose shopping behavior confidence is greater than or equal to the shopping behavior confidence threshold is determined as a valid encoding image.

[0080] Specifically, the method of determining an image whose shopping behavior confidence is less than the shopping behavior confidence threshold as an invalid encoding image may be: determining an image whose shopping behavior confidence is less than the shopping behavior confidence threshold in the second image set as an invalid encoding image.

[0081] Specifically, if there are a continuous preset number of invalid coding images, the first invalid coding image is determined as a valid coding image in the following manner: if there are a continuous preset number of invalid coding images in the second image set, the first invalid coding image among the continuous preset number is determined as a valid coding image; if there are invalid coding images and there are not a continuous preset number of invalid coding images, the invalid coding images are discarded. For example, the preset number may be 3, and if there are 3 consecutive invalid coding images, the first invalid coding image among the three invalid coding images is determined as a valid coding image.

[0082] Specifically, the method of generating the target image set according to the coding effective images may be: generating the target image set from all the coding effective images in the second image set.

[0083] It should be noted that in the actual shopping process, after the user scans the code to unlock the door, it is possible that the user has been observing the product but has not made any effective shopping behavior. During this period of time, the camera has been recording invalid images, which leads to a large video file. By using the shopping behavior confidence threshold in the embodiment of the present invention to determine whether the image captured by the camera is a valid encoding image, it is possible to effectively reduce the number of invalid encoding images and thus reduce the size of the video file.

[0084] By determining the first invalid door opening image as a valid door opening image if there are a preset number of consecutive invalid door opening images, and determining the first invalid encoding image as a valid encoding image if there are a preset number of consecutive invalid encoding images, it can effectively prevent uneven timestamps of image frames during the encoding process and frame skipping during playback of video files.

[0085] Optionally, after the first image set is screened according to the threshold parameter to obtain the target image set, the method further includes:

[0086] Obtain a first timestamp corresponding to the unlocking signal and a second timestamp corresponding to the closing signal;

[0087] Determine a total duration of a first shopping trip according to the first timestamp and the second timestamp;

[0088] Get the number of images in the target image collection;

[0089] Determining a first video recording frame rate according to the number of images and the total duration of the first shopping trip;

[0090] Encoding each frame of the target image set to obtain a first target code stream;

[0091] The first target code stream is transcoded into a first target video file according to a first video frame rate.

[0092] The first timestamp is the timestamp corresponding to the unlock signal, and the second timestamp is the timestamp corresponding to the door closing signal. The first total shopping duration is the duration between the first timestamp and the second timestamp. The number of images is the number of all images in the target image set, the first video frame rate is the frame rate of the first target video file, the first target code stream is the cached code stream after encoding each frame of the target image set, and the first target video file is the video file converted from the cached code stream obtained according to the unlock signal and the door closing signal.

[0093] Specifically, the method for obtaining the first timestamp corresponding to the unlocking signal and the second timestamp corresponding to the door closing signal can be: if the heartbeat status remains normal, the camera can receive the unlocking signal sent by the controller, the first timestamp corresponding to the unlocking signal, the door closing signal indicating that the user has finished shopping, and the second timestamp corresponding to the door closing signal.

[0094] Specifically, the method for determining the total duration of the first shopping according to the first timestamp and the second timestamp can be: the duration between the first timestamp and the second timestamp is determined as the total duration of the first shopping, that is, the total duration of the first shopping is equal to the second timestamp minus the first timestamp.

[0095] Specifically, the method for obtaining the number of images in the target image set may be: the number of images in the target image set may be obtained by using a counter or other counting tools.

[0096] Specifically, the method for determining the first video recording frame rate according to the number of images and the total duration of the first shopping can be: the formula for calculating the first video recording frame rate according to the number of images in the target image set and the total duration of the first shopping can be recorded as: first video recording frame rate = number of images in the target image set / total duration of the first shopping.

[0097] Specifically, each frame of the target image set is encoded to obtain the first target code stream in a manner that: each frame of the target image set can be encoded, and the code stream is cached during the encoding process to obtain the first target code stream. For example, each frame of the target image set can be encoded with H264 and the code stream is cached.

[0098] Specifically, the method of transcoding the first target code stream into the first target video file according to the first video frame rate can be: converting the encoded first target code stream into the first target video file according to the first video frame rate, for example, converting the cached H264 code stream into an mp4 video file according to the first video frame rate.

[0099] The first shopping duration is determined by the first timestamp corresponding to the unlocking signal and the second timestamp corresponding to the door closing signal. The images in the target image set at this time are images captured by the camera during the first shopping duration, which are both valid images for opening the door and valid images for encoding. Each frame of the image in the target image set is encoded and the first target code stream is cached, and then the first target code stream is converted into a first target video file. This can reduce the possibility of video anomalies, directly discard invalid images during the period when the user closes the door and locks the door, and thus reduce the size of the video file.

[0100] Optionally, also include:

[0101] If the heartbeat state is abnormal, the second total shopping time is obtained, and the first image set is obtained according to the unlocking signal and the second total shopping time.

[0102] Among them, the abnormal state is the state when the camera and the controller cannot communicate normally, that is, the state when the network fails. It should be noted that the heartbeat state is monitored in real time by the controller and the camera. If the heartbeat state is abnormal when the user scans the QR code, the shopping cabinet will be locked and the after-sales service will be notified; if the heartbeat state becomes abnormal during the shopping process after the user has unlocked the door, the camera can calculate the current shopping duration based on the timestamp of the current image captured, and determine whether the shopping is over based on the current shopping duration and the threshold parameters. Among them, the second total shopping duration is the duration between the timestamp corresponding to the current image captured by the camera and the first timestamp corresponding to the unlocking signal when the heartbeat state is abnormal after the user unlocks the door.

[0103] Specifically, if the heartbeat status is abnormal, the second total shopping time is obtained, and the method for obtaining the first image set according to the unlocking signal and the second total shopping time can be: if the heartbeat status is abnormal after the user unlocks the door, the camera calculates the second total shopping time according to the timestamp corresponding to the current image captured and the first timestamp corresponding to the unlocking signal, and obtains the first image set during the second total shopping time.

[0104] Optionally, the threshold parameters include: a door opening confidence threshold, a shopping behavior confidence threshold, and a time threshold.

[0105] Acquiring the second shopping duration, and acquiring the first image set according to the unlocking signal and the second shopping total duration, including:

[0106] Get the current image and the current timestamp;

[0107] Obtaining the second total shopping duration according to the first timestamp corresponding to the unlocking signal and the current timestamp;

[0108] If the second total shopping time is greater than or equal to the time threshold, a first image set is acquired according to the first timestamp and the second total shopping time.

[0109] Among them, the current timestamp is the timestamp corresponding to the current image. The time threshold is a preset threshold parameter, which is a preset timeout threshold for user shopping. If the total duration of the second shopping is greater than or equal to the time threshold, it means that the time spent by the user during shopping has reached the time threshold, and the shopping is forced to end. It should be noted that when the user's shopping exceeds the time threshold, the system will sound a prompt to prompt the user to complete the shopping as soon as possible.

[0110] Specifically, the current image and the current timestamp may be obtained in the following manner: a camera may capture the current image and the timestamp corresponding to the current image.

[0111] Specifically, the second shopping total time can be obtained according to the first timestamp corresponding to the unlocking signal and the current timestamp: the second shopping total time is the difference between the current timestamp and the first timestamp, that is, the second shopping total time = current timestamp - first timestamp.

[0112] Specifically, if the second total shopping time is greater than or equal to the time threshold, the method for obtaining the first image set according to the first timestamp and the second total shopping time can be: if the calculated second total shopping time is greater than or equal to the time threshold, it means that the user's shopping time has reached the time threshold, and the shopping and image capture are forced to end. According to the first timestamp and the second total shopping time, the camera captures the first image set during the second total shopping time. It should be noted that if the total duration of the second shopping trip obtained according to the first timestamp and the current timestamp is less than the time threshold, it is necessary to start the door opening and closing algorithm detection for the current image captured by the camera. If the door opening confidence of the current image is greater than or equal to the door opening confidence threshold, the camera continues to acquire images until the camera detects an image (first end image) whose door opening confidence is less than the door opening confidence threshold or detects that the difference between the timestamp corresponding to the image (second end image) and the first timestamp is greater than or equal to the time threshold, then the camera stops capturing images and updates the total duration of the second shopping trip according to the timestamp corresponding to the first end image and the first timestamp corresponding to the unlocking signal, or updates the total duration of the second shopping trip according to the timestamp corresponding to the second end image and the first timestamp corresponding to the unlocking signal; if the door opening confidence of the current image is less than the door opening confidence threshold, it is determined that the current closed-door shopping is over, and image capture stops. The total duration of the second shopping trip is the difference between the current timestamp and the first timestamp corresponding to the unlocking signal.

[0113] Optionally, after the first image set is screened according to the threshold parameter to obtain the target image set, the method further includes:

[0114] Get the number of images in the target image collection;

[0115] Determining a second video recording frame rate according to the number of images and the second total shopping duration;

[0116] Encoding each frame of the target image set to obtain a second target code stream;

[0117] The second target code stream is transcoded into a second target video file according to a second video frame rate.

[0118] Among them, the second recording frame rate is the frame rate of the second target recording file, the second target code stream is a cached code stream obtained by obtaining a first image set according to the first timestamp and the second total shopping duration, filtering the first image set according to a threshold parameter, and encoding each frame image in the obtained target image set, and the second target recording file is a recording file converted from the cached code stream obtained according to the unlocking signal and the second total shopping duration.

[0119] Specifically, the method for obtaining the number of images in the target image set can be: the number of images in the target image set can be obtained by using a counter or other counting tools. It should be noted that the target image set here is a first image set obtained based on the first timestamp and the second total shopping duration, and the first image set is screened to obtain the target image set.

[0120] Specifically, the method for determining the second recording frame rate according to the number of images and the second total shopping duration can be: the formula for calculating the second recording frame rate according to the number of images and the second total shopping duration can be recorded as: second recording frame rate = number of images in the target image set / second total shopping duration.

[0121] Specifically, each frame of the target image set is encoded to obtain the second target code stream in the following manner: each frame of the target image set may be encoded, and code streams may be cached during the encoding process to obtain the second target code stream.

[0122] Specifically, the method of transcoding the second target code stream into the second target video file according to the second video frame rate may be: converting the second target code stream obtained by encoding into the second target video file according to the second video frame rate.

[0123] By obtaining the second total shopping time when the heartbeat status is abnormal, and obtaining the first image set based on the unlocking signal and the second total shopping time, the integrity of the video can be guaranteed even when the network is unstable, solving the problems of traditional video such as screen distortion and frame loss, thereby reducing traffic consumption and improving video stability.

[0124] The technical solution of this embodiment obtains a threshold parameter and an unlocking signal; obtains a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set, which solves the problem of a large number of invalid video clips caused by the user's failure to open the door in time when unlocking, failure to shop in time and effectively after opening the door, and failure to close the door in time after shopping, which in turn leads to large video files and excessive traffic consumption. Using the threshold parameter to filter the first image set to obtain the target image set can reduce invalid video clips, reduce the size of video files, and thus reduce the traffic consumption of transmitting video files.

[0125] Embodiment 2

[0126] Figure 2 is a flow chart of an image processing method in the second embodiment of the present invention. This embodiment is applicable to the case where a user records a video while shopping at a self-service vending machine. The method can be executed by the image processing device in the fourth embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 2 As shown, the method is executed by a controller and specifically includes the following steps:

[0127] S210, obtaining a threshold parameter and an unlocking signal;

[0128] It should be noted that the controller can be the Android main control terminal of the self-service vending machine, or the main controller of other self-service machines. The controller and the camera can have built-in http services, and the camera can provide capability interfaces to the outside world. For example, the camera can provide capability interfaces such as querying firmware versions, obtaining threshold parameters, obtaining video files, synchronizing time, and heartbeat monitoring, and the controller can integrate the interface capabilities in the camera.

[0129] Specifically, the method for obtaining the threshold parameter and the unlocking signal may be: the threshold parameter may be preset inside the controller, and when the user scans the QR code on the vending machine to open the cabinet lock, the controller obtains the unlocking signal.

[0130] S220, sending the threshold parameter and the unlocking signal to the camera, so that the camera acquires a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set.

[0131] Specifically, the threshold parameters and the unlocking signal are sent to the camera so that the camera obtains a first image set according to the unlocking signal, and filters the first image set according to the threshold parameters. The method for obtaining the target image set can be: after obtaining the threshold parameters and the unlocking signal, the controller can send the threshold parameters and the unlocking signal to the camera, and the camera can obtain the first image set according to the unlocking signal, and filter the first image set according to the threshold parameters to obtain the target image set.

[0132] Optionally, also include:

[0133] Get the heartbeat status;

[0134] If the heartbeat state is normal, a door closing signal is acquired and sent to the camera, so that the camera acquires a first image set according to the unlocking signal and the door closing signal.

[0135] Specifically, the way to obtain the heartbeat status is: both the controller and the camera have built-in heartbeat monitoring threads, set a cycle time, the controller sends heartbeat data to the camera within the cycle time, and the camera responds after receiving it. The heartbeat status at this time is normal. If the communication between the controller and the camera is abnormal, the heartbeat status is abnormal.

[0136] Specifically, if the heartbeat state is normal, a door closing signal is obtained, and the door closing signal is sent to the camera, so that the camera can obtain the first image set according to the unlocking signal and the door closing signal. If the heartbeat state remains normal, the controller obtains the door closing signal when the user closes the door, and sends the door closing signal to the camera, and the camera can obtain the first image set according to the unlocking signal and the door closing signal. It should be noted that if the heartbeat state is abnormal when the user scans the QR code, the shopping cabinet is directly locked and the after-sales service is notified; if the heartbeat state becomes abnormal during the shopping process after the user has unlocked the door, the controller can send an alarm to the background to notify the after-sales staff to manually take out the cached video file for order settlement.

[0137] Optionally, if the heartbeat status remains normal, the controller may send a video acquisition command to the camera, so that the camera sends the video file to the controller.

[0138] In a specific embodiment, Figure 3 This is a flow chart of obtaining a video file in Embodiment 2 of the present invention. Figure 3As shown in the figure, the camera can be an IP camera of a certain intelligent shopping cabinet, and the controller can be the Android main control terminal in this intelligent shopping cabinet. Among them, the IP camera integrates an HTTP server and realizes the defined external interface capabilities. The main control terminal integrates an HTTP client and realizes the corresponding protocol interface capabilities. After the Android main control terminal completes initialization, it can use the HTTP protocol to send threshold parameters to the IP camera. For example, the threshold parameters can include: the door opening confidence threshold QTH1, the shopping behavior confidence threshold QTH2, and the time threshold T. The IP camera and the Android main control terminal respectively start heartbeat monitoring threads. The main control terminal can send heartbeat data to the IP camera every 3 seconds. The IP camera responds to this heartbeat after receiving it. Both ends set a variable heart_flg, which is set to true when the heartbeat communication is normal and set to false when the heartbeat communication is abnormal. If a communication abnormality occurs between the main control terminal and the IP camera, both sides will detect a heartbeat abnormality and set the heart_flg variable respectively; when the user scans the code to unlock the door, if heart_flg == true, the main control terminal will send the unlock signal, the shopping event id, and the first timestamp t_sys1 corresponding to the unlock signal to the IP camera. If heart_flg == false, the unlock fails and the after-sales management personnel are notified to handle it; after the IP camera receives the unlock signal, the shopping event id, and the first timestamp corresponding to the unlock signal sent by the main control terminal, it starts to capture YUV images, and can record the acquisition timestamp t_yuv of each frame of YUV image. The IP camera starts the door opening and closing algorithm detection to obtain the door opening confidence of the acquired image. When the door opening confidence >= QTH1, it is determined that the door is open, and the current image is set as a valid door opening image. When the door opening confidence < QTH1, the current image is set as an invalid door opening image, and the invalid door opening image counter m is incremented. When m >= 3, it is cleared and the image is reset as a valid door opening image, otherwise the image is directly discarded; for the valid door opening images, the shopping behavior algorithm detection is started to obtain the shopping behavior confidence of the valid door opening images. When the shopping behavior confidence >= QTH2, it is determined that the shopping is valid, and this valid door opening image is set as a valid encoded image. When the shopping behavior confidence < QTH2, it is determined that the shopping is invalid, and this valid door opening image is set as an invalid encoded image, and the invalid encoded image counter n is incremented. When n >= 3, it is cleared and the image is reset as a valid encoded image, otherwise the image is directly discarded; at the same time, the valid encoded images are encoded in H264, the code stream is cached, and the valid encoded image counter frm_valid is incremented;When the master control terminal receives the door closing signal, it sends the door closing signal and the second timestamp t_sys2 corresponding to the door closing signal to the IP camera. At this time, the shopping ends. Calculate the first total shopping duration m_elapse1 = t_sys2 - t_sys1. The IP camera stops algorithm detection. According to the first total shopping duration m_elapse1 and the encoded valid image counter frm_valid++, calculate the video frame rate fps1 = frm_valid / m_elapse1. Transcode the H.264 video stream into an MP4 video file at this frame rate, and name the video file with the shopping event id and the first timestamp t_sys1. The master control terminal sends a command to obtain the video. The IP camera sends the video file to the master control terminal. If when the IP camera captures a YUV image, it detects that heart_flg == false, it is determined that the IP camera and the master control terminal are disconnected during the shopping process. Calculate the second total shopping duration m_elapse2 = t_yuv - t_sys1 using the timestamp t_yuv of the currently captured image. When the opening confidence of the currently captured image < QTH1, or the second total shopping duration >= the time threshold T, force the shopping to end. The IP camera stops algorithm detection. According to the second total shopping duration m_elapse2 and the encoded valid image counter frm_valid++, calculate the video frame rate fps2 = frm_valid / m_elapse2. Transcode the H.264 video stream into an MP4 video file at this frame rate, and name the video file with the shopping event id and the timestamp t_yuv. The master control terminal issues an abnormal order alarm and disables the vending machine, and notifies the after-sales personnel to manually obtain the order video corresponding to the timestamp at the IP camera end according to the timestamp; otherwise, continue to capture images until the opening confidence of the currently captured image < QTH1, or the second total shopping duration >= the time threshold T.;

[0139] The technical solution of this embodiment solves the problem of a large number of invalid video segments generated due to the user's failure to open the door in time when unlocking, failure to shop effectively in time after opening the door, and failure to close the door in time after shopping ends, resulting in a large video file size and excessive traffic consumption. By obtaining the threshold parameter and the unlocking signal and sending them to the camera, the camera can obtain the first image set according to the unlocking signal and filter the first image set according to the threshold parameter to obtain the target image set. Filtering the first image set with the threshold parameter to obtain the target image set can reduce invalid video segments, reduce the size of the video file, and thus reduce the traffic consumption for transmitting the video file.

[0140] Embodiment III

[0141] Figure 4is a schematic diagram of the structure of an image processing device in the third embodiment of the present invention. This embodiment is applicable to the case where a user records a video while shopping at a self-service vending machine. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides image processing functions, such as Figure 4 As shown, the image processing device specifically includes: a first acquisition module 310 and a obtaining module 320.

[0142] The first acquisition module 310 is used to acquire a threshold parameter and an unlocking signal;

[0143] The obtaining module 320 is used to obtain a first image set according to the unlocking signal, and filter the first image set according to a threshold parameter to obtain a target image set.

[0144] Optionally, the obtaining module is specifically used for:

[0145] Get the heartbeat status;

[0146] If the heartbeat state is normal, a door closing signal is acquired, and a first image set is acquired according to the unlocking signal and the door closing signal.

[0147] Optionally, the threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold;

[0148] The obtaining module is specifically used for:

[0149] Obtaining a door opening confidence corresponding to each frame of the first image set;

[0150] The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set;

[0151] Obtaining a shopping behavior confidence corresponding to each frame of the second image set;

[0152] The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

[0153] Optionally, the obtaining module is specifically used for:

[0154] Determine an image in the first image set whose door opening confidence is greater than or equal to the door opening confidence threshold as a door opening valid image;

[0155] An image whose door opening confidence is less than a door opening confidence threshold is determined as an invalid door opening image;

[0156] If there are a preset number of consecutive invalid door opening images, the first invalid door opening image is determined as a valid door opening image;

[0157] A second image set is generated according to the door opening valid image.

[0158] Optionally, the obtaining module is specifically used for:

[0159] Determine an image in the second image set whose shopping behavior confidence is greater than or equal to a shopping behavior confidence threshold as a coding valid image;

[0160] Determine an image whose shopping behavior confidence is less than a shopping behavior confidence threshold as an encoding invalid image;

[0161] If there are a continuous preset number of coding invalid images, the first coding invalid image is determined as a coding valid image;

[0162] Generate a set of target images based on the encoded valid images.

[0163] Optionally, the obtaining module is further used for:

[0164] Obtain a first timestamp corresponding to the unlocking signal and a second timestamp corresponding to the closing signal;

[0165] Determine a total duration of a first shopping trip according to the first timestamp and the second timestamp;

[0166] Get the number of images in the target image collection;

[0167] Determining a first video recording frame rate according to the number of images and the total duration of the first shopping trip;

[0168] Encoding each frame of the target image set to obtain a first target code stream;

[0169] The first target code stream is transcoded into a first target video file according to a first video frame rate.

[0170] Optionally, the obtaining module is further used for:

[0171] If the heartbeat state is abnormal, the second total shopping time is obtained, and the first image set is obtained according to the unlocking signal and the second total shopping time.

[0172] Optionally, the threshold parameters include: a door opening confidence threshold, a shopping behavior confidence threshold, and a time threshold.

[0173] The obtaining module is also used for:

[0174] Get the current image and the current timestamp;

[0175] Obtaining the second total shopping duration according to the first timestamp corresponding to the unlocking signal and the current timestamp;

[0176] If the second total shopping time is greater than or equal to the time threshold, a first image set is acquired according to the first timestamp and the second total shopping time.

[0177] Optionally, the obtaining module is further used for:

[0178] Get the number of images in the target image set;

[0179] Determining a second video recording frame rate according to the number of images and the second total shopping duration;

[0180] Encoding each frame of the target image set to obtain a second target code stream;

[0181] The second target code stream is transcoded into a second target video file according to a second video frame rate.

[0182] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0183] Embodiment 4

[0184] Figure 5 is a schematic diagram of the structure of an image processing device in the fourth embodiment of the present invention. This embodiment is applicable to the case where a user records a video while shopping at a self-service vending machine. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides image processing functions, such as Figure 5 As shown, the image processing device specifically includes: a second acquisition module 410 and a first sending module 420.

[0185] The second acquisition module 410 is used to acquire the threshold parameter and the unlocking signal;

[0186] The first sending module 420 is used to send the threshold parameter and the unlocking signal to the camera, so that the camera obtains a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set.

[0187] Optionally, also include:

[0188] The third acquisition module is used to obtain the heartbeat status;

[0189] The second sending module is used to obtain a door closing signal if the heartbeat state is normal, and send the door closing signal to the camera, so that the camera obtains a first image set according to the unlocking signal and the door closing signal.

[0190] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0191] Embodiment 5

[0192] Figure 6 : is a structural diagram of an electronic device in Embodiment 5 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0193] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0194] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0195] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image processing method.

[0196] In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0197] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0199] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0201] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0202] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0203] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0204] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: Executed by a camera, the method includes: Get the threshold parameters and unlocking signal sent by the controller; Acquire a first image set according to the unlocking signal, and filter the first image set according to a threshold parameter to obtain a target image set; The acquiring a first image set according to the unlocking signal comprises: Get the heartbeat status; If the heartbeat state is normal, a door closing signal is obtained, and a first image set is obtained according to the unlocking signal and the door closing signal; The threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold; The step of screening the first image set according to the threshold parameter to obtain a target image set includes: Obtaining a door opening confidence corresponding to each frame of the first image set; The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set; Obtaining a shopping behavior confidence corresponding to each frame of the second image set; The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

2. The method according to claim 1, characterized in that The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set, including: Determine an image in the first image set whose door opening confidence is greater than or equal to the door opening confidence threshold as a door opening valid image; An image whose door opening confidence is less than a door opening confidence threshold is determined as an invalid door opening image; If there are a preset number of consecutive invalid door opening images, the first invalid door opening image is determined as a valid door opening image; A second image set is generated according to the door opening valid image.

3. The method according to claim 1, characterized in that The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set, including: Determine an image in the second image set whose shopping behavior confidence is greater than or equal to a shopping behavior confidence threshold as a coding valid image; Determine an image whose shopping behavior confidence is less than a shopping behavior confidence threshold as an encoding invalid image; If there are a continuous preset number of coding invalid images, the first coding invalid image is determined as a coding valid image; Generate a set of target images based on the encoded valid images.

4. The method according to claim 1, characterized in that: After the first image set is screened according to the threshold parameter to obtain the target image set, the method further includes: Obtain a first timestamp corresponding to the unlocking signal and a second timestamp corresponding to the closing signal; Determine a total duration of a first shopping trip according to the first timestamp and the second timestamp; Get the number of images in the target image collection; Determining a first video recording frame rate according to the number of images and the total duration of the first shopping trip; Encoding each frame of the target image set to obtain a first target code stream; The first target code stream is transcoded into a first target video file according to a first video frame rate.

5. The method according to claim 1, characterized in that Also includes: If the heartbeat state is abnormal, the second total shopping time is obtained, and the first image set is obtained according to the unlocking signal and the second total shopping time.

6. The method according to claim 5, characterized in that The threshold parameters include: door opening confidence threshold, shopping behavior confidence threshold and time threshold; Acquiring the second shopping duration, and acquiring the first image set according to the unlocking signal and the second shopping total duration, including: Get the current image and the current timestamp; Obtaining the second total shopping duration according to the first timestamp corresponding to the unlocking signal and the current timestamp; If the second total shopping time is greater than or equal to the time threshold, a first image set is acquired according to the first timestamp and the second total shopping time.

7. The method according to claim 6, characterized in that After the first image set is screened according to the threshold parameter to obtain the target image set, the method further includes: Get the number of images in the target image collection; Determining a second video recording frame rate according to the number of images and the second total shopping duration; Encoding each frame of the target image set to obtain a second target code stream; The second target code stream is transcoded into a second target video file according to a second video frame rate.

8. An image processing method, characterized in that: Executed by a controller, the method includes: Get threshold parameters and unlocking signal; Sending the threshold parameter and the unlocking signal to a camera, so that the camera acquires a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set; The method further comprises: Get the heartbeat status; If the heartbeat state is normal, a door closing signal is obtained, and the door closing signal is sent to the camera, so that the camera obtains a first image set according to the unlocking signal and the door closing signal; The threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold; The step of screening the first image set according to the threshold parameter to obtain a target image set includes: Obtaining a door opening confidence corresponding to each frame of the first image set; The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set; Obtaining a shopping behavior confidence corresponding to each frame of the second image set; The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

9. An image processing device, characterized in that: The image processing device comprises: A first acquisition module, used to acquire a threshold parameter and an unlocking signal sent by a controller; an obtaining module, configured to obtain a first image set according to the unlocking signal, and filter the first image set according to a threshold parameter to obtain a target image set; The obtaining module is specifically used for: Get the heartbeat status; If the heartbeat state is normal, a door closing signal is obtained, and a first image set is obtained according to the unlocking signal and the door closing signal; The threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold; The obtaining module is specifically used for: Obtaining a door opening confidence corresponding to each frame of the first image set; The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set; Obtaining a shopping behavior confidence corresponding to each frame of the second image set; The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

10. An image processing device, characterized in that: The image processing device comprises: A second acquisition module, used to acquire a threshold parameter and an unlocking signal; A first sending module, configured to send the threshold parameter and the unlocking signal to a camera, so that the camera acquires a first image set according to the unlocking signal, and filters the first image set according to the threshold parameter to obtain a target image set; The image processing device further comprises: The third acquisition module is used to obtain the heartbeat status; A second sending module is used to obtain a door closing signal if the heartbeat state is normal, and send the door closing signal to the camera, so that the camera obtains a first image set according to the unlocking signal and the door closing signal; The threshold parameters include: a door opening confidence threshold and a shopping behavior confidence threshold; The step of screening the first image set according to the threshold parameter to obtain a target image set includes: Obtaining a door opening confidence corresponding to each frame of the first image set; The first image set is screened according to the door opening confidence and the door opening confidence threshold corresponding to each frame of the image in the first image set to obtain a second image set; Obtaining a shopping behavior confidence corresponding to each frame of the second image set; The second image set is screened according to the shopping behavior confidence corresponding to each frame image in the second image set and the shopping behavior confidence threshold to obtain a target image set.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image processing method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 8 when executed.

Images