Water drinking behavior determination method and device, electronic equipment and storage medium

Abstract

The invention discloses a water drinking behavior determination method and device, electronic equipment and a storage medium, and relates to the computer technology. The method comprises the following steps: determining an initial frame image and a target frame image according to a video stream; determining an initial liquid volume and an initial liquid level area from the initial frame image, and determining a current liquid level area from the target frame image; and determining a liquid volume change rate according to the initial liquid volume, the initial liquid level area and the current liquid level area, and determining whether the user drinks water according to the liquid volume change rate. According to the method, whether the user drinks water or not can be determined only by analyzing the liquid level change conditions in the initial frame image and the target frame image, the body features or facial features of the user do not need to be recognized, and the beneficial effect of protecting the privacy of the user is achieved while the scheme implementation process is simplified; and the embodiment does not need to perform water drinking behavior analysis on each frame of image in the video stream in real time, so that the beneficial effects of saving the system overhead and reducing the scheme implementation cost are achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for determining drinking behavior. Background Art

[0002] With the improvement of vehicle intelligence, monitoring user behavior while driving and providing in-vehicle linkage services based on the behavior monitoring results is an effective way to improve user driving experience.

[0003] Existing behavior monitoring systems mainly rely on behavior recognition technology to determine whether the user performs specific activities while driving. Taking the identification of the user's drinking behavior as an example, existing solutions generally rely on detecting the user's physical and facial features to determine the drinking behavior. For example, when it is detected that the user's hand key points leave the steering wheel and the lip key points are blocked, the drinking detection model is called to determine whether the image contains water bottle features based on the drinking detection model. If the water bottle features are contained, it is determined that the user has drunk water.

[0004] The above solution, which relies on identifying the user's physical and facial features to determine drinking behavior, has the problem of easily infringing the user's privacy; and the implementation logic of the above solution is relatively complex, and the implementation cost of the solution is relatively high. Summary of the invention

[0005] The present application provides a drinking behavior determination method, device, electronic device and storage medium, which can simplify the solution implementation process while protecting user privacy.

[0006] In a first aspect, the present application provides a method for determining drinking behavior, comprising:

[0007] An initial frame image and a target frame image are determined according to the video stream, wherein the initial frame image is the first frame image in the video stream that represents the possible drinking behavior of the user, and the target frame image is the last frame image in the video stream that represents the possible drinking behavior of the user; an initial liquid volume and an initial liquid surface area are determined from the initial frame image, and a current liquid surface area is determined from the target frame image; a liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate.

[0008] Optionally, determining the initial frame image and the target frame image based on the video stream includes: obtaining the liquid surface angles in several consecutive frame images starting from the current frame image in the video stream; when it is detected that the liquid surface angle is in an increasing state, determining that the current frame image is the initial frame image; when it is detected that the liquid surface angle is in a decreasing state, determining that the current frame image is the target frame image.

[0009] Optionally, determining the initial liquid volume and the initial liquid surface area from the initial frame image includes: inputting the initial frame image into an image segmentation model to obtain a target segmentation map, wherein the target segmentation map includes a liquid area and a bottle body area; determining the initial liquid volume according to a liquid radius and a liquid height of the liquid area in the bottle body area; and determining the initial liquid surface area according to a pixel area corresponding to the liquid area.

[0010] Optionally, determining the liquid volume change rate based on the initial liquid volume, the initial liquid surface area and the current liquid surface area includes: determining a liquid surface change coefficient based on the initial liquid volume and the initial liquid surface area; determining the current liquid volume based on the current liquid surface area and the liquid surface change coefficient; and determining the liquid volume change rate based on the current liquid volume and the initial liquid volume.

[0011] Optionally, determining whether the user has drunk water based on the liquid volume change rate includes: determining a target bottle mouth type based on the initial frame image; determining a target rate interval from a preset mapping table based on the target bottle mouth type, wherein one bottle mouth type corresponds to one rate interval in the preset mapping table; determining that the user has drunk water when the liquid volume change rate is within the target rate interval; and determining that the user has not drunk water when the liquid volume change rate is not within the target rate interval.

[0012] Optionally, before determining whether the user has drunk water according to the liquid volume change rate, it also includes: for the preset frame image before the target frame image, obtaining the liquid surface inclination direction and the liquid surface inclination angle in each frame image; when the liquid surface inclination direction is the preset direction and the liquid surface inclination angle is within the preset angle range, performing the operation of determining whether the user has drunk water according to the liquid volume change rate; when the liquid surface inclination direction is not the preset direction, or the liquid surface inclination angle is not within the preset angle range, it is determined that the user does not have the drinking behavior.

[0013] Optionally, after determining that the user has drunk water, it also includes: determining at least one target hardware information to be linked; calling at least one target hardware device corresponding to the target hardware information to provide the user with a corresponding vehicle-mounted linkage service based on at least one target hardware device.

[0014] In a second aspect, the present application provides a drinking behavior determination device, the device comprising:

[0015] An image determination module, used to determine an initial frame image and a target frame image according to a video stream, wherein the initial frame image is the first frame image in the video stream indicating that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream indicating that the user may have a drinking behavior;

[0016] a parameter determination module, used to determine an initial liquid volume and an initial liquid surface area from the initial frame image, and to determine a current liquid surface area from the target frame image;

[0017] The behavior determination module is used to determine the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and determine whether the user has a drinking behavior according to the liquid volume change rate.

[0018] In a third aspect, the present application further provides an electronic device, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drinking behavior determination method described in any embodiment of the present application.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining drinking behavior described in any embodiment of the present application when executed.

[0023] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining drinking behavior described in any embodiment of the present application.

[0024] The drinking behavior determination scheme provided in the embodiment of the present application first determines the initial frame image and the target frame image according to the video stream, wherein the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior; then the initial liquid volume and the initial liquid surface area are determined from the initial frame image, and the current liquid surface area is determined from the target frame image; finally, the liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area, and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate. This embodiment can determine whether the user has a drinking behavior by simply analyzing the liquid surface changes in the initial frame image and the target frame image, without identifying the user's physical features or facial features, simplifying the implementation process of the scheme while achieving the beneficial effect of protecting the user's privacy; and this embodiment does not need to perform drinking behavior analysis on each frame image in the video stream in real time, achieving the beneficial effect of saving system overhead and reducing the implementation cost of the scheme.

[0025] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the drinking behavior determination device, or may be packaged separately from the processor of the drinking behavior determination device, and this application does not limit this.

[0026] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0028] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 is a flow chart of a method for determining drinking behavior provided in an embodiment of the present application;

[0031] Figure 2 is another flow chart of the method for determining drinking behavior provided in an embodiment of the present application;

[0032] Figure 3 is a structural schematic diagram of a drinking behavior determination device provided in an embodiment of the present application;

[0033] Figure 4 It is a structural schematic diagram of an electronic device for a method for determining drinking behavior provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the present application will be clearly and completely described below in conjunction with the drawings in the present embodiment. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0037] Figure 1This is a flow chart of a method for determining drinking behavior provided in an embodiment of the present application. This embodiment is applicable to the case where a user's drinking behavior is determined while the user is driving a vehicle. The method can be executed by a drinking behavior determination device, which can be implemented in the form of hardware and / or software and integrated into an electronic device that executes the method. Preferably, the electronic device in the embodiment of the present application can be a server, or a computer device, etc.

[0038] refer to Figure 1 The drinking behavior determination method of this embodiment includes but is not limited to the following steps:

[0039] S110 . Determine an initial frame image and a target frame image according to the video stream.

[0040] The video stream represents the video data received by the server during the driving process of the vehicle. The server may receive the video stream by installing a high-resolution camera at an appropriate position inside the vehicle (such as the A-pillar or the steering wheel) to capture the image in the cockpit. In this embodiment, the camera is preferably able to capture the position of the drinking water container in the cockpit. Generally, the position of the drinking water container is the middle position between the main driving position and the co-pilot position in the cockpit, which helps to analyze whether the user has drinking behavior based on the captured liquid level changes in the drinking water container. The camera should be configured to continuously record the video stream. When the drinking water container is captured to be moved, the video stream captured by the camera is sent to the server, so that the received video stream is processed at the server, at least one frame of image is extracted from the video stream, and then at least one frame of image is analyzed to determine whether there is drinking behavior in the current video stream.

[0041] It should be noted that this embodiment analyzes the changes in the liquid level in the drinking container captured in the video stream to determine whether the user has a drinking behavior. Therefore, the drinking container analyzed in this embodiment is a transparent water cup or water bottle with a clearly visible liquid level, and the specific type of drinking container is not limited here.

[0042] In this embodiment, the initial frame image is the first frame image in the video stream that indicates that the user may drink water, that is, when the initial frame image that meets the relevant requirements is detected, it indicates that the user may drink water next. The initial frame image can be determined by detecting a frame image when the drinking container is moved to the first preset position. The current first preset position can be the driver's chest position, indicating that the drinking container is picked up and the user may drink water next. Therefore, the current frame image can be determined as the initial frame image; it can also be a frame image when it is detected that the drinking container is at a first preset angle with a horizontal line parallel to the ground. The current first preset angle can be 10° or 15°, etc., indicating that the drinking container is tilted and the user may drink water next. Therefore, the current frame image can be determined as the initial frame image; it can also be obtained by obtaining the liquid surface angle in a number of continuous frame images starting from the current frame image in the video stream. The current liquid surface angle indicates the angle between the liquid surface in the drinking container and the parallel to the ground. When the liquid surface angle is detected to be in an increasing state, it indicates that the drinking container has a trajectory state from being picked up to tilting. Therefore, the current frame image can be determined as the initial frame image. The specific method of determining the initial frame image from the video stream is not limited here.

[0043] In this embodiment, the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior, that is, when the target frame image that meets the relevant requirements is detected, it indicates that the user's drinking behavior may have ended. The target frame image may be determined by detecting the frame image when the drinking container is moved to the second preset position. The current second preset position may be lower than the first preset position, and the second time sequence corresponding to the frame image at the second preset position is greater than the first time sequence corresponding to the first preset position. That is, when the drinking container is first detected at the first preset position and then detected at the second preset position, it indicates that the drinking container has a tendency to be put back, and the user may end the drinking behavior next. Therefore, the current frame image may be determined as the target frame image; it may also be the frame image when the drinking container is detected at the second preset angle to the horizontal line parallel to the ground. Frame image, the current second preset angle is lower than the first preset angle, when it is detected that the first preset angle decreases to the second preset angle, it indicates that the drinking container has a tendency to be put back, and the user may end the drinking behavior next, so the current frame image can be determined as the target frame image; it can also be to obtain the liquid surface angle in several consecutive frame images starting from the current frame image in the video stream, the current liquid surface angle indicates the angle between the liquid surface in the drinking container and the ground, when it is detected that the liquid surface angle is decreasing, it indicates that the drinking container has a trajectory state from being tilted to being put back, so the current frame image can be determined as the target frame image. The specific method of determining the target frame image from the video stream is not limited here.

[0044] In this embodiment, if the initial frame image is determined in the video stream, but the target frame image cannot be determined based on the above rules, it indicates that the current user does not have any driving behavior and there is no need to perform subsequent steps. The reason why the target frame image cannot be determined may be that the driver picked up the drinking water container and handed it to the passenger in the car.

[0045] The purpose of determining the initial frame image and the target frame image in this embodiment is that only the intermediate frame image between the initial frame image and the target frame image needs to be analyzed to determine whether the user has a drinking behavior. There is no need to execute the drinking behavior determination scheme provided by this embodiment on all frame images in the video stream, thereby achieving the beneficial effect of saving system overhead and reducing the cost of implementing the scheme.

[0046] Optionally, the drinking behavior determination scheme provided in this embodiment is mainly used to determine the driver's drinking behavior. Under normal circumstances, there are also cases where passengers in the car drink water. To avoid misidentifying the passengers' drinking behavior as the driver's drinking behavior, this embodiment can pre-mark the driving area and passenger area in the cabin, so that after determining the initial frame image and the target frame image according to the video stream, when it is further determined that the initial frame image and the target frame image are images corresponding to the driving area, the subsequent step S120 is executed.

[0047] S120, determining an initial liquid volume and an initial liquid surface area from the initial frame image, and determining a current liquid surface area from the target frame image.

[0048] The initial liquid volume and the initial liquid surface area represent, in the current video stream, the liquid volume and the liquid surface area corresponding to the current liquid in the drinking container when the user triggers the drinking behavior.

[0049] Among them, in this embodiment, the method for determining the initial liquid volume from the initial frame image can be that, since the initial frame image is the image corresponding to the triggering of the drinking behavior, in the initial frame image, the drinking container may be in a vertical state or in a tilted state. In order to improve the accuracy of the initial liquid volume calculation, in this embodiment, the initial frame image is preferably determined as an image of moving the drinking container to the first preset position. According to the user's drinking habits, the drinking container is in a vertical state at this time. Then, the shape of the liquid in the drinking container in this state is close to the shape of the drinking container. The conventional drinking container is cylindrical, so the initial liquid volume can be obtained by calculating the volume calculation formula of the cylinder.

[0050] The method for determining the initial liquid surface area can be that, if the shape of the liquid in the container in the initial frame image is relatively regular, then the initial liquid surface area can be obtained by calculating the surface area corresponding to the liquid shape; or the area of ​​the pixel points corresponding to the liquid in the container can be calculated, etc. The specific method for determining the initial liquid surface area is not limited here.

[0051] Accordingly, when determining the current liquid surface area from the target frame image, since the liquid may be tilted to a large extent and the liquid shape may be irregular, the calculation result may be inaccurate when the current liquid surface area is calculated directly by the shape of the liquid in the container. Therefore, it is preferred to obtain the current liquid surface area by calculating the area of ​​the corresponding pixel points of the liquid in the container in the target frame image. The method of calculating the initial liquid surface area is consistent with the method of calculating the current liquid surface area.

[0052] S130, determining the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and determining whether the user has a drinking behavior according to the liquid volume change rate.

[0053] The liquid volume change rate is used to indicate the changing relationship between the liquid volume and the liquid area in the drinking container. In this embodiment, it is used to indicate the liquid change rate, that is, when the user drinks water, it indicates how fast the user drinks water. The current liquid volume change rate can be indicated by a specific change value.

[0054] Generally, the method of determining whether the user has a drinking behavior based on the liquid volume change rate is to pre-set a rate change threshold, and if the current liquid volume change rate is within the rate change threshold, it indicates that the user has a drinking behavior; if the current liquid volume change rate is not within the rate change threshold, there is no drinking behavior. The rate change threshold is a threshold corresponding to the normal human drinking rate.

[0055] Among them, the above-mentioned method of determining the liquid volume change rate based on the initial liquid volume, the initial liquid surface area and the current liquid surface area can be, first, determining the change coefficient based on the initial liquid volume and the initial liquid surface area in the initial frame image, the current change coefficient is the relationship coefficient between volume and area, and then the current liquid surface volume in the target frame image can be calculated based on the change coefficient and the current liquid surface area, and finally the liquid volume change rate can be obtained through the initial liquid volume and the current liquid surface volume. Thus, it is possible to determine whether the user has a drinking behavior based on the liquid volume change rate. The drinking behavior determination solution provided in this embodiment only realizes the determination of driving behavior by identifying the liquid parameters (liquid volume and liquid area) of the liquid in the container, without being affected by the external light environment and different driving postures of the user. The direct detection of the liquid surface parameters ensures the objectivity and accuracy of the monitoring results.

[0056] The drinking behavior determination method provided in the embodiment of the present application first determines the initial frame image and the target frame image according to the video stream, wherein the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior; then the initial liquid volume and the initial liquid surface area are determined from the initial frame image, and the current liquid surface area is determined from the target frame image; finally, the liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area, and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate. This embodiment can determine whether the user has a drinking behavior by simply analyzing the liquid surface changes in the initial frame image and the target frame image, without identifying the user's physical features or facial features, simplifying the implementation process of the solution while achieving the beneficial effect of protecting the user's privacy; and this embodiment does not need to perform drinking behavior analysis on each frame image in the video stream in real time, achieving the beneficial effect of saving system overhead and reducing the implementation cost of the solution.

[0057] Figure 2 It is another flow chart of the method for determining drinking behavior provided in an embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiments. Specifically, in this embodiment, the related processes of "determining the initial frame image and the target frame image according to the video stream", "obtaining the initial liquid volume and the initial liquid surface area from the initial frame image", "determining the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area" and "determining whether the user has drinking behavior according to the liquid volume change rate" are explained in detail.

[0058] See also Figure 2 The drinking behavior determination method provided in this embodiment includes but is not limited to the following steps:

[0059] S210: Determine an initial frame image and a target frame image according to the video stream.

[0060] In one implementation, the method of determining the initial frame image and the target frame image according to the video stream in this embodiment can be implemented by the following steps:

[0061] a) Obtain the liquid surface angles in a number of consecutive frame images starting from the current frame image in the video stream.

[0062] The current frame image may be the frame image that is arranged first in sequence in the video stream, or may be any frame image in the video stream.

[0063] In this embodiment, the purpose of analyzing the liquid level angles in several consecutive frame images starting from the current frame image in the video stream is to analyze the angle changes of the liquid level angles in several consecutive frame images one by one, so as to determine whether the drinking water specifications are met according to the angle changes, and further determine the initial frame image and the target frame image if they are met, so as to determine whether the user has a drinking behavior according to the changes in the liquid level parameters in the drinking water container in the initial frame image and the target frame image.

[0064] Among them, the above-mentioned drinking water specification is obtained by the change of the liquid surface angle in the drinking water container when the user actually drinks water. Generally, when the drinking water container is placed in the car, it is usually placed vertically. At this time, the liquid surface angle is horizontal relative to the ground or the bottom of the water bottle, and the liquid surface angle is 0 degrees; when the user starts to pick up the drinking water container, the bottom of the container will tilt forward or upward, which causes the liquid surface to begin to tilt relative to the bottom of the container, and the liquid surface angle will begin to increase from 0 degrees, indicating that the liquid begins to tilt to one side of the container; as the container further tilts enough to allow the liquid to flow into the mouth, the liquid surface angle reaches the maximum value, and the current liquid surface angle is approximately 30° to 60°, and the specific angle depends on the design of the container and the driver's drinking habits; after the user finishes drinking water, the drinking water container will be put back in place. At this time, the drinking water container gradually returns to a more vertical position, and then the liquid surface angle will also gradually decrease accordingly, and finally to 0 degrees, to return to a horizontal state, etc.

[0065] Optionally, the method for obtaining the liquid surface angle in each frame of the image in this embodiment can be to identify the drinking water container in each frame of the image through traditional image recognition technology. Furthermore, since the liquid surface in the drinking water container is clearly visible in each frame of the image, the liquid surface in the drinking water container can be segmented based on image segmentation technology, and then the liquid surface angle in each frame of the image can be obtained according to the angle of the liquid surface relative to the ground.

[0066] b) When it is detected that the liquid surface angle is in an increasing state, the current frame image is determined to be the initial frame image.

[0067] Taking the first frame image in the video stream as the current frame image as an example, the liquid surface angle of the drinking container in the first frame image is obtained, and then the several consecutive frame images after the first frame image are analyzed in sequence to obtain the liquid surface angles corresponding to the several consecutive frame images. When it is detected that the liquid surface angle is in an increasing state, the first frame image can be determined to be the initial frame image. If the several consecutive frame images after the first frame image are not in an increasing state, then the judgment operation of whether the liquid surface angle in the several consecutive frame images after the next frame is in an increasing state is performed. In this embodiment, the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior.

[0068] Conventionally, a 1-second video stream may contain 25 frames of images, and the current number of consecutive frame images may be 10 frames, 15 frames, or 20 frames, etc. The specific number of frame images is not limited here.

[0069] c) When it is detected that the liquid surface angle is in a decreasing state, the current frame image is determined to be the target frame image.

[0070] When a decreasing state is detected, it indicates that the user has a tendency to put the drinking container back. Then, when the liquid surface angles in a number of consecutive frames starting from the current frame image are detected to be decreasing, the current frame image is determined as the target frame image. In this embodiment, the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior.

[0071] When the target frame image can be determined, it indicates that there is no need to execute the drinking behavior determination scheme of this embodiment on all frame images after the target frame image in the video stream, so as to further reduce computing power and release system resources.

[0072] S220: Input the initial frame image into the image segmentation model to obtain a target segmentation map, where the target segmentation map includes a liquid area and a bottle body area.

[0073] In this embodiment, the liquid level parameters of the liquid in the drinking water container are analyzed based on the image segmentation model. The page parameters in this embodiment include but are not limited to liquid volume, liquid area, liquid height, liquid radius, etc.

[0074] Specifically, the image segmentation model of this embodiment can be trained in the following ways: first, a large number of images containing liquids in different types of transparent containers are collected, and these images should include different liquid surface heights and inclination angles; second, these images are accurately annotated to distinguish between containers, liquids and other background parts, and these annotated data will be used to train the model; then a suitable deep learning model is selected for semantic segmentation, such as this embodiment can be implemented based on the instance segmentation algorithm (Mask R-CNN); finally, the annotated image data set is used to train the model so that it can recognize and distinguish between drinking water containers and liquids in the image. The training process should include adjusting model parameters and optimizing the segmentation accuracy of liquids. The image segmentation model training is completed until the final model output result reaches a certain convergence accuracy.

[0075] In this embodiment, after the initial frame image is input into the image segmentation model, a target segmentation map with the liquid area and the bottle area marked is obtained as output by the model. The liquid area and the bottle area can be marked and distinguished using different colors, such as marking the liquid area in yellow and the bottle area in red, etc. The specific marking method is not limited here. The purpose of marking the liquid area and the bottle area in this embodiment is to facilitate the accurate calculation of the liquid surface volume and liquid surface area in subsequent steps.

[0076] S221. Determine an initial liquid volume according to a liquid radius and a liquid height of the liquid area in the bottle body area.

[0077] In this embodiment, the initial liquid volume can be obtained by the following formula:

[0078] V=πr 2 h (1)

[0079] In formula (1), V represents the initial liquid volume, r represents the liquid radius, which is equivalent to the radius of the drinking water container, and h is equivalent to the liquid height. The liquid radius and liquid height can be obtained based on the actual data measured in the initial image frame; or they can be obtained based on the pixel height corresponding to the liquid area and the bottle area obtained based on the image segmentation model; the specific measurement method of the liquid radius and liquid height is not limited here.

[0080] S222. Determine the initial liquid surface area according to the pixel area corresponding to the liquid area.

[0081] The method of determining the initial liquid surface area according to the pixel area corresponding to the liquid area can be to count the total number of pixels in the liquid area, and then obtain the initial liquid surface area according to the product of the total number of pixels and the area of ​​a single pixel. The size of the single pixel area is related to the image resolution in the obtained video frame image. The image resolution was previously 300 dots per inch (DPI), that is, there are 300 pixels in 1 inch, and there are 300*300=90,000 pixels in 1 square inch.

[0082] S230, obtaining the current liquid surface area from the target frame image.

[0083] In this embodiment, the method for obtaining the current liquid surface area from the target frame image is as follows: after the initial target frame image is input into the image segmentation model, a segmentation map output by the model with the liquid area and the bottle body area marked thereon is obtained; and then the current liquid surface area is determined based on the pixel area corresponding to the current liquid area in the segmentation map.

[0084] S240. Determine a liquid level variation coefficient according to the initial liquid volume and the initial liquid surface area.

[0085] The liquid level variation coefficient is used to represent the correlation between the liquid volume and the liquid area in the drinking water container in the image. The liquid level variation coefficient can be specifically determined by dividing the initial liquid volume by the initial liquid surface area to represent the liquid level variation coefficient.

[0086] S241. Determine the current liquid volume according to the current liquid surface area and the liquid surface variation coefficient.

[0087] Since the liquid surface in the drinking water container in the target frame image has a certain liquid surface angle, it is not convenient to obtain the current liquid volume in the target frame image by mathematical calculation or pixel point calculation. In this embodiment, for the target frame image, since the drinking water container has not changed, the current liquid volume can be obtained by calculating the product of the current liquid surface area and the liquid surface change coefficient.

[0088] S242. Determine the liquid volume change rate according to the current liquid volume and the initial liquid volume.

[0089] When the user drinks water, the current liquid volume must be smaller than the initial liquid volume. The liquid volume change rate can be calculated by dividing the current liquid volume by the initial liquid volume, and the liquid volume change rate is less than 1.

[0090] S250 . For the preset frame images before the target frame image, obtain the liquid surface inclination direction and the liquid surface inclination angle in each frame image.

[0091] In order to further improve the accuracy of determining drinking behavior, this embodiment, based on obtaining the liquid volume change rate, further obtains the liquid surface inclination direction and liquid surface inclination angle of the preset frame image before the target frame image, so as to determine whether it meets the relevant characteristics of drinking behavior.

[0092] When drinking water, the liquid surface should be tilted towards the user, which can be determined by the camera installation position combined with the coordinate quadrant to which the liquid surface belongs. When the driver's lips are taken as the coordinate origin, the direction perpendicular to the ground is taken as the ordinate, and the direction parallel to the ground is taken as the abscissa, then the liquid surface tilt direction should be in the first quadrant.

[0093] The liquid surface inclination angle indicates the angle between the liquid surface and the horizontal line, which can be obtained according to the coordinates of the two end points of the liquid surface in the container, wherein the coordinates of the two end points in the container include the first coordinate and the second coordinate of the liquid surface, wherein the first coordinate can be the contact coordinate between the container and the lips when the user drinks water, and the second coordinate can be the coordinate corresponding to the bottom of the liquid surface in the container. The liquid surface inclination angle can be calculated by:

[0094]

[0095] Among them, the above (x1, y1) and (x2, y2) represent the first coordinate and the second coordinate respectively.

[0096] Since the target frame image is the first frame image when the drinking behavior ends, the preset frame image before the target frame image is the image corresponding to when the user has the drinking behavior. Among them, the current preset frame image can be a continuous 5-frame image or 10-frame image, etc., and the specific number of preset frame images is not limited here.

[0097] Optionally, for the preset frame image before the target frame image, this embodiment can also provide a basis for determining the drinking behavior, such as using the Euclidean distance formula to determine the liquid surface length according to the first coordinate and the second coordinate, so as to determine whether to execute the subsequent steps based on the change in the liquid surface length recorded in the preset frame image. If the user has a drinking behavior, the liquid surface length will usually increase as the container tilts, and then gradually decrease as the liquid surface drops. By monitoring the change in the liquid surface length in the container, the expansion and contraction of the liquid surface length when the container changes from static to tilted can be monitored. The change in the liquid surface length can directly reflect the movement of the liquid and is an important parameter for determining the driver's drinking behavior.

[0098] S251. Determine whether the liquid surface inclination direction is a preset direction, and whether the liquid surface inclination angle is within a preset angle range.

[0099] The preset direction indicates that the direction is located in the first quadrant in the coordinate system and is toward the user; the preset angle is the angle at which the user tilts the drinking container when drinking water, which can generally be 30° to 60°.

[0100] If the liquid surface inclination direction is the preset direction and the liquid surface inclination angle is within the preset angle range, that is, yes, then execute step S260; if the liquid surface inclination direction is not the preset direction, or the liquid surface inclination angle is within the preset angle range, that is, no, then execute step S271.

[0101] S260: Determine the target bottle mouth type according to the initial frame image.

[0102] Since the amount of liquid a user drinks within a certain period of time will vary depending on the bottle mouth type, the corresponding liquid volume change rate will also vary depending on the bottle mouth type.

[0103] The method of determining the target bottle mouth type from the initial frame image may be to obtain the bottle mouth diameter and then determine the target bottle mouth type based on the bottle mouth diameter.

[0104] S261. Determine a target rate interval from a preset mapping table according to the target bottleneck type.

[0105] In this embodiment, a preset mapping table is pre-set, in which one bottle mouth type corresponds to one rate interval. Among them, the current bottle mouth type may include at least a straw type (0.5 cm-1.0 cm), a squeeze bottle mouth type, a narrow bottle mouth type (diameter 1.0 cm-2.8 cm) and a wide bottle mouth type (2.8 cm and above), etc., and then the corresponding rate interval is obtained for each bottle mouth type through testing, and the bottle mouth type corresponding to the specific drinking container is not limited here.

[0106] S262: Determine whether the liquid volume change rate is within the target rate range.

[0107] If the liquid volume change rate is within the target rate range, that is, yes, indicating that the liquid volume change rate is consistent with the normal user drinking habits, then execute step S270; if the liquid volume change rate is not within the target rate range, that is, no, indicating that it is not a drinking behavior, then execute step S271.

[0108] S270: Determine whether the user drinks water.

[0109] S271. Determine whether the user has drunk water.

[0110] After determining that the user has drunk water, this embodiment is also used to realize intelligent linkage through human-computer interaction, and remind the user to drive safely on the basis of improving the user's driving experience. The specific method is to determine at least one target hardware information to be linked; call the target hardware device corresponding to the at least one target hardware information, so as to provide the user with the corresponding vehicle linkage service based on the at least one target hardware device.

[0111] Among them, the at least one target hardware information to be linked can be voice module information and recording program identification information, etc. After determining that the user has drunk water, the voice component can be called based on the voice module information for voice reminder. The current reminder method can be "It is detected that you are currently drinking water while driving, please drive carefully". After detecting that the user has finished drinking water, it can interact with the customer based on the voice component, such as "It is detected that you have finished drinking water, do you need to store the drinking record on the application?" If the user's voice is "yes", the component corresponding to the recording program identification information is called for vehicle linkage, so as to record the user's drinking time and estimated drinking amount on the application of the user's mobile terminal, as one of the auxiliary judgment factors of his health status. Specifically, in this embodiment, after determining that the user has drunk water, the vehicle linkage service that can be provided is not limited here.

[0112] The drinking behavior determination method provided in the embodiment of the present application first determines the initial frame image and the target frame image according to the video stream, wherein the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior; then the initial liquid volume and the initial liquid surface area are determined from the initial frame image, and the current liquid surface area is determined from the target frame image; finally, the liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area, and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate. This embodiment can determine whether the user has a drinking behavior by simply analyzing the liquid surface changes in the initial frame image and the target frame image, without identifying the user's physical features or facial features, simplifying the implementation process of the solution while achieving the beneficial effect of protecting the user's privacy; and this embodiment does not need to perform drinking behavior analysis on each frame image in the video stream in real time, achieving the beneficial effect of saving system overhead and reducing the implementation cost of the solution.

[0113] Figure 3 is a schematic diagram of a structure of a drinking behavior determination device provided in an embodiment of the present application, and the device is suitable for executing a drinking behavior determination method provided in an embodiment of the present application. Figure 3 As shown, the device may specifically include: an image determination module 310, a parameter determination module 320 and a behavior determination module 330, wherein:

[0114] An image determination module 310 is used to determine an initial frame image and a target frame image according to a video stream, wherein the initial frame image is the first frame image in the video stream indicating that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream indicating that the user may have a drinking behavior;

[0115] A parameter determination module 320, for determining an initial liquid volume and an initial liquid surface area from the initial frame image, and determining a current liquid surface area from the target frame image;

[0116] The behavior determination module 330 is used to determine the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and determine whether the user has a drinking behavior according to the liquid volume change rate.

[0117] The drinking behavior determination device provided in the embodiment of the present application first determines the initial frame image and the target frame image according to the video stream, wherein the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior; then the initial liquid volume and the initial liquid surface area are determined from the initial frame image, and the current liquid surface area is determined from the target frame image; finally, the liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate. This embodiment can determine whether the user has a drinking behavior by simply analyzing the liquid surface changes in the initial frame image and the target frame image, without identifying the user's physical features or facial features, simplifying the implementation process of the solution while achieving the beneficial effect of protecting the user's privacy; and this embodiment does not need to perform drinking behavior analysis on each frame image in the video stream in real time, achieving the beneficial effect of saving system overhead and reducing the implementation cost of the solution.

[0118] In one embodiment, the image determination module 310 includes an angle acquisition unit and an image determination unit, wherein:

[0119] An angle acquisition unit, used to acquire the liquid surface angles in a plurality of consecutive frame images starting from a current frame image in the video stream;

[0120] The image determination unit is used to determine the current frame image as the initial frame image when the liquid surface angle is detected to be in an increasing state; and is also used to determine the current frame image as the target frame image when the liquid surface angle is detected to be in a decreasing state.

[0121] In one embodiment, the parameter determination module 320 includes an image input unit, a first volume determination unit and an area determination unit, wherein:

[0122] An image input unit, used for inputting the initial frame image into an image segmentation model to obtain a target segmentation map, wherein the target segmentation map includes a liquid area and a bottle body area;

[0123] A first volume determination unit, configured to determine the initial liquid volume according to a liquid radius and a liquid height of the liquid region in the bottle body region;

[0124] An area determination unit is used to determine the initial liquid surface area according to the pixel area corresponding to the liquid area.

[0125] In one embodiment, the behavior determination module 330 includes a coefficient determination unit, a second volume determination unit, and a change rate determination unit, wherein:

[0126] a coefficient determination unit, configured to determine a liquid level variation coefficient according to the initial liquid volume and the initial liquid surface area;

[0127] a second volume determination unit, configured to determine a current liquid volume according to the current liquid surface area and the liquid surface variation coefficient;

[0128] A change rate determining unit is used to determine the liquid volume change rate according to the current liquid volume and the initial liquid volume.

[0129] In one embodiment, the behavior determination module 330 is specifically used to determine the target bottle mouth type according to the initial frame image; determine the target rate interval from a preset mapping table according to the target bottle mouth type, in which one bottle mouth type corresponds to one rate interval; when the liquid volume change rate is within the target rate interval, determine that the user has the drinking behavior; when the liquid volume change rate is not within the target rate interval, determine that the user does not have the drinking behavior.

[0130] In one embodiment, the behavior determination module 330 is further specifically used to obtain the liquid surface inclination direction and the liquid surface inclination angle in each frame image of the preset frame image before the target frame image; when the liquid surface inclination direction is the preset direction and the liquid surface inclination angle is within the preset angle range, an operation of determining whether the user has the drinking behavior based on the liquid volume change rate is performed; when the liquid surface inclination direction is not the preset direction, or the liquid surface inclination angle is not within the preset angle range, it is determined that the user does not have the drinking behavior.

[0131] In one embodiment, the apparatus further includes an information determination module and a device calling module, wherein:

[0132] An information determination module, used to determine at least one target hardware information to be linked;

[0133] The device calling module is used to call at least one target hardware device corresponding to the target hardware information, so as to provide the user with a corresponding vehicle linkage service based on at least one target hardware device.

[0134] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0135] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drinking behavior determination method described in any embodiment of the present application.

[0136] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining drinking behavior described in any embodiment of the present application when executed.

[0137] Reference below Figure 4 , Figure 4 1 is a schematic diagram of a structure of an electronic device for a method for determining drinking behavior provided in an embodiment of the present application, and shows a schematic diagram of a structure of a computer system 500 suitable for implementing the electronic device of the embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0138] like Figure 4 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0139] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0140] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0141] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the above.

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The modules and / or units involved in the embodiments described in the present application may be implemented by software or hardware. The modules and / or units described may also be set in a processor, for example, it may be described as: a processor includes an image determination module, a parameter determination module, and a behavior determination module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0144] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: determining an initial frame image and a target frame image according to a video stream, wherein the initial frame image is the first frame image in the video stream that represents the user's possible drinking behavior, and the target frame image is the last frame image in the video stream that represents the user's possible drinking behavior; determining an initial liquid volume and an initial liquid surface area from the initial frame image, and determining a current liquid surface area from the target frame image; determining a liquid volume change rate according to the initial liquid volume, the initial liquid surface area, and the current liquid surface area, and determining whether the user has a drinking behavior according to the liquid volume change rate.

[0145] According to the technical solution of this embodiment, it is possible to determine whether the user has drunk water simply by analyzing the changes in the liquid levels in the initial frame image and the target frame image, without identifying the user's physical or facial features. This simplifies the implementation process of the solution while achieving the beneficial effect of protecting user privacy. Moreover, this embodiment does not need to perform real-time drinking behavior analysis on each frame of the video stream, thereby achieving the beneficial effect of saving system overhead and reducing the cost of implementing the solution.

[0146] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for determining drinking behavior, characterized in that: include: Determine an initial frame image and a target frame image according to the video stream, wherein the initial frame image is the first frame image in the video stream that indicates that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream that indicates that the user may have a drinking behavior; Determining an initial liquid volume and an initial liquid surface area from the initial frame image, and determining a current liquid surface area from the target frame image; The liquid volume change rate is determined according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and whether the user has a drinking behavior is determined according to the liquid volume change rate.

2. The method for determining drinking behavior according to claim 1, characterized in that: The step of determining the initial frame image and the target frame image according to the video stream includes: Obtaining liquid surface angles in a plurality of consecutive frame images starting from a current frame image in the video stream; When it is detected that the liquid surface angle is in an increasing state, determining the current frame image as the initial frame image; When it is detected that the liquid surface angle is in a decreasing state, the current frame image is determined to be the target frame image.

3. The method for determining drinking behavior according to claim 1, characterized in that: Determining the initial liquid volume and the initial liquid surface area from the initial frame image includes: Inputting the initial frame image into an image segmentation model to obtain a target segmentation map, wherein the target segmentation map includes a liquid area and a bottle body area; Determine the initial liquid volume according to the liquid radius and liquid height of the liquid area in the bottle body area; The initial liquid surface area is determined according to the pixel area corresponding to the liquid region.

4. The method for determining drinking behavior according to claim 1, characterized in that: The determining the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area comprises: Determining a liquid level variation coefficient according to the initial liquid volume and the initial liquid surface area; Determine the current liquid volume according to the current liquid surface area and the liquid surface variation coefficient; The liquid volume change rate is determined according to the current liquid volume and the initial liquid volume.

5. The method for determining drinking behavior according to claim 1, characterized in that: The determining whether the user has a drinking behavior according to the liquid volume change rate includes: Determine the target bottle mouth type according to the initial frame image; Determine a target speed interval from a preset mapping table according to the target bottleneck type, wherein one bottleneck type corresponds to one speed interval in the preset mapping table; When the liquid volume change rate is within the target rate interval, determining that the user has the drinking behavior; When the liquid volume change rate is not within the target rate interval, it is determined that the user does not have the water drinking behavior.

6. The method for determining drinking behavior according to claim 1, characterized in that: Before determining whether the user has a drinking behavior according to the liquid volume change rate, the method further includes: For the preset frame images before the target frame image, obtaining the liquid surface inclination direction and the liquid surface inclination angle in each frame image; When the liquid surface inclination direction is a preset direction and the liquid surface inclination angle is within a preset angle range, an operation of determining whether the user has a drinking behavior according to the liquid volume change rate is performed; When the liquid surface inclination direction is not the preset direction, or the liquid surface inclination angle is not within the preset angle range, it is determined that the user does not have the drinking behavior.

7. The method for determining drinking behavior according to claim 1, characterized in that: After determining that the user has drunk water, the following steps are also included: Determine at least one target hardware information to be linked; At least one target hardware device corresponding to the target hardware information is called to provide the user with a corresponding vehicle linkage service based on the at least one target hardware device.

8. A device for determining drinking behavior, characterized in that: include: An image determination module, configured to determine an initial frame image and a target frame image according to a video stream, wherein the initial frame image is the first frame image in the video stream indicating that the user may have a drinking behavior, and the target frame image is the last frame image in the video stream indicating that the user may have a drinking behavior; A parameter determination module, used to determine an initial liquid volume and an initial liquid surface area from the initial frame image, and to determine a current liquid surface area from the target frame image; The behavior determination module is used to determine the liquid volume change rate according to the initial liquid volume, the initial liquid surface area and the current liquid surface area, and determine whether the user has a drinking behavior according to the liquid volume change rate.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the drinking behavior determination method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for determining drinking behavior as described in any one of claims 1 to 7 is implemented.

Images