Method, device, equipment and medium for generating electronic fence based on scene adaptation
By using scene adaptive electronic fence generation method in infant care technology, real-time video data and pre-trained models determine the scene type and fence location, the problem of low adaptability and accuracy of electronic fence generation in the prior art is solved. Achieve more efficient and safe infant and toddler monitoring.
Patent Information
- Application Number
- CN202411080715.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-08
AI Technical Summary
Among the existing infant and child care technologies, electronic fence generation has weak adaptability and low accuracy, and it is impossible to effectively ensure the safety of infants and young children in various scenarios.
Using an electronic fence generation method based on scene adaptation, by obtaining real-time video data, using pre-trained object detection model and scene recognition large model, determining the scene type of infants and young children, and processing images according to the preset fence positioning strategy, dynamically Adjust the position and range of the electronic fence.
Improve the adaptability and accuracy of electronic fences, reduce the dependence on manual monitoring, and improve the efficiency and safety of care, especially when caregivers cannot monitor them at full time.
Smart Images

Figure CN118609035B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of infant care, and in particular to a scene-adaptive electronic fence generation method, device, equipment and medium. Background Art
[0002] With the development of society, family structure and parenting methods have changed significantly. More and more parents are busy with work and cannot monitor the safety of infants and young children around the clock. In this case, the application of technology in infant and young child safety monitoring becomes particularly important.
[0003] Although traditional physical fences can provide protection to a certain extent, their fixedness and dependence on the environment make them inconvenient and limited. Therefore, the development of an adaptive and highly accurate automated virtual fence generation system has become an urgent need in the field of infant safety monitoring. At present, there are some technologies and equipment for monitoring infant safety on the market. For example, video surveillance systems can monitor infant activities in real time, but their disadvantage is that parents or guardians need to pay attention to the screen at all times, and they cannot achieve true automated monitoring. In addition, although smart bracelets and positioning devices can provide positioning and activity monitoring functions, their coverage is limited and they cannot dynamically adjust specific areas.
[0004] The existing Chinese patent CN116416291A discloses an automatic generation method of an electronic fence, a real-time detection method and a device, and specifically discloses the following technical solutions: obtaining an environmental image, which is obtained by shooting a scene in which a user is located; generating spatial structure information according to the environmental image, wherein the spatial structure information includes information of a 3D point cloud and at least one plane, wherein the plane is determined by the distribution of points of the 3D point cloud on the plane; generating an electronic fence according to the spatial structure information; constructing an optimization equation according to the 3D point cloud and the information of the plane; optimizing the plane according to the optimization equation to obtain an optimized plane; determining a reference plane according to the optimized plane corresponding to the ground; projecting the optimized plane representing the boundary onto the reference plane in the direction of gravity to determine the reference plane boundary, and obtaining a reference horizontal plane with a boundary corresponding to the ground; projecting the optimized plane representing the obstacle onto the reference horizontal plane in the direction of gravity to determine the obstacle area of the reference horizontal plane; generating the electronic fence according to the reference horizontal plane and the obstacle area. The posture information and the 3D point cloud are obtained according to the environmental image and the IMU data; plane detection is performed on the posture information and the 3D point cloud to obtain the plane; depth detection is performed on the environmental image to obtain the depth data; the Mesh recognition data and the 3D object recognition information are processed according to the depth data; the electronic fence is generated according to the 3D point cloud, the plane, and the Mesh recognition data and the 3D object recognition information processed with the depth data. Although the above scheme provides a systematic method to generate electronic fences by using advanced technologies such as 3D point clouds, optimization algorithms and depth detection, it may face some defects. First, this scheme has high requirements for hardware devices and requires relatively expensive and advanced sensors and processors to generate and process 3D point clouds and execute complex optimization algorithms in real time. Second, the complexity of the scheme may affect real-time performance, especially in dynamic environments that require high response in infant care, where the processing speed and response time cannot meet actual application requirements. In addition, although this method may perform well in specific environments, its applicability may be limited by the complexity and diversity of the scenes. For example, different scenes involve different terrain features, lighting conditions and obstacle types, which will lead to unstable 3D point cloud generation and inaccurate plane detection. Therefore, this scheme will face the problems of low efficiency and reduced accuracy when performing electronic fence recognition across multiple different environments or in a wider range of application scenarios, thereby limiting its possibility of wide application.
[0005] Therefore, how to provide an automatic virtual electronic fence generation method with strong adaptability and high accuracy in various scenarios of infant care to ensure the safe supervision of infants and young children is an urgent problem to be solved. Summary of the invention
[0006] In view of this, the present invention provides a scene-adaptive electronic fence generation method, device, equipment and medium to solve the problem in the prior art that in various scenarios of infant care, the electronic fence generation has weak adaptability and low accuracy, and the safety of infants and young children cannot be guaranteed.
[0007] The technical solution adopted by the present invention is:
[0008] In a first aspect, the present invention provides a method for generating an electronic fence based on scene adaptation, the method comprising:
[0009] Acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images;
[0010] Inputting the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in;
[0011] The real-time image is processed according to the scene type and a preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area.
[0012] Preferably, the step of sequentially inputting the real-time image into a pre-trained target detection model and a scene recognition model to determine the type of scene in which the infant is located comprises:
[0013] Inputting the real-time image into a pre-trained target detection model and outputting infant detection results;
[0014] According to the infant detection result, if an infant is successfully detected, obtaining the infant position information;
[0015] According to the infant position information, the preset prompt information is input into the scene recognition model, and the scene type is output, wherein the scene type at least includes: infant bed, big bed and living room.
[0016] Preferably, the processing of the real-time image according to the scene type and the preset fence analysis strategy corresponding to the scene type to determine the electronic fence area location information includes:
[0017] If the scene type is a baby bed, target detection is performed on the real-time image according to a first fence positioning strategy corresponding to the baby bed to determine the location information of the electronic fence area;
[0018] If the scene type is a large bed, depth detection and gradient analysis are performed on the real-time image according to the second fence positioning strategy corresponding to the large bed to determine the location information of the electronic fence area;
[0019] If the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room to determine the electronic fence area location information.
[0020] Preferably, if the scene type is a baby bed, target detection is performed on the real-time image according to the first fence positioning strategy corresponding to the baby bed, and the electronic fence area location information is determined, including:
[0021] Inputting the real-time image into a pre-trained infant bed detection model to output infant bed position information;
[0022] The position information of the infant bed is calculated to determine the position information of the electronic fence area.
[0023] Preferably, if the scene type is a large bed, depth detection and gradient analysis are performed on the real-time image according to the second fence positioning strategy corresponding to the large bed to determine the electronic fence area location information, including:
[0024] Inputting the real-time image into a pre-trained depth detection model and outputting a scene depth map;
[0025] Processing the scene depth map using a Sobel gradient algorithm to output a Sobel gradient map;
[0026] Scanning the Sobel gradient image line by line to determine fence boundary position information;
[0027] The electronic fence area location information is determined based on the fence boundary location information.
[0028] Preferably, scanning the Sobel gradient map line by line to determine the fence boundary position information includes:
[0029] Scan the highest row of the Sobel gradient image from left to right, and obtain the position information when the maximum gradient value appears for the first time as the left boundary position information;
[0030] Scan the highest row of the Sobel gradient image from right to left, and obtain the position information when the maximum gradient value appears for the first time as the right boundary position information;
[0031] Continue to scan the Sobel gradient map row by row from top to bottom, obtain the position information when the maximum gradient value appears for the first time as the upper boundary position information, and obtain the position information when the maximum gradient value appears for the last time as the lower boundary position information;
[0032] Compare the left boundary position sequence and the right boundary position sequence obtained after scanning row by row from top to bottom to obtain the maximum left boundary position and the minimum right boundary position;
[0033] The fence boundary position information is determined according to the maximum left boundary position, the minimum right boundary position, the upper boundary position information and the lower boundary position information.
[0034] Preferably, if the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room, and determining the electronic fence area location information includes:
[0035] Inputting the real-time image into a pre-trained infant detection model to output infant position information;
[0036] Inputting the real-time image into a pre-trained ground segmentation model and outputting a ground image sequence;
[0037] Performing contour detection on the ground image sequence and outputting a ground contour sequence;
[0038] According to the infant position information and the ground contour sequence, when it is determined that the infant is in a target contour area in the ground contour sequence, the target contour area position information is acquired as the electronic fence area position information.
[0039] In a second aspect, the present invention provides a scene-adaptive electronic fence generation device, the device comprising:
[0040] A real-time image acquisition module, used to acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images;
[0041] A scene classification module, used to input the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in;
[0042] The fence positioning module is used to process the real-time image according to the scene type and a preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area.
[0043] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.
[0044] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiment is implemented.
[0045] In summary, the beneficial effects of the present invention are as follows:
[0046] The present invention provides a scene-adaptive electronic fence generation method, device, equipment and medium, the method comprising: obtaining real-time video data in an infant care scene, decomposing the real-time video data into multiple frames of real-time images; inputting the real-time images into a pre-trained target detection model and a scene recognition model in sequence to determine the scene type in which the infant is located; processing the real-time images according to the scene type and the preset fence positioning strategy corresponding to the scene type, and determining the electronic fence area position information. The present invention uses the pre-trained target detection model and the scene recognition model to classify the scene in which the infant is located, so that the generation of the electronic fence can be dynamically adjusted according to different environmental conditions and scenes, such as applying different fence strategies in different scenes such as infant beds, large beds and living rooms. The real-time image is processed according to the scene type and the preset fence positioning strategy to more accurately determine the position of the electronic fence, thereby effectively limiting the range of activities of infants and preventing them from entering potential dangerous areas. The automatic image processing and fence generation reduce the dependence on manual monitoring, and improve the efficiency and safety of care, especially when the caregiver cannot monitor full-time. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.
[0048] Figure 1 It is a schematic diagram of the overall working process of the method for generating an electronic fence based on scene adaptation in Embodiment 1 of the present invention;
[0049] Figure 2 A schematic diagram of a process of inputting a real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene in which an infant is located in Embodiment 1 of the present invention;
[0050] Figure 3 It is a schematic diagram of a process of processing a real-time image according to a scene type and a preset fence analysis strategy corresponding to the scene type in Embodiment 1 of the present invention;
[0051] Figure 4 This is a schematic diagram of a process of performing target detection on a real-time image according to a first fence positioning strategy corresponding to a baby bed in Embodiment 1 of the present invention;
[0052] Figure 5 This is a flow chart of performing depth detection and gradient analysis on a real-time image according to the second fence positioning strategy corresponding to the large bed in Example 1 of the present invention;
[0053] Figure 6 A schematic diagram of a process of scanning a Sobel gradient image line by line to determine fence boundary position information in Embodiment 1 of the present invention;
[0054] Figure 7 This is a flow chart of performing target detection and image segmentation on a real-time image according to the third fence positioning strategy corresponding to the living room in Embodiment 1 of the present invention;
[0055] Figure 8 It is a structural block diagram of a scene-adaptive electronic fence generation device in Embodiment 2 of the present invention;
[0056] Fig. 9 It is a schematic diagram of the structure of the electronic device in Embodiment 3 of the present invention. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the phrase "comprising..." do not exclude the existence of other identical elements in the process, method, article or device comprising the elements. If there is no conflict, the embodiments of the present invention and the various features in the embodiments can be combined with each other, all within the protection scope of the present invention.
[0058] Example 1
[0059] See also Figure 1 Embodiment 1 of the present invention discloses a method for generating an electronic fence based on scene adaptation, the method comprising:
[0060] Acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images;
[0061] Specifically, in an infant care scenario, real-time video data is collected by a care camera, wherein the infant care scene includes at least an infant bed, a large bed and a living room scene, and the real-time video data is decomposed into multiple frames of real-time images to provide more comprehensive monitoring. The real-time camera is installed in an inclined manner to capture the activities and environment of the infant at the best angle. Since the inclined installation method can provide a wider field of view, it ensures that the camera can capture details in every corner of the care scene, thereby improving the monitoring effect and reliability.
[0062] Inputting the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in;
[0063] Specifically, the decomposed real-time image is sent to the pre-trained target detection model and scene recognition model for processing to identify and determine the specific scene type in which the infant is located, such as in a crib, a big bed, or a living room. The target detection model includes: YOLO, Faster R-CNN and SSD, and the scene recognition model includes at least: GPT series models, BERT models, XLNet and Blip series models. Combining the target detection model and the scene recognition model to identify the scene can more comprehensively understand the scene of the real-time image, thereby accurately identifying the current environment of the infant, and providing accurate scene information and positioning strategies for the subsequent electronic fence generation. Such a processing process enables the system to automatically adapt to different monitoring scenarios and provide more personalized and accurate safety protection for infants.
[0064] In one embodiment, see Figure 2 The step of sequentially inputting the real-time image into the pre-trained target detection model and the scene recognition model to determine the scene type in which the infant is located includes:
[0065] Inputting the real-time image into a pre-trained target detection model and outputting infant detection results;
[0066] Specifically, the real-time image is input into a pre-trained target detection model, such as YoloV8s, and the infant target in the scene is detected through the reasoning process of the target detection model. The YoloV8s model has the characteristics of high detection speed and high real-time response. It can accurately locate and detect the position of infants in real-time images, thereby providing the necessary basic information for subsequent safety monitoring.
[0067] According to the infant detection result, if an infant is successfully detected, obtaining the infant position information;
[0068] Specifically, according to the infant detection results, based on the successfully detected infant target, its position information is obtained from the target detection model. Once the infant is identified, the relevant position information will be extracted from the output of the target detection model, including the coordinates or bounding box information of the image where the infant is located. These position information will be used for subsequent safety monitoring measures, such as determining the specific scene type in which the infant is located, generating a corresponding electronic fence, etc.
[0069] According to the infant position information, the preset prompt information is input into the scene recognition model, and the scene type is output, wherein the scene type at least includes: infant bed, big bed and living room.
[0070] Specifically, the preset prompt information is input into the Blip2 scene recognition model, and combined with the location information of the infant, the current scene is intelligently analyzed to determine the specific type of environment the infant is in. The setting of the prompt information includes the recognition conditions and output results for different scenes. For example, the prompt information can be set to "If the infant is identified as being in a crib, the system outputs the crib mode; if it is in a big bed, the system outputs the big bed mode; if it is in the living room, the system outputs the living room mode." The current scene type is quickly and accurately determined based on the infant's location information and preset conditions, providing an accurate scene judgment basis for subsequent safety monitoring and fence generation.
[0071] The real-time image is processed according to the scene type and a preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area.
[0072] Specifically, the real-time image is processed according to the acquired scene type and the corresponding pre-set fence positioning strategy to determine the regional location information of the electronic fence. By combining the scene type and the corresponding fence positioning strategy, the location and range of the fence can be accurately determined according to the characteristics of the specific scene and safety requirements to ensure the safe monitoring of infants and young children in different environments.
[0073] In one embodiment, see Figure 3 The processing of the real-time image according to the scene type and the preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area includes:
[0074] If the scene type is a baby bed, target detection is performed on the real-time image according to a first fence positioning strategy corresponding to the baby bed to determine the location information of the electronic fence area;
[0075] Specifically, if the scene type is determined to be a baby bed, the real-time image will be detected according to the first fence positioning strategy corresponding to the baby bed. The first fence positioning strategy includes detecting the position of the baby bed through the baby bed detection model, and setting the electronic fence area position information by accurately identifying the position and boundary of the baby bed. By automatically generating an electronic fence suitable for the scene, the range of infant activities on the bed is effectively limited to ensure their safety.
[0076] In one embodiment, see Figure 4 If the scene type is a baby bed, target detection is performed on the real-time image according to the first fence positioning strategy corresponding to the baby bed, and the electronic fence area location information is determined, including:
[0077] Inputting the real-time image into a pre-trained infant bed detection model to output infant bed position information;
[0078] Specifically, first, the real-time image is sent to a pre-trained infant bed detection model, such as the YoloV8s infant bed detection module. The infant bed detection model has been trained and has the ability to identify infant beds. During the model's reasoning process, it will detect the infant bed target in the image and output the bed's location information, usually in the form of a rectangular box, including the coordinates of the upper left corner of the bed (x, y) and the width (w) and height (h). In this way, the accurate location information of the infant bed in the real-time image can be obtained.
[0079] The position information of the infant bed is calculated to determine the position information of the electronic fence area.
[0080] Specifically, based on the position information of the infant bed, the regional position of the electronic fence is calculated. Starting from the coordinates of the upper left corner of the bed (x, y), plus the width (w) and height (h) of the bed, the coordinates of the four vertices of the electronic fence can be determined, that is, the four vertices are P1 (x, y), P2 (x+w, y), P3 (x+w, y+h) and P4 (x, y+h), which constitute the boundary of the electronic fence. Therefore, according to the calculation of the position information of the infant bed, the regional position information of the electronic fence can be accurately determined for subsequent safety monitoring measures.
[0081] If the scene type is a large bed, depth detection and gradient analysis are performed on the real-time image according to the second fence positioning strategy corresponding to the large bed to determine the location information of the electronic fence area;
[0082] Specifically, if the scene type is determined to be a large bed, depth detection and gradient analysis will be performed on the real-time image according to the second fence positioning strategy corresponding to the large bed. Through depth detection, the depth information of objects in the image can be obtained, and the height difference between the bed and the ground can be distinguished. Using gradient analysis, the changes between the edge of the bed and the ground can be detected, thereby determining the position and boundary of the large bed. Combined with the results of depth detection and gradient analysis, the position and outline of the large bed can be accurately determined, and then the regional position information of the electronic fence can be calculated to ensure the safety of infants and young children on the large bed.
[0083] In one embodiment, see Figure 5 If the scene type is a large bed, the second fence positioning strategy corresponding to the large bed is used to perform depth detection and gradient analysis on the real-time image to determine the location information of the electronic fence area, including:
[0084] Inputting the real-time image into a pre-trained depth detection model and outputting a scene depth map;
[0085] Specifically, the real-time image is input into a pre-trained depth detection model, such as the MonoDepth monocular depth detection model, to obtain the scene depth map Depth(x,y) of the big bed scene. The scene depth map Depth(x,y) contains the depth information of each pixel in the real-time image. Through the scene depth map, the distance relationship between objects at different positions in the real-time image and the camera can be understood, so as to better understand the spatial structure and layout characteristics of the big bed scene, and provide important basic data for subsequent fence positioning.
[0086] Processing the scene depth map using a Sobel gradient algorithm to output a Sobel gradient map;
[0087] Specifically, the scene depth map is processed by the Sobel gradient algorithm to generate a Sobel gradient map. In this process, the Sobel operator is first calculated in the horizontal direction on the depth map sequence Depth(x,y), thereby obtaining the depth gradient information of each pixel in the horizontal direction. These gradient values represent the depth change degree of each position in the depth map. By analyzing the gradient map, the depth change between the bed and the ground can be identified, and then the edge contour of the big bed can be determined. This processing process provides important data support for the subsequent positioning of the big bed fence, which helps to more accurately protect the safety of infants and young children on the big bed.
[0088] Scanning the Sobel gradient image line by line to determine fence boundary position information;
[0089] Specifically, the Sobel gradient map is scanned row by row to determine the boundary position information of the fence. In this process, the Sobel gradient map is traversed row by row to find the position with the largest depth change in each row. These positions usually correspond to the boundary between the big bed and the ground. By finding the position with the maximum gradient in each row, the edge position of the bed is determined, and thus the upper and lower boundaries of the fence are obtained. In this way, the boundary position of the fence can be accurately determined according to the gradient change of the edge of the bed, which provides an important position information basis for the subsequent fence generation to ensure the safe monitoring of infants and young children in the big bed.
[0090] In one embodiment, see Figure 6 The step of scanning the Sobel gradient map line by line to determine the fence boundary position information includes:
[0091] Scan the highest row of the Sobel gradient image from left to right, and obtain the position information when the maximum gradient value appears for the first time as the left boundary position information;
[0092] Specifically, when the highest row of the Sobel gradient map is scanned from left to right, it checks the gradient value of each pixel one by one. In this process, the position where the maximum gradient value first appears is recorded. This position represents the left boundary X1 of the bed. The coordinate information of this position is saved and used as the left boundary position of the fence.
[0093] Scan the highest row of the Sobel gradient image from right to left, and obtain the position information when the maximum gradient value appears for the first time as the right boundary position information;
[0094] Specifically, the highest row of the Sobel gradient map is scanned from right to left. Similarly, the gradient value of each pixel is checked one by one, and the position where the maximum gradient value first appears is recorded. This position represents the right boundary X2 of the bed. The coordinate information of this position is saved and used to determine the right boundary position of the fence.
[0095] Continue to scan the Sobel gradient map row by row from top to bottom, obtain the position information when the maximum gradient value appears for the first time as the upper boundary position information, and obtain the position information when the maximum gradient value appears for the last time as the lower boundary position information;
[0096] Specifically, the Sobel gradient map is then scanned row by row from top to bottom to find the first position of the maximum gradient value in each row and regard it as the upper boundary Ymin of the bed. At the same time, the position of the last maximum gradient value in each row is found and regarded as the lower boundary Ymax of the bed, thereby determining the vertical range of the bed in the image.
[0097] Compare the left boundary position sequence and the right boundary position sequence obtained after scanning row by row from top to bottom to obtain the maximum left boundary position and the minimum right boundary position;
[0098] Specifically, the left boundary position sequence (X1 sequence) and the right boundary position sequence (X2 sequence) are obtained after scanning row by row from top to bottom. When comparing the left and right boundary position sequences, the maximum value X1_max in the left boundary position sequence and the minimum value X2_max in the right boundary position sequence are found, and X1_max is used as the maximum left boundary of the bed and X2_max is used as the minimum right boundary of the bed, which provides an accurate horizontal range for the subsequent positioning of the fence.
[0099] The fence boundary position information is determined according to the maximum left boundary position, the minimum right boundary position, the upper boundary position information and the lower boundary position information.
[0100] Specifically, finally, based on the acquired maximum left boundary, minimum right boundary, upper boundary and lower boundary position information, the coordinates of the four vertices of the fence are calculated to determine the boundary position of the fence. The boundary positions of the fence are: P1 (X1_max, Ymin), P2 (X2_min, Ymin), P3 (X2_min, Ymax), P4 (X1_max, Ymax), thereby generating a fence that conforms to the layout of the big bed and ensures the safe supervision of infants and young children on the big bed.
[0101] The electronic fence area location information is determined based on the fence boundary location information.
[0102] Specifically, the electronic fence area location information is determined based on the fence boundary location information. By using the above-mentioned gradient scanning method to determine the electronic fence area location information, the boundary position of the bed can be accurately identified without relying on complex sensor equipment or manual calibration. This method is not affected by changes in ambient light and is suitable for beds of different shapes and sizes. It has strong adaptability and versatility. Moreover, this automated boundary recognition process can achieve real-time monitoring, timely detect changes in the position of infants and young children, and take safety measures when necessary, thereby improving the safety of infants and young children. In addition, the method of using gradient scanning to determine the electronic fence can greatly reduce labor costs, improve the degree of automation, and make the monitoring process more convenient and efficient.
[0103] If the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room to determine the electronic fence area location information.
[0104] Specifically, when the scene type is determined to be a living room, the third fence positioning strategy corresponding to the living room is adopted to perform target detection and image segmentation on the real-time image. Through target detection, target objects in the living room, such as infants or other furniture, are identified, and through image segmentation, different areas in the living room image, such as the ground and furniture, are separated. By combining this information, the location information of the electronic fence is determined to ensure the safe activity range of infants in the living room, and safety measures can be taken when necessary.
[0105] In one embodiment, see Figure 7 If the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room, and the location information of the electronic fence area is determined, including:
[0106] Inputting the real-time image into a pre-trained infant detection model to output infant position information;
[0107] Specifically, the real-time image is input into a pre-trained infant detection model, for example, the YoloV8s detection model, to accurately locate the position information of the infant in the image, including Baby(x, y, w, h), where x and y represent the horizontal and vertical coordinates of the upper left corner of the infant human frame, and w and h represent the width and height of the human frame, respectively.
[0108] Inputting the real-time image into a pre-trained ground segmentation model and outputting a ground image sequence;
[0109] Specifically, in the living room scene, infants and young children are young and have no awareness of self-protection. Guardians usually do not place infants and young children independently on chairs, sofas and other areas where they are prone to falling and getting injured. Therefore, in actual infant care, setting up electronic fences does not need to consider the situations of the above-mentioned chairs and sofas, because the above-mentioned situations already have manual care by guardians. Instead, it is only necessary to consider the situation where the infants and young children are in the ground area and set up electronic fences. The real-time image is input into a pre-trained ground segmentation model, such as the SAM segmentation model, to effectively separate the ground in the living room image from other areas, and obtain a ground image sequence consisting of multiple frames of ground images.
[0110] Performing contour detection on the ground image sequence and outputting a ground contour sequence;
[0111] Specifically, contour detection is performed on the ground image sequence, and the ground image sequence is processed using a contour detection algorithm to obtain a ground contour sequence Contour(N), where N represents N ground contours. Each contour represents a continuous area on the ground.
[0112] According to the infant position information and the ground contour sequence, when it is determined that the infant is in a target contour area in the ground contour sequence, the target contour area position information is acquired as the electronic fence area position information.
[0113] Specifically, based on the position information of the infant and the ground contour sequence, it is determined whether the position of the infant is within a target contour area in the ground contour sequence. If so, the position information of the target contour area is used as the position information of the electronic fence area to ensure the safety range of the infant's activities. For example, the center coordinates of the infant human frame are C(x+w / 2, y+h / 2), and the target ground contour containing the center coordinates is Contour(i). By calculating the circumscribed rectangle of the target ground contour Contour(i), the area range of the contour is determined, and then the position information of the electronic fence is determined.
[0114] In one embodiment, the processing of the real-time image according to the scene type and the preset fence positioning strategy corresponding to the scene type to determine the electronic fence area location information further includes:
[0115] If the scene type is that there are both a baby bed and a big bed, input the real-time image into a pre-trained baby bed detection model and a big bed detection model, and output baby bed position information and big bed position information;
[0116] Specifically, the scene of both a baby bed and a big bed is very common in family bedrooms, especially for parents to take care of their babies at night or to ensure the safety and comfort of their babies. This arrangement allows parents and babies to rest in the same room, which is convenient for responding to the needs of babies immediately, and also provides independent sleeping space to avoid mutual interference. In the scene of both a baby bed and a big bed, the captured real-time image is first processed and input into two independent pre-trained detection models: a baby bed detection model and a big bed detection model. Both models are based on the Yolo8s architecture. After learning a large amount of training data, they can accurately identify and locate the positions of baby beds and big beds. When the real-time image is input into these models, they analyze the features in the image, identify the baby bed and the big bed, and output their location information. This location information usually includes the coordinates of each bed in the image (such as the coordinates of the upper left corner and the lower right corner of the rectangular box) and the size (width and height), so as to accurately identify the position and range of the baby bed and the big bed, and lay the foundation for subsequent processing and analysis by obtaining clear and accurate location information of the baby bed and the big bed in the scene.
[0117] Determine relative position information between the infant bed and the large bed based on the infant bed position information and the large bed position information;
[0118] Specifically, after obtaining the position information of the crib and the bed, the relative position information between the crib and the bed is determined by calculating the relative coordinate difference between them in the image. Specifically, the boundary coordinates of the crib and the bed are first extracted (such as the coordinate points of the upper left corner and the lower right corner), and then the horizontal and vertical distances between the two are calculated. The position of the crib relative to the bed is determined, such as whether the crib is on the left, right, above or below the bed, and the actual distance between them. This relative position information is crucial for subsequent safety monitoring and electronic fence setting, because it can accurately describe the spatial relationship between the crib and the bed, thereby helping the system make accurate protection decisions.
[0119] When it is determined that the infant bed is adjacent to the large bed according to the relative position information, an image of the infant bed and an image of the large bed are acquired according to the position information of the infant bed and the position information of the large bed;
[0120] Specifically, after determining the relative position information of the infant bed and the large bed, the relative position information is analyzed to determine whether the two are adjacent. If the distance between the infant bed and the large bed is within a preset distance threshold, they will be determined to be adjacent. At this time, the previously detected position information will be used to extract the image parts of the infant bed and the large bed from the original image. Specifically, based on the boundary coordinates of the infant bed and the large bed, sub-images containing the infant bed and the large bed are cropped from the overall image. These sub-images will contain detailed visual information of each bed, which will be provided to subsequent depth detection models for further analysis, thereby ensuring that the actual scene layout and safety requirements are taken into account when setting up the electronic fence.
[0121] Inputting the baby bed image and the large bed image into a pre-trained depth detection model, and outputting a baby bed depth value and a large bed depth value;
[0122] Specifically, after judging the crib and the big bed as adjacent and extracting their images, these sub-images are respectively input into a pre-trained depth detection model, such as the MonoDepth monocular depth detection model. This depth detection model has learned how to estimate the depth information of each pixel from a two-dimensional image through a large amount of training data. After the sub-image is input, the model will analyze the visual features in the image and calculate the depth values of the crib and the big bed. These depth values represent the distance from each point in the image to the camera, thereby reflecting the height and position of the crib and the big bed in three-dimensional space. By obtaining the depth information of the crib and the big bed, the height difference between them can be further analyzed, providing a basis for subsequent safety measures and electronic fence settings.
[0123] Determine the relative depth between the infant bed and the large bed according to the infant bed depth value and the large bed depth value;
[0124] Specifically, after obtaining the depth values of the crib and the large bed, the relative depth value between them is calculated to determine their height difference in three-dimensional space. Specifically, the average depth value of the crib and the large bed or the depth values of certain key points (such as edge points) are first extracted from the output of the depth detection model. Then, by comparing these depth values, the depth difference between the crib and the large bed, that is, the relative depth value, is calculated. The purpose of this step is to understand the relative height of the crib and the large bed in the vertical direction, so as to assess the risk of the baby falling from the crib to the large bed. The relative depth value is very important for safety monitoring, because a smaller height difference may require more protective measures, while a larger height difference may indicate a higher risk and require a stricter safety limit.
[0125] When the relative depth value is less than a preset depth threshold, the electronic fence area position information is determined according to the infant bed position information and the large bed position information.
[0126] Specifically, when the calculated relative depth value is less than the preset depth threshold, it is determined that the height difference between the infant bed and the large bed is small and there is no potential safety risk. At this time, first, the output position information of the infant bed (such as the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2)) is used. The four vertices of the area are P1(x1, y1), P2(x2, y1), P3(x2, y2), and P4(x1, y2). Then, the large bed image is sent to the MonoDepth monocular depth detection module to obtain the scene depth map sequence Depth(x, y), calculate the Sobel gradient map in the horizontal direction of the depth map, scan the gradient map from left to right to obtain the first maximum gradient value, determine the X1 coordinate of the left boundary of the big bed, then scan from right to left to obtain the first maximum gradient value, determine the X2 coordinate of the right boundary of the big bed, and scan from top to bottom to obtain these X1 and X2 sequences. When a row cannot obtain the maximum gradient value, it is determined as the lower boundary Ymax of the bed, and when the maximum gradient value is obtained for the first time, it is determined as the upper boundary Ymin of the big bed. Finally, the maximum value X1_max in the X1 sequence and the minimum value X2_min in the X2 sequence are statistically obtained to form the four vertices P1 (X1_max, Ymin), P2 (X2_min, Ymin), P3 (X2_min, Ymax), and P4 (X1_max, Ymax) of the big bed. According to the four vertex coordinates of the baby bed and the four vertex coordinates of the big bed, determine a minimum area that can surround the baby bed and the big bed, form an electronic fence area, ensure that the baby bed and the big bed are completely covered, and thus provide a safe protection area.
[0127] When the relative depth value is greater than or equal to a preset depth threshold, the real-time image is input into a pre-trained infant detection model, and the infant position information is output.
[0128] The electronic fence area location information is determined based on the infant location information, the infant bed location information and the large bed location information.
[0129] Specifically, when the calculated relative depth value is greater than or equal to a preset depth threshold, it is determined that the height difference between the infant bed and the large bed is large, and there is a potential safety risk of the infant falling from the large bed to the infant bed. Based on the infant position information, the current position of the infant is first determined. If the infant is detected in the infant bed, an electronic fence is defined on the infant bed according to the boundary coordinates of the infant bed (for example, the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2)). The four vertices of the fence are P1(x1, y1), P2(x2, y1), P3(x2, y2), and P4(x1, y2). If an infant is detected on the big bed, an electronic fence will be drawn on the big bed based on the boundary coordinates of the big bed (the left boundary X1, right boundary X2, upper boundary Ymin and lower boundary Ymax of the big bed calculated by the MonoDepth monocular depth detection module and the Sobel gradient map). The four vertices of the fence are P1 (X1_max, Ymin), P2 (X2_min, Ymin), P3 (X2_min, Ymax), and P4 (X1_max, Ymax). In this way, an electronic fence is dynamically set on the corresponding bed according to the actual position of the infant to ensure the safety of the infant.
[0130] Example 2
[0131] See also Figure 8 Embodiment 2 of the present invention further provides a scene-adaptive electronic fence generation device, the device comprising:
[0132] A real-time image acquisition module, used to acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images;
[0133] A scene classification module, used to input the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in;
[0134] The fence positioning module is used to process the real-time image according to the scene type and a preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area.
[0135] Specifically, the scene-adaptive electronic fence generation device provided by the embodiment of the present invention is adopted, and the device includes: a real-time image acquisition module, which is used to acquire real-time video data in the infant care scene, and decompose the real-time video data into multiple frames of real-time images; a scene classification module, which is used to input the real-time image into the pre-trained target detection model and the scene recognition model in sequence to determine the scene type of the infant; and a fence positioning module, which is used to process the real-time image according to the scene type and the preset fence positioning strategy corresponding to the scene type, and determine the location information of the electronic fence area. The device uses the pre-trained target detection model and the scene recognition model to classify the scene where the infant is located, so that the generation of the electronic fence can be dynamically adjusted according to different environmental conditions and scenes, such as applying different fence strategies in different scenes such as infant beds, large beds and living rooms. The real-time image is processed according to the scene type and the preset fence positioning strategy to more accurately determine the location of the electronic fence, thereby effectively limiting the range of activities of infants and preventing them from entering potential dangerous areas. Through automated image processing and fence generation, the dependence on manual monitoring is reduced, and the efficiency and safety of care are improved, especially when the caregiver cannot monitor full-time.
[0136] Example 3
[0137] In addition, combined Figure 1 The scenario-adaptive electronic fence generation method described in Embodiment 1 of the present invention may be implemented by an electronic device. Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in Embodiment 3 of the present invention is shown.
[0138] The electronic device may include a processor and a memory storing computer program instructions.
[0139] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.
[0140] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.
[0141] The processor implements any one of the scene-adaptive electronic fence generation methods in the above embodiments by reading and executing computer program instructions stored in the memory.
[0142] In one example, the electronic device may further include a communication interface and a bus. Fig. 9 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.
[0143] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.
[0144] The bus includes hardware, software or both, and the components of the device are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention contemplates any suitable bus or interconnection.
[0145] Example 4
[0146] In addition, in combination with the scene-adaptive electronic fence generation method in the above embodiment 1, the present invention can also provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any scene-adaptive electronic fence generation method in the above embodiments is implemented.
[0147] In summary, the embodiments of the present invention provide a method, device, equipment and medium for generating an electronic fence based on scene adaptation.
[0148] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.
[0149] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0150] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant location, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0151] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.
[0152] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.
Claims
1. A method for generating an electronic fence based on scene adaptation, characterized in that: The method comprises: Acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images; Inputting the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in; Processing the real-time image according to the scene type and a preset fence positioning strategy corresponding to the scene type to determine the location information of the electronic fence area; The processing of the real-time image according to the scene type and the preset fence positioning strategy corresponding to the scene type to determine the electronic fence area location information includes: If the scene type is a baby bed, target detection is performed on the real-time image according to a first fence positioning strategy corresponding to the baby bed to determine the location information of the electronic fence area; If the scene type is a large bed, depth detection and gradient analysis are performed on the real-time image according to the second fence positioning strategy corresponding to the large bed to determine the location information of the electronic fence area; If the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room to determine the location information of the electronic fence area; If the scene type is a large bed, then according to the second fence positioning strategy corresponding to the large bed, depth detection and gradient analysis are performed on the real-time image to determine the location information of the electronic fence area, including: Inputting the real-time image into a pre-trained depth detection model and outputting a scene depth map; Processing the scene depth map using a Sobel gradient algorithm to output a Sobel gradient map; Scanning the Sobel gradient image line by line to determine fence boundary position information; The electronic fence area location information is determined based on the fence boundary location information.
2. The method for generating an electronic fence based on scene adaptation according to claim 1, characterized in that: The step of sequentially inputting the real-time image into a pre-trained target detection model and a scene recognition model to determine the scene type in which the infant is located comprises: Inputting the real-time image into a pre-trained target detection model and outputting infant detection results; According to the infant detection result, if an infant is successfully detected, obtaining the infant position information; Input the prompt information corresponding to the infant position information into the scene recognition large model, and output the scene type, wherein the scene type at least includes: infant bed, large bed and living room.
3. The scene-adaptive electronic fence generation method according to claim 1, characterized in that: If the scene type is a baby bed, then according to the first fence positioning strategy corresponding to the baby bed, target detection is performed on the real-time image to determine the electronic fence area location information, including: Inputting the real-time image into a pre-trained infant bed detection model to output infant bed position information; The position information of the infant bed is calculated to determine the position information of the electronic fence area.
4. The scene-adaptive electronic fence generation method according to claim 1, characterized in that: Scanning the Sobel gradient map line by line to determine the fence boundary position information includes: Scan the highest row of the Sobel gradient image from left to right, and obtain the position information when the maximum gradient value appears for the first time as the left boundary position information; Scan the highest row of the Sobel gradient image from right to left, and obtain the position information when the maximum gradient value appears for the first time as the right boundary position information; Continue to scan the Sobel gradient map row by row from top to bottom, obtain the position information when the maximum gradient value appears for the first time as the upper boundary position information, and obtain the position information when the maximum gradient value appears for the last time as the lower boundary position information; Compare the left boundary position sequence and the right boundary position sequence obtained after scanning row by row from top to bottom to obtain the maximum left boundary position and the minimum right boundary position; The fence boundary position information is determined according to the maximum left boundary position, the minimum right boundary position, the upper boundary position information and the lower boundary position information.
5. The method for generating an electronic fence based on scene adaptation according to claim 1, characterized in that: If the scene type is a living room, then according to the third fence positioning strategy corresponding to the living room, target detection and image segmentation are performed on the real-time image to determine the electronic fence area location information, including: Inputting the real-time image into a pre-trained infant detection model to output infant position information; Inputting the real-time image into a pre-trained ground segmentation model and outputting a ground image sequence; Performing contour detection on the ground image sequence and outputting a ground contour sequence; According to the infant position information and the ground contour sequence, when it is determined that the infant is in a target contour area in the ground contour sequence, the target contour area position information is acquired as the electronic fence area position information.
6. A scene-adaptive electronic fence generation device, characterized in that: The device comprises: A real-time image acquisition module, used to acquire real-time video data in an infant care scene, and decompose the real-time video data into multiple frames of real-time images; A scene classification module, used to input the real-time image into a pre-trained target detection model and a scene recognition model in sequence to determine the type of scene the infant is in; A fence positioning module, used to process the real-time image according to the scene type and a preset fence positioning strategy corresponding to the scene type, and determine the location information of the electronic fence area; The processing of the real-time image according to the scene type and the preset fence positioning strategy corresponding to the scene type to determine the electronic fence area location information includes: If the scene type is a baby bed, target detection is performed on the real-time image according to a first fence positioning strategy corresponding to the baby bed to determine the location information of the electronic fence area; If the scene type is a large bed, depth detection and gradient analysis are performed on the real-time image according to the second fence positioning strategy corresponding to the large bed to determine the location information of the electronic fence area; If the scene type is a living room, target detection and image segmentation are performed on the real-time image according to the third fence positioning strategy corresponding to the living room to determine the location information of the electronic fence area; If the scene type is a large bed, then according to the second fence positioning strategy corresponding to the large bed, depth detection and gradient analysis are performed on the real-time image to determine the location information of the electronic fence area, including: Inputting the real-time image into a pre-trained depth detection model and outputting a scene depth map; Processing the scene depth map using a Sobel gradient algorithm to output a Sobel gradient map; Scanning the Sobel gradient image line by line to determine fence boundary position information; The electronic fence area location information is determined based on the fence boundary location information.
7. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 5.
8. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.
