Multi-modal data-based unpowered park operation processing method and system

Abstract

The invention provides an unpowered park operation processing method and system based on multi-modal data, and the method comprises the steps: determining a monitoring equipment group corresponding to a playing end according to the code scanning input information of the playing end, and enabling the monitoring equipment group to comprise monitoring sub-equipment and monitoring mother equipment; acquiring user information input by the playing terminal, and determining a playing monitoring area corresponding to the playing terminal according to the user information; determining a transition mode of the playing monitoring area according to the movement information of the monitoring equipment group, and updating the playing monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; and determining that the monitoring equipment group which does not meet the fence monitoring information of the dynamic monitoring area is warning equipment, generating early warning information corresponding to the warning equipment, and sending the early warning information to the corresponding playing terminal.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to an operation processing method and system of a non-powered park based on multimodal data. Background Art

[0002] A non-powered park is a special type of amusement park, which mainly includes active experience equipment that does not rely on any non-natural external force and energy (such as electricity, fuel, etc.), such as rope net equipment, slide equipment, rock climbing equipment, trampoline series, etc. With the progress of society and the improvement of people's safety awareness, the public's safety requirements for non-powered parks are getting higher and higher.

[0003] At present, non-powered parks are generally equipped with professional safety management personnel to monitor the tourists' play through manual monitoring to ensure that tourists use the amusement facilities in accordance with regulations. However, due to the large number of tourists in the park, the management personnel may not be able to take care of all tourists, so when tourists behave abnormally, they may not be able to remind the corresponding tourists in time, which reduces the safety of tourists when playing.

[0004] Therefore, how to automatically monitor tourists in non-powered parks and improve their safety has become an urgent problem that needs to be solved. Summary of the invention

[0005] The present invention provides an operation processing method and system for a non-powered park based on multimodal data, which can automatically monitor tourists in the non-powered park and improve the safety of tourists when playing.

[0006] A first aspect of the present invention provides an operation processing method of a non-powered park based on multimodal data, comprising: Determine the monitoring device group corresponding to the gaming terminal according to the scanned code input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device; Acquire user information input by the gaming terminal, and determine a gaming monitoring area corresponding to the gaming terminal according to the user information; Determine a transition mode of the play monitoring area according to the movement information of the monitoring device group, and update the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; The monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area is determined as a warning device, and early warning information corresponding to the warning device is generated and sent to the corresponding playing terminal.

[0007] Optionally, in a possible implementation manner of the first aspect, obtaining user information input by the gaming terminal, and determining a gaming monitoring area corresponding to the gaming terminal according to the user information includes: Obtaining area restriction conditions corresponding to each play area in the play area, wherein the area restriction conditions at least include height restriction and / or age restriction; Comparing the area restriction conditions based on the user information, and when the user information satisfies the area restriction conditions, determining the corresponding play area as a to-be-screened monitoring area, wherein the user information includes at least height information and age information; Acquire the play attributes corresponding to each of the to-be-screened monitoring areas, wherein the play attributes include low-risk attributes and high-risk attributes, and determine that the to-be-screened monitoring area corresponding to the low-risk attribute is a planned monitoring area; Receive area adjustment information of the planned monitoring area from the player terminal, and update the planned monitoring area according to the area adjustment information to obtain a playing monitoring area.

[0008] Optionally, in a possible implementation manner of the first aspect, receiving area adjustment information of the planned monitoring area by the player terminal, and updating the planned monitoring area according to the area adjustment information to obtain the playing monitoring area includes: Retrieving a site layout diagram corresponding to the play site, highlighting a reference area in the site layout diagram corresponding to the planned monitoring area to obtain a play planning diagram and sending it to the play terminal; receiving a customized area demarcated by the player terminal based on the game planning map, and determining a planned monitoring area completely located within the customized area as a game monitoring area; Acquire a planned monitoring area that is not completely located in the customized area as a determined monitoring area, and count the number of players corresponding to the determined monitoring area at the current moment; The monitoring area where the number of players is less than the dangerous number threshold is determined as the playing monitoring area.

[0009] Optionally, in a possible implementation manner of the first aspect, determining a transition mode of the play monitoring area according to movement information of the monitoring device group, and updating the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area includes: Parsing the movement information to obtain a device distance between the moving positions of the monitoring child device and the monitoring mother device; When the device spacing is less than or equal to the standard spacing, determining that the transition mode is a fixed mode, and determining that the play monitoring area in the fixed mode remains unchanged; When the device spacing is greater than the standard spacing within a preset time period, determining that the transition mode is a dynamic mode; The current number of players in each of the play monitoring areas is counted according to the dynamic mode, and the play monitoring area where the current number of players is less than a threshold value of the number of dangerous players is determined as a dynamic monitoring area.

[0010] Optionally, in a possible implementation of the first aspect, determining that a monitoring device group that does not satisfy the fence monitoring information of the dynamic monitoring area is a warning device, generating early warning information corresponding to the warning device and sending it to the corresponding player terminal includes: Acquire the positioning information of the monitoring sub-device, and when the positioning information is not within the dynamic monitoring area, determine that the monitoring sub-device does not meet the fence monitoring information of the dynamic monitoring area, and determine that the corresponding monitoring sub-device is a warning device; Determine the play area where the positioning information of the warning device passes within the historical time period as the warning monitoring area, and obtain the sub-historical time period where the warning device passes through each of the warning monitoring areas; Obtaining a historical monitoring video of the warning monitoring area in the sub-historical time period, and identifying an action person in the historical monitoring video; Determine the person who is in action and corresponds to the warning device as the target person, and obtain the current playing area where the positioning information is located as the warning prompt area; The current monitoring video of the warning prompt area is obtained, and the target person in the current monitoring video is highlighted to obtain early warning information and send it to the player end.

[0011] Optionally, in a possible implementation manner of the first aspect, determining the action person corresponding to the warning device as the target person includes: Acquire the action personnel existing in each of the historical monitoring videos as the candidates, and when the number of the candidates is equal to the reference number, determine the candidates as the target personnel; When the number of the candidates is greater than the reference number, obtaining a plurality of human body detection points corresponding to each of the candidates; Input the human body detection points into a height detection model, output the predicted height of the candidate, and obtain the height difference between the predicted height and the height information corresponding to the user information; The candidate whose height difference is within the normal difference range is obtained as the target person.

[0012] Optionally, in a possible implementation of the first aspect, in response to a monitoring request from the monitoring terminal, a first distance between the monitoring child device and the monitoring parent device in each play area is obtained, and a monitoring device group whose first distance is smaller than a standard distance is determined as an accompanying device group; Acquire the current captured video shot by the monitoring equipment of each of the play areas, and extract the outline of the person in the video frame corresponding to the current captured video; Performing attribute recognition on the person profile, determining the person profile with adult attributes as the first profile, and determining the person profile with child attributes as the second profile; Acquire a first contour and a second contour whose contour interval is smaller than a reference contour interval as the same accompanying contour group, and count a first number of the accompanying device groups and a second number of the accompanying contour groups; When the first number is greater than the second number, the outline of the abnormal person in the currently collected video is determined, the outline of the abnormal person is highlighted and displayed, and the abnormal monitoring data is obtained and sent to the monitoring end.

[0013] Optionally, in a possible implementation manner of the first aspect, when the first number is greater than the second number, determining the profile of the abnormal person in the currently collected video includes: When the first quantity is greater than the second quantity, obtaining a quantity difference between the first quantity and the second quantity; Acquire the first contour in the video frame except the accompanying contour group as the first contour to be detected, and the second contour as the second contour to be detected; Determine that the first to-be-tested contour and the second to-be-tested contour whose contour spacing is greater than the reference contour spacing are the same to-be-tested group, and the number of the first to-be-tested contour and the second to-be-tested contour in the to-be-tested group is equal to the reference number; Counting the number of people to be inspected in the group to be inspected, and if the number of people to be inspected is equal to the number difference, determining that the person profile corresponding to the group to be inspected is an abnormal person profile; If the number to be inspected is not equal to the number difference, the historical acquisition video of the corresponding monitoring device is obtained, and the group to be inspected is re-matched according to the historical acquisition video to obtain multiple abnormal person contours that match each other in the current acquisition video.

[0014] Optionally, in a possible implementation manner of the first aspect, if the number to be inspected is not equal to the number difference, obtaining a historical acquisition video of the corresponding monitoring device, re-matching the group to be inspected according to the historical acquisition video, and obtaining multiple abnormal person profiles that match each other in the current acquisition video, including: If the number to be inspected is not equal to the number difference, obtaining the historical collected video of the monitoring device within the historical monitoring time period; Extracting a historical group to be inspected corresponding to the group to be inspected from a plurality of historical frames corresponding to the historical acquisition video, and obtaining the historical group to be inspected whose contour spacing is smaller than the reference contour spacing as an accompanying group to be inspected; Deleting the remaining groups to be inspected in which the first profile to be inspected and the second profile to be inspected exist in the accompanying group to be inspected, and determining that the first profile to be inspected and the second profile to be inspected in the accompanying group to be inspected are profiles of abnormal persons that match each other; An abnormality display frame corresponding to each of the accompanying inspection groups is generated, and different preset pixel values ​​are called to update the abnormality display frames corresponding to different accompanying inspection groups.

[0015] A second aspect of the present invention provides an operation processing system for a non-powered park based on multimodal data, comprising: An input module, used to determine a monitoring device group corresponding to the gaming terminal according to the scanned code input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device; A planning module, used to obtain user information input by the game terminal, and determine a game monitoring area corresponding to the game terminal according to the user information; A monitoring module, configured to determine a transition mode of the play monitoring area according to movement information of the monitoring device group, and to update the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; The early warning module is used to determine that the monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area is a warning device, generate early warning information corresponding to the warning device and send it to the corresponding playing terminal.

[0016] The beneficial effects of the present invention are as follows: The present invention can automatically monitor tourists in a non-powered park and improve the safety of tourists when they play. The present invention configures a corresponding monitoring device group for parents and children entering the non-powered park, and determines whether the child is accompanied by a parent based on the movement information of the monitoring device group, and promptly reminds the parent when the child acts alone, so that the parent can pay attention to the child's playing safety in time, thereby improving the safety of the child when playing.

[0017] The present invention can combine the user information input by the game player to quickly match the area where the corresponding user can play for reference, thereby improving the user's gaming efficiency. In addition, the present invention will also update the recommended area based on the user's own needs and movement information, thereby improving the flexibility of area recommendation.

[0018] The present invention will also verify the surveillance videos of each play area, determine the personnel who do not correspond to the positioning information of the monitoring equipment group and highlight them, so that the abnormal personnel can be displayed to the supervisor for review, so that they can intuitively see the abnormal personnel and give corresponding reminders to them, thereby reducing the user's play risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of an operation processing method of a non-powered park based on multimodal data provided by an embodiment of the present invention; Figure 2is a schematic diagram of a monitoring device group provided by an embodiment of the present invention; Figure 3 is a schematic diagram of interaction between a monitoring device group and a server provided by an embodiment of the present invention; Figure 4 It is a structural schematic diagram of an operation processing system of a non-powered park based on multimodal data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1 , is a flow chart of an operation processing method of a non-powered park based on multimodal data provided by an embodiment of the present invention, Figure 1 The execution subject of the method shown may be a software and / or hardware device. The execution subject of the present application may include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, user equipment may include but is not limited to computers, smart phones, personal digital assistants (PDA) and the electronic devices mentioned above. Network equipment may include but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. It includes steps S1 to S4, as follows: S1, determining a monitoring device group corresponding to the gaming terminal according to the code scanning input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device.

[0022] It is understandable that non-powered parks are generally attended by parents and children together. Therefore, when monitoring tourists in non-powered parks, this solution will configure corresponding monitoring equipment groups for parents and children, and then combine the movement information of the monitoring equipment group to determine whether the child is accompanied by the parent, and remind the parents in time when the child acts alone, so that parents can pay attention to the safety of their children's play in time and improve the safety of their children when playing.

[0023] Among them, the monitoring device group can be a positioning bracelet, the monitoring child device can be a positioning bracelet worn by the child, the monitoring mother device can be a positioning bracelet worn by the parent, and the playing end refers to the terminal carried by the parent, such as a mobile phone or other electronic device.

[0024] See also Figure 2 , is a schematic diagram of a monitoring device group provided by an embodiment of the present invention. Figure 2 As shown, parents can scan the code and enter information to bind the monitoring device group with the gaming terminal, so that when an abnormality occurs in the monitoring device group, the corresponding gaming terminal can be reminded in time.

[0025] In actual application, before entering the park, parents can scan the code through the game terminal, and then bind the scan code information with the corresponding monitoring device group. After binding, parents can carry the monitoring mother device and children can carry the monitoring child device, so that the movements of the corresponding parents and children can be obtained in time through the monitoring device group.

[0026] S2, obtaining user information input by the gaming terminal, and determining a gaming monitoring area corresponding to the gaming terminal according to the user information.

[0027] Non-powered parks generally include multiple play areas, such as rope net areas, slide areas, rock climbing areas, trampoline areas, etc. The restrictions corresponding to different areas may be different. For example, each area may have age and height restrictions. Some areas with higher risk factors may restrict young children from playing, thereby improving the safety of users when playing.

[0028] Therefore, in order to improve the user's play efficiency, this solution will combine the user information input at the play end to quickly match the area where the corresponding user can play for guidance reference, so that parents can take their children to the corresponding area for play quickly, reducing the time added due to parents' unfamiliarity with the layout of the non-powered park.

[0029] The user information specifically includes the child information entered by the parents, such as the child's height, age, etc. The play monitoring area refers to the area where the user corresponding to the play end can play in the non-powered park.

[0030] Based on the above embodiment, the specific implementation of step S2 may be: S21, obtaining area restriction conditions corresponding to each play area in the play area, wherein the area restriction conditions at least include height restriction and / or age restriction.

[0031] In practical applications, the play venue refers to a non-powered park, the play area refers to the area for play in the non-powered park, and the area restriction conditions can be the conditions for restricting the number of people playing in each play area configured in advance by the management personnel, which at least include height restrictions and / or age restrictions in this solution.

[0032] S22, comparing each of the area restriction conditions based on the user information, and when the user information satisfies the area restriction conditions, determining the corresponding play area as a monitoring area to be screened, wherein the user information includes at least height information and age information.

[0033] It is worth mentioning that when parents enter their children's information, they can be prompted that height information and age information are required, so that the play area that meets the user information can be automatically screened out. Specifically, the play area where the height information meets the height limit and the age information meets the age limit can be used as the monitoring area to be screened. The monitoring area to be screened is the play area that is screened out at one time based on the user information.

[0034] S23, obtaining the play attributes corresponding to each of the to-be-screened monitoring areas, wherein the play attributes include low-risk attributes and high-risk attributes, and determining the to-be-screened monitoring area corresponding to the low-risk attribute as the planned monitoring area.

[0035] It is understandable that for safety reasons, some areas in non-powered parks may require parents to accompany them to play, such as areas with higher risk projects such as rock climbing, while some areas may have lower risk factors such as areas with slides, and children may be allowed to play alone in such areas.

[0036] Therefore, in order to improve the safety of users when playing, when automatically planning the recommended area, this solution will also combine the play attributes of the monitored area to be screened for secondary screening, and use the monitored area to be screened with low-risk attributes as the planned monitoring area. Among them, the corresponding play attributes can be configured in advance according to the risk factor of each play area, so that when making recommendations, areas that meet user needs and have a higher safety factor can be screened for recommendation.

[0037] S24, receiving area adjustment information of the planned monitoring area from the player terminal, and updating the planned monitoring area according to the area adjustment information to obtain a playing monitoring area.

[0038] In actual applications, parents may also adjust the recommended areas according to their own needs. Therefore, when monitoring the corresponding users on the gaming end, risk monitoring can be performed on the areas after the users’ final adjustments. Among them, risk adjustment information refers to the information when the users adjust the planned monitoring areas.

[0039] In some embodiments, step S24 may be implemented by the following steps: S241, retrieve the site layout map corresponding to the play site, highlight the reference area corresponding to the planned monitoring area in the site layout map to obtain a play planning map and send it to the play terminal.

[0040] In order to allow users to intuitively view the specific distribution of the planned monitoring area, the planned monitoring area can be displayed through the site layout map of the playground. When highlighting the planned monitoring area, the planned monitoring area can be updated with different pixel values ​​from other areas, so as to obtain a play planning map and send it to the user for area determination. Among them, the play planning map refers to the layout display map of the play area automatically planned in combination with user information.

[0041] In practical applications, a site layout corresponding to the play area may be configured in advance, and reference areas corresponding to various play areas may be defined in the layout.

[0042] S242, receiving a customized area demarcated by the player terminal based on the game planning map, and determining a planned monitoring area completely located in the customized area as a game monitoring area.

[0043] In order to quickly determine the planned monitoring area selected by the user, this solution will allow the user to delineate the selected area, and then select the final area based on the area delineated by the user.

[0044] In some embodiments, when the user has a selection requirement, he can define an area by himself, that is, the above-mentioned customized area. It is understandable that if the planned monitoring area is completely located in the positioning area, it means that the area is completely selected by the user, so it can be used as the final selected play monitoring area.

[0045] S243, obtaining a planned monitoring area that is not completely located in the customized area as a determined monitoring area, and counting the number of players corresponding to the determined monitoring area at the current moment.

[0046] If the planned monitoring area is not completely located within the positioning area, such planned monitoring area can be screened again. The reason for screening by the current number of visitors is that when the number of visitors is large, the danger may also be greater, such as congestion, etc. Therefore, the area with a lower risk factor at the current moment can be screened out by the number of visitors as the final selected area.

[0047] The monitoring area is determined to be a planned monitoring area for re-screening. When determining the number of visitors in the monitoring area, the number of monitoring equipment groups in the monitoring area can be determined accordingly.

[0048] S244, determining the monitoring area where the number of players is less than the threshold of the number of dangerous players as the playing monitoring area.

[0049] Among them, the dangerous number threshold refers to the number threshold corresponding to the corresponding area. If it exceeds this threshold, it means that there are too many people in the corresponding area, and the danger may be greater. Therefore, when the number of players is less than this threshold, the corresponding judgment monitoring area can be used as the playing monitoring area.

[0050] Through the above method, the play area for risk monitoring of the user can be quickly determined, thereby improving the efficiency of the user in selecting the play area.

[0051] S3, determining a transition mode of the play monitoring area according to the movement information of the monitoring device group, and updating the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area.

[0052] See also Figure 3 , which is a schematic diagram of the interaction between a monitoring device group and a server provided in an embodiment of the present invention. This solution will obtain the location information of each monitoring device group in real time through the server, and determine the movement information of each monitoring setting group through the location information, wherein the movement information refers to the interval distance between the monitoring device groups determined by the point movement information of the monitoring device groups.

[0053] It is understandable that children may be safer when accompanied by their parents. When selecting the play monitoring area, this solution gives priority to areas with lower risk factors. However, in actual applications, children may go to areas with higher risk factors to play alone. In this case, problems may easily arise without the company of their parents. Therefore, this solution will combine the mobile information of the monitoring equipment group to update the play monitoring area in real time, so that early warning reminders can be issued as soon as possible when there is a risk.

[0054] The transition mode refers to an area update mode that updates the play monitoring area in real time in combination with the movement information of the monitoring device. The dynamic monitoring area refers to an area that is updated in real time in combination with the movement information. The risk factor of this area is relatively low for users.

[0055] Based on the above embodiment, the specific implementation of step S3 may be: S31, parsing the movement information to obtain the device distance between the movement positions of the monitoring child device and the monitoring parent device.

[0056] In practical applications, when obtaining the device spacing, it can be determined by monitoring the distance between the positioning positions of the child device and the parent device at the same time.

[0057] S32, when the device distance is less than or equal to the standard distance, determining that the transition mode is a fixed mode, and determining that the play monitoring area in the fixed mode remains unchanged.

[0058] The standard spacing refers to the threshold for distance judgment. If it is less than the threshold, it means that the distance between the monitoring device groups may be close. In this case, the child may be playing with his parents, which is safer. Therefore, the play monitoring area can be changed differently. The fixed mode is a mode in which the play monitoring area is not changed.

[0059] S33: When the device distance is greater than the standard distance within a preset time period, determine that the transition mode is a dynamic mode.

[0060] When the distance between devices is greater than the standard distance within the preset time period, it means that the child may leave the parents to play alone. In this case, the play monitoring area can be updated in real time to determine the areas with lower risk factors when the child plays alone, so that parents can be warned in time when the child is not in these areas.

[0061] In actual applications, the preset time period can be a pre-configured time threshold. If the distance between the child and the parents continues to be far during this time period, it means that the child may be playing alone. The dynamic mode refers to a mode in which the play monitoring area is redetermined when the distance between the monitoring device groups is far.

[0062] S34, counting the current number of players in each of the game monitoring areas according to the dynamic mode, and determining the game monitoring areas where the current number of players is less than a dangerous number threshold as dynamic monitoring areas.

[0063] When the play monitoring area is updated according to the dynamic mode, corresponding updates can be made according to the current number of players in each dynamic monitoring area, and the play monitoring area with the number of players within the safe value is used as the dynamic monitoring area.

[0064] Through the above method, the flexibility of monitoring user gameplay can be improved.

[0065] S4, determining that the monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area is a warning device, generating early warning information corresponding to the warning device and sending it to the corresponding playing terminal.

[0066] After determining the dynamic monitoring area, if the child is not in these areas, in order to improve the safety of the child when playing, an early warning reminder can be issued to the parents. Fence monitoring information refers to the regional fence information of the dynamic monitoring area. When the location of the device held by the child is not in the regional fence corresponding to the dynamic monitoring area, it can be determined that it does not meet the fence monitoring information, and the monitoring sub-device in the corresponding monitoring device group is used as a warning device, that is, a monitoring device for warning reminders. Early warning information refers to information that warns the corresponding user of danger, prompting the corresponding user to take corresponding measures in time to reduce the risk of children playing.

[0067] Based on the above embodiment, the specific implementation of step S4 may be: S41, obtaining the positioning information of the monitoring sub-device, and when the positioning information is not within the dynamic monitoring area, determining that the monitoring sub-device does not meet the fence monitoring information of the dynamic monitoring area, and determining that the corresponding monitoring sub-device is a warning device.

[0068] It is understandable that the monitoring sub-device is a device carried by the child. Therefore, when the positioning information of the monitoring sub-device is not in the dynamic monitoring area, it means that the child may be in other areas with higher risk factors. In this case, in order to promptly remind the corresponding parents, it can be determined that it does not meet the fence monitoring information and the corresponding monitoring sub-device is determined to be a warning device.

[0069] S42, determining the play area through which the positioning information of the warning device passes within the historical time period as the warning monitoring area, and obtaining the sub-historical time period in which the warning device passes through each of the warning monitoring areas.

[0070] In order to enable parents to check their children’s current situation, the child’s current play video can be sent to the parents for viewing, ensuring that parents can take appropriate measures as soon as the child is in danger, thereby improving the safety of the child while playing.

[0071] Specifically, since there are a large number of people in each area, in order to determine the specific person corresponding to the warning device in the video, the specific person may be determined through the video data of the area through which the warning device passed in the historical time period.

[0072] The historical time period may be a pre-configured time period, for example, the time period corresponding to the past 10 minutes based on the current moment. If the warning device has passed through play area 1 and play area 2 in the past 10 minutes, play area 1 and play area 2 may be used as warning monitoring areas, i.e., areas for determining the warning personnel, and then the sub-time periods in which the warning device has passed through play area 1 and play area 2 in the past 10 minutes, i.e., the above-mentioned sub-historical time periods, are obtained, and the corresponding video segments are subsequently retrieved through the sub-time periods to determine the personnel.

[0073] S43, obtaining the historical monitoring video of the warning monitoring area in the sub-historical time period, and identifying the action personnel in the historical monitoring video.

[0074] Historical monitoring video refers to the video obtained by retrieving the monitoring video of the corresponding area in combination with the sub-historical time period. The action personnel refers to the personnel data in the historical monitoring video, such as the personnel outlines in the historical monitoring video. Specifically, the personnel outlines in the historical monitoring video can be identified through the outline recognition technology in the existing technology, and the personnel outlines specifically corresponding to the warning device can be determined in the subsequent process through the identified personnel outlines.

[0075] S44, determining the person corresponding to the warning device as the target person, and obtaining the current playing area where the positioning information is located as the warning prompt area.

[0076] Among them, the target person is the target object for warning corresponding to the warning device, and the warning prompt area is the area where the warning reminder is currently being given.

[0077] In some embodiments, the target person may be determined by the following steps: S441, obtaining the action personnel existing in each of the historical monitoring videos as candidates for selection, and when the number of the candidates for selection is equal to the reference number, determining the candidates for selection as target personnel.

[0078] It is understandable that since the personnel corresponding to the warning device will pass through the warning monitoring area, the monitoring videos in the warning monitoring area can identify the corresponding personnel. Therefore, the action personnel existing in each historical monitoring video can be determined as candidate personnel, that is, personnel waiting for secondary screening, and the personnel corresponding to the warning device can be screened out from these personnel.

[0079] The benchmark number is 1. It can be understood that if the number of candidates is equal to 1, it means that only this person has passed through the warning monitoring area during the historical time period, and his movement trajectory is the same as that of the person corresponding to the warning device. Therefore, the candidate can be directly determined as the target person.

[0080] S442: When the number of the candidates is greater than the reference number, a plurality of human body detection points corresponding to each of the candidates is acquired.

[0081] If the number of people is greater than the benchmark number, it means that more than one person has the same action trajectory as the person corresponding to the warning device in the historical time period. At this time, a corresponding determination can be made based on the user information corresponding to the warning device, specifically, based on height.

[0082] It is understandable that since the height information of different people may be different, the height information can be used to screen the corresponding people. When determining the height, the human body detection point can be used for corresponding determination. The human body detection point is the point where the height of the person is detected, for example, it can be the two characteristic points of the human head and the foot vertical point.

[0083] It is worth mentioning that when obtaining the human body detection points corresponding to each candidate, the video frame that best reflects the height of the person when standing can be selected for determination.

[0084] S443, inputting the human body detection points into a height detection model, outputting the predicted height of the candidate, and obtaining the height difference between the predicted height and the height information corresponding to the user information.

[0085] The height detection model is a model that determines the height of a person based on the body detection points. When determining the predicted height of the candidate, the vertical distance between the body detection points can be obtained first, and then the vertical distance can be actually converted to obtain the predicted height, that is, the height predicted by the height detection model for each candidate.

[0086] It is understandable that since errors may occur during prediction, in order to improve the accuracy of processing, the target person corresponding to the warning device can be ultimately determined by the difference between the height differences.

[0087] S444, obtaining the candidate whose height difference is within the normal difference range as the target person.

[0088] The normal difference interval is the difference interval corresponding to the pre-configured normal range. If the height difference is within this interval, it means that the predicted height and the height information are very close. Therefore, the candidate corresponding to the predicted height can be used as the target person.

[0089] Through the above method, the person corresponding to the warning device can be quickly determined, and the accuracy of data processing can be improved.

[0090] S45, obtaining the current monitoring video of the warning prompt area, highlighting the target person in the current monitoring video to obtain warning information and sending it to the player end.

[0091] When highlighting the target person in the current monitoring video, a red warning box corresponding to the target person can be generated to highlight him / her, so that parents can intuitively view their children's play situation and take timely corresponding measures when the child is in danger.

[0092] Through the above method, users who are at risk of danger can be reminded in time, thereby improving safety during play.

[0093] In addition, based on the above embodiments, this solution also includes the following embodiments: S46, responding to the monitoring request of the monitoring terminal, obtaining the first distance between the monitoring child device and the monitoring main device in each play area, and determining the monitoring device group whose first distance is smaller than the standard distance as the accompanying device group.

[0094] In actual applications, parents may carry both a sub-monitoring device and a main monitoring device at the same time. In this case, the distance between the monitoring device groups is small, but the child may be far away from the parent. Therefore, in order to enable supervisors to remind the corresponding parents in time and reduce the danger when the child is playing, abnormal persons can be displayed to supervisors for viewing, so that they can intuitively see the abnormal persons and give corresponding reminders.

[0095] Specifically, the distance between the monitoring child device and the monitoring mother device, i.e., the first distance, can be obtained first, and the first distance is determined to be less than the standard distance, i.e., the monitoring device group with a closer distance is the accompanying device group, and the accompanying setting group is used to screen abnormal persons in the monitoring video corresponding to the corresponding play area in the subsequent process. Among them, the monitoring terminal is the terminal held by the supervisor, the monitoring request is the supervisor's request for abnormal supervision of each play area, and the accompanying device group is the monitoring device group with a closer distance.

[0096] S47, obtaining the current captured video shot by the monitoring equipment in each of the play areas, and extracting the contours of people in the video frames corresponding to the current captured video.

[0097] In practical applications, the monitoring device may be a surveillance camera, and the currently collected video refers to the video data collected at the current moment in each play area, and each play area may be configured with a corresponding monitoring device. When obtaining the person's outline, the outline extraction technology in the prior art may be used for corresponding determination.

[0098] S48, performing attribute recognition on the person profile, determining the person profile with adult attributes as the first profile, and determining the person profile with child attributes as the second profile.

[0099] When identifying the attributes of a person's outline, it can be determined through multiple dimensions such as the height, body shape, and facial features of each person's outline. The attributes in this solution specifically include adult attributes and child attributes. Different attributes can be used to determine whether the outline corresponds to an adult or a child. Therefore, the attributes and positions of multiple outlines and the accompanying equipment group can be combined to determine whether there are any person outlines in the play area that do not match the positioning information.

[0100] S49, acquiring a first contour and a second contour whose contour interval is smaller than a reference contour interval as the same accompanying contour group, and counting a first number of the accompanying device groups and a second number of the accompanying contour groups.

[0101] The profile spacing refers to the spacing between the first profile and the second profile. The reference profile spacing is a threshold used for distance determination. If the spacing is less than the threshold, it means that the spacing between the corresponding profiles is close, and they may be playing together, so they can be regarded as the same accompanying profile group. The first number refers to the number of accompanying device groups, and the second number refers to the number of accompanying profile groups.

[0102] S410, when the first number is greater than the second number, determining the outline of the abnormal person in the currently collected video, highlighting the outline of the abnormal person to obtain abnormal monitoring data and sending it to the monitoring end.

[0103] If the first number is greater than the second number, it means that the number of accompanying device groups is greater than the number of accompanying profile groups. There may be a situation where the distance between the parent and the child in the video is far, but the distance between their monitoring devices is close. In this case, the abnormal person profile in the current collected video can be determined, and after highlighting it, the abnormal monitoring data is obtained and sent to the monitoring end. Among them, the abnormal person profile refers to the abnormal person profile, and when it is highlighted, its corresponding red warning box can be generated. Abnormal monitoring data refers to the monitoring video data with abnormal person data.

[0104] In some embodiments, the above abnormal person profile can be determined by the following steps: S4101: When the first quantity is greater than the second quantity, obtain a quantity difference between the first quantity and the second quantity.

[0105] It is understandable that the reason for obtaining the quantity difference is to determine the number of user pairs with abnormalities, so that abnormal users with the corresponding number of pairs can be found later to provide abnormal reminders.

[0106] S4102: Acquire a first contour other than the accompanying contour group in the video frame as a first contour to be detected, and a second contour as a second contour to be detected.

[0107] It can be understood that the spacing between the accompanying profile groups is relatively close, and only the remaining personnel profiles except the accompanying profile group are likely to be abnormal personnel profiles. Therefore, the first profile except the accompanying profile group can be used as the first profile to be inspected, that is, the adult profile waiting for abnormal verification, and the second profile except the accompanying profile group can be used as the second profile to be inspected, that is, the child profile waiting for abnormal verification.

[0108] S4103: Determine that a first contour to be detected and a second contour to be detected whose contour interval is greater than the reference contour interval are in the same group to be detected, and the number of the first contour to be detected and the second contour to be detected in the group to be detected is equal to the reference number.

[0109] Specifically, there is only one adult and one child in each of the inspection groups. If the distance between the first inspection contour and the second inspection contour is too far, it means that they are likely to meet the abnormal condition, so they can be regarded as the same inspection group.

[0110] S4104, counting the number of persons to be inspected in the group to be inspected, and if the number of persons to be inspected is equal to the number difference, determining that the person profile corresponding to the group to be inspected is an abnormal person profile.

[0111] If the number to be inspected and the number difference are equal, for example, both are equal to 1, it means that the group of abnormal people in the video and the number difference that appears abnormal during comparison correspond to each other, so the person outline corresponding to the group to be inspected can be directly determined as the abnormal person outline.

[0112] S4105, if the number of persons to be inspected is not equal to the number difference, the historical acquisition video of the corresponding monitoring device is obtained, and the group to be inspected is re-matched according to the historical acquisition video to obtain multiple abnormal person outlines that match each other in the current acquisition video.

[0113] If the number to be checked is not equal to the number difference, it means that the abnormal group of people in the video and the abnormal number difference when compared do not correspond. For example, the same person outline may exist in multiple groups to be checked. When the number of abnormal groups is greater than 1, the corresponding parents and children may not be determined. In this case, the same person outline may exist in multiple groups to be checked, resulting in an inequality between the number to be checked and the number difference. Therefore, in this case, it is necessary to further match multiple adult outlines and child outlines, determine the corresponding abnormal person outlines and display them to the management personnel, so that the management personnel can remind the corresponding parents.

[0114] Specifically, in some embodiments, step S4105 may be implemented in the following manner: If the number to be inspected is not equal to the number difference, the historically collected video of the monitoring device within the historical monitoring time period is obtained.

[0115] Among them, the historical monitoring time period refers to the pre-configured time period for abnormal verification, for example, it can be the time period corresponding to 10 minutes traced back based on the current time, and the historical collected video refers to the monitoring video data retrieved by the monitoring equipment within the historical monitoring time period.

[0116] A historical group to be inspected corresponding to the group to be inspected in a plurality of historical frames corresponding to the historical acquisition video is extracted, and the historical group to be inspected whose contour spacing is smaller than the reference contour spacing is obtained as an accompanying group to be inspected.

[0117] It is understandable that if the corresponding adults and children are playing together, they may be close to each other at a certain moment in history. For example, they may be holding hands at the beginning and then play separately at the current moment. Therefore, after obtaining the historical collection video, multiple historical frames can be analyzed frame by frame to determine the historical inspection group whose contour spacing is less than the benchmark contour spacing as the accompanying inspection group.

[0118] It is worth mentioning that the baseline contour spacing can be set to the spacing between parents and children when they hold hands or engage in other contact behaviors, thereby improving the accuracy of the analysis.

[0119] The remaining groups to be inspected in which the first profile to be inspected and the second profile to be inspected exist in the accompanying group to be inspected are deleted, and the first profile to be inspected and the second profile to be inspected in the accompanying group to be inspected are determined to be profiles of abnormal persons that match each other.

[0120] It is understandable that since there may be personnel outlines corresponding to the accompanying group to be inspected in other groups to be inspected, in order to improve the accuracy of the display, other groups to be inspected with corresponding personnel outlines can be deleted, and then the personnel outlines in the accompanying group to be inspected can be determined as the corresponding abnormal personnel outlines.

[0121] An abnormality display frame corresponding to each of the accompanying inspection groups is generated, and different preset pixel values ​​are called to update the abnormality display frames corresponding to different accompanying inspection groups.

[0122] In actual applications, the abnormal display frame can be an outer rectangular frame corresponding to the corresponding person's outline. Since there may be multiple groups of abnormal people, in order to distinguish between parents and children in different groups, the abnormal display frame can be updated with different pixel values. For example, when there are two groups of abnormal people, the abnormal display frames of different groups can be updated with red and green respectively. Among them, the abnormal display frame refers to a rectangular frame that highlights the outline of the abnormal person.

[0123] Through the above method, supervisors can be reminded of abnormalities in a timely manner, thereby improving the accuracy of supervision of non-powered parks and reducing the risks of users' play.

[0124] See also Figure 4 , is a structural diagram of an operation processing system of a non-powered park based on multimodal data provided by an embodiment of the present invention, and the operation processing system of the non-powered park based on multimodal data includes: An input module, used to determine a monitoring device group corresponding to the gaming terminal according to the scanned code input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device; A planning module, used to obtain user information input by the game terminal, and determine a game monitoring area corresponding to the game terminal according to the user information; A monitoring module, configured to determine a transition mode of the play monitoring area according to movement information of the monitoring device group, and to update the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; The early warning module is used to determine that the monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area is a warning device, generate early warning information corresponding to the warning device and send it to the corresponding playing terminal.

[0125] Figure 4 The apparatus of the embodiment shown can be used to perform Figure 1 The implementation principles and technical effects of the steps in the method embodiment shown are similar and will not be repeated here.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for operating a non-powered park based on multimodal data, characterized in that: include: Determine the monitoring device group corresponding to the gaming terminal according to the scanned code input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device; Acquire user information input by the gaming terminal, and determine a gaming monitoring area corresponding to the gaming terminal according to the user information; Determine a transition mode of the play monitoring area according to the movement information of the monitoring device group, and update the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; Determine the monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area as a warning device, generate early warning information corresponding to the warning device and send it to the corresponding playing terminal; Determining a transition mode of the play monitoring area according to the movement information of the monitoring device group, and updating the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area, including: Parsing the movement information to obtain a device distance between the moving positions of the monitoring child device and the monitoring mother device; When the device spacing is less than or equal to the standard spacing, determining that the transition mode is a fixed mode, and determining that the play monitoring area in the fixed mode remains unchanged; When the device spacing is greater than the standard spacing within a preset time period, determining that the transition mode is a dynamic mode; The current number of players in each of the play monitoring areas is counted according to the dynamic mode, and the play monitoring area where the current number of players is less than a threshold value of the number of dangerous players is determined as a dynamic monitoring area.

2. The method according to claim 1, characterized in that Acquiring user information input by the gaming terminal, and determining a gaming monitoring area corresponding to the gaming terminal according to the user information, including: Obtaining area restriction conditions corresponding to each play area in the play area, wherein the area restriction conditions at least include height restriction and / or age restriction; Comparing the area restriction conditions based on the user information, and when the user information satisfies the area restriction conditions, determining the corresponding play area as a to-be-screened monitoring area, wherein the user information includes at least height information and age information; Acquire the play attributes corresponding to each of the to-be-screened monitoring areas, wherein the play attributes include low-risk attributes and high-risk attributes, and determine that the to-be-screened monitoring area corresponding to the low-risk attribute is a planned monitoring area; Receive area adjustment information of the planned monitoring area from the player terminal, and update the planned monitoring area according to the area adjustment information to obtain a playing monitoring area.

3. The method according to claim 2, characterized in that Receiving area adjustment information of the planned monitoring area from the player terminal, and updating the planned monitoring area according to the area adjustment information to obtain a playing monitoring area, including: Retrieving a site layout diagram corresponding to the play site, highlighting a reference area in the site layout diagram corresponding to the planned monitoring area to obtain a play planning diagram and sending it to the play terminal; receiving a customized area demarcated by the player terminal based on the game planning map, and determining a planned monitoring area completely located within the customized area as a game monitoring area; Acquire a planned monitoring area that is not completely located in the customized area as a determined monitoring area, and count the number of players corresponding to the determined monitoring area at the current moment; The monitoring area where the number of players is less than the dangerous number threshold is determined as the playing monitoring area.

4. The method according to claim 1, characterized in that: Determining a monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area as a warning device, generating early warning information corresponding to the warning device and sending it to the corresponding player terminal, including: Acquire the positioning information of the monitoring sub-device, and when the positioning information is not within the dynamic monitoring area, determine that the monitoring sub-device does not meet the fence monitoring information of the dynamic monitoring area, and determine that the corresponding monitoring sub-device is a warning device; Determine the play area where the positioning information of the warning device passes within the historical time period as the warning monitoring area, and obtain the sub-historical time period where the warning device passes through each of the warning monitoring areas; Obtaining a historical monitoring video of the warning monitoring area in the sub-historical time period, and identifying an action person in the historical monitoring video; Determine the person who is in action and corresponds to the warning device as the target person, and obtain the current playing area where the positioning information is located as the warning prompt area; The current monitoring video of the warning prompt area is obtained, and the target person in the current monitoring video is highlighted to obtain early warning information and send it to the player end.

5. The method according to claim 4, characterized in that Determining the action personnel corresponding to the warning device as the target personnel includes: Acquire the action personnel existing in each of the historical monitoring videos as the candidates, and when the number of the candidates is equal to the reference number, determine the candidates as the target personnel; When the number of the candidates is greater than the reference number, obtaining a plurality of human body detection points corresponding to each of the candidates; Input the human body detection points into a height detection model, output the predicted height of the candidate, and obtain the height difference between the predicted height and the height information corresponding to the user information; The candidate whose height difference is within the normal difference range is obtained as the target person.

6. The method according to claim 1, characterized in that Also includes: In response to a monitoring request from the monitoring terminal, a first distance between the monitoring child device and the monitoring parent device in each play area is obtained, and a monitoring device group whose first distance is smaller than a standard distance is determined as an accompanying device group; Acquire the current captured video shot by the monitoring equipment of each of the play areas, and extract the outline of the person in the video frame corresponding to the current captured video; Performing attribute recognition on the person profile, determining the person profile with adult attributes as the first profile, and determining the person profile with child attributes as the second profile; Acquire a first contour and a second contour whose contour interval is smaller than a reference contour interval as the same accompanying contour group, and count a first number of the accompanying device groups and a second number of the accompanying contour groups; When the first number is greater than the second number, the outline of the abnormal person in the currently collected video is determined, the outline of the abnormal person is highlighted and displayed, and the abnormal monitoring data is obtained and sent to the monitoring end.

7. The method according to claim 6, characterized in that When the first number is greater than the second number, determining the outline of the abnormal person in the currently collected video includes: When the first quantity is greater than the second quantity, obtaining a quantity difference between the first quantity and the second quantity; Acquire the first contour in the video frame except the accompanying contour group as the first contour to be detected, and the second contour as the second contour to be detected; Determine that the first to-be-tested contour and the second to-be-tested contour whose contour spacing is greater than the reference contour spacing are the same to-be-tested group, and the number of the first to-be-tested contour and the second to-be-tested contour in the to-be-tested group is equal to the reference number; Counting the number of people to be inspected in the group to be inspected, and if the number of people to be inspected is equal to the number difference, determining that the person profile corresponding to the group to be inspected is an abnormal person profile; If the number to be inspected is not equal to the number difference, the historical acquisition video of the corresponding monitoring device is obtained, and the group to be inspected is re-matched according to the historical acquisition video to obtain multiple abnormal person contours that match each other in the current acquisition video.

8. The method according to claim 7, characterized in that If the number to be inspected is not equal to the number difference, the historical acquisition video of the corresponding monitoring device is obtained, and the group to be inspected is re-matched according to the historical acquisition video to obtain multiple abnormal person profiles that match each other in the current acquisition video, including: If the number to be inspected is not equal to the number difference, obtaining the historical collected video of the monitoring device within the historical monitoring time period; Extracting a historical group to be inspected corresponding to the group to be inspected from a plurality of historical frames corresponding to the historical acquisition video, and obtaining the historical group to be inspected whose contour spacing is smaller than the reference contour spacing as an accompanying group to be inspected; Deleting the remaining groups to be inspected in which the first profile to be inspected and the second profile to be inspected exist in the accompanying group to be inspected, and determining that the first profile to be inspected and the second profile to be inspected in the accompanying group to be inspected are profiles of abnormal persons that match each other; An abnormality display frame corresponding to each of the accompanying inspection groups is generated, and different preset pixel values ​​are called to update the abnormality display frames corresponding to different accompanying inspection groups.

9. An operation processing system for a non-powered park based on multimodal data, characterized in that: include: An input module, used to determine a monitoring device group corresponding to the gaming terminal according to the scanned code input information of the gaming terminal, wherein the monitoring device group includes a monitoring child device and a monitoring parent device; A planning module, used to obtain user information input by the game terminal, and determine a game monitoring area corresponding to the game terminal according to the user information; A monitoring module, configured to determine a transition mode of the play monitoring area according to movement information of the monitoring device group, and to update the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area; An early warning module, used to determine the monitoring device group that does not meet the fence monitoring information of the dynamic monitoring area as a warning device, generate early warning information corresponding to the warning device and send it to the corresponding playing terminal; Determining a transition mode of the play monitoring area according to the movement information of the monitoring device group, and updating the play monitoring area in real time based on the transition mode to obtain a dynamic monitoring area, including: Parsing the movement information to obtain a device distance between the moving positions of the monitoring child device and the monitoring mother device; When the device spacing is less than or equal to the standard spacing, determining that the transition mode is a fixed mode, and determining that the play monitoring area in the fixed mode remains unchanged; When the device spacing is greater than the standard spacing within a preset time period, determining that the transition mode is a dynamic mode; The current number of players in each of the play monitoring areas is counted according to the dynamic mode, and the play monitoring area where the current number of players is less than a threshold value of the number of dangerous players is determined as a dynamic monitoring area.

Images